Sign up
View stargazersSign up

Session import ignores compressed Codex rollouts (`.jsonl.zst`), so their conversations are never discovered#21755

opened10d agobySSriCharanK2002

What happened

During onboarding, the user imported their existing Codex and Claude Code history into the desktop app. Many recent Codex conversations never showed up, and no older Codex history came in for any project. Re-running the import from /welcome brought in nothing more for Codex.

Diagnosis

Codex now stores older rollouts zstd-compressed, as rollout-*.jsonl.zst next to the plain rollout-*.jsonl files under $CODEX_HOME/sessions/YYYY/MM/DD/. On this machine, 1,157 of 1,324 rollouts were .jsonl.zst. Every rollout older than about two weeks was compressed, along with some recent ones.

discoverCodexTranscripts only accepts plain .jsonl files (apps/server/src/project/AgentSessionScanner.ts:1014 at v0.1.0):

ts
1if (!entry.startsWith("rollout-") || !entry.endsWith(".jsonl")) continue;

Compressed rollouts are never discovered. They don't appear in the scan's thread counts, aren't counted as skipped, and the UI never mentions them. The rest of the pipeline also assumes plain text: readCwd and readTranscript read raw bytes, and fallbackSessionId strips .jsonl.

A second limit stacks on top. prepareRecentThreads only takes transcripts whose file mtime is within RECENT_THREAD_WINDOW_MS (30 days) of the moment the import runs. Even with .zst support, history older than 30 days can't be imported, and there's no setting or retry path to reach it.

Steps to reproduce

  1. Use Codex CLI 0.162.x until it has compressed some rollouts to .jsonl.zst under ~/.codex/sessions/. Some of these can be recent, under 30 days old.
  2. In T3 Code 0.1.0, go through onboarding (or open #/welcome in the desktop app) and import a project those sessions ran in.
  3. Only the conversations whose rollouts are still plain .jsonl are imported. Conversations that exist only as .jsonl.zst are silently missing.

Version

0.1.0 (desktop app "T3 Code (Beta)", local server 0.1.0)

Environment

macOS (Darwin 27, arm64), Node 26.8.2, codex-cli 0.162.1, Claude Code 2.1.303

Evidence

shell
1# Rollout files under ~/.codex/sessions
21324 total = 1157 *.jsonl.zst + 167 *.jsonl
3
4# Top-level Codex conversations (excluding subagent rollouts) with at least one user message,
5# last activity in the 30 days before import: 129
6# imported: 98
7# missing because every rollout segment is .jsonl.zst: 20
8# missing for other reasons (folder not selected / deleted): 11
9
10# All-time, per project (top-level conversations / imported / missing only as .zst)
11project A 196 / 44 / 152
12project B 45 / 11 / 34
13project C 89 / 15 / 74
14project D 36 / 21 / 15
15# 6 more projects whose Codex history exists only as .jsonl.zst

Related issues

#17484 and #16222 cover other import limits (the record/transcript caps, and subagent rollouts imported as threads). #10961 covers re-running import after onboarding. None of them covers compressed rollouts or the 30-day mtime window.

Fix applied or workaround

This confirms the diagnosis and gives a manual workaround. Opening an old conversation in the Codex app rewrites its rollout from .jsonl.zst back to a plain .jsonl with a fresh mtime. After the user opened a batch of old threads in the Codex app, 40 rollouts dated months earlier became plain .jsonl files modified today, and re-running the T3 import picked up 37 of them. The other 3 were in folders without a T3 project. That only works one conversation at a time, so it isn't practical for hundreds of sessions.

No fix was applied for this. A DB backup was taken, and the import was re-run via location.hash = '#/welcome' in the desktop DevTools. That recovered conversations missed for other reasons, but no .zst Codex history. A possible fix: discover rollout-*.jsonl.zst, stream-decompress it in readCwd/readTranscript (with the existing byte budgets applied to the decompressed size), and treat .jsonl and .jsonl.zst segments of one session as a single conversation. Separately, consider letting the user choose the import window instead of a fixed 30 days.

Filed by

Claude Code (claude-opus-5-5) via t3 triage

JJulius Marminge10d ago

Note

Grok responding on behalf of Julius.

Thanks for the careful counts. Confirmed on current main.

Codex discovery only accepts files that start with rollout- and end in .jsonl (apps/server/src/project/AgentSessionScanner.ts:945), and the session id fallback strips only .jsonl (AgentSessionScanner.ts:1372). Compressed rollout-*.jsonl.zst files are skipped without being counted or reported. Codex has compressed older rollouts this way for months, so this is a gap in how we read Codex's normal storage, not an unusual setup. Accepting the compressed rollout part as a bug.

The 30 day limit is intended. Import only looks at transcripts modified in the last 30 days (AgentSessionScanner.ts:57, :1256). Importing older history, or making that window configurable, would be new behavior, so please open an Idea for it: https://github.com/pingdotgg/t3code/discussions/new?category=ideas

For now, opening a conversation in Codex writes it back as plain .jsonl with a fresh date, as you found, and import picks it up after that.

JJulius Marminge8d ago

Reproduced on Linux with main 70e5e384da, running the web dev server against isolated state. In /welcome → Projects, a fresh plain Codex rollout appears with one conversation; an equivalent .jsonl.zst-only project is absent. Decompressing that same fixture while preserving its mtime makes the project appear. Two focused scanner tests also fail with empty discovery/import results. The intended 30-day window is outside this fix.