[Bug]: Orchestrator V2 nightly silently reuses a preview-created database and misses newer V1 threads#15017

openedOct 3, 2026byPParanjayy

TL;DR: An earlier Orchestrator V2 preview left a September 24 database on disk. The October 3 Orchestrator V2 nightly reused it, so newer stable V1 threads and changes did not appear. A local repair recovered 16 missing threads and 4 projects; detecting this stale preview database or explaining recovery would help other affected users.

Area: apps/server, V1 → Orchestrator V2 migration
Impact: Recent local threads, projects, messages, and thread status changes are silently absent after upgrading to the Orchestrator V2 nightly.

Steps to reproduce

  1. Run an earlier Orchestrator V2 alpha/preview once. This creates ~/.t3/userdata/statev2.sqlite from the then-current V1 state.sqlite.
  2. Continue using stable V1, creating threads and changing thread state there. The V1 database advances, while the preview Orchestrator V2 database remains on disk.
  3. Install and start the first Orchestrator V2 nightly, 0.0.46-nightly.20261003.2610, on the same Mac and data directory.
  4. Inspect the sidebar or compare the local V1 and Orchestrator V2 database inventories. Restarting the nightly does not bring in the V1 changes made since the preview's copy.

Expected behavior

The first public Orchestrator V2 nightly should either migrate the current V1 state, or clearly detect and explain that an older preview-created Orchestrator V2 database is taking precedence and newer V1 data will be omitted. Existing Orchestrator V2-only work must be preserved.

Actual behavior

The nightly silently reuses the September 24 Orchestrator V2 database. Its legacy tables are a snapshot of V1 as of that date, so the lazy importer cannot see later V1 records. Recent threads are missing, and existing threads can retain old snooze/settlement metadata.

Evidence from one affected Mac

  • The original statev2.sqlite was created September 24, before this Orchestrator V2 nightly. It contained V1 events through sequence 141672.
  • The current stable V1 state.sqlite contained events through sequence 149248 on October 2: 7576 later V1 events.
  • Before repair, the Orchestrator V2 store lacked 16 V1 thread records and 4 projects. 18 existing threads had later V1 messages, and 15 had differing settlement/snooze metadata.
  • A backup-based offline reconciliation, followed by restarting the nightly, brought the local store to 0 missing threads, 0 missing projects, 0 pending transcripts, and 0 incorrect non-settled statuses. The recovered threads appeared in the app. This confirms the V1 records were present on disk and the normal Orchestrator V2 startup had not imported them.

The shipped nightly bundle's initializeV2Database returns immediately when statev2.sqlite already exists:

js
1if (yield* fs.exists(destinationPath)) return;

The Orchestrator V2 importer then reads the V1 projection tables inside that existing Orchestrator V2 database, not the newer V1 state.sqlite. This explains why restarting cannot catch up.

Scope / possible fix

I understand that the migration guide documents a one-time copy and intentional divergence between Orchestrator V1 and Orchestrator V2. This case is specifically a Orchestrator V2 database seeded by pre-release preview use before the first broadly released Orchestrator V2 nightly, followed by continued stable V1 use. Silently treating that preview database as the release migration source leaves the user with an apparently successful but stale import.

An age/source check with a clear migration choice or recovery guidance would prevent silent loss. Selective catch-up might also work, provided it does not overwrite Orchestrator V2-only data.

The separate behavior of snoozed threads disappearing during ordinary use is not part of this report; the snooze differences above are attributable to this stale migration snapshot.

Environment

  • macOS, local T3 Code data store
  • T3 Code nightly 0.0.46-nightly.20261003.2610
  • Earlier Orchestrator V2 preview/alpha use, followed by continued stable V1 use

Local recovery / attribution

The local workaround was carried out with a Codex agent inside T3 Code: it prepared an offline repair script, backed up both databases, reconciled the missing records, and verified the import after restart. The integrated agent workflow made the investigation and recovery straightforward. This was a local data repair, not an upstream code fix.

AI-assisted investigation and report through Codex in T3 Code; this revision was prepared by GPT-6.1-Sol.

Synced from GitHub

One additional observation from the same upgrade: saved, unsent composer drafts that were visible in the previous version no longer appeared in the Orchestrator V2 nightly.

This is a user-observed symptom so far. We have not checked where those drafts were stored, whether they remain recoverable, or whether the cause is the stale database described above, the desktop profile change, or another migration path. The verified local thread repair did not verify draft recovery.

It may be useful to check draft preservation alongside the V1 → Orchestrator V2 migration. I am adding it here for context; it would need separate storage evidence or a reproduction before treating it as the same confirmed defect.

This comment was prepared by GPT-6.1-Sol through Codex in T3 Code, based on the user's report.

Note

Grok responding on behalf of Julius.

Triage

Thanks for such a careful report, @Paranjayy. The before-and-after counts and the offline reconciliation made this straightforward to confirm.

Confirmed on 0.0.46-nightly.20261003.2610. If a statev2.sqlite file already exists, the nightly treats it as the finished V2 store and never reads the live V1 database again. It can't tell a file left by a preview build apart from one the nightly made itself. That means threads and thread-state changes you made in stable V1 after the preview get skipped without any warning, and restarting won't catch them up. Your state.sqlite isn't modified or deleted. The newer V1 rows are still there, they just never get read.

  • initializeV2Database returns as soon as the destination file exists (source).
  • The lazy importer reads projection_threads from inside that V2 file, not from state.sqlite (source).
  • Metadata repair only fills fields that are still empty on threads that were already imported. It never refreshes settlement or snooze state from a newer V1 database. That lines up with what you saw: threads and projects missing, later messages absent, and stale snooze and settlement metadata.

How this relates to #14871: copying once is intentional after a real V2 start, and the thread migration guide says later V1 and V2 changes don't sync. This report covers an earlier gap. V2-branch preview builds such as v0.0.43-preview.20260923.2177 already wrote userdata/statev2.sqlite and had the same early return. The stable nightly from that week didn't use that path at all. So if a preview created that file on September 24, this nightly keeps it and treats it as the migration source.

Until there's a fix, and keeping any V2-only work safe:

  • If the preview database has nothing you need to keep: stop T3 Code. Move statev2.sqlite and its -wal and -shm files somewhere else, but don't delete them yet and don't touch state.sqlite. Then start the nightly, which will take a fresh copy of your current V1 data. Once your threads look right, you can delete the moved files.
  • If the preview database has V2-only threads you need: leave it where it is. There's no supported catch-up command yet, so recovery takes an offline merge like the one you did.

Possible fixes: detect a V2 snapshot whose copied V1 tables are older than the current state.sqlite and warn before continuing, or selectively import the missing V1 threads, messages, and metadata without overwriting V2-only data. Picking between those is a maintainer decision. Adding a note to #14871 alone wouldn't help installs that have already launched this nightly.

Synced from GitHub

Hit a variant of this on Windows 11. I never ran a preview build. Here statev2.sqlite came from the first public V2 nightly (orchestration_v2_projection_metadata updated 2026-10-03T07:07:42Z).

Then I lost my sidebar to #15001 and rolled back to 0.0.45-nightly.20261002.2595 to keep working. That puts me in the same trap this issue describes, just without a preview involved. Any V1 work I do now will be missing the next time I start V2, because V2 sees the existing statev2.sqlite and never reads state.sqlite again.

The drift has already started. One day after the cutover, V1 state.sqlite has 428 threads and 12,493 messages, while statev2.sqlite has 422 threads and 12,267 messages.

Anyone rolling back to V1 to dodge a V2 bug will hit this. To work around it, I moved statev2.sqlite* out of ~/.t3/userdata so the next V2 launch migrates from current V1 data. It would help if V2 compared V1's last-write time or event sequence with the snapshot it imported from, and re-imported or at least warned when V1 is newer.

Synced from GitHub

Still reproducible on stable 0.1.0 (macOS). The earlier reports here were on nightlies and a manual rollback; this one came through the normal stable release, so the trap is no longer nightly-only.

Shape of this instance: a 0.0.46-nightly ran once on this machine, seeded statev2.sqlite, and the machine returned to stable 0.0.45 about 2.5 minutes later. Stable then wrote V1 for ~37 hours. When 0.1.0 arrived, it opened the Oct 14 snapshot and six threads were simply gone from the sidebar.

Timeline (from logs/desktop.trace.ndjson.7, local EDT / UTC)

LocalUTCEvent
07:57:3511:57:35Zdesktop.updates.applyUpdateAvailable → 0.0.46-nightly.20261014.3105
07:57:4411:57:44ZapplyUpdateDownloaded
07:58:4111:58:41Znightly boots: migrations 55–62 (OrchestrationV2 … ThreadFailureLookupIndex) and statev2.sqlite seeded
08:00:0812:00:08ZapplyUpdateAvailable → 0.0.45 (back to stable)
08:00:1612:00:16ZapplyUpdateDownloaded
—Oct 14 12:31Z → Oct 16 01:00Zstable 0.0.45 writes V1 only: events 1000 → 6261, 6 new threads
Oct 15 ~20:26Oct 16 ~00:26Z0.1.0 first appears in traces, reads the stale statev2.sqlite

Divergence measured on disk

state.sqlite (V1)statev2.sqlite
projection_threads93
orchestration_events62611151
projection_state.last_applied_sequence6261999 (frozen at 2026-10-14T02:28:20.146Z)
orchestration_v2_events—0
auth_sessions2517
auth_pairing_links2011

orchestration_v2_projection_metadata reads last_sequence = 1151, updated_at = 2026-10-14T11:58:42.752Z — not one V2 write in the ~37 hours between the seed and the 0.1.0 upgrade.

The seed itself was correct — the defect is purely the missing re-sync

Worth isolating, because 999 looks like a truncation bug and isn't one. In V1, event 999 occurred at 2026-10-14T02:28:20.146Z and event 1000 at 2026-10-14T12:31:49.514Z. The seed ran at 11:58:41Z, between them, so V1 held exactly 999 events at that instant. Sequences 1–999 in statev2.sqlite match V1 one-for-one on both event_id and sequence (999/999).

The copy was therefore complete and correct for the moment it was taken. Everything lost is a consequence of there being no staleness check afterward. Thread-level cut:

  • Imported (all events ≤ 999): 2f9e8b79… (seq 2–33), 5650d604… (36–999), 0d912b99… (751–966)
  • Never imported (all start ≥ 1000): 95b9898f… (1000), 089fff1c… (1069), 57dfed50… (1175), 6fdad9c0… (1244), 65753d74… (1265), 0bf63032… (1404)

statev2.sqlite also carries 152 appended events (seq 1000–1151) whose event_ids appear nowhere in V1 (0/152) and which cover only those same 3 threads plus 2 projects — the migration's own re-emitted thread.created / message.updated / turn-item.updated records, ending with two thread.visited at 11:58:42Z.

Code is unchanged at the v0.1.0 tag

Both mechanisms @juliusmarminge confirmed are still present:

Suggested detection

Picking up @kharitonov-egor's suggestion, the comparison needs no new bookkeeping — both numbers already exist. At startup, compare V1 projection_state.last_applied_sequence (or max(sequence) from its orchestration_events) against orchestration_v2_projection_metadata.last_sequence. Here that is 6261 vs 1151: a 5110-event gap that could have been surfaced as a prompt or a warning instead of silence. A one-time guard would also help: refuse to seed, or mark the store provisional, when the build writing it is a pre-release and a stable build may follow.

Recovery on this machine

The maintainer workaround in this thread worked unchanged. With the app fully quit, state.sqlite verified integrity_check: ok at 9 threads / 6261 events; statev2.sqlite* was moved out of userdata/ (not deleted) and the app restarted to re-seed from current V1. state.sqlite was never written to. Re-pairing one device was expected, since 4 auth_sessions rows existed only in the old V2 file.

Environment

  • macOS 27.0.0, darwin arm64, Node v26.8.2
  • Bug surfaced on: desktop app 0.1.0 against its local server (serverVersion 0.1.0, orchestrationProtocolVersion 2)
  • Drift accumulated on: stable 0.0.45
  • statev2.sqlite seeded by: 0.0.46-nightly.20261014.3105
  • updateChannel: "latest", updateChannelConfiguredByUser: true
  • Triage CLI that produced the context file: t3 0.1.0 via npx t3 triage

Investigated with Claude Opus 5 (claude-opus-5) in Claude Code, run from t3 triage. Diagnosis read against a v0.1.0 tag checkout; counts are from this machine's own databases, with the home directory path redacted.

Synced from GitHub

Another stable 0.1.0 case (macOS, arm64), with a different starting point: I never rolled back by hand. The switch from Nightly back to the V1 build arrived as an ordinary in-app update.

Timeline (from ~/Library/Caches/com.t3tools.t3code.ShipIt/ShipIt_stderr.log and file timestamps, UTC)

UTCEvent
Oct 12 18:01:13ZA Nightly build seeds statev2.sqlite from state.sqlite (all orchestration_v2_legacy_imports.shell_imported_at = 2026-10-12T18:01:13.615Z; v1-local-storage-imported marker written at the same time)
Oct 12 18:14:42ZShipIt: Moving bundle from T3 Code (Nightly).app to T3 Code (Alpha).app, so the machine is back on the V1 build about 13 minutes later
Oct 12 → Oct 16 10:01ZV1 keeps writing to state.sqlite for ~3.5 days
Oct 16 10:01:25ZShipIt: Moving bundle from T3 Code (Alpha).app to T3 Code (Beta).app (0.1.0), which opens the Oct 12 snapshot

Divergence measured on disk

state.sqlite (V1)statev2.sqlite
projection_threads385354
projection_thread_messages13,07312,029
  • 31 threads created in V1 after the seed have no V2 thread.created event. Their created_at values run from 2026-10-12T18:16:53Z to 2026-10-16T08:42:09Z. The newest seeded thread was created at 2026-10-12T18:02:03Z, so the cut is exact.
  • 5 seeded threads gained 381 message rows in V1 after the seed (162 of them user/assistant). Their transcript_imported_at was already set on Oct 12, so ensureTranscript never looks at them again.
  • 0.1.0 added nothing to those 5 threads, so their gap is pure V1 drift.

Same root cause, confirmed in the shipped 0.1.0 bundle

initializeV2Database still returns when statev2.sqlite exists ("Seed V2 once"). reconcileShells and ensureTranscript only ever read the legacy tables inside statev2.sqlite.

Why this one matters

The issue was opened for preview builds, then manual rollbacks. Here there was no manual downgrade: the app's own updater (ShipIt) swapped the bundle from Nightly to Alpha 13 minutes after the seed, and I carried on working normally. Anyone who briefly ran a V2 Nightly and then moved back to the stable channel before 0.1.0 is likely in the same position without knowing it.

Local recovery (offline, preserves V2-only data)

Instead of moving statev2.sqlite aside, which would drop work done in 0.1.0 since the upgrade, I used the importer's own idempotency:

  1. Quit T3 Code and back up both databases.
  2. Copy the missing legacy rows (projection_threads for the 31 threads; INSERT OR IGNORE of projection_thread_messages, _pull_requests, _activities, _turns, _proposed_plans, _sessions) from state.sqlite into statev2.sqlite. The table layouts are identical in 0.1.0.
  3. Set transcript_imported_at = NULL on orchestration_v2_legacy_imports for the 5 seeded threads that gained messages. ensureTranscript skips messages whose import event already exists, so it only adds the missing ones.
  4. Start 0.1.0. reconcileShells imports the 31 threads at startup, and the transcripts fill in.

Result after one restart of 0.1.0:

CheckBeforeAfter
Legacy threads with no V2 thread.created310
orchestration_v2_projection_threads355386 (385 from V1 + 1 created in 0.1.0, untouched)
orchestration_v2_legacy_imports with transcript_imported_at IS NULL—0, with 0 last_error
V1 user/assistant messages without a migration:v1:message:<id> event4840 of 8,959
V1 user/assistant messages without a V2 message row4840 of 8,959

Of the 1,044 legacy message rows copied across, 484 are user/assistant (322 in the 31 new threads, 162 in the 5 continued ones), and all 484 are now in V2. The other 560 are tool and system rows, which the importer skips by design (role IN ('user', 'assistant')). The most-affected continued thread (300 missing rows) now ends at its last V1 message, a few seconds before the 0.1.0 update was applied. No restored thread came back archived or deleted.

That is effectively the "selective catch-up" from the triage comment, and it takes about 20 lines of SQL. Running it automatically at startup whenever V1's max(orchestration_events.sequence) is ahead of the snapshot's would close this without touching V2-only data.

Synced from GitHub

Same shape on stable 0.1.0, Windows 11, local desktop install. A V2 nightly ran once on 2026-10-03 and seeded statev2.sqlite, then the machine went back to 0.0.45 (to dodge #15001) and kept writing V1 until 2026-10-15. After the auto-update to 0.1.0, the app opened the Oct 3 snapshot.

Before: state.sqlite 53 threads / 27 projects; statev2.sqlite 37 threads / 25 projects, all orchestration_v2_legacy_imports rows dated 2026-10-03, no errors, 0 orchestration_v2_events (nothing V2-native).

Workaround from the triage comment above worked: app quit, statev2.sqlite plus -wal and -shm moved aside, relaunch. Result: 53 threads, 53 transcripts imported, 0 errors, 27 projects. Took a few seconds.

Filed by Claude Fable 5.1 via t3 triage.

Synced from GitHub

Another way into this, with no channel switching and no manual rollback: an in-app Update server from 0.0.x to 0.1.x that the service launcher rolls back.

  1. The 0.1.x server runs as a trial under the launcher the unit names. That is still the old 0.0.x launcher, because the unit stays on the runtime that installed it (#17490). It gives the trial 120 s to report prepared. The 30 minutes from #19311 only apply from 0.1.0 on.
  2. Before it listens, the trial seeds userdata/statev2.sqlite from state.sqlite (Sqlite.layerConfig → initializeV2Database). On a big V1 database the copy alone eats a good part of those 120 s. We measured 33 s for 5.2 GB on an Apple Silicon SSD, and a VM disk is slower.
  3. On rollback the launcher restores only the database the requesting server named. That is state.sqlite (restoreDatabaseBackup on pending.dbPath, the same in 0.0.45). statev2.sqlite and its -wal/-shm stay, outside the rollback boundary that docs/internals/server-updates.md describes.
  4. The 0.0.x server comes back and keeps writing state.sqlite. The next update finds statev2.sqlite and takes it as is, so everything written between the two attempts is missing.

Two things that might help pick a fix:

  • A launcher-side fix can't cover this transition. The pre-0.1.0 launcher does the rollback, so the fix has to live in the 0.1.x server's seeding or startup.
  • In this path the leftover copy is disposable. A trial that never reported prepared never opened its command gate. As far as we can tell, its copy holds only the seed, the migrations and the import, and no user work. Seeding again when state.sqlite changed after the seed, and the copy has no V2-native writes yet, would cover it without risking V2-only data.

This comes from reading the code at main 5b72385 and v0.1.0, not from an end-to-end reproduction. On our hosted machines we move statev2.sqlite* aside after a rolled-back update.

Synced from GitHub

I'm seeing this on 0.1.9-nightly.20261110.4478 (macOS arm64).

  • statev2.sqlite was created on Oct 19. All 117 rows in orchestration_v2_legacy_imports have shell_imported_at = 2026-10-19T17:56:39Z.
  • After that I used Beta 0.1.8, which wrote only to state.sqlite. Today's nightly reused the Oct 19 statev2.sqlite, so 98 threads (about 10k messages) were missing.
  • reconcileShells runs on every startup, but it reads the legacy projection_* tables inside statev2.sqlite. Those tables are never refreshed from state.sqlite.

Related problem: thread ID collision. A new v2 thread created today got the same thread_id as a v1 thread from Oct 19 (c4904b8b-7902-48cd-9eeb-fae564b33b2d). reconcileShells finds a thread.created event for that ID, so it can't import the v1 thread.

Workaround (quit the app fully first):

  1. Back up both databases with sqlite3 <db> ".backup <file>".
  2. Run ATTACH 'state.sqlite' AS v1. Then insert the v1 rows into the legacy projection_threads, projection_thread_messages and projection_thread_pull_requests tables in statev2.sqlite. Copy only rows whose thread_id is in neither main.projection_threads nor orchestration_v2_projection_threads.
  3. Some threads were already imported but changed in v1 later. Copy their new messages with INSERT OR IGNORE, then set transcript_imported_at = NULL in orchestration_v2_legacy_imports so they re-hydrate.
  4. Start the app again. reconcileShells imports the copied rows.