[Bug]: migrate-dev-db silently empties every thread table when --source is a V1 state.sqlite#19018

opened12d agobyRrenepf

Area: apps/server Impact: Major degradation or frequent failure — the command exits 0 and reports success, so the data loss is only visible by counting rows afterwards.

I searched existing issues (migrate-dev-db, V1 state.sqlite, empty dev database) and found no duplicate. #15017 is a different defect in the same area.

Steps to reproduce

1# 1. A V1 source: a database from before the V2 thread tables.
2sqlite3 "file:$HOME/.t3/userdata/state.sqlite?mode=ro" \
3 "select count(*) from projection_threads;" # 105
4sqlite3 "file:$HOME/.t3/userdata/state.sqlite?mode=ro" \
5 "select count(*) from sqlite_master where name='orchestration_v2_projection_threads';" # 0
6
7# 2. From a linked worktree, with the dev server stopped:
8vp run migrate-dev-db --source "$HOME/.t3/userdata/state.sqlite" \
9 --projects 5 --threads-per-project 10
10
11# 3. Count what arrived:
12sqlite3 <worktree>/.t3/userdata/statev2.sqlite \
13 "select count(*) from projection_threads;" # 0

A synthetic source works too: build one with runMigrations({ toMigrationInclusive: 54 }) and insert one row into projection_threads before running the command.

Expected behavior

Either the V1 thread data survives into the dev database, or the command fails loudly. The script's own documentation promises the first: apps/server/scripts/migrate-dev-db.ts:276-277 says "A source from before the V2 thread tables has no visibility to filter on, so it is copied whole", and :544-545 says migrations run before the prune so "a source older than this checkout would otherwise crash the prune queries". Both read as deliberate V1 support.

Actual behavior

The command exits 0, prints Dev database ready: … (844 KB) and lists every kept project with 0 threads. Every thread table is empty.

1Copying visible threads from /Users/.../.t3/userdata/state.sqlite (read-only)...
2Running migrations on the snapshot...
3INFO (#4): Migrations ran successfully {
4 migrations: [ '55_OrchestrationV2', ..., '61_PullRequestReviewGuides' ] }
5Pruning to 5 projects, 10 stopped threads each...
6Compacting into .../.t3/userdata/statev2.sqlite...
7
8Dev database ready: .../.t3/userdata/statev2.sqlite (844 KB)
9 t3code: 0 threads
10 ops: 0 threads
11 processes: 0 threads
12 zoho-api: 0 threads
13 website: 0 threads
14 10 orchestration events kept

Measured, source against result:

tableV1 sourceresult
projection_threads1050
projection_thread_messages96720
projection_thread_activities26570
projection_turns830
projection_thread_pull_requests90
projection_thread_sessions60
orchestration_events where aggregate_kind='thread'156150
orchestration_events where aggregate_kind='project'1310

Cause

The copy phase does the right thing: a source without orchestration_v2_projection_threads sets filtered = false (:290) and is copied whole. Migration 55_OrchestrationV2 then populates part of the V2 world — ApplicationEventSource.ts:82 writes orchestration_events from orchestration_v2_events (:63), :165 and :210 write project events from projection_projects (:139), :251 writes orchestration_command_receipts — which is why the result still carries 5 projects and 10 project events.

What no migration does is import V1 threads. That is a server boot step: serverRuntimeStartup.ts:507-509 runs legacyV1ThreadImporter.reconcileShells, which reads FROM projection_threads (orchestration-v2/legacy/LegacyV1ThreadImporter.ts:575, second site :666). migrate-dev-db never calls it, so after the migrate phase orchestration_v2_projection_threads is empty.

pruneSnapshot then builds thread_families, stopped_families and kept_threads from that one empty table (:358-399) and sweeps every table carrying thread_id (:402-408, :412-425), plus every thread event (:428-432), with WHERE thread_id NOT IN (SELECT thread_id FROM kept_threads). With a V1 source the keep set is necessarily empty, so the sweep deletes exactly the rows the copy phase went out of its way to bring along.

The comment at :402-404 names the trap without seeing it: "Every V2 table and every V1 table the lazy importer reads is keyed by thread_id, so one sweep covers both." The sweep covers both. The keep set is V2-only.

Why the test suite does not catch it

apps/server/scripts/migrate-dev-db.test.ts:185-202, it.effect("upgrades a source from before the V2 thread tables"), builds its source with runMigrations({ toMigrationInclusive: 54 }) and inserts no rows at all, then asserts only assert.include(result.executedMigrations, "55_OrchestrationV2"). It proves the migrate phase runs and nothing about what survives it. It is green because the fixture had nothing to lose. Adding a single projection_threads row to that fixture and asserting it survives would fail today.

Environment

  • macOS 26.3.2 (build 25D2150, Darwin 25.3.0), Apple Silicon
  • Node 24.13.1 (package.json engines ^24.13.1)
  • pingdotgg/t3code at 7b6d907574, run from a linked git worktree
  • Source: a real ~/.t3/userdata/state.sqlite, 53 MB, 105 V1 threads, no statev2.sqlite alongside it

Suggested fixes, for maintainers to judge

  1. A guard, cheapest and smallest. A new error class in the row at :50-150 that aborts when orchestration_v2_projection_threads is empty while the source carried projection_threads rows. It turns silent loss into a clear refusal. The check has to hang off the source, not the result, or it would also abort on a legitimately empty database.
  2. Make the V1 path real. Run the headless equivalent of the boot import inside the script — persistence/initializeV2Database.ts:20-24 plus orchestration-v2/legacy/LegacyV1ThreadImporter.ts:575 — between the migrate and prune phases. This creates a second entry point into the V1-to-V2 transition that you would have to maintain, so it is your architectural call, not something I would send as a PR unasked.

In either case the vacuous test above should gain a data row.

Reproduced on Linux/Node 24.21.0 at 70e5e384da. Created a migration-54 source with one project and one legacy thread, then ran vp run migrate-dev-db --source /tmp/triage-19018/source/state.sqlite --projects 5 --threads-per-project 10 from a linked worktree with no dev server running. The command exited 0 and printed Repro: 0 threads; SQLite counts were source 1, destination 0. No matching open duplicate found.