Sign up
View stargazersSign up

Prevent workspace ownership changes racing checkpoint file restore#12341

openedSep 18, 2026byJJulius Marminge

Checkpoint file-restore isolation is checked against recorded thread/workspace ownership at command admission and again before provider rollback. These are observations, not an atomic workspace reservation.

An existing race remains: restore A passes its last check; thread B then registers/starts in an overlapping directory; A restores the checkout and can overwrite B's files. This also existed for equal paths before the parent/child ownership fix in #12338. CheckpointService's workspace mutex serializes checkpoint operations, while thread creation and provider work do not acquire it, so merely changing its key to a canonical path does not close the admission race.

Coordinate workspace ownership registration and filesystem restore. Cover thread creation, workspace/path changes, and provider cwd registration, including canonical aliases and parent/child directories. A new owner must not start writing while a destructive restore is in progress. Avoid holding the SQLite transaction or a global command lock across provider rollback or slow Git work, which would block unrelated threads and may deadlock provider event persistence.

Add a receipt/Deferred-based regression that pauses a restore after its isolation check, attempts an overlapping thread start, and proves the other thread's files remain intact. Shared main behavior needs the protection; v2 requires its own admission integration when inheriting it.

JJulius MarmingeSep 18, 2026

Triage

Confirmed. Isolation at command admission and again before rollback is an observation of recorded owners, not an atomic workspace reservation. The remaining race is:

  1. Restore A passes its last isolation check.
  2. Thread B registers or starts in an overlapping directory (same path, parent, child, or canonical alias).
  3. A restores the checkout (git restore / git clean) and can overwrite B's files.

This already existed for equal paths before the parent/child containment work. #12306 (main) and #12338 (v2) reject more already-recorded owners; they do not reserve the workspace for the duration of the restore. Macroscope's High on #12338 (isCheckpointRestoreIsolated vs checkpoints.restore) is the same window; changing CheckpointService's per-cwd semaphore key to a canonical path is not enough, because thread creation, workspace/path changes, and provider cwd registration do not take that lock.

Where it lives

Main

  • apps/server/src/orchestration/Layers/CheckpointReactor.ts — isRestoreWorkspaceIsolated then restoreCheckpoint with a gap
  • apps/server/src/orchestration/decider.ts — thread.create, thread.meta.update (worktreePath), thread.checkpoint.revert (admission does not reserve)
  • apps/server/src/orchestration/Layers/ProviderCommandReactor.ts + apps/server/src/workspace/workspaceLease.ts — provider start leases the exact resolved cwd only; restore does not take it
  • apps/server/src/vcs/GitVcsDriver.ts — destructive restore/clean of .

V2 (t3code/codex-turn-mapping)

  • apps/server/src/orchestration-v2/CheckpointRestoreSafety.ts
  • apps/server/src/orchestration-v2/CheckpointRollbackService.ts / Orchestrator.ts
  • apps/server/src/orchestration-v2/CheckpointService.ts — withWorkspaceLock on the literal cwd

Constraints

Do not hold the SQLite dispatch transaction or the global command queue across provider rollback or slow Git. Unrelated threads would stall, and provider event persistence can deadlock.

Suggested fix

Coordinate ownership registration and filesystem restore under one reservation that covers thread create, workspace/path changes, and provider cwd registration, including canonical aliases and parent/child directories. A new owner must not start writing while a destructive restore is in progress.

Add a receipt/Deferred regression that pauses a restore after its isolation check, attempts an overlapping thread start, and proves the other thread's files remain intact.

Shared main behavior needs the protection first. V2 needs its own admission integration when inheriting it.

Related: #12306 (merged), #12338 (open, observation-only), #2829 (v2), #10608 (pending-turn serialization — different race).

Severity: high (data-loss TOCTOU). Type: bug. Not a duplicate of #12338.