[Bug]: Codex command cancellation on server restart leaves its caller asleep#24503

opened4d agobyMMaximilianMauroner

🤖 Comment made by gpt-6.1-sol acting on behalf of @MaximilianMauroner.

Related fix: https://github.com/pingdotgg/t3code/pull/23585

This report records the defect and evidence already described in that PR. Issue filing did not rerun the app or its tests. Maintainer triage and acceptance remain pending.

Before submitting

  • I searched existing issues and did not find a duplicate.
  • I included enough detail to reproduce or investigate the problem.

Area

apps/server

Steps to reproduce

  1. Use a root native Codex command with continuation support. Let the agent turn end while the command still runs and owes a completion notification.
  2. Restart the T3 server before that command finishes, with restart continuation enabled and its normal eligibility conditions met.
  3. Let recovery cancel the command, then wait without sending a new message.

This is the failure scenario described and covered by the PR tests. It was not independently reproduced with a real server restart during issue filing. The PR does not supply a deterministic live-app script or failure frequency.

Expected behavior

The caller receives one restart continuation that names the cancelled command, so it can resume or restart needed work. The existing late-completion notification promise must survive recovery.

Actual behavior

Recovery loses the command’s notification ownership. It cancels the command but can leave the caller asleep after its agent turn has ended. No late completion arrives to resume the caller.

Impact

Major degradation: an affected workflow waits for a notification that will never arrive. Frequency is not established.

Version or commit

Affected release not specified in the PR. The linked PR records the proposed repair and its review history.

Environment

Codex provider with root native command continuation support. The PR reports local checks on Coding VM and an isolated Effect service runtime. The affected release, OS, and Codex app-server version are not specified.

Workaround

Send a new message to the caller to resume the workflow, then inspect and restart cancelled work as needed. This workaround was not tested during issue filing.

Related issues and search

Related reports were checked: #19185 concerns live Claude subagents, #21865 concerns invisible Codex goal turns, and #23399 concerns retries after provider failures and follow-up child notifications. Their triggers and repair scopes differ from this root-command ownership defect.

Evidence, proposed repair, and verification limits

The following snapshot preserves the PR description, including its reported checks, scope, and evidence. All results are attributed to the PR authors; they are not new verification from issue filing.

PR #23585: fix(server): preserve Codex command wakeups across restarts

Problem

A T3 restart can cancel a Codex command after its agent turn ends and leave the caller asleep. The adapter promises a late completion notification, but restart recovery loses that ownership.

Change

Persist the existing notification ownership on root command items and cancellation notes. Keep it through updates and duplicate-note merges. Reuse the existing restart continuation outbox and eligibility gates.

Scope and approval

This is a focused repair of an existing notification promise under CONTRIBUTING.md's obvious-bug exception. It adds no queue, caller API, configuration, or command classifier. Root native Codex commands with continuation support qualify. A marked server command can also report restart cancellation. Old unmarked records retain their behavior. Generic completion-delivery retries remain outside scope. Maintainer acceptance is pending.

Verification

  • A separate test-only commit reproduced three missing-wake failures before the fix. Five paired unowned cases passed.
  • On Coding VM, vp run check:changed upstream/main through heavy-check passed format, lint, knip, generated code, 29 affected package typechecks, and 146 tests across four files after the final cleanup.
  • Sol and DeepSeek reviewed the full change. DeepSeek's test-compile finding is fixed. Fresh Sol reviews found no production or Effect-convention issue and approved the typed-access cleanup.
  • The captured isolated Effect service runtime below passed its two owned/unowned cases. It uses the real recovery service, projection reducer, and continuation logic. Projection storage and final dispatch are stubs. It does not prove SQLite persistence or a subprocess restart.
text
1EVIDENCE {"kind":"isolated-effect-service-runtime","phase":"before-recovery","work":{"kind":"shell","label":"pnpm test","id":"turn-item:dev-server","completionWake":"always"},"status":"running"}
2EVIDENCE {"phase":"after-recovery","status":"cancelled","cancelledWork":[{"kind":"shell","label":"pnpm test","id":"turn-item:dev-server","completionWake":"always"}],"continuationEffects":[{"type":"provider-runtime.continue","sourceRunId":"run:restart"}]}
3EVIDENCE {"phase":"after-repeated-recovery","commits":1,"shutdownWrites":0}
4EVIDENCE {"phase":"continuation-dispatch-at-fake-io-boundary","messages":[{"messageId":"message:restart-continuation:run:restart","text":"Note: the T3 server restarted, and this background work was cancelled before it finished. It will not report back on its own, so resume or restart what you still need and wait for its result:\n- shell: pnpm test"}]}
5EVIDENCE {"phase":"duplicate-continuation-with-existing-message","messages":[]}
6EVIDENCE {"phase":"after-recovery","status":"cancelled","cancelledWork":[{"kind":"shell","label":"pnpm test","id":"turn-item:dev-server"}],"continuationEffects":[]}
7EVIDENCE {"phase":"continuation-dispatch-at-fake-io-boundary","messages":[]}

Full hosted CI is not verified. No installed service restart, deployment, or merge ran. Opus and Sonnet attempts hit Claude rate limits. GLM could not run due provider balance. None supplied a review pass.

Implementation by GPT-6.1-Sol in the Codex harness through T3 Code. Completed source reviews by GPT-6.1-Sol and DeepSeek V4.1 Flash. Guideline and final cleanup reviews by GPT-6.1-Sol.