fix(server): stop abandoned turns before continuing threads#23186

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Repair the abandoned provider turn after ingestion failure.

RunExecutionService interrupts the provider after ingestion fails, then finalizes the failed run without writing provider-turn.updated. The provider-turn projection can remain running.

On the next continuation, ProviderTurnStartService calls interruptTurn again with requestRuntimeRestart: true. Muse rejects an inactive turn with ProviderAdapterInterruptError. The recovery branch then fails or settles the new run before it writes the repair event. This can block continuations for providers with this behavior.

Write the interrupted provider-turn.updated event in the ingestion-failure path, or handle the adapter’s explicit inactive-turn result as successful recovery. Do not treat unrelated interrupt errors as successful.

✅ Confirmed as addressed by @Yash-Singh1

Confirmed and fixed in f33f3bf5d0. Ingestion failure finalization now records a successful provider interruption in the same guarded transaction as the failed run. This prevents the next continuation from sending a second interrupt to an inactive turn. Unrelated interrupt failures still propagate through continuation recovery.

The regression failed before the fix and now passes. Tests also cover failed interruptions, lost ownership, and continuation after an interrupted turn. All 75 focused tests and vp run check:changed pass.

@Yash-Singh1 Verified the fix in the current code. Successful interruption now records provider-turn.updated with status interrupted in the same guarded write as the failed run.

The regression tests assert that interruption failure omits the repair and lost ownership commits no final events. Continuation recovery still propagates unrelated interrupt failures.

This addresses the finding. I inspected the code and test assertions; I did not rerun the tests.

✅ Review thread resolved.

_{You are interacting with an AI system.}

🟠 High orchestration-v2/RunExecutionService.ts:1471

When ingestion fails while background subagent calls or wakes exist, this interrupts only the foreground turn, then finalizes the run while that work continues. Recovery skips the terminal turn, leaving background work able to modify the workspace and causing rollback to fail; pass requestRuntimeRestart: true so interruptTurn stops the background work before finalization.

Confirmed and fixed in 92cafceae3. Ingestion-failure cleanup now passes requestRuntimeRestart: true, using the same adapter behavior as Stop to stop background work before finalization.

The regression models background work that survives a foreground interruption and records whether it is still running when finalization begins. It failed for both ingestion failures and defects before this fix. All 75 focused tests and vp run check:changed now pass. Interruption errors still leave the provider turn unrepaired rather than recording a successful stop.

🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift

Stop accepted provider work when its turn ID is not yet routed.

New attempts start with providerTurnId: null. Codex sends client.request("turn/start", turnStartParams) before calling registerRootTurn(...). Event routing runs in a separate fiber, and provider events are queued. A stream failure in this window can leave the native turn running while routing.rootProviderTurnId remains null. The failure handler then skips interruptTurn, and recovery cannot find the unprojected turn.

Record the accepted turn ID before provider work continues, or stop the accepted native turn when registration cannot complete.

This stream-failure scenario is already handled outside RunExecutionService. ProviderSessionManager.startEventPump publishes the failure to subscribers, removes the session, and calls releaseEntry with runtime_error. releaseEntry closes the adapter scope, so a Codex stream failure before registerRootTurn does not depend on a routed turn ID to close its runtime.

The separate persistence-failure case is covered by this PR: registerRootTurn publishes provider_turn.updated as its first event, and RunExecutionService updates eventRouting and latestRootProviderTurn before calling ingestNormalized. If that first write fails, cleanup has the accepted turn ID and interrupts it.

Re-ran the existing session-stream-failure test, which asserts that the session is removed and its runtime closes once, along with the ingestion cleanup and unknown-turn tests. All six focused cases pass. No change for this finding.

@Yash-Singh1 Thanks for the clarification. I verified both paths:

• For a provider stream failure, ProviderSessionManager.startEventPump calls releaseEntry with runtime_error. That path removes the live session and closes its adapter scope without requiring a routed turn ID.
• For a persistence failure, registerRootTurn first emits provider_turn.updated. RunExecutionService updates routing and latestRootProviderTurn before ingestNormalized, so cleanup has the accepted turn ID if that write fails.

My finding overlooked the session-level cleanup. I withdraw it. No code change is needed for this finding. I did not rerun the tests.

✅ Review thread resolved.

_{You are interacting with an AI system.}