fix(server): bound stalled Claude requests and recover sessions#25147

open4
main
fix/issue-24842
opened2d agobyJJulius Marminge
View on GitHubTwo-way sync enabledPR last imported 2d ago

Problem

Claude turns can remain active indefinitely while the native SDK repeats requesting, retry, or compaction status without making progress. Stop also hangs when the SDK interrupt or close never resolves, leaving later work blocked.

Change

Bound request and compaction phases at ten minutes without root-turn progress; repeated status frames do not extend the deadline. Fail stalled turns with a retryable native_session_stalled error and release query ownership before bounded cleanup.

Bound Stop and close operations, preserve one terminal outcome when Stop crosses the stall deadline, and mark unresponsive native sessions broken. Refusing to resume their existing error state reuses the fresh-session and full-summary fallback. The fix changes only the Claude adapter and its tests.

Scope and approval

Fixes pingdotgg/t3code#24842

Triaged bug via the automated issue flow: #24842. Ordinary bug fixes and fix PRs are authorized for this flow.

Verification

The read-only scripts used to inspect provider frame timing are also available: audit scripts. No real provider logs are included.

No render paths change, so the visual evidence matrix is empty. Real large-context compaction and hung SDK shutdown are covered by deterministic tests rather than runtime reproduction. A healthy request without progress for more than ten minutes will also reach the deadline. Packaged desktop and mobile clients were not separately exercised; this is shared server behavior.

Models: GPT-6.1-sol through the Codex harness and Claude Opus 5.5 through the Claude Agent harness.

Checks

20 successful · 10 skipped · 1 neutral
All checks passed
d5f70ee

Added 4 commits

  • fix(server): stalled Claude turns fail and the next message starts a fresh session3e5eaa4
  • refactor(server): drop turn-start stall check now that Claude refuses to resume8892c55
  • Merge remote-tracking branch 'origin/main' into fix/issue-24842bcdc4a8
  • fix(server): a Stop near the stall deadline settles its Claude turn onced5f70ee
Synced from GitHub

Approvability

Verdict: Needs Human Approval

Macroscope's review found this PR requires human approval — This PR adds a production watchdog and recovery state machine that can terminate stalled Claude turns, alter Stop/close behavior, and mark native sessions unrecoverable for normal resume. Although the change is focused and extensively tested, its concurrency and session-lifecycle impact merits human review.

You can add or adjust custom eligibility rules. Learn more.

Synced from GitHub

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ProviderMetricMain baselineThis PRImpactPR ceiling
CodexTotal thread wire5.0 KiB5.0 KiB+7 B (+0.1%)6.8 KiB✅
CodexThread snapshot wire3.8 KiB3.8 KiB0 B (0.0%)4.9 KiB✅
CodexLive turn WebSocket wire1.2 KiB1.2 KiB+7 B (+0.6%)2.0 KiB✅
CodexLive turn WebSocket decoded20.9 KiB20.9 KiB0 B (0.0%)29.3 KiB✅
CodexLive turn messages220 (0.0%)8✅
ClaudeTotal thread wire5.0 KiB5.0 KiB0 B (0.0%)6.8 KiB✅
ClaudeThread snapshot wire3.8 KiB3.8 KiB0 B (0.0%)4.9 KiB✅
ClaudeLive turn WebSocket wire1.2 KiB1.2 KiB0 B (0.0%)2.0 KiB✅
ClaudeLive turn WebSocket decoded21.2 KiB21.2 KiB0 B (0.0%)29.3 KiB✅
ClaudeLive turn messages220 (0.0%)8✅

Baseline: 1a13127 · PR result: d5f70ee · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Synced from GitHub

Review in Change Stack →

No actionable comments were generated in the recent review.

External image Recent review info
External image Run configuration
External image Commits

Reviewing files that changed from the base of the PR and between 1a13127c15b24132a896eb72fda060d36f705c08 and d5f70eefb9f607dedb1c55170994021334e9468f.

External image Files selected for processing: 2
  • apps/server/src/orchestration-v2/Adapters/ClaudeAdapterV2.test.ts
  • apps/server/src/orchestration-v2/Adapters/ClaudeAdapterV2.ts

Limit details: You’ve used all 10 included reviews currently available.

Walkthrough

ClaudeAdapterV2 now detects turns without root-level progress, fails stalled turns, and bounds waits for SDK interruption and closure. It marks affected provider threads as broken and rejects attempts to resume them. Tests cover watchdog deadlines and shutdown behavior.

Changes: Claude native session stall handling

Layer / File(s)Summary
Track root-turn progress
apps/server/src/orchestration-v2/Adapters/ClaudeAdapterV2.ts, apps/server/src/orchestration-v2/Adapters/ClaudeAdapterV2.test.ts
The adapter starts a ten-minute watchdog for stall phases and cancels it when root-turn progress occurs. Tests check repeated status frames, subagent activity, and deadline resets.
Finalize stalled turns and bound shutdown
apps/server/src/orchestration-v2/Adapters/ClaudeAdapterV2.ts, apps/server/src/orchestration-v2/Adapters/ClaudeAdapterV2.test.ts
A stall produces a retryable failure and marks the provider thread broken. Stop and close waits are bounded, and broken threads cannot be resumed. Tests cover stalled failures and shutdown outcomes.

Priority: External image Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix · Severity of issue fixed: Medium

Sequence Diagram(s)

mermaid
1sequenceDiagram
2 participant ClaudeSDK
3 participant ClaudeAdapterV2
4 participant ActiveTurn
5 participant ProviderThread
6 ClaudeSDK->>ClaudeAdapterV2: Send status or api_retry frames
7 ClaudeAdapterV2->>ActiveTurn: Start watchdog after stall phase
8 ActiveTurn-->>ClaudeAdapterV2: Report timeout without root progress
9 ClaudeAdapterV2->>ProviderThread: Mark thread broken
10 ClaudeAdapterV2->>ClaudeSDK: Attempt bounded close

Suggested labels: bugfix

Suggested reviewers: t3dotgg, maria-rcks, macroscopeapp

Merge Risk: External image Minimal · up to d5f70

No actionable issue is established that would prevent merging after normal checks.

Pre-merge checks: Passed 4
Check nameStatusExplanation
Title checkPassedThe title is concise, uses the repository's conventional format, and clearly identifies the main change: bounding stalled Claude requests and recovering sessions.
Description checkPassedThe description includes the required Problem, Change, Scope and approval, and Verification sections. It explains the behavior, linked issue, test results, limitations, and agent provenance.
Linked Issues checkPassedIssue [#24842] requires bounded request and compaction phases, visible retryable failure, released query ownership, fresh-session recovery, and bounded Stop/close operations. ClaudeAdapterV2.ts adds…
Out of Scope Changes checkPassedThe diff is limited to apps/server/src/orchestration-v2/Adapters/ClaudeAdapterV2.ts and its focused test file. The implementation changes the Claude adapter behavior required by [#24842]. The added …
Full details: Linked Issues check

Explanation

Issue [#24842] requires bounded request and compaction phases, visible retryable failure, released query ownership, fresh-session recovery, and bounded Stop/close operations. ClaudeAdapterV2.ts adds a ten-minute CLAUDE_STALLED_TURN_TIMEOUT, ignores repeated status and api_retry frames as progress, emits retryable native_session_stalled, releases query ownership before bounded close, and persists broken thread status. It bounds interrupt and close waits and preserves one Stop outcome. The existing error-state resume path is preserved through the new resume refusal. ClaudeAdapterV2.test.ts covers compacting, requesting, api_retry, ambient frames, subagent frames, root progress, later turns, hung close, hung Stop, ordinary Stop, and single terminalization.

Full details: Out of Scope Changes check

Explanation

The diff is limited to apps/server/src/orchestration-v2/Adapters/ClaudeAdapterV2.ts and its focused test file. The implementation changes the Claude adapter behavior required by [#24842]. The added tests directly verify the stall watchdog, recovery state, Stop and close bounds, and race behavior. No unrelated product area or render path changes appear.

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Usage-based review receipt

Note

This review was completed with usage-based billing: files reviewed beyond your plan's included limits are billed at $0.25/file. View usage-based billing.

Comment @coderabbitai help to get the list of available commands.