fix(opencode): preserve owned turns during server recovery#25596

open1
main
maria/takeover-pr-24158
opened21h agobyMmaria
View on GitHubTwo-way sync enabledPR last imported 20h ago

Problem

A slow but live T3-owned OpenCode 2 server can exhaust five recovery attempts and fail active turns that are still running. Recovery also holds the event lock across history reads, delaying Stop.

Change

Preserve @Zheruel's recovery design on current main: retry owned recovery with backoff capped at 30 seconds until process exit, Stop, or shutdown; retain bounded external-server recovery. Keep HTTP reads outside the state lock, recheck turn/request ownership before applying snapshots, isolate deleted sessions, and scope unread SSE requests to their connection. Recovery never resends prompts.

The policy deliberately has no elapsed-time cutoff for a live owned process. This is the maintainer decision to review.

Compared with the original diff, all recovery mechanisms and regression cases remain. Main's surrounding changes are retained, the history comment is shorter, and a test now asserts terminal status in its main fiber so a failure cannot strand a Deferred.

Verification

  • Regression fails on current main: stalled recovery ends failed instead of completed.
  • 120 focused tests pass, covering recovery, Stop, process exit, shutdown, pending requests, deleted sessions, SSE cancellation, server ownership, orchestration and text generation.
  • Blacksmith vp run check:changed passes: format, lint, both knip checks, GraphQL, affected-package typechecks and adjacent tests. The isolated worker's origin/main ref was set to the exact baseline commit because its snapshot has no remote refs.
  • Real OpenCode 2.1.0 on Linux x64, Node 24.22.1: main fails both active turns while native history reports success on the same live PID; the replacement completes both. Both runs pause the captured owned process for 65 seconds and delay each reconciliation response by 8 seconds. Two initial prompts, no resubmission. Uses a local model fixture and real shell tools, isolated provider state, no credentials or live user data.
  • Real process death fails both turns. Runtime harness, adapted from the original author's harness.
  • Two independent GPT-6.1-Sol reviews approve head b8a9888760628cfd256049ab9e374ec7aadb1ac4 with no findings. All 28 CI/check contexts are successful, with no pending or failing checks. Macroscope correctness and Effect checks pass; its approvability result requests human review of the production recovery policy. CodeRabbit reports success but skipped review because its included review budget was exhausted. No unresolved correctness findings remain.
  • Unverified: exact NixOS/systemd/zram starvation, upstream database-lock cause, desktop/mobile and rendered web client. No UI rendering code changes; provider runtime transcripts are the behavioral evidence.

Takeover

Replaces #24158, with thanks to @Zheruel for the original implementation.

The fork has maintainerCanModify: false, so this is a new branch from current main with the contributor credited in the commit trailer. The original PR remains open.

Built with GPT-6.1-Sol in the Codex harness on T3 Code.

Checks

21 successful · 6 skipped · 1 neutral
All checks passed
b8a9888

Added 1 commit

  • fix(opencode): preserve owned turns during server recoveryb8a9888
Synced from GitHub

Warning

Review limit reached

Only developers with an assigned seat can use this organization's usage-based review budget, and seats here are assigned manually. Ask an admin to assign a seat, or change the review continuation mode in Billing.

Next included review available in 15 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

Learn how review limits work.

Review configuration:

External image Run configuration
External image Commits

Reviewing files that changed from the base of the PR and between 4e42eaf4293c51c5b9eac601b7ab346aa19d08af and b8a9888760628cfd256049ab9e374ec7aadb1ac4.

External image Files selected for processing: 8
  • apps/server/src/orchestration-v2/Adapters/OpenCode2AdapterV2.test.ts
  • apps/server/src/orchestration-v2/Adapters/OpenCode2AdapterV2.testkit.ts
  • docs/user/providers-opencode.md
  • packages/provider-opencode/src/server/v2/OpenCode2Client.test.ts
  • packages/provider-opencode/src/server/v2/OpenCode2Client.ts
  • packages/provider-opencode/src/server/v2/OpenCode2Server.test.ts
  • packages/provider-opencode/src/server/v2/OpenCode2Server.ts
  • packages/provider-opencode/src/server/v2/adapter.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

Synced from GitHub

Approvability

Verdict: Needs Human Approval

Macroscope's review found this PR requires human approval — This changes the production OpenCode recovery workflow across reconnect retries, process lifetime handling, synchronization, history backfill, and pending-request restoration for both owned and external servers. Despite extensive tests, the concurrency and recovery-state changes are broad enough to warrant human review.

You can add or adjust custom eligibility rules. Learn more.

Synced from GitHub

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ProviderMetricMain baselineThis PRImpactPR ceiling
CodexTotal thread wire5.0 KiB5.0 KiB+7 B (+0.1%)6.8 KiB✅
CodexThread snapshot wire3.8 KiB3.8 KiB0 B (0.0%)4.9 KiB✅
CodexLive turn WebSocket wire1.2 KiB1.2 KiB+7 B (+0.6%)2.0 KiB✅
CodexLive turn WebSocket decoded20.9 KiB20.9 KiB0 B (0.0%)29.3 KiB✅
CodexLive turn messages220 (0.0%)8✅
ClaudeTotal thread wire5.0 KiB5.0 KiB+7 B (+0.1%)6.8 KiB✅
ClaudeThread snapshot wire3.8 KiB3.8 KiB0 B (0.0%)4.9 KiB✅
ClaudeLive turn WebSocket wire1.2 KiB1.2 KiB+7 B (+0.6%)2.0 KiB✅
ClaudeLive turn WebSocket decoded21.2 KiB21.2 KiB0 B (0.0%)29.3 KiB✅
ClaudeLive turn messages220 (0.0%)8✅

Baseline: 4e42eaf · PR result: b8a9888 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.