fix(server): messages cut by a restart no longer stay marked as streaming#28646

merged1

main

GitHub fork: fix/recovery-cancelled-items-stop-streaming

View on GitHubTwo-way sync enabledPR last imported 5d ago

Problem

When a server restart or shutdown cuts a run, recovery cancels its open turn items but leaves streaming: true on assistant_message, reasoning and proposed_plan items. The item ends up cancelled and still streaming, and nothing clears it later because the run is over. Clients build the assistant row from that flag, so a partial reply keeps rendering as live output.

Expected: a cancelled item is not streaming, as already happens on the other terminal paths.

Change

cancelledItem in ProviderRuntimeRecoveryService.ts now sets streaming: false on any item that has the field, using the same "streaming" in item check as RunExecutionService. Those three types are the only turn items with the field, and other items keep their shape.

Scope and approval

No triaged issue for this code path, under the very small, focused fix for an obvious bug route: a cancelled item cannot still be streaming, and the fix is one spread in one helper.

Related history, stated plainly so nobody has to dig for it:

#14433 by @kiiwee made this fix for assistant and reasoning items. It was closed for its model during the open PR cleanup, with a recommendation to rebuild it on a newer model. This is that rebuild, and it also covers proposed_plan.
#21610 contains the same hunk, but it is an XL PR about queued steering, currently conflicting. This is the small standalone version. If #21610 is about to land, closing this one is fine.
#14572 reported the same symptom on the V1 code path and was closed as resolved by Orchestrator v2. This is the V2 recovery path, and the flag still sticks there.

Verification

Extended the existing "cancels the complete in-flight subtree" case in ProviderRuntimeRecoveryService.test.ts with one streaming item of each type.

cd apps/server && vp test run src/orchestration-v2/ProviderRuntimeRecoveryService.test.ts -t "cancels the complete in-flight subtree"

Before the fix: fails, each of the three items is cancelled with streaming: true.
After the fix: passes, all three are false.

To check the client side, I ran a throwaway test (not committed) that applied a running, streaming assistant_message to the real SQLite projection store, ran recovery.reconcile("startup"), read the row back from getThreadProjection, and passed it to the web deriveTimelineEntriesFromVisibleTurnItems:

	stored item	timeline `message.streaming`
Before	`cancelled`, `streaming: true`	`true`
After	`cancelled`, `streaming: false`	`false`

The recovery, regression, projection recovery and restart continuation test files pass (75 tests), and so does vp run check:changed.

Not checked: a real provider turn killed mid-stream in a running app, and the rendered result. From reading TimelineAssistantRows.tsx, a streaming row hides its timestamp and keeps ChatMarkdown in streaming mode, but I did not see that on screen. Mobile maps the flag the same way in threadActivity.ts; I read that and did not run it.

Claude Opus 5.5 via Claude Code

Checks

24 successful · 14 skipped

All checks passed

Added 1 commit8d ago

fix(server): messages cut by a restart no longer stay marked as streaming812cc1a

MmacroscopeappBotapproved these changes8d ago

MmacroscopeappBot8d ago

Approvability

Verdict: Approved at 812cc1a

Macroscope's review found this PR approvable — Recovery now clears the streaming flag on interrupted turn items already being cancelled, preventing partial replies from continuing to appear live. Existing cancellation rules and message contents are unchanged, keeping the impact narrowly bounded.

^{You can add or adjust custom eligibility rules. Learn more.}

CcoderabbitaiBot8d ago

Review in Change Stack →

No actionable comments were generated in the recent review.

External image Recent review info

External image Run configuration

Source mode: Auto
Repository: pingdotgg/t3code
Branch: main
File: .coderabbit.config.ts
Commit: 8760d06
Layers: Repository TS config
Review profile: CHILL
Plan: Advanced
Run ID: b27d172a-bd5f-45dd-ac24-a894ebf5c191

External image Commits

Reviewing files that changed from the base of the PR and between 8760d063e903406c8e8f695b0852c0a15fc1e00f and 812cc1ae9075689a14da6f5bd993172b2fa7aee1.

External image Files selected for processing: 2

apps/server/src/orchestration-v2/ProviderRuntimeRecoveryService.test.ts
apps/server/src/orchestration-v2/ProviderRuntimeRecoveryService.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 3 remain after this review.

Walkthrough

Recovery cancellation now clears the streaming flag on cancelled turn items that have that property. The recovery test covers streaming assistant_message, reasoning, and proposed_plan items.

Changes: Recovery cancellation

Layer / File(s)	Summary
Clear streaming state on cancelled items `apps/server/src/orchestration-v2/ProviderRuntimeRecoveryService.ts`, `apps/server/src/orchestration-v2/ProviderRuntimeRecoveryService.test.ts`	Cancellation clears `streaming` when the item has that property. The test verifies this for `assistant_message`, `reasoning`, and `proposed_plan` items.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Suggested reviewers: juliusmarminge

Merge Risk: External image Minimal · up to 812cc

The recovery change correctly clears streaming state for cancelled items and is ready to merge.

Pre-merge checks: ^Passed 4

Check name	Status	Explanation
Title check	Passed	The title clearly and concisely describes the main fix: clearing the streaming state for messages interrupted by a server restart.
Description check	Passed	The description includes all required sections. It explains the problem, change, scope rationale, verification results, limitations, and agent details.
Linked Issues check	Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	Passed	Check skipped because no linked issues were found for this pull request.

Finishing Touches

Generate unit tests (beta)

Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Mmaria8d ago

Note

Written by gpt-6.1-sol on behalf of Maria

Verified in a real dev environment

verified as-is at 812cc1ae9075689a14da6f5bd993172b2fa7aee1 on a disposable linux vm. no code changes.

used repo Playwright 1.60.0 with /usr/bin/google-chrome, the running dev web client, and real Codex gpt-6.1-sol turns through the local proxy. added a local git project through the app and selected the model in the composer. ran one dev server at a time with isolated state.

reproduced on github/main (82b10d546c): interrupting a streaming reply with server shutdown left the assistant item cancelled with streaming: true after restart. its timestamp and copy control stayed absent.
on the pr, shutdown recovery cleared streaming, preserved the partial reply, and the client returned to idle.
separately saved a read-only sqlite snapshot during a real in-flight main turn and started the pr against a copy of that state. startup recovery cancelled the same 825-character reply with streaming: false. the matching captures below show its timestamp and copy control restored. no turn items were fabricated or edited.
sent a follow-up in the recovered thread: the real provider replied recovery follow-up works, completed normally, and persisted streaming: false.
started another long reply and clicked stop: the client showed Run interrupted · Stopped by user, preserved partial text, and persisted streaming: false.
ran the recovery, restart continuation, projection recovery, and regression suites through blacksmith: all 75 tests passed. the existing head CI checks also passed.

read-only runtime transcript (type | status | streaming | text length):

text

1	main after shutdown/restart:
2	assistant_message\|cancelled\|1\|825
3	pr startup recovery of the same in-flight reply:
4	assistant_message\|cancelled\|0\|825
5	pr independent shutdown:
6	assistant_message\|cancelled\|0\|3622
7	pr follow-up:
8	assistant_message\|completed\|0
9	pr user stop:
10	assistant_message\|completed\|0
11	run_interrupt_result\|interrupted

before, main with the partial reply still marked streaming:

main recovery leaves cancelled reply without timestamp or copy control

after, pr startup recovery of the identical partial reply:

pr recovery restores timestamp and copy control on the same cancelled reply

real streaming reply, server shutdown, reconnect, and idle state:

real codex reply interrupted by server shutdown and recovered to idle

neighboring stop interaction:

stop generation interrupts a real codex reply and preserves partial output

Not verified

native ios/android and electron shells, macos/windows, remote relay/tunnel connections, and other providers were not exercised on this linux vm. real interruption of streaming reasoning and proposed-plan items was not exercised; those types are covered by the passing focused recovery test. startup was checked with a copy of actual in-flight state, rather than a machine crash. automatic restart continuation was disabled for the matching comparison; its existing tests passed.