perf(web): streamed assistant text re-renders only its own row#27419

merged1
t3/streaming-perf-3
t3/streaming-perf-4
opened1d agobyJJulius Marminge
View on GitHubTwo-way sync enabledPR last imported 21h ago

Each assistant delta changed the visible turn items, so ThreadTimeline, ConversationTimeline and the virtual list re-rendered and the list ran three extra layout commits before the streaming row itself updated (about 5 React commits per delta).

The timeline now reads visible items that hold while only streamed assistant text moves (timelineVisibleTurnItemsAtom), and a streaming assistant row reads its own text through assistantMessageTextAtom. Any structural change, including the message finishing, still updates the timeline with the full text. Reasoning items are unchanged. While a message streams, timeline-level readers of its text (minimap preview, find over off-screen rows) see the text from the last structural change; the copy button is already hidden while streaming.

Commits per delta drop from about 5 to 2. The stream render scenario asserts the timeline renders at most 3 times across 30 deltas (32 on main). Checked in the dev app with the Replay provider: streamed text grows and the finished reply renders in full.

Benchmark

Measured with the PandaCode vs T3 Code streaming bench (W1: a 50 KB Markdown reply delivered as 500 whole-text updates over 32 s, production build, headless Chromium on cups, median of 3 runs). Each row adds one layer of this stack:

T3 buildMain-thread taskScriptWorst delivery lateness
main21.0 s14.8 s73 ms
+ timeline rows (this stack, 1)19.3 s13.0 s26 ms
+ Markdown blocks (2)12.1 s4.4 s10 ms
+ git actions panel (3)10.4 s3.4 s9 ms
+ streamed text bypasses the timeline (4)9.5 s2.6 s9 ms
PandaCode, same machine6.6 s0.5 s2 ms

The bench delivers an update every 64 ms. In the app the default paragraph streaming mode sends at most one every 400 ms, so real streams hit these paths less often, but each update costs what it does here.

Implemented by Claude Opus 5.5 in Claude Code, running in T3 Code.

🤖 Generated with Claude Code

Checks

25 successful · 14 skipped
All checks passed
52a14d6

Added 1 commit

  • perf(web): streamed assistant text re-renders only its own row52a14d6
Synced from GitHub

Approvability

Verdict: Approved at 52a14d6

Macroscope's review found this PR approvable — Streaming assistant text now updates its own row rather than re-rendering the entire timeline. The change is limited to client-side subscriptions, with structural updates still reaching the timeline and no changes to stored data or product defaults.

You can add or adjust custom eligibility rules. Learn more.

Synced from GitHub

Review in Change Stack →

No actionable comments were generated in the recent review.

External image Recent review info
External image Run configuration
External image Commits

Reviewing files that changed from the base of the PR and between 67613870641071f312e0b7739a55149e28c3e8cd and 52a14d6d3d21792ed95d08fbae9f88ef67b01293.

External image Files selected for processing: 6
  • apps/web/src/components/chat/ThreadTimeline.tsx
  • apps/web/src/components/chat/TimelineAssistantRows.tsx
  • apps/web/src/state/entities.ts
  • apps/web/test/appScenarios.renders.tsx
  • packages/client-runtime/src/state/threadDetail.test.ts
  • packages/client-runtime/src/state/threadDetail.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 6 remain after this review.

Walkthrough

The change separates timeline-visible items from live assistant text during streaming. The timeline atom preserves its array when only streaming assistant text changes, while a per-item text atom supplies current text to the assistant row. Web hooks and tests cover the state and rendering behavior.

Changes: Streaming assistant text

Layer / File(s)Summary
Stable timeline state
packages/client-runtime/src/state/threadDetail.ts, packages/client-runtime/src/state/threadDetail.test.ts
The timeline atom retains its previous array when changes are limited to streamed assistant text. A separate atom returns text for an assistant item by ID. Tests cover text updates during streaming and the completed item.
Web timeline rendering
apps/web/src/state/entities.ts, apps/web/src/components/chat/ThreadTimeline.tsx, apps/web/src/components/chat/TimelineAssistantRows.tsx, apps/web/test/appScenarios.renders.tsx
Web hooks expose timeline-visible items and per-item assistant text. The timeline uses the timeline-specific hook, and assistant rows prefer live text when available. The render scenario permits up to three timeline renders.

Priority: External image Low

Estimated code review effort: 2 (Simple) | ~12 minutes

Change: Refactor

Sequence Diagram(s)

mermaid
1sequenceDiagram
2 participant ThreadTimeline
3 participant ThreadDetailAtoms
4 participant AssistantTimelineRow
5 ThreadTimeline->>ThreadDetailAtoms: Read timeline-visible items
6 ThreadDetailAtoms-->>ThreadTimeline: Return stable array for text-only changes
7 AssistantTimelineRow->>ThreadDetailAtoms: Read assistant text by item ID
8 ThreadDetailAtoms-->>AssistantTimelineRow: Return current streamed text

Possibly related issues

  • pingdotgg/t3code#24120: The issue concerns re-derivation during streamed reasoning and plan text. This change addresses streamed assistant text, but does not apply the optimization to reasoning or plan text.

Suggested labels: size:S

Suggested reviewers: t3dotgg

Merge Risk: External image Minimal · up to 52a14

Streaming text updates are isolated to their row. No concrete merge-blocking issue was established.

Pre-merge checks: Passed 4
Check nameStatusExplanation
Title checkPassedThe title clearly and concisely describes the primary change: streamed assistant text now re-renders only its own row. It uses a conventional commit format.
Description checkPassedThe description explains the performance problem, the implementation, the behavior trade-offs, and focused verification results. It includes benchmark data and manual validation. It does not provide a…
Linked Issues checkPassedCheck skipped because no linked issues were found for this pull request.
Out of Scope Changes checkPassedCheck skipped because no linked issues were found for this pull request.
Full details: Description check

Explanation

The description explains the performance problem, the implementation, the behavior trade-offs, and focused verification results. It includes benchmark data and manual validation. It does not provide an explicit Scope and approval section or issue link, but it is otherwise sufficiently complete and focused.

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.