perf: opening a thread loads 85% less data#24899

merged1
main
perf/provider-snapshots-on-demand
opened7d agobyTtheo
View on GitHubTwo-way sync enabledPR last imported 7d ago

Problem

Opening a thread loaded 0.3 to 3 MB, even when the thread itself was only 16 to 75 KB. Mobile felt it most.

The real cost was provider config. Each provider keeps per-folder "workspace snapshots" (slash commands and skills, 8 to 27 KB each, up to 16 per provider). Every thread open asked the server to scan the thread's folder. The reply held every provider with every snapshot, and the server also pushed that same list to every connected client. A deleted worktree never finished scanning, so its composer retried every 10 seconds, forever.

Images above the visible area also loaded at full size, because the timeline mounts nearby rows.

Fix

  • Snapshots on demand. Clients send workspaceSnapshotsOnDemand. The config stream then leaves snapshots out, so a rescan sends nothing. A folder refresh returns only that provider's snapshot for that folder. The client keeps what it fetched and merges it back into config, so web and mobile readers do not change.
  • Compatible both ways. Old clients send no flag and get the same payload as before. New clients merge fetched snapshots only when the server advertises providerWorkspaceSnapshotsOnDemand. Against older servers they use the config as is.
  • No retry loop. Claude reports a deleted folder as scanned, not pending. Web and mobile composers stop after 3 failed tries per folder. The 3-try cap is what limits other providers.
  • Shared scans. A second request for a folder that is being scanned waits for that scan instead of returning empty. The scan keeps running if the client that started it disconnects.
  • Fresh data. A fetched snapshot is dropped when the provider-wide commands, skills, or enabled state change. Examples: Antigravity's first live session, a late /usage-limits, a sign-out. The composer then fetches the snapshot again at once.
  • Lazy images. Web timeline images and attachment thumbnails use loading="lazy". Mobile's list already draws only rows near the screen.

Result

Real data copy (325 threads), dev server restarted and client caches cleared for each side, mobile viewport, same 7 thread opens. Bytes are WebSocket JSON plus HTTP, without dev-mode JS.

Loadmainthis PRcut
App load (new thread)521 KB393 KB25%
Open thread 11423 KB203 KB86%
Open thread 2277 KB94 KB66%
Open thread 3708 KB40 KB94%
Open thread 4802 KB88 KB89%
Open thread 5808 KB48 KB94%
Open thread 6943 KB137 KB85%
Open thread 7 (71 child threads)3046 KB293 KB90%
Total8528 KB1296 KB85%

App load includes 228 KB of dev-only brand icons. Production icons are 114 KB and cached. Without them, app load goes from 298 KB to 170 KB. App load plus a typical thread open is now about 260 KB. JS bundle size is a separate track and is not counted here.

Limits

  • Another device's scan, or a provider rebuilt after a settings change, shows on this client's next refresh, at most one 5 minute TTL later.
  • Fetched snapshots stay in memory. After a cold start, the composer shows provider-wide skills until its first refresh returns.
  • Still large on some threads: getThreadFamily sends about 2.5 KB per child thread (176 KB for 71 children). That is a follow-up.

Proof

  • New tests, each fails without its fix:
    • ws.test.ts: the stream drops snapshots, a snapshot-only rescan sends nothing, and a scoped refresh returns one folder.
    • ProviderRegistry.test.ts: a second request waits for the running scan, still gets the result after the first request leaves, and only one scan runs.
    • server.test.ts: the merge, its capability gate, identity reuse, and dropping a snapshot when provider-wide commands change.
    • ClaudeCapabilitiesProbe.test.ts: a deleted folder is not pending.
  • vp run check:changed passes.
  • Real app on the branch: the A/B above shows one refresh per thread open and no config pushes. The new-thread / menu still lists repo-scoped skills from the folder snapshot, and /usage-limits.

Surfaces: web and desktop share the client-runtime path. Mobile uses the same path and gets the same retry cap.

Builds on main, not #24692. The council measured the thread payload at 1 to 8% of a thread open, so lazy folded rows were not worth the extra protocol.

Council review

Rounds: 4. A round 0 before any code also chose the cuts.

DelegateRound 1Round 2Round 3Round 4
Claude Opus 5.57/108.5/109/109.5/10
Muse Wonder Full8/109/109.5/109.5/10
Codex GPT-6.1 Sol7/108/109/109/10

Created with Claude Opus 5.5 in Claude Code.

🤖 Generated with Claude Code

Checks

23 successful · 12 skipped · 1 neutral
All checks passed
67b8ab3

Added 1 commit

  • perf: thread opens load 85% less data67b8ab3
Synced from GitHub

Approvability

Verdict: Needs Human Approval

Macroscope's review found this PR requires human approval — This PR introduces a cross-cutting on-demand workspace-snapshot protocol, client-side cache/merge logic, and changed ProviderRegistry scan lifecycle behavior. It also enables the new transport mode by default, so the resulting production behavior and default configuration path merit human review.

You can add or adjust custom eligibility rules. Learn more.

Synced from GitHub

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ProviderMetricMain baselineThis PRImpactPR ceiling
CodexTotal thread wire5.0 KiB5.0 KiB0 B (0.0%)6.8 KiB✅
CodexThread snapshot wire3.8 KiB3.8 KiB0 B (0.0%)4.9 KiB✅
CodexLive turn WebSocket wire1.2 KiB1.2 KiB0 B (0.0%)2.0 KiB✅
CodexLive turn WebSocket decoded20.9 KiB20.9 KiB0 B (0.0%)29.3 KiB✅
CodexLive turn messages220 (0.0%)8✅
ClaudeTotal thread wire5.0 KiB5.0 KiB0 B (0.0%)6.8 KiB✅
ClaudeThread snapshot wire3.8 KiB3.8 KiB0 B (0.0%)4.9 KiB✅
ClaudeLive turn WebSocket wire1.2 KiB1.2 KiB0 B (0.0%)2.0 KiB✅
ClaudeLive turn WebSocket decoded21.2 KiB21.2 KiB0 B (0.0%)29.3 KiB✅
ClaudeLive turn messages110 (0.0%)8✅

Baseline: 5b35402 · PR result: 67b8ab3 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Synced from GitHub

Review in Change Stack →

No actionable comments were generated in the recent review.

External image Recent review info
External image Run configuration
External image Commits

Reviewing files that changed from the base of the PR and between 5b35402623d7d1d88eaa472807272a6c2e78012a and 67b8ab3b6b6b18016d2e543b889188c075909df4.

External image Files selected for processing: 16
  • apps/mobile/src/features/threads/use-composer-command-menu.ts
  • apps/server/src/environment/ServerEnvironment.ts
  • apps/server/src/provider/ClaudeCapabilitiesProbe.test.ts
  • apps/server/src/provider/ClaudeProvider.ts
  • apps/server/src/provider/ProviderRegistry.test.ts
  • apps/server/src/provider/ProviderRegistry.ts
  • apps/server/src/ws.test.ts
  • apps/server/src/ws.ts
  • apps/web/src/components/chat/ChatMarkdownMedia.tsx
  • apps/web/src/components/chat/TimelineUserMessageRow.tsx
  • apps/web/src/components/chat/useProviderWorkspaceSnapshotRefresh.ts
  • packages/client-runtime/src/rpc/session.ts
  • packages/client-runtime/src/state/server.test.ts
  • packages/client-runtime/src/state/server.ts
  • packages/contracts/src/environment.ts
  • packages/contracts/src/rpc.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

Walkthrough

The change adds on-demand workspace snapshot delivery, client-side snapshot caching, and limits on failed refresh retries. It also updates workspace scan coordination and Claude probing, and enables lazy loading for chat images.

Changes: On-demand workspace snapshots

Layer / File(s)Summary
Snapshot contract and scan handling
packages/contracts/src/environment.ts, packages/contracts/src/rpc.ts, apps/server/src/environment/ServerEnvironment.ts, apps/server/src/provider/ProviderRegistry.ts, apps/server/src/provider/ProviderRegistry.test.ts, apps/server/src/provider/ClaudeProvider.ts, apps/server/src/provider/ClaudeCapabilitiesProbe.test.ts
The environment and RPC contracts add on-demand snapshot options. Workspace scan refreshes coordinate through deferreds, and a test covers a waiting refresh after interruption. Claude probing skips a nonexistent working directory, with a test for slashCommandsPending.
Server snapshot delivery
apps/server/src/ws.ts, apps/server/src/ws.test.ts
Config subscriptions can omit workspace snapshots. On-demand refreshes return only the requested instance and working-directory snapshot; provider-status updates omit snapshots when requested. WebSocket tests cover these responses and updates.
Client snapshot fetching and cache
packages/client-runtime/src/rpc/session.ts, packages/client-runtime/src/state/server.ts, packages/client-runtime/src/state/server.test.ts, apps/mobile/src/features/threads/use-composer-command-menu.ts, apps/web/src/components/chat/useProviderWorkspaceSnapshotRefresh.ts
Client subscriptions request on-demand snapshots. The client records and merges fetched snapshots when the saved provider basis matches, and tests cover cache behavior. Mobile and web refresh hooks stop retrying after three failures for a key.

Chat image loading

Layer / File(s)Summary
Lazy loading for chat images
apps/web/src/components/chat/ChatMarkdownMedia.tsx, apps/web/src/components/chat/TimelineUserMessageRow.tsx
Markdown images and attachment previews set image loading to lazy.

Priority: External image Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

mermaid
1sequenceDiagram
2 participant ClientRuntime
3 participant WebSocketServer
4 participant ProviderRegistry
5 participant ClientSnapshotState
6 ClientRuntime->>WebSocketServer: Subscribe with workspaceSnapshotsOnDemand
7 WebSocketServer-->>ClientRuntime: Config without workspace snapshots
8 ClientRuntime->>WebSocketServer: Refresh requested instance and cwd
9 WebSocketServer->>ProviderRegistry: Refresh workspace snapshot
10 ProviderRegistry-->>WebSocketServer: Workspace snapshot
11 WebSocketServer-->>ClientRuntime: Return requested snapshot
12 ClientRuntime->>ClientSnapshotState: Record and merge snapshot

Possibly related issues

  • Issue 24375: The changes add per-working-directory snapshot delivery and refresh; the reported wrong-working-directory discovery objective may remain unresolved.
  • Issue 24378: The changes add per-working-directory snapshot delivery and refresh; the reported wrong-working-directory discovery objective may remain unresolved.
  • Issue 23507: The changes affect when workspace skill snapshots load, but the reported plugin-discovery gap may remain unresolved.

Suggested reviewers: juliusmarminge, exotic209093, justyannicc

Merge Risk: External image Minimal · up to 67b8a

The supplied evidence does not establish a merge-blocking problem. Complete normal checks before merging.

Pre-merge checks: Passed 4
Check nameStatusExplanation
Title checkPassedThe title clearly states the primary performance improvement: opening a thread transfers substantially less data. It is concise and relevant to the changes.
Description checkPassedThe description is detailed and covers the problem, implementation, compatibility behavior, limitations, verification, test coverage, and measured results. The scope and approval section does not expl…
Linked Issues checkPassedCheck skipped because no linked issues were found for this pull request.
Out of Scope Changes checkPassedCheck skipped because no linked issues were found for this pull request.
Full details: Description check

Explanation

The description is detailed and covers the problem, implementation, compatibility behavior, limitations, verification, test coverage, and measured results. The scope and approval section does not explicitly link a triaged issue or maintainer approval, but the overall description is complete enough to pass.

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.