perf(mobile): thread list resumes from the saved cache, and large gaps reload over gzip HTTP#32287

open1
main
t3/shell-cold-resume
opened1d agobyJJulius Marminge
View on GitHubTwo-way sync enabledPR last imported 10h ago

On a cold start every client downloaded the full thread list, even when its saved copy was a few events behind. When a client did resume from a cursor beyond the server's replay cap (1,000 events, 8 MB), the server answered with a full snapshot over the socket. iOS does not compress that, so it arrived at 1.37 MB instead of the 88 KB the gzip HTTP route costs.

Fix

  • Resume from the saved cache. The shell cache now records the snapshot its rows came from as origin: server version, database migration, omission flags, and snapshotAt. A client whose cache matches its subscription and is under 12 hours old subscribes with afterSequence instead of loading a fresh snapshot. Replay never renews snapshotAt, so drift stays bounded.
  • snapshot-required. A resuming client sends httpSnapshotFallback: true. If the server cannot replay that cursor (over the gap cap or pruned), it answers with one snapshot-required item and doesn't send the snapshot. The client then loads the gzip HTTP snapshot and subscribes again from it.
    • A cursor from a snapshot that was just loaded never sends the flag, so a refusal cannot loop.
    • Old servers ignore the flag, and old clients never send it.
  • Items buffered from a superseded session are dropped, and a resume that catches up bumps generation.

Scope and approval

Maintainer-authored. It replaces #30732 at the maintainers' direction, after measuring replay against a fresh snapshot. Layer 1 of 2; #32288 builds on it.

Measurements

All numbers come from a sanitized copy of a heavy real database (2,279 threads, 2.8 M events). Clients ran each build's exact flow, using its own requests and flags, through a throttling proxy that counts bytes. Every cell is the median of 3 runs.

Time until the thread list is current, and bytes down:

scenariomainthis PRLTE (20 Mbit/s, 50 ms)slow 3G (1.6 Mbit/s, 300 ms)
cold start, 300 events behindHTTP snapshot, 92 KBreplay, 20 KB408 → 194 ms1.89 → 1.05 s
warm reconnect, past the replay capsocket snapshot, 1,338 KBsnapshot-required → HTTP, 92 KB854 → 467 ms7.93 → 2.19 s
cold start, past the replay capHTTP snapshot, 92 KBsnapshot-required → HTTP, 92 KB417 → 480 ms1.89 → 2.21 s
first load, or warm reconnect 300 events behindunchangedunchanged424 / 203 ms1.88 / 1.05 s
  • Cold starts with a small gap: 4.6× fewer bytes and half the wait on LTE.
  • Warm reconnects past the cap: 14× fewer bytes and 3.6× faster on slow 3G. Main sent these uncompressed over the iOS socket.
  • The one cost: a cold start past the cap now pays one extra round trip for the refused resume, about 60 ms on LTE and 300 ms on 3G. The 12-hour cache limit keeps that to returns where the gap is likely to be replayable.

CPU for each path, on the same database:

pathwireserver CPUclient CPU (JIT / interpreter)
HTTP snapshot (gzip)88.5 KB~210 ms11.6 / 40 ms
replay ≤ 1,000 events (deflate)7 KB~125 ms0.6 / 3.2 ms
over-cap snapshot over the iOS socket (main)1.37 MB~160 ms—

Replay beats a snapshot on every metric up to the cap. Past roughly 1,200 dense events it costs about as much as a snapshot, which is why the cap stays at 1,000.

How often each case happens is decided by real history. Replay covers 96% of returns after 5 minutes away, 50% after 30 minutes, 25% after 2 hours, and 4% after 8 hours.

The proxy does not model TCP or TLS handshakes. Both builds open the same connections in each scenario, so the comparison holds.

Verification

  • Tests:
    • shell-sync.test.ts gains a resume table and a snapshot-required case, covering cold and warm. Reverting the loop guard fails 6 cases.
    • ws.test.ts gains a pruned-boundary row.
    • The cache-store test covers the origin round-trip.
  • Web, against a copy of a 2.1 GB real database:
    • Reloading made no shell snapshot request.
    • On a warm reconnect against pruned history, the trace showed a refused subscribe (13 ms), then one HTTP snapshot (112 ms), then a resubscribe without the flag.

Web client after a warm reconnect over pruned history: thread list intact

Credit

This replaces #30732 by @Bil0000 and builds on #30317 by @emmems. Both found the cold-start cost and the uncompressed socket fallback.

Built by Claude Fable 5.1 and Claude Opus 5.5 in Claude Code.

🤖 Generated with Claude Code

Checks

24 successful · 6 skipped
All checks passed
460a65b

Added 1 commit

  • perf(mobile): thread list resumes from the saved cache, and large gaps reload over gzip HTTP460a65b
Synced from GitHub

Approvability

Verdict: Approved at 460a65b

Macroscope's review found this PR approvable — Thread lists resume from recent, compatible cached snapshots and reload unreplayable gaps through HTTP. Provenance checks, a 12-hour age limit and authoritative snapshot fallback keep the synchronization change bounded.

You can add or adjust custom eligibility rules. Learn more.

Synced from GitHub

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ProviderMetricMain baselineThis PRImpactPR ceiling
CodexTotal thread wire5.0 KiB5.0 KiB0 B (0.0%)6.8 KiB✅
CodexThread snapshot wire3.8 KiB3.8 KiB0 B (0.0%)4.9 KiB✅
CodexLive turn WebSocket wire1.2 KiB1.2 KiB0 B (0.0%)2.0 KiB✅
CodexLive turn WebSocket decoded21.5 KiB21.5 KiB0 B (0.0%)24.4 KiB✅
CodexLive turn messages110 (0.0%)8✅
ClaudeTotal thread wire5.0 KiB5.0 KiB0 B (0.0%)6.8 KiB✅
ClaudeThread snapshot wire3.8 KiB3.8 KiB0 B (0.0%)4.9 KiB✅
ClaudeLive turn WebSocket wire1.2 KiB1.2 KiB0 B (0.0%)2.0 KiB✅
ClaudeLive turn WebSocket decoded21.9 KiB21.9 KiB0 B (0.0%)24.4 KiB✅
ClaudeLive turn messages110 (0.0%)8✅

Baseline: 23bdb9e · PR result: 460a65b · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Synced from GitHub

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

Shell cache records now include origin metadata. The client uses this metadata to decide whether to resume a cached snapshot. When a resume requires a snapshot, the server can signal the client to load one over HTTP and resubscribe.

Changes: Shell resume and snapshot fallback

Layer / File(s)Summary
Resume contracts and cache metadata
packages/contracts/src/orchestrationV2.ts, packages/client-runtime/src/platform/orchestrationCache.ts, packages/client-runtime/src/platform/persistence.ts, apps/mobile/src/connection/environment-cache-store.ts, apps/mobile/src/connection/environment-cache-store.test.ts, apps/web/src/connection/storage.ts
The shell stream contract adds snapshot-required and the optional httpSnapshotFallback input. Cache types and mobile and web stores support optional origin metadata. The mobile test checks that the metadata survives a save and load.
Server snapshot fallback
apps/server/src/ws.ts, apps/server/src/ws.test.ts
When a shell resume requests HTTP snapshot fallback, the server emits snapshot-required instead of loading and streaming a shell snapshot. Tests cover both fallback and the existing snapshot behavior.
Client cache validation and reload
packages/client-runtime/src/state/shell.ts, packages/client-runtime/src/state/shellReducer.ts, packages/client-runtime/src/state/shell-sync.test.ts
The client resumes cached rows only when origin identity matches and the snapshot is less than 12 hours old. It handles snapshot-required by reloading over HTTP, updates cache origin metadata, and filters stream items by active subscription. Tests cover eligible and ineligible cached origins and cold and warm fallback paths.

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

mermaid
1sequenceDiagram
2 participant ShellSync
3 participant subscribeOrchestrationV2Shell
4 participant HTTPSnapshotLoader
5 participant EnvironmentCacheStore
6 ShellSync->>subscribeOrchestrationV2Shell: Subscribe with cursor and HTTP fallback
7 subscribeOrchestrationV2Shell-->>ShellSync: Emit snapshot-required
8 ShellSync->>HTTPSnapshotLoader: Load authoritative snapshot
9 HTTPSnapshotLoader-->>ShellSync: Return snapshot
10 ShellSync->>EnvironmentCacheStore: Save snapshot with origin
11 ShellSync->>subscribeOrchestrationV2Shell: Resubscribe from snapshot sequence

Possibly related PRs

Suggested reviewers: bil0000, saphid, t3dotgg

Merge Risk: External image Minimal · up to 460a6

The change lets cold starts resume from the saved cache and falls back to the gzip HTTP snapshot when replay is not possible. No concrete merge-blocking defect was established.

Pre-merge checks: Passed 3 · Inconclusive 1

Failed Failed checks: 1 inconclusive

Check nameStatusExplanationResolution
Description checkInconclusiveThe description follows the repository template closely. It covers the problem (cold-start full thread-list downloads and uncompressed over-cap socket snapshots), the change (cache origin, afterSequen…Add a link to the triaged issue or to the discussion that records explicit maintainer approval of the direction and scope, including the approval comment. The author states the work was done at the maintainers' direction, but the template r…
Passed Passed checks: 3 passed

Check nameStatusExplanation
Title checkPassedThe title names the two main changes: the thread list resumes from the saved cache on cold start, and large replay gaps reload over gzip HTTP. The raw summary confirms both: cache origin with afterSeq…
Linked Issues checkPassedCheck skipped because no linked issues were found for this pull request.
Out of Scope Changes checkPassedCheck skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the two main changes: the thread list resumes from the saved cache on cold start, and large replay gaps reload over gzip HTTP. The raw summary confirms both: cache origin with afterSequence resume, and the snapshot-required fallback that loads an HTTP snapshot. The title follows the conventional commit format and is specific to the changeset.

Full details: Description check

Explanation

The description follows the repository template closely. It covers the problem (cold-start full thread-list downloads and uncompressed over-cap socket snapshots), the change (cache origin, afterSequence resume, httpSnapshotFallback and snapshot-required, loop guard, generation bump and buffered-item drop), and scope with a link to the superseded PR and a statement that the change is maintainer-authored. Verification is detailed: named test files and cases, a mutation-style check (reverting the loop guard fails 6 cases), web manual traces against a copy of a real database, and a screenshot. Measurements are reported with medians and methods. Gaps: the Scope and approval section does not link a triaged issue or a discussion with explicit maintainer approval comment. It states the work is maintainer-authored and was done at the maintainers' direction, but no approval comment is linked. The template requires that link or the explanation that the change qualifies as a small obvious fix, and this change is a broad performance and protocol change, so the approval evidence matters. The template also asks for the model and harness used, and the description ends with a 'Built by' line that names models and Claude Code, which satisfies that request. The screenshot is hosted on a file-drop URL, not GitHub, and the template says to upload evidence to GitHub. The UI change here is minor, since the PR is primarily protocol and sync behavior, so the screenshot is supplementary.

Resolution

Add a link to the triaged issue or to the discussion that records explicit maintainer approval of the direction and scope, including the approval comment. The author states the work was done at the maintainers' direction, but the template requires the approval evidence itself. Alternatively, explain why this change qualifies without prior approval. Because this PR changes the stream protocol (a new httpSnapshotFallback flag and a snapshot-required item), it is not a small obvious fix. Also move the screenshot from the file-drop host to GitHub, or confirm that the screenshot is evidence the template accepts. Keep the existing verification, measurements, and model/harness attribution.

Finishing Touches

Generate docstrings

  • Commit to this branch
  • Create a new PR
Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
External image Recent review info

External image Run configuration

External image Commits

Reviewing files that changed from the base of the PR and between 23bdb9ef1cc39ddb21a3e077f97f580d50b8fc22 and 460a65be963cf7a0d9b400c7cc15b7fc77e55aed.

External image Files selected for processing: 11

  • apps/mobile/src/connection/environment-cache-store.test.ts
  • apps/mobile/src/connection/environment-cache-store.ts
  • apps/server/src/ws.test.ts
  • apps/server/src/ws.ts
  • apps/web/src/connection/storage.ts
  • packages/client-runtime/src/platform/orchestrationCache.ts
  • packages/client-runtime/src/platform/persistence.ts
  • packages/client-runtime/src/state/shell-sync.test.ts
  • packages/client-runtime/src/state/shell.ts
  • packages/client-runtime/src/state/shellReducer.ts
  • packages/contracts/src/orchestrationV2.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.

Synced from GitHub
Action not completed

Already reviewed the last commit. Use @coderabbitai full review to rerun a review of the entire changeset.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.