fix(server): agent recordings honor the browser frame rate setting#23895

open5
main
fix/issue-19442
opened16h agobyJJulius Marminge
View on GitHubTwo-way sync enabledPR last imported 16h ago

Problem

Selecting 60 fps in Browser recording settings still encoded agent preview recordings with a fixed 30 fps capture limit. Reproduced on Linux desktop with the reporter’s animation-frame counter and actual agent recording start/stop calls.

Change

Send the existing recording-rate setting with the existing browser-profile report. The server uses the latest explicitly reported rate for each new recording’s capture stream and bitrate, retaining the default of 30 until a client reports. Older reports can omit the field. Existing settings exports remain compatible.

Scope and approval

Fixes #19442

Triaged bug via the automated issue flow. Existing Browser recording configuration now applies to the server encoder used by agents. No new RPC, tool input, permission, persisted setting or recording option.

Verification

  • Focused server/browser tests: 43 passed; contracts preview/settings tests: 27 passed after merging main.
  • Contracts, server, web and desktop typechecks passed; changed-file lint/format passed; final desktop build passed.
  • Two independent reviewers returned MERGE on the complete final diff.
  • Committed replay fixture: scripts/fixtures/issue-19442/.

Actual Electron recordings, Linux/KVM, Xvfb, GPU disabled, one light theme:

Content viewportBefore / controlAfter at 60Observed result
1280×800Before recording, built from 11adcb0e25After recording, built from 07d7b3c83620.79 → 29.62 fps average; busiest one-second window 28 → 37 frames
800×45030 fps control, after build60 fps recording, same after build29.14 → 45.54 fps average; median frame interval 32.22 → 19.77 ms

Steps: open the counter fixture, start recording, wait four seconds, stop, and inspect decoded timestamps with ffprobe. Source animation frames stayed approximately 60 Hz. The setting is a maximum; these runs do not demonstrate sustained 60 fps. Later commits preserve the captured encoder source and add compatibility exports, tests and replay files.

Spatial samples show different counter values and codec noise; the small 60 fps control also includes the representative click’s blue pointer. These are capture-content differences, not additional UI changes. Full-frame diffs use absolute RGB ×8 at zero threshold and strict max-channel delta >16/255; they inspect spatial content and do not prove cadence.

1280×800, lightBeforeAfterDiff0DiffT16
Complete frames at ≥1 sBefore
11adcb0e25
After
07d7b3c836
Diff0 amplifiedDiffT16
800×450, light, both after implementation30 fps control60 fps controlDiff0DiffT16
Complete frames at ≥1 s30 fps control
07d7b3c836
60 fps control
07d7b3c836
Control Diff0 amplifiedControl DiffT16

800×450 control frames and both diffs, pixel measurements and capture environment.

Full evidence index, lifecycle recordings, measurements and limitations. Actual setting reset/selection, duplicate start, changing settings during recording, subsequent recording and immediate idle export were exercised.

Independent audit of the rebuilt desktop repeated the 800×450 recording at 44.55 fps, peak 53 frames/second, with source RAF at 60 Hz: actual recording, probe, runtime responses. The audited app was built at 877c7da3d5; 3bb66f26c9 only changes the replay probe.

Additional live web and lifecycle evidence at 3bb66f26c9 confirms the normal web reporter sends 60, a server-owned page exports, the encoder retains 60 after client disconnect, and a restarted server selects 30 with no client connected. All three recordings exported; their delivered cadence is routing/lifecycle evidence, not a sustained-60 claim.

Not exercised live: permission/hydration transitions, multi-client/multi-environment delivery, or the reporter’s Wayland compositor. Existing guards and destination routing are unchanged; focused tests cover default, explicit rates, omitted-field retention and updates. The most recent explicit client report controls the environment’s rate for subsequent recordings.

Implemented and verified with GPT-6.1 Sol and Claude Opus 5.5 through T3 Code’s Codex and Claude Code harnesses.

Checks

All checks passed

Added 5 commits

  • fix(server): agent browser recordings honor the recording frame rate setting07d7b3c
  • test(preview): scan recording timestamps in linear time3bb66f2
  • fix(contracts): keep recording rate exports on the settings subpath70acaad
  • Merge main into fix/issue-19442877c7da
  • test(preview): add reproducible recording rate fixtureb66487e
Synced from GitHub

Approvability

Verdict: Needs Human Approval

Macroscope's review found this PR requires human approval — This fix carries the existing 30/60 fps browser setting into server-side agent recording, changing capture cadence and bitrate while preserving the 30 fps default and older-client compatibility. Because it alters a non-trivial recording pipeline, the runtime impact merits human review.

You can add or adjust custom eligibility rules. Learn more.

Synced from GitHub

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ProviderMetricMain baselineThis PRImpactPR ceiling
CodexTotal thread wire5.0 KiB5.0 KiB0 B (0.0%)6.8 KiB✅
CodexThread snapshot wire3.8 KiB3.8 KiB0 B (0.0%)4.9 KiB✅
CodexLive turn WebSocket wire1.2 KiB1.2 KiB0 B (0.0%)2.0 KiB✅
CodexLive turn WebSocket decoded20.9 KiB20.9 KiB0 B (0.0%)29.3 KiB✅
CodexLive turn messages220 (0.0%)8✅
ClaudeTotal thread wire5.0 KiB5.0 KiB−24 B (−0.5%)6.8 KiB✅
ClaudeThread snapshot wire3.8 KiB3.8 KiB0 B (0.0%)4.9 KiB✅
ClaudeLive turn WebSocket wire1.2 KiB1.2 KiB−24 B (−2.0%)2.0 KiB✅
ClaudeLive turn WebSocket decoded21.2 KiB21.2 KiB−41 B (−0.2%)29.3 KiB✅
ClaudeLive turn messages21−1 (−50.0%)8✅

Baseline: c7e3993 · PR result: 3bb66f2 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Acknowledged; human approval remains outstanding. Focused coverage checks the default, older-client reports and setting updates. Matched 800×450 recordings delivered 29.14 fps at 30 and 45.54 fps at 60; sustained 60 fps is not claimed. The PR includes the recordings, lifecycle evidence and verification limits.

Synced from GitHub

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
📥 Commits

Reviewing files that changed from the base of the PR and between c7e3993fe9fb8e75ccfb1ac64fb360c130d91689 and 3bb66f26c9baefc2541974ce8809eb30c45cbc1d.

📒 Files selected for processing: 10
  • apps/server/src/preview/ServerBrowser.test.ts
  • apps/server/src/preview/ServerBrowser.ts
  • apps/server/src/preview/ServerBrowserPage.ts
  • apps/web/src/browser/BrowserProfileReporter.tsx
  • packages/contracts/src/preview.ts
  • packages/contracts/src/settings.ts
  • scripts/fixtures/issue-19442/README.md
  • scripts/fixtures/issue-19442/idle.html
  • scripts/fixtures/issue-19442/moving.html
  • scripts/fixtures/issue-19442/replay.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 4 remain after this review.

📝 Walkthrough

Agent preview recordings now use the browser recording frame rate reported by the client. The server defaults to 30 fps until it receives a rate. The changes also add tests and fixture tools for inspecting recording cadence.

Changes: Agent recording frame rate

Layer / File(s)Summary
Define and report the recording rate
packages/contracts/src/preview.ts, packages/contracts/src/settings.ts, apps/web/src/browser/BrowserProfileReporter.tsx
The preview profile contract accepts an optional 30 or 60 fps rate. The client includes its configured rate in reports.
Configure server-side recordings
apps/server/src/preview/ServerBrowserPage.ts, apps/server/src/preview/ServerBrowser.ts, apps/server/src/preview/ServerBrowser.test.ts
The server defaults to 30 fps and retains the latest reported rate. The encoder uses that rate for capture and bitrate calculations. Tests cover the default, reported rates, and bitrate values.
Reproduce and inspect recording cadence
scripts/fixtures/issue-19442/*
The fixture adds idle and animated pages, instructions for comparing recordings, and commands to serve the fixture and inspect video frame timing.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Low

Sequence Diagram(s)

mermaid
1sequenceDiagram
2 participant BrowserProfileReporter
3 participant PreviewReportProfilesInput
4 participant ServerBrowser
5 participant recordingEncoderScript
6 BrowserProfileReporter->>PreviewReportProfilesInput: Report recordingFrameRate
7 PreviewReportProfilesInput->>ServerBrowser: Provide optional recordingFrameRate
8 ServerBrowser->>recordingEncoderScript: Pass stored frame rate
9 recordingEncoderScript->>recordingEncoderScript: Set capture rate and bitrate

Possibly related issues

Suggested reviewers: maria-rcks

Merge Risk: ⚪ Minimal · up to 3bb66

Agent recordings can use the selected frame-rate maximum. No unresolved issue identified here prevents merging after normal checks.

Pre-merge checks: Passed 4
Check nameStatusExplanation
Title checkPassedThe title clearly and concisely identifies the main change: server agent recordings now honor the browser frame rate setting.
Description checkPassedThe description includes the required Problem, Change, Scope and approval, and Verification sections. It explains the issue, implementation, linked issue, compatibility behavior, focused tests, manual…
Linked Issues checkPassedIssue #19442 is open and directly linked. The web reporter sends the existing browserRecordingFrameRate setting in reportProfiles. The server retains the latest defined rate and passes it to the e…
Out of Scope Changes checkPassedThe changed server, web, and contract code implements the rate-reporting and rate-aware encoding required by issue #19442. The tests verify the behavior. The settings re-exports preserve compatibility…
Full details: Description check

Explanation

The description includes the required Problem, Change, Scope and approval, and Verification sections. It explains the issue, implementation, linked issue, compatibility behavior, focused tests, manual evidence, limitations, and agent usage.

Full details: Linked Issues check

Explanation

Issue #19442 is open and directly linked. The web reporter sends the existing browserRecordingFrameRate setting in reportProfiles. The server retains the latest defined rate and passes it to the encoder. The encoder uses that rate for canvas.captureStream() and bitrate calculation. The optional contract field preserves older reports. Focused tests cover the 30 fps default, explicit 60 fps, omitted-field retention, updates, and bitrate behavior.

Full details: Out of Scope Changes check

Explanation

The changed server, web, and contract code implements the rate-reporting and rate-aware encoding required by issue #19442. The tests verify the behavior. The settings re-exports preserve compatibility. The fixture README, HTML pages, and replay probe support reproduction and verification. No unrelated product behavior, RPC, permission, persisted setting, or recording option is added.

✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.