fix(preview): improve browser recording quality#8839

merged11
main
GitHub fork: t3code/improve-browser-recording-quality
openedAug 31, 2026byMmaria
View on GitHubTwo-way sync enabledPR last imported 22d ago

browser recordings previously serialized jpeg screenshots through ipc at a fixed 12 fps and re-encoded them through a canvas, which caused low and uneven frame rates. this switches recording to electron's native tab media stream, preserves the preview's current dimensions, and negotiates the best available mediarecorder codec while saving the recorder's actual output format.\n\nrecordings default to 30 fps and can be set to 30 or 60 fps in Settings > Integrations > Browser. hidden tabs remain offscreen; recording gives each guest two bounded paint opportunities before native source acquisition, then primes it with a best-effort non-encoded capture with one retry. this allows multiple tabs to record without opening their threads. native stream acquisition has a five-second bound and disposes streams that resolve after timeout.\n\nfocused tests, lint, formatting, and affected package typechecks pass. an animated-page capture produced av1 at 1000x620, 60.01 fps, and bt.709 color; a two-tab pass confirmed both recordings continued while both previews were hidden. built with gpt-5.6-sol in the codex harness.


Note

Medium Risk Recording is a full pipeline swap with an IPC return-type change and new failure modes (dimensions, capture timeout, missing MIME); desktop tab/webview lifecycle races are explicitly serialized but remain sensitive.

Overview Replaces the old 12 fps JPEG screencast โ†’ canvas โ†’ MediaRecorder path with native Chromium tab capture: desktop startRecording now returns a DesktopPreviewRecordingSource (sourceId, dimensions), and the renderer acquires that stream via getUserMedia and encodes with MediaRecorder.

Desktop PreviewManager measures viewport size, optionally warms the guest with capturePage, resolves getMediaSourceId, and keeps only a background-throttling lease for recordingโ€”the periodic capturePage loop is picture-in-picture only. Saved artifacts pick file extensions from the recorder MIME via recordingFileExtension. New errors include PreviewRecordingSourceSizeUnavailableError; start/stop recording use tab lifecycle locking and race-handling tests were updated accordingly.

Web browserRecording drops frame subscription and canvas sizing; it holds browser surface activity, waits two animation frames (bounded timeout), applies browserRecordingFrameRate (30 or 60 fps, default 30 in contracts), negotiates preferred codecs, and saves the recorderโ€™s actual output type. Startup capture is bounded (5s) with cleanup for late streams.

Settings & contracts: browserRecordingFrameRate client setting, Integrations UI, search/restore labels; IPC/previewBridge startScreencast return type changes to DesktopPreviewRecordingSource.

Reviewed by Cursor Bugbot for commit 385b43162ea5ab6ff17ecec115842b2540491297. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Improve browser recording quality with native MediaStream capture

  • Replaces canvas-based JPEG frame rendering with native MediaStream capture using MediaRecorder.
  • PreviewManager.startRecording measures source dimensions and returns DesktopPreviewRecordingSource (sourceId, width, height) instead of void.
  • Adds browserRecordingFrameRate client setting (30 or 60 fps) configurable via the Integrations settings panel.
  • Risk: desktop.ipc.preview.startRecording and DesktopPreviewBridge.recording.startScreencast now return DesktopPreviewRecordingSource; out-of-tree callers must handle the new return type and PreviewRecordingSourceSizeUnavailableError.

Macroscope summarized 385b431.

Added 1 commit

  • fix(preview): improve browser recording quality7c0541f
Synced from GitHub

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

โš™๏ธ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: cefe4431-da73-464e-849d-85405318640f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • ๐Ÿ” Trigger review

Warning

Your free Security trial is over. An organization admin can activate Security or dismiss this notice.


Comment @coderabbitai help to get the list of available commands.

apps/desktop/src/preview/Manager.tsline 3151Outdated ยท 7c0541fOpen thread
โ€ฆโ€ฆ@@ -3091,6 +3101,54 @@ const makeNativeOperations = Effect.fn("PreviewManager.makeOperations")(function
3148+ () => wc.getMediaSourceId(requestWebContents),
3149+ );
3150+ return { sourceId, ...size } satisfies DesktopPreviewRecordingSource;
3151+ }).pipe(Effect.onError(() => stopFrameCapture(tabId, "recording").pipe(Effect.ignore)));
MmacroscopeappBoton line 3151Aug 31, 2026

๐ŸŸก Medium preview/Manager.ts:3151

A failed startRecording can stop a newer recording: its Effect.onError unconditionally calls stopFrameCapture(tabId, "recording"), which removes whichever recording lease currently exists. If the first start fails after a second start succeeds, the second caller has a source ID but its capture lease and background-throttling protection are released. Associate cleanup with this invocationโ€™s lease or serialize start/stop operations.

๐Ÿค– Copy this AI Prompt to have your agent fix this:
text
1In file @apps/desktop/src/preview/Manager.ts around line 3151:
2
3A failed `startRecording` can stop a newer recording: its `Effect.onError` unconditionally calls `stopFrameCapture(tabId, "recording")`, which removes whichever recording lease currently exists. If the first start fails after a second start succeeds, the second caller has a source ID but its capture lease and background-throttling protection are released. Associate cleanup with this invocationโ€™s lease or serialize start/stop operations.

fixed in d876adc0e by serializing start and stop through the existing per-tab lifecycle lock. the new regression test holds the first measurement open, starts a second recording, fails the first, and verifies the second lease remains active through stop.

apps/desktop/src/preview/Manager.tsline 3142Outdated ยท 7c0541fOpen thread
โ€ฆโ€ฆ@@ -3091,6 +3101,54 @@ const makeNativeOperations = Effect.fn("PreviewManager.makeOperations")(function
3139+ return { width: measuredSize.width, height: measuredSize.height };
3140+ },
3141+ );
3142+ const sourceId = yield* attempt(
MmacroscopeappBoton line 3142Aug 31, 2026

๐ŸŸ  High preview/Manager.ts:3142

startRecording can return a media sourceId for a detached guest after the tab has registered a replacement webview, causing recording to target the old page or fail to acquire the source. executeJavaScript yields before wc.getMediaSourceId(requestWebContents), but the captured wc is never revalidated against the tab's current webContentsId; re-check that identity before obtaining the source ID (or serialize recording startup with webview replacement).

๐Ÿค– Copy this AI Prompt to have your agent fix this:
text
1In file @apps/desktop/src/preview/Manager.ts around line 3142:
2
3`startRecording` can return a media `sourceId` for a detached guest after the tab has registered a replacement webview, causing recording to target the old page or fail to acquire the source. `executeJavaScript` yields before `wc.getMediaSourceId(requestWebContents)`, but the captured `wc` is never revalidated against the tab's current `webContentsId`; re-check that identity before obtaining the source ID (or serialize recording startup with webview replacement).

fixed in d876adc0e by serializing source acquisition with webview registration and revalidating the current WebContents after the async size read. the replacement-race test verifies registration cannot replace the guest while the source is being acquired.

MmacroscopeappBotreviewed ยท 1 inline threadAug 31, 2026
Synced from GitHub

One convention finding in apps/desktop/src/preview/Manager.ts: the new recording-source size validation is a pure domain/validation failure but is modeled by throwing a synthetic Error inside attempt so it surfaces as a generic PreviewOperationError whose only real information lives in a manufactured cause. Details inline.

The rest of the change looks consistent with the repo's Effect conventions: namespace imports (* as Effect, * as PreviewManager), the new IPC method uses DesktopIpc.makeIpcMethod with Effect.fn and resolves PreviewManager.PreviewManager from the environment, no new ManagedRuntime.make / runPromise / service-instance injection was introduced, and the widened startRecording return type is reflected in the inline Context.Service interface and the contracts schema.

Posted via Macroscope โ€” Effect Service Conventions

apps/desktop/src/preview/Manager.tslines 3118โ€“3140Outdated ยท 7c0541fOpen thread
โ€ฆโ€ฆ@@ -3091,6 +3101,54 @@ const makeNativeOperations = Effect.fn("PreviewManager.makeOperations")(function
3137+ throw new Error("Preview media source dimensions are unavailable.");
3138+ }
3139+ return { width: measuredSize.width, height: measuredSize.height };
3140+ },
MmacroscopeappBoton lines 3118โ€“3140Aug 31, 2026

This guard is a pure validation failure, but it is expressed by throwing a bare Error inside attempt, so the only failure that reaches callers is a generic PreviewOperationError ("Desktop preview operation failed: recording.measureSource") whose real reason exists solely as a manufactured cause. Convention here is to define the failure with Schema.TaggedErrorClass and structured attributes and derive message from them, rather than fabricating an Error just to populate cause (a validation error created without an underlying failure needs no cause).

Consider adding a dedicated error next to the other preview errors, e.g.

ts
1export class PreviewRecordingSourceSizeUnavailableError extends Schema.TaggedErrorClass<PreviewRecordingSourceSizeUnavailableError>()(
2 "PreviewRecordingSourceSizeUnavailableError",
3 { tabId: Schema.String, webContentsId: Schema.Number },
4) {
5 override get message(): string {
6 return `Preview media source dimensions are unavailable for tab ${this.tabId}`;
7 }
8}

and failing with return yield* new PreviewRecordingSourceSizeUnavailableError({ tabId, webContentsId: wc.id }) after a plain (non-attempt) shape check, keeping attempt/attemptPromise for the calls that can actually throw (executeJavaScript, getMediaSourceId).

Posted via Macroscope โ€” Effect Service Conventions

fixed in d876adc0e with PreviewRecordingSourceSizeUnavailableError, a structured Schema.TaggedErrorClass carrying tabId and webContentsId. invalid dimensions now fail directly without a manufactured cause, with focused coverage.

apps/desktop/src/preview/Manager.tsline 3148Outdated ยท 7c0541fOpen thread
โ€ฆโ€ฆ@@ -3091,6 +3101,54 @@ const makeNativeOperations = Effect.fn("PreviewManager.makeOperations")(function
3145+ tabId,
3146+ webContentsId: wc.id,
3147+ },
3148+ () => wc.getMediaSourceId(requestWebContents),
CcursorBoton line 3148Aug 31, 2026

Recording uses stale webview source

Medium Severity

startRecording keeps the WebContents from requireWebContents across executeJavaScript and then calls getMediaSourceId on that snapshot. A guest swap in that window, or later in registerWebview, leaves the renderer stream on the old sourceId, so the recording can capture a destroyed tab.

Additional Locations (1)

Reviewed by Cursor Bugbot for commit 7c0541f366aec0bfcd2004564d5d0d389f37a8c0. Configure here.

Added 1 commit

  • fix(preview): guard recording startup racesd876adc
MmacroscopeappBotreviewed ยท 1 inline threadAug 31, 2026
Synced from GitHub

One convention issue found in the changed error-handling code. Everything else (the new PreviewRecordingSourceSizeUnavailableError tagged error, its union registration, the DesktopIpc.makeIpcMethod conversion for startRecording, and the contracts schema) follows the service conventions.

Posted via Macroscope โ€” Effect Service Conventions

apps/web/src/browser/browserRecording.tslines 444โ€“448Outdated ยท d876adcOpen thread
โ€ฆโ€ฆ@@ -533,10 +441,16 @@ const finalizeBrowserRecording = async (
445+ recording.recorder.mimeType ||
446+ recording.chunks.find((chunk) => chunk.type.length > 0)?.type;
447+ if (!mimeType) {
448+ throw new Error("MediaRecorder did not report its output format.");
MmacroscopeappBoton lines 444โ€“448Aug 31, 2026

This is a pure domain condition (the recorder never reported an output format), but it is expressed as a bare Error inside the try that maps everything to operation: "save-artifact". Callers therefore only ever see Browser recording operation save-artifact failed for tab X., with the real reason surviving solely as a manufactured cause โ€” the convention is to model such failures with Schema.TaggedErrorClass and structured attributes and derive message from them, not to fabricate an Error to populate cause.

Consider resolving the mime type before the try, and failing with a dedicated error, e.g. add beside the other error classes:

ts
1export class BrowserRecordingFormatUnavailableError extends Schema.TaggedErrorClass<BrowserRecordingFormatUnavailableError>()(
2 "BrowserRecordingFormatUnavailableError",
3 { tabId: Schema.String },
4) {
5 override get message(): string {
6 return `MediaRecorder did not report an output format for tab ${this.tabId}.`;
7 }
8}

and then throw new BrowserRecordingFormatUnavailableError({ tabId }) outside the save-artifact wrapper so the structured failure reaches callers unwrapped.

Posted via Macroscope โ€” Effect Service Conventions

fixed in 08bbfe04e with BrowserRecordingFormatUnavailableError, raised before the save-artifact wrapper so the structured failure reaches callers unchanged. the new test verifies the error, skipped save, and recording-slot cleanup.

Added 1 commit

  • fix(preview): report missing recording format08bbfe0
Synced from GitHub

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable โ€” This PR replaces the existing recording pipeline with native tab capture and changes desktop IPC, stream lifecycle, codec negotiation, artifact formats, and persisted recording settings. It also introduces a 30 FPS product default and a new user-facing frame-rate control, so the breadth and runtime impact warrant human review.

You can add or adjust custom eligibility rules. Learn more.

Added 2 commits

  • fix(preview): record hidden tabs at configured frame rate32bb4fb
  • test(desktop): include recording frame rate settingc716da2
apps/web/src/browser/browserRecording.tsline 213Outdated ยท c716da2Open thread
โ€ฆโ€ฆ@@ -226,17 +198,23 @@ const stopMediaRecorder = async (recorder: MediaRecorder | null): Promise<void>
237210 publishActiveRecordingTabIds();
238211};
239212
213+const waitForBrowserRecordingPaint = async (): Promise<void> => {
MmacroscopeappBoton line 213Aug 31, 2026

๐ŸŸ  High browser/browserRecording.ts:213

When the renderer is hidden or minimized, startBrowserRecording never settles, and a subsequent stop times out while retaining the active recording and native capture lease indefinitely. The two requestAnimationFrame callbacks are paused for hidden documents, so this wait needs a bounded timer fallback that cancels the pending frame work when it settles.

๐Ÿค– Copy this AI Prompt to have your agent fix this:
text
1In file @apps/web/src/browser/browserRecording.ts around line 213:
2
3When the renderer is hidden or minimized, `startBrowserRecording` never settles, and a subsequent stop times out while retaining the active recording and native capture lease indefinitely. The two `requestAnimationFrame` callbacks are paused for hidden documents, so this wait needs a bounded timer fallback that cancels the pending frame work when it settles.

fixed in 5f9370b20. compositor warmup now races the two frame callbacks against a 250 ms fallback and cancels any still-pending frame callbacks on either path. the focused test covers a renderer where requestAnimationFrame never fires.

Added 1 commit

  • fix(preview): bound recording paint warmup5f9370b
apps/web/src/browser/hostedBrowserWebviewStyle.tsline 46Outdated ยท 5f9370bOpen thread
โ€ฆโ€ฆ@@ -39,8 +39,11 @@ export function resolveHostedBrowserWebviewWrapperStyle(input: {
43+ // Keep active background work behind the app so native tab capture can
44+ // start before the user has ever opened this preview.
45+ left: renderingActive ? 0 : HIDDEN_BROWSER_WEBVIEW_OFFSET,
46+ top: renderingActive ? 0 : HIDDEN_BROWSER_WEBVIEW_OFFSET,
CcursorBoton line 46Aug 31, 2026

Hidden webviews show through chrome

Medium Severity

Hidden renderingActive guests now sit at the viewport origin with visibility: visible and z-index: -1. That stacks them above the root background, so glass and other translucent chrome reveal the guest and the wrapperโ€™s bg-muted/35 for the whole recording, picture-in-picture, or background-activity lifetime.

Reviewed by Cursor Bugbot for commit 5f9370b201281b43121c29ce814ca65292d90a3f. Configure here.

fixed in 14cb6ffa3. hidden guests are back at the original offscreen position, so they cannot bleed through translucent chrome. native recording now primes the guest with one non-encoded capturePage call before requesting its media source; the steady-state recording remains the native stream.

Added 1 commit

  • fix(preview): prime hidden native recordings14cb6ff
CcursorBotreviewed ยท 1 inline threadAug 31, 2026
Synced from GitHub

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

โŒ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 14cb6ffa38975f6a1282c256ccc5911e8d02f0f0. Configure here.

apps/desktop/src/preview/Manager.tsline 3147Outdated ยท 14cb6ffOpen thread
โ€ฆโ€ฆ@@ -3090,11 +3100,77 @@ const makeNativeOperations = Effect.fn("PreviewManager.makeOperations")(function
3144+ webContentsId: wc.id,
3145+ },
3146+ () => wc.capturePage().then(() => undefined),
3147+ );
CcursorBoton line 3147Aug 31, 2026

Background recording warmup can fail

Medium Severity

startRecording now requires a one-shot capturePage warmup, and that call is fatal. It runs inside startScreencast before the renderer applies visibility: visible or waits for compositor frames, while hidden guests stay at HIDDEN_BROWSER_WEBVIEW_OFFSET. A transient compositor miss on a never-shown tab fails the whole start, and the later paint wait no longer composites an on-screen guest.

Additional Locations (1)

Reviewed by Cursor Bugbot for commit 14cb6ffa38975f6a1282c256ccc5911e8d02f0f0. Configure here.

fixed in 4db19c241. the renderer now waits for the hidden surface to receive two paint opportunities before requesting the native source. the one-shot capture warmup retries once and is best-effort, so a transient or permanent capturePage rejection no longer aborts native acquisition. focused tests cover paint-before-source ordering, paused-frame fallback, and warmup rejection.

Added 1 commit

  • fix(preview): stabilize hidden recording startup4db19c2
apps/web/src/browser/browserRecording.tsline 393Outdated ยท 4db19c2Open thread
โ€ฆโ€ฆ@@ -425,34 +389,28 @@ export async function startBrowserRecording(
430- if (!hasFirstFrame) {
431- const cause = new Error(`No valid recording frame arrived for tab ${tabId}.`);
392+ try {
393+ recording.stream = await captureTabMediaStream(source, frameRate);
MmacroscopeappBoton line 393Aug 31, 2026

๐ŸŸ  High browser/browserRecording.ts:393

startBrowserRecording() can remain stuck indefinitely at captureTabMediaStream() when the getUserMedia() permission prompt is ignored, so its finally never runs and the active-recording slot plus surface-activity lease remain held. Add a timeout/cancellation path that releases the recording and lease, and stops any stream that resolves after cancellation.

๐Ÿค– Copy this AI Prompt to have your agent fix this:
text
1In file @apps/web/src/browser/browserRecording.ts around line 393:
2
3`startBrowserRecording()` can remain stuck indefinitely at `captureTabMediaStream()` when the `getUserMedia()` permission prompt is ignored, so its `finally` never runs and the active-recording slot plus surface-activity lease remain held. Add a timeout/cancellation path that releases the recording and lease, and stops any stream that resolves after cancellation.

fixed in ff7e4b9d0. native stream acquisition now shares the existing five-second startup bound. timeout runs the normal failed-start cleanup, which stops the desktop capture lease and releases the browser surface/activity slot. if getUserMedia resolves after timeout, every late track is stopped immediately. the regression test covers the timeout, released state, and late-stream disposal.

Added 1 commit

  • fix(preview): bound recording stream startupff7e4b9
MmacroscopeappBotreviewed ยท 1 inline threadAug 31, 2026
Synced from GitHub

One error-modeling convention issue in the new browser recording capture path. The two findings from earlier runs (the fabricated Error for the missing output format, and the bare Error for invalid measured source dimensions in PreviewManager.startRecording) are both resolved โ€” BrowserRecordingFormatUnavailableError and PreviewRecordingSourceSizeUnavailableError now model those failures structurally.

Posted via Macroscope โ€” Effect Service Conventions

apps/web/src/browser/browserRecording.tslines 219โ€“221Outdated ยท ff7e4b9Open thread
โ€ฆโ€ฆ@@ -226,17 +199,69 @@ const stopMediaRecorder = async (recorder: MediaRecorder | null): Promise<void>
218+ streamPromise,
219+ new Promise<never>((_, reject) => {
220+ timeoutId = window.setTimeout(
221+ () => reject(new Error("Browser recording media capture did not settle.")),
MmacroscopeappBoton lines 219โ€“221Aug 31, 2026

The acquisition timeout is a pure domain condition (no underlying failure exists), but it is expressed by manufacturing an Error whose message is the only record of what happened. The call site then wraps it as BrowserRecordingOperationError({ operation: "capture-media-stream" }), whose message is derived solely from operation/tabId, so a stalled getUserMedia and a rejected getUserMedia become indistinguishable to callers except through the fabricated cause string.

Consider modeling it the same way this PR already models the missing output format: a Schema.TaggedErrorClass (e.g. BrowserRecordingCaptureTimeoutError with tabId and timeoutMs, no cause) thrown from this helper, and let the capture-media-stream catch block pass an already-structured recording error through instead of re-wrapping it.

Posted via Macroscope โ€” Effect Service Conventions

fixed in 34406a298. capture timeout is now BrowserRecordingCaptureTimeoutError with structured tabId and timeoutMs fields and no fabricated cause. after the existing failed-start cleanup succeeds, the typed timeout passes through unchanged; a real cleanup failure still preserves its cause in BrowserRecordingOperationError. the focused timeout test now asserts the tagged error shape.

Added 1 commit

  • fix(preview): type recording capture timeouts34406a2

Added 1 commit

  • fix(preview): default browser recordings to 30 fps385b431