fix(desktop): capture covered preview surfaces without hanging#33719

open2
main
GitHub fork: fix/desktop-preview-capture
opened1d agobyAandrew-metaplanet
View on GitHubTwo-way sync enabledPR last imported 1d ago

Problem

Agent snapshots and a recording's first frame can hang when an Electron preview guest is covered by the app. The existing painting lease moves the guest into the window, but Chromium's Page.captureScreenshot can still wait indefinitely for a presented frame. A stuck first frame then holds the server capture queue and later snapshots hit the broker timeout.

Reproduced on macOS with Electron 44.10.1: with the lease held and the guest covered, 2 of 5 baseline CDP captures never answered within the fixture deadline. The same guest captured successfully when uncovered.

Change

Use Electron's webContents.capturePage() for plain viewport PNG/JPEG requests in the existing guest command relay. Retry rejected or empty surface copies at 50 ms intervals, with a five-second deadline. Serialize native copies per webContents across request timeouts; canceled waiters cannot start extra work, and a later live request captures a fresh frame after the earlier copy settles.

Clipped, beyond-viewport, WebP, fromSurface: false, child-session and unrecognized-option requests retain their existing CDP path. The existing painting/focus leases and idle suspension remain in place.

Scope and approval

This is a focused reliability fix for the existing screenshot operation, submitted under the obvious-bug exception in CONTRIBUTING.md. Three files change: the guest relay, its regression tests, and the existing host test fake needed by that relay. No setting, permission, protocol, always-on rendering or unrelated cleanup is introduced.

Related work: #21534 also addresses background captures with a renderer preparation handshake and CDP nudges; #27092 includes native capture as part of a macOS screen-lock solution that keeps guests inside the window between captures. This change keeps the current on-demand painting lifecycle and addresses the covered, unlocked-window repro. It does not claim screen-lock support. The server queue/deadline work in #24523 is complementary.

Verification

Tested tree: d21df352534b68fe3a60ff02810e867a92089678 (identical to the locally tested implementation tree).

  • vp run check:changed: format, lint, knip, affected typechecks and tests passed.
  • Focused GuestInput and DesktopBrowserHost suites: 53 tests passed in independent review. Regressions cover rejected/empty copies, a never-settling native copy, canceled requests sharing that pending copy, and a fresh capture after it settles.
  • Real Electron comparison using the production sendPageCommand and a synthetic local page: all 5 covered-window captures passed in 20–23 ms. A guest that never paints failed cleanly in 5003 ms; the same guest then recovered for 3 captures. JPEG first-frame capture returned a valid image. Baseline covered captures included two unanswered requests.
  • Integrated desktop check used the actual compiled main process, preload, web client, bundled backend, free Replay adapter and MCP tools against a synthetic Japanese fixture. With preview_status reporting visible:false: snapshot 124 ms, recording start 3807 ms, snapshot during recording 67 ms, recording stop 51 ms with a valid MP4, then tab close. Both returned PNGs were visually inspected.

These timings are local observations, not guarantees. Linux, Windows, screen lock, remote viewing on a second device and packaged release updates were not verified. The integrated run used the local desktop build, not a released installer. Already-issued native capture promises cannot be canceled; the map prevents overlapping copies while the request deadline frees its caller.

Synthetic fixture evidence (no accounts, external sites or private data):

Recording produced by the hidden-preview agent workflow

Sanitized before/after and integrated timing receipt

Update at 28f73fc9: merged upstream main 4c924ff4. The only conflict was in the DesktopBrowserHost test double; both upstream loading/destruction state and the native capture mock are retained. Capture implementation and GuestInput regression tests are unchanged. vp run check:changed origin/main passes (56 focused tests); the full desktop preview directory passes 271/271, including upstream first-page attachment cases.

Independent code review: GPT-6.1 Sol, APPROVE, no blocker or should-fix findings.

Implemented with Claude Opus and GPT-6.1 Sol through T3 Code, using Claude Code and Codex harnesses.

Checks

21 successful · 10 skipped
All checks passed
28f73fc

Added 1 commit

  • fix(desktop): capture covered preview surfaces without hangingd21df35
Synced from GitHub

Approvability

Verdict: Approved at d21df35

Macroscope's review found this PR approvable — Preview viewport screenshots use Electron surface capture to avoid hanging when the preview is covered. Retries are bounded, copies are serialized, and specialized screenshot requests retain their existing routing.

You can add or adjust custom eligibility rules. Learn more.

Synced from GitHub

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

Eligible viewport screenshot requests now use Electron capturePage() with retries and a five-second deadline. Other screenshot requests continue through the existing command path.

Changes: Desktop viewport screenshot capture

Layer / File(s)Summary
Capture eligibility and retry behavior
apps/desktop/src/preview/GuestInput.ts
The capture path accepts supported viewport parameters, serializes captures per guest, retries empty or failed captures, and encodes successful images as PNG or JPEG.
Command routing and screenshot tests
apps/desktop/src/preview/GuestInput.ts, apps/desktop/src/preview/GuestInput.test.ts, apps/desktop/src/preview/DesktopBrowserHost.test.ts
Eligible requests route to viewport capture. Tests cover retries, clipped requests, timeout behavior, and fresh captures after an unresolved capture settles.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Sequence Diagram(s)

mermaid
1sequenceDiagram
2 participant Request
3 participant sendPageCommand
4 participant captureViewport
5 participant webContents
6 Request->>sendPageCommand: Page.captureScreenshot
7 sendPageCommand->>captureViewport: eligible viewport request
8 captureViewport->>webContents: capturePage()
9 loop empty or failed capture
10 webContents-->>captureViewport: empty image or error
11 captureViewport->>webContents: retry after 50 ms
12 end
13 webContents-->>captureViewport: painted image
14 captureViewport-->>Request: encoded image or timeout

Possibly related issues

  • pingdotgg/t3code#23022: This change updates viewport capture; the issue's remote-viewer captureLock path may remain unaffected.
  • pingdotgg/t3code#32410: Eligible plain viewport PNG/JPEG requests now use capturePage() instead of CDP.
  • pingdotgg/t3code#23936: This change updates eligible viewport requests; child-session requests retain the existing CDP path.

Suggested labels: bugfix

Suggested reviewers: juliusmarminge

Merge Risk: External image Minimal · up to 28f73

Viewport captures now time out rather than hang. No material merge-blocking risk was established.

Pre-merge checks: Passed 4

Check nameStatusExplanation
Title checkPassedThe title clearly identifies the desktop fix and its purpose: capturing covered preview surfaces without hanging.
Linked Issues checkPassedCheck skipped because no linked issues were found for this pull request.
Out of Scope Changes checkPassedCheck skipped because no linked issues were found for this pull request.
Description checkPassedThe description covers the problem, change, scope and approval rationale, and verification. It reports focused test results and integrated checks, and it states the limits of testing. It also includes…
Full details: Description check

Explanation

The description covers the problem, change, scope and approval rationale, and verification. It reports focused test results and integrated checks, and it states the limits of testing. It also includes the required UI evidence and agent model and harness.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
External image Recent review info

External image Run configuration

External image Commits

Reviewing files that changed from the base of the PR and between d21df352534b68fe3a60ff02810e867a92089678 and 28f73fc90aa82da6143fefbf91e736f9fe85ec0a.

External image Files selected for processing: 1

  • apps/desktop/src/preview/DesktopBrowserHost.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

Added 1 commit

  • Merge upstream main into fix/desktop-preview-capture28f73fc