Sign up
View stargazers13Sign up

fix(desktop): agent preview tabs keep working while T3 is hidden#22743

draft1
main
t3/preview-background-rendering-lease
opened3d agobyJJulius Marminge
22 files changed
View on GitHubTwo-way sync enabledPR last imported 2h ago

Agent preview_snapshot calls time out whenever the T3 window isn't on screen, while preview_evaluate and clicks keep working. This has been fixed and has come back several times.

Why it keeps coming back

Every preview tab is a <webview> inside the main window. To save energy, inactive tabs are parked off-screen (#8567, #16961). A screenshot needs a freshly drawn frame, and Chromium stops drawing a page that sits outside its window. Earlier fixes added opt-in "keep drawing" leases, and each refactor that moved capture to another layer lost them. #15328 dropped the automation lease when agent automation moved to the server.

I tested this in real Electron 44.9.3 on macOS, with one fresh launch per condition, screenshots taken over CDP, and the page changed before each capture to tell fresh frames from stale ones:

Where the agent's page liveswindow visiblehiddenminimized
<webview> parked off-screen (today)times outtimes outtimes out
parked, then moved back inside the window to capturefreshneverabout half the time
kept inside the window, created while the window was visiblefreshfreshfresh
kept inside the window, created while the window was hiddenfreshnevernever
any of the above with throttling off or Chromium's no-backgrounding switchesno helpno helpno help
its own never-shown BrowserWindowfreshfreshfresh

Agents open tabs whenever they like, including while you're away, so no lease or placement trick on a <webview> can work.

Fix

Agent-opened tabs (sessions with automationOwner) no longer get a <webview>. On desktop:

  • Desktop: the desktop app hosts each one in a never-shown BrowserWindow with the same partition (your logins), preload, sandbox and passkey bridge as a panel guest (preview/BackgroundTabWindow.ts). The manager registers it through the same path a <webview> uses, so listeners, zoom, mute, the debugger and the relay attach are all unchanged. It's destroyed when the tab closes. Off macOS it also closes with the main window, so the app can still quit.
  • Web: the panel shows these tabs through the server stream view, as web and mobile already do for server tabs.
  • Server: the relay's attached event now carries background. The server sizes those tabs itself (viewport, zoom, viewer resize), handles their file choosers, and closes them when idle, the same as headless tabs.
  • Errors: a timeout that comes from the browser host no longer claims the broker's "timed out after 15000ms".

Tabs you open yourself are unchanged.

Agent tabs stay on desktop rather than moving to the server's headless browser because agent tabs default to your default profile, and the headless browser has separate profile storage. Moving them would log agents out of sites you're signed into.

What changes for agent tabs: they look like server tabs do on web. That means no native DevTools, element picker or desktop recording for them in the panel. Agent recording is server-side and still works.

Regression guard

  • New CI job, Desktop Background Preview (macOS): runs apps/desktop/scripts/verify-background-preview.mjs. It launches real Electron, hides and then minimizes the app window, and creates a tab window after that, using the exact options the manager ships. It fails unless every screenshot returns fresh pixels. It runs when preview code, this check, the Electron pin in apps/desktop/package.json, or the CI files change. It fails when painting is disabled.
  • Focused tests, each confirmed to fail without the change:
    • the manager creates and registers the hidden window and destroys it on close
    • the browser host mounts no <webview> and asks for a background window
    • the server sizes background desktop tabs and leaves panel tabs alone

Verification

  • Every preview test file in the desktop package passes, plus the preview IPC tests, ServerBrowser.test.ts, and the browser host browser test. Typecheck is clean for contracts, server, web and desktop. Changed files lint clean.
  • The real-Electron check passes locally.
  • Not verified end to end in the full app. I tried a real agent turn against vp run dev:desktop. To let the web UI pair, I had to switch the dev desktop to network-accessible mode, and in that mode the dev desktop didn't host any tab, including a human <webview> tab, which my change doesn't touch. The server fell back to headless and every snapshot succeeded, but that says nothing about this change. Before merging, someone should run a desktop build, minimize T3, have an agent open a tab, and take snapshots.

Opus 5.5 via Claude Code.

🤖 Generated with Claude Code

Checks

Some checks failed

Added 1 commit

  • fix(desktop): agent preview tabs keep working while T3 is hidden6a3a2cd
Ggithub-actionsBot3d ago
Synced from GitHub

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ProviderMetricMain baselineThis PRImpactPR ceiling
CodexTotal thread wire5.0 KiB5.0 KiB0 B (0.0%)6.8 KiB✅
CodexThread snapshot wire3.8 KiB3.8 KiB0 B (0.0%)4.9 KiB✅
CodexLive turn WebSocket wire1.2 KiB1.2 KiB0 B (0.0%)2.0 KiB✅
CodexLive turn WebSocket decoded20.9 KiB20.9 KiB0 B (0.0%)29.3 KiB✅
CodexLive turn messages220 (0.0%)8✅
ClaudeTotal thread wire5.0 KiB5.0 KiB0 B (0.0%)6.8 KiB✅
ClaudeThread snapshot wire3.8 KiB3.8 KiB0 B (0.0%)4.9 KiB✅
ClaudeLive turn WebSocket wire1.2 KiB1.2 KiB0 B (0.0%)2.0 KiB✅
ClaudeLive turn WebSocket decoded21.2 KiB21.2 KiB0 B (0.0%)29.3 KiB✅
ClaudeLive turn messages220 (0.0%)8✅

Baseline: 618761a · PR result: 6a3a2cd · Source CI: failure

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Aajpemok2d ago
Synced from GitHub

Windows Background Capture Evidence

Historical Windows evidence related to #16567 and the work in #21534, #22547 and #22743:

After restarting our locally repaired installation based on nightly 0.1.2-nightly.20261021.3457, snapshots completed through the installed preview API. Two background captures of an owned synthetic page succeeded. The second showed a changed colour and "Synthetic Frame2", confirming that the captured content had updated.

These results come from our local repair and do not test any of the three upstream PR heads. They do not establish reliability across minimisation, cancellation, navigation or guest replacement. Page Visibility behaviour remains unverified.

A matched Windows test of the chosen implementation would be the useful next check.

Prepared with GPT-6.1-Sol through the Codex harness in T3 Code.

Sshivam1d ago

#23936 is the case this PR targets: on macOS 0.1.1, preview_snapshot times out for tabs opened with open:false and for every tab a delegate_task child opens, even after preview_open({tabId, open:true}). Evaluate, click and navigate still work in those tabs.

I didn't run anything. I read the diff, and both cases have automationOwner set, so they'd take the new never-shown window path. The error-message change also covers the issue's second point (the host fails at 10 s but the message says 15000ms).

The PR says it hasn't been checked end to end in the full app. The issue's repro steps would do for that check: open a tab with open:false and snapshot it, then have a delegate_task child open its own tab and snapshot that. The PR body doesn't link #23936 yet.

DDerpyCrabs1d ago
Synced from GitHub

Tested this approach on Linux/Wayland with Electron 44.9.3 and opened a focused follow-up, #24065, targeting this PR's branch at 6a3a2cd473c404b17eaaa97a472080fad23b58dc.

Two additions were needed locally: offscreen: true for reliable fresh captures, and loading about:blank before prepareWebview so the initial debugger command does not wait for a document that is only loaded after preparation. The follow-up also covers the startup ordering with a failing-before/passing-after test and isolates the native fixture's user data. All six native captures passed with the app hidden/minimized.

Single cherry-pickable commit: https://github.com/DerpyCrabs/t3code/commit/b3a82b88f03116353726a12591cc7f92bc4ff6d3

Prepared with GPT-6-Astra through the Codex harness in T3 Code.

WWosca18h ago
Synced from GitHub

Evidence from a narrower attempt at #23936 (#24133, now a draft in favour of this PR), in case it helps.

On macOS with vp run dev:desktop (Electron 44.9.3), against https://todomvc.com/examples/react/dist/, a capture-scoped acquireBrowserSurfaceActivity lease around each Page.captureScreenshot gave these results:

CaseInstalled nightlyCapture-scoped lease
Parent thread, open:false tab, 3 snapshotstimes out (as in #23936)3 of 3 fresh
Delegated child thread, its own open:false tab, 3 snapshotstimes out (as in #23936)3 of 3 fresh
On-screen tab, T3 minimized during the snapshotworksreadiness never arrives (no frames); works only by falling back to a plain capture

That matches your table: leases help only while the main window is painting, so minimized, hidden and locked hosts need what this PR does. Screenshots are on https://github.com/pingdotgg/t3code/pull/24133#issuecomment-6366981098. A remote-viewer gap in the lease approach is noted on #24133 too: the screencast freezes once the lease is released.

You noted this hasn't been checked end to end in a desktop build. I'm happy to run it (minimize T3, have an agent open a tab, take snapshots, plus a delegated child) and post the results here.