Sign up
View stargazersSign up

Desktop shows a fatal "HTTP 500" screen when the backend is busy for more than about 10 s after startup, because the bearer bootstrap budget allows only one attempt#23378

opened14d agobyRrealhasanshoaib

What happened

On a launch where the backend had just become ready, the desktop window went straight to "Something went wrong. Primary environment request failed during fetch-session-state (HTTP 500)." Clicking "Reload app" about a minute later loaded the app normally, with nothing else changed.

Diagnosis

The desktop's bearer bootstrap gives up before one slow attempt can be retried. DesktopLocalEnvironmentAuth.getBearerToken (apps/desktop/src/backend/DesktopLocalEnvironmentAuth.ts:99-117) retries transient errors every 500 ms, but wraps the whole retry loop in Effect.timeout(BOOTSTRAP_TRANSIENT_RETRY_TIMEOUT), which is 15 s (line 19). Each /oauth/token attempt has its own 10 s timeout (DEFAULT_REMOTE_REQUEST_TIMEOUT_MS, packages/client-runtime/src/authorization/remote.ts:24). So when the backend is briefly unresponsive, the budget covers one full attempt plus half of a second. The trace shows exactly that, twice: a 10.0 s RemoteEnvironmentAuthTimeoutError, then a retry interrupted at 4.5 s.

The web side does not retry either. readDesktopPrimaryBearerToken (apps/web/src/environments/primary/desktopAuth.ts:12-15) clears its cached promise and rethrows, and the route's beforeLoad turns that into the full-screen error.

The error screen also reports a status the server never sent. PrimaryEnvironmentRequestError.fromCause (apps/web/src/environments/primary/auth.ts:54) uses readHttpApiStatus(cause) ?? 500, so the IPC timeout is shown as "HTTP 500".

Why the backend was unresponsive: right after startup, spans across unrelated subsystems stretched to about 35 s (08:51:02 to 08:51:37), and then to about 28 s (08:51:38 to 08:52:06). event-loop-stalls.ndjson has one sample in this period, at 08:52:07, with a 27 s max delay at 100% use. The second stall lines up with ProjectionStore.readShellSnapshotRows (apps/server/src/orchestration-v2/ProjectionStore.ts:6289), a json_extract query over every thread's payload_json that ran 27 to 28 s, and 15.6 s again at 08:52:25. Unrelated spans (git status, GitHub GraphQL, simple ProjectService.getShell selects) stretched to the same length. I could not identify the operation behind the first stall. The state database is 17 GB, so slow queries are expected here. The bug is that a backend that is slow for 30 s turns into a fatal screen.

Of the 32 commits on main after this build (b5a226e4..3b95e783), one touches these files, 6f75bcb3 (ChatGPT Business usage limits), and it does not change this behavior.

Steps to reproduce

Not reproduced on demand. Any way of making /oauth/token take longer than about 10 s during the desktop's first load should show it, for example a large state database (17 GB here) where readShellSnapshotRows takes longer than 10 s, or pausing the backend process for 20 s right after it reports ready.

Version

0.1.2-nightly.20261021.3499 (desktop app and its bundled backend)

Environment

macOS 26.6.2, MacBook Air M4 (16 GB), desktop app, Local environment on, no background service installed

Evidence

shell
1desktop.trace.ndjson, local time UTC+4
208:51:03 clientRuntime.authorization.bootstrapRemoteBearerSession 10.0 s Failure RemoteEnvironmentAuthTimeoutError: .../oauth/token timed out after 10000ms
308:51:14 clientRuntime.authorization.bootstrapRemoteBearerSession 4.5 s Interrupted
408:51:03 desktop.localEnvironmentAuth.getBearerToken 15.0 s Failure DesktopLocalEnvironmentAuthSessionBootstrapError
508:51:19 (second load) same pattern: 10.0 s timeout, retry interrupted at 4.5 s, getBearerToken fails at 15.0 s
6
7server.trace.ndjson, same run
808:50:52 server.startup 9.75 s (ready at 08:51:02)
908:51:37 sqlite <- ProjectionStore.readShellSnapshotRows 28.6 s
1008:51:38 sqlite <- ProjectionStore.readShellSnapshotRows 27.4 s
1108:51:38 sqlite <- ProjectService.getShell (x20) 27.8-28.4 s
1208:52:25 sqlite <- ProjectionStore.readShellSnapshotRows 15.6 s
13
14event-loop-stalls.ndjson
1504:52:07Z delayMaxMs 27001 utilization 1
16
17Same endpoints one minute later: GET /api/auth/session 200 in 1.5 ms, POST /oauth/token 415 in 1.5 ms (dummy body).

#12918 (closed, with #19215 as its duplicate) is the same error chain caused by a transient /oauth/token failure. Its fix, #12919, added the retry described above, but the 15 s overall budget is shorter than what one 10 s attempt plus a retry needs, so a slow backend still reaches the error screen. #23372 is the same install's previous launch, where startup recovery held a write transaction for 11 minutes. That is a different stall: there, readiness never passed.

Fix applied or workaround

Clicked "Reload app" once the backend had settled, about a minute later.

Filed by

claude (claude-opus-5-5) via Claude Code, following the t3 triage playbook

OOfficiallySp7d ago
Synced from GitHub

One more consequence of the readHttpApiStatus(cause) ?? 500 mapping in PrimaryEnvironmentRequestError.fromCause (apps/web/src/environments/primary/auth.ts:54, v0.1.3): it stops the web side from ever retrying "can't connect" errors.

retryTransientBootstrap treats a TypeError (Failed to fetch) as transient (auth.ts:294-305). But fetchSessionState and fetchPrimaryEnvironmentDescriptor (context.ts:12-27) wrap the error in PrimaryEnvironmentRequestError inside the retried operation. A transport error has no response, so it becomes status: 500, which is not in {502, 503, 504}, and the retry loop throws it on the first attempt. When the backend isn't listening yet, the route's beforeLoad goes straight to the fatal screen instead of waiting out the 15 s budget.

#26604 hit this after the backend was killed from outside and restarted: the renderer reloaded about 4.5 s before the backend was back, failed once, and stayed on the error screen until the user clicked Reload. That issue has the trace and log evidence.

Claude Code (claude-opus-5-5) via t3 triage