After an update restart, resumed threads sit in "Working" for 9 minutes: provider-turn.start waits behind a retrying thread-title.generate effect#25962

opened5d agobyUUkrainianCitizen

Alpha tester reporting for duty.

Version

0.1.4-nightly.20261027.3770 (macOS desktop, nightly channel).

Setup

The desktop app self-updated and restarted while four threads had active turns (Claude Agent provider). After the final relaunch, the app tried to resume those threads.

Observed

From the server trace log and the state database:

  • About 12 seconds after the server started, EnvironmentCommands.resumeThreadQueue ran once per thread. Each one dispatched a queued-run.start command. The receipt is accepted with no error, and an Orchestrator.startNextQueuedRun span opens.
  • The thread list and the thread view switch to "Working" with a spinner.
  • For about 15 minutes no provider session exists:
    • In the state database, provider_session_runtime and the provider sessions projection have every row for these threads at stopped.
    • Their last provider turn is cancelled (by the pre-restart reconcile).
    • No new provider event log is written under logs/provider/.
    • There is no agent child process on the machine.
    • Nothing in the server trace, the command receipts or the UI reports an error.
  • A user message typed into one of these threads ("continue where you left off") is shown in the transcript but gets no reply during that time, because the thread believes a run is in progress.
  • Roughly 15 minutes after the relaunch, the threads started working on their own, with no user action.

Expected

queued-run.start should start the provider session promptly, or fail visibly and return the thread to idle. If a start is delayed or queued behind something, the UI should say so instead of showing "Working" with nothing running.

Workaround

Wait. A separate issue covers the update itself killing the in-flight turns and relaunching three times.

Tried to reproduce this on main at e9236274a3, with the Claude Agent provider and "Continue threads after server update" on (the default):

  1. A thread running a long foreground Bash turn, with a second message queued. Graceful server stop, then restart. The restart continuation started its run within a second. Resuming the queue from the web client (Send now, which sends queued-run.start) started the queued run at once.
  2. The same setup, but the server process was killed with SIGKILL mid-turn and then relaunched. Startup recovery queued one provider-runtime.continue per thread. Both were claimed within about 0.1 s, and their provider-turn.start effects ran within 0.4 s.

In neither case did a start sit idle.

The server runs a thread's effects one at a time, in order. So a queued-run.start that is accepted but never spawns an agent usually means its provider-turn.start effect is waiting behind another effect on the same thread. To tell which one held yours, could you share the following?

  • The effect rows for one of the affected threads, from statev2.sqlite in your T3 data folder (userdata). Please send them from a copy, not the live file, and redact paths as needed:
    sql
    1select rowid, effect_type, status, attempt_count, created_at, available_at, lease_expires_at, completed_at, substr(last_error, 1, 200)
    2from orchestration_v2_effect_outbox
    3where thread_id = '<thread id>'
    4order by rowid desc limit 20;
    If those rows are already gone, send them after it happens again.
  • From the server trace, the OrchestrationEffectExecutorV2.execute spans, with their effect type and duration, between the relaunch and the moment the threads started (about 15 minutes later).
  • Whether "continue where you left off" was typed by you, or whether it is the "T3 Code restarted and resumed this turn" notice that the server adds after a restart.
  • Roughly how large the affected projects' working trees are, including untracked files.
Synced from GitHub

You called it. Effect rows for one of the four threads, UTC, from a copy of statev2.sqlite. The other three match within a second.

rowideffect_typestatusattemptscreated_atavailable_atcompleted_atlast_error
7028thread-title.generatesucceeded513:06:16.76713:18:07.45813:28:43.801
7035provider-runtime.continuesucceeded113:13:51.46513:13:51.46513:14:02.381
7039provider-turn.startcancelled013:14:01.93213:14:01.93213:14:46.021Cancelled because the server shut down before the provider work completed.
7042provider-runtime.continuesucceeded113:14:45.99713:14:45.99713:14:50.210
7046provider-turn.startcancelled013:14:50.01413:14:50.01413:16:57.427same
7050provider-runtime.continuesucceeded113:16:57.40713:16:57.40713:17:10.448
7054provider-turn.startcancelled013:17:10.24113:17:10.24113:18:07.445same
7058provider-runtime.continuesucceeded113:18:07.41413:18:07.41413:19:35.141
7062provider-turn.startsucceeded113:19:47.92613:19:47.92613:28:47.569

provider-turn.start (7062) was accepted at 13:19:47 and completed at 13:28:47. The effect ahead of it was thread-title.generate (7028), created with the thread's first message, on its fifth attempt, made available again at 13:18:07 by the restart recovery, and completed at 13:28:43. The same row exists for all four affected threads plus a fifth idle thread. All five completed between 13:28:43.729 and 13:28:43.852, within 120 ms of each other, and the four provider-turn.start rows completed 1.5 to 4 seconds later. Five independent title calls finishing together looks like a shared timeout or a worker pool freeing at once, possibly the mechanism in #16636.

The server trace for 13:19 to 13:28 has rotated out (10 files of 10 MB cover about 20 minutes at this activity level), so I cannot provide the OrchestrationEffectExecutorV2.execute spans. Tell me what to capture and I will try to reproduce.

"continue where you left off" was typed by me, after the third "T3 Code restarted and resumed this turn" notice. The notices themselves are the server's. Working trees are about 155 MB each, with no uncommitted or untracked files at the time.

Correction to my title: the wait was about 9 minutes, not 15.