Sign up
View stargazersSign up

[Bug]: Muse thread stays "Waiting on background task" after Workflow completes and parent run finishes; Stop finds nothing#19287

opened17d agobySscotty-git

Before submitting

  • I searched existing issues and did not find a duplicate.
  • I included enough detail to reproduce or investigate the problem.

Searched for "Waiting on background task", Muse + waiting/workflow, and work_stop. No standalone issue covers this Muse + Workflow route; related issues are listed at the end. This is a second reproduced case, without a restart, of a route previously only hypothesised in a comment on #15239.

Area

Not sure (suspected packages/provider-muse, server-side background-work tracking)

Steps to reproduce

  1. In a Muse thread (provider muse, model muse-wonder-full-1.0), ask the agent to launch a native Workflow that outlives the parent turn, so the parent turn ends with the background task pending.
  2. Let the Workflow run to completion so it wakes/resumes the parent.
  3. Let the parent reply and its run complete.
  4. Observe the sidebar and composer: the thread still shows "Waiting" / "Waiting on background task …" with a Stop control.
  5. Try Stop: it targets nothing (details below).

Expected behavior

Once the Workflow has finished, the parent has replied, and the run has completed, the "Waiting on background task" state clears and the thread shows as completed/idle.

Actual behavior

The thread stays on "Waiting" indefinitely after everything has finished, and Stop cannot clear it. Reproduced twice (all times UTC). In neither case was there a T3 restart, settle, fork, or provider switch.

Case 1 — thread c33df331

  • Workflow: 8/8 children finished 2026-10-13T14:15:05Z.
  • Parent replied 2026-10-13T14:20:03Z; the run completed durably.
  • UI still showed "Waiting" ~14 hours later (screenshot 2026-10-14T04:14Z).
  • Muse-side work_list was empty; work_stop for the workflow returned not_found; interrupt hit the already-completed run. The user could not stop it.

Case 2 — thread b6edc5ef

  • An 8-agent Workflow: 8/8 agents result_ready, finished 2026-10-14T04:52:44Z.
  • Parent replied 2026-10-14T04:53:32Z; run completed 2026-10-14T04:53:42Z. Checking afterwards, no background work was still running on the Muse side.
  • UI still showed "Waiting on background task " at 2026-10-14T05:11:21Z (screenshot).

Impact

Major degradation or frequent failure

The work itself completes, but the thread looks permanently busy and offers a Stop that does nothing, so there's no way from the UI to tell it's actually done or to clear the state.

Version or commit

T3 Code Nightly 0.0.46_nightly.20261013.3053 (t3code-nightly-bin)

Checked the release notes for nightlies .3066, .3080, and .3093: no waiting/workflow/Muse fix found. The only adjacent fix in that range is PR #18996 ('Subagent needs input' no longer sticks after the subagent finishes, .3093), which is a different banner. The bug is present on .3053 and not yet retested on .3093 (the AUR package is still on .3053 at time of writing).

Environment

Arch Linux / Omarchy, x86_64; T3 Code Nightly (desktop); Muse Code 1.4.7 (1.4.7-R6101.1); provider muse, model muse-wonder-full-1.0

Logs or stack traces

shell
1# Case 1 (c33df331): workflow 8/8 finished 2026-10-13T14:15:05Z,
2# parent replied 14:20:03Z, run completed; UI "Waiting" 2026-10-14T04:14Z;
3# work_list empty; work_stop(work.v1.workflow…) = not_found
4# Case 2 (b6edc5ef): workflow finished 2026-10-14T04:52:44Z,
5# parent replied 04:53:32Z, run completed 04:53:42Z;
6# UI "Waiting on background task <workflow title>" 05:11:21Z

No stack trace; this is stale UI state, not a crash. Screenshots for both timestamps above can be attached on request (redacted).

Screenshots, recordings, or supporting files

Screenshots showing the stale "Waiting" state at 2026-10-14T04:14Z (case 1) and 2026-10-14T05:11:21Z (case 2), after the runs had completed.

Workaround

None found. Stop, work_stop, and interrupting the run do not clear it, because the work and the run have already finished.

Related issues (similar family, different trigger/mode)

  • #15239 — same family; a 2026-10-10 comment hypothesised this Muse Workflow route from code reading, restart-conditioned and unreproduced. This report is reproduced evidence without a restart.
  • #15993 — same banner from a Claude worktree handoff; there, Stop clears it.
  • #19061 — inverse: Muse monitors are never tracked as pending work; notes the Muse adapter never calls task/stop.
  • #17804 — Muse precedent for a finished turn never being reconciled (missed terminal events).

Suspected area (read on main — hypothesis, not proven)

From reading packages/provider-muse/src/server/adapter.ts on main (which may differ from the .3053 nightly): in publishItem/syncBackground/observedChildren, a non-running publishItem appears to be the only normal way a tracked workflow/subagent item is cleared, so a stuck "Waiting" suggests that terminal update never ran for that item. Candidate drop points in handleNotification worth checking: the owner fast-path's turnId condition, wake holding, the owns() gate, and the revision guard. One shape not covered by the existing fixture (apps/server/src/orchestration-v2/testkit/fixtures/muse_workflow/, where the terminal item arrives under the original turnId before the report turn starts) is a terminal item arriving under a different turnId, or after the report turn has started. It's equally possible Muse never emits a terminal item here — I can't distinguish emit-vs-drop from outside.

What would decide it: an MSP transcript of a stuck run (via apps/server/scripts/record-muse-msp-replay-fixture.ts) — is a terminal item/completed emitted, and under which turnId? Happy to capture a transcript, and to attempt a PR from a fork once maintainers confirm the shape/fix direction.

JJulius Marminge17d ago

Note

Grok responding on behalf of Julius.

This needs an MSP transcript before we can call it a bug. On main, the adapter paths you pointed to do clear a finished Workflow.

In packages/provider-muse/src/server/adapter.ts, a terminal item/completed for a tracked workflow clears it in either of two cases:

  • it arrives under the original turn id, through the owner fast path (~L1318), or
  • it arrives under the report turn id while that turn is running (~L1378). publishItem deletes by itemId, whichever turn owns it.

It would only be dropped if it arrived under some other turn id while no T3 run was active (~L1370), or if Muse never sent it. I can't tell which from the outside either, so please send:

  1. The record-muse-msp-replay-fixture.ts transcript of a stuck run, showing whether a terminal item/completed for the workflow item arrives, and under which turnId.
  2. Whether it still happens on .3093 or newer.

Stop doing nothing for a listed workflow is a known gap (task/stop isn't wired up; see #19061 / #19086). The family is tracked in #15239.

Sscotty-git17d ago
Synced from GitHub

Thanks. Transcripts: https://gist.github.com/scotty-git/b3c9fbf6b411e04808d4fc57e0adf02f (unlisted gist).

Short version: in both stuck threads, Muse sent the terminal item/completed for the workflow under the original turn id, while no T3 run was active. In an instrumented replay on main @614b432a, the adapter clears the workflow and emits provider_thread.updated with pendingBackgroundTasks: [], but the orchestrator doesn't commit that update, so the roster stays stuck.

Not retested in the app on .3093+. My install is still .3053. packages/provider-muse/src/server/adapter.ts, apps/server/src/orchestration-v2/RunExecutionService.ts and EventSink.ts are identical in .3053, .3093 and main. The replays below ran main.

What the real threads show (c33df331, from T3's provider event log and DB):

114:15:05.319 item/completed workflow 30bc740b rev 42 status=completed turnId=01a13599 (original turn, ended 14:13:12)
214:15:05.331 turn/started turnId=01a1359d (Muse's report turn; T3 run 3 requested 14:15:05.350)
3DB afterwards: workflow turn item = completed; provider thread pendingBackgroundTasks = [30bc740b]

b6edc5ef has the same shape: item/completed for 6b0843d2 under the original turn 01a138bb at 04:52:44.619, then turn/started for the report turn 11 ms later.

What the replay shows (those real inbound frames through main, with debug prints):

  1. publishItem removes the workflow from observedChildren, and syncBackground emits provider_thread.updated with pendingBackgroundTasks: [].
  2. The launching run ingests that update after its terminal event, through writeIfProviderThreadOwner (RunExecutionService ~L1263). By then the report run has claimed the provider thread, so EventSink (~L488) doesn't commit the write: expectedLastRunOrdinal=1 actualLastRunOrdinal=2. Those are harness ordinals; on the real thread they were runs 2 and 3.
  3. The report run's turn start sets thread = turnInput.providerThread (adapter L1979). That snapshot still lists the workflow, so the stale roster is saved again and nothing clears it afterwards.

Muse leaves only 10-30 ms between the terminal item and the next turn/started. The workflow's ~40 item/updated revisions, each re-sending every child row, look like the backlog that lets the report run win that race: replaying the same frames with those intermediate updates removed clears normally.

Files in the gist

  • 01_real_c33df331_frames_STUCK_redacted.ndjson, 02_real_b6edc5ef_frames_STUCK_redacted.ndjson: the clean evidence. T3's provider log holds only Muse's inbound notifications, so each is a splice: the stock muse_workflow handshake and turn/start (recorded on muse-spark-1.3-contributor), followed by the real inbound frames from the stuck thread (muse-wonder-full-1.0, Muse 1.4.7). Ids are remapped and free text is replaced with <redacted>. Child-approval server requests are left out. Both replay to a stuck roster on main.
  • 03_control_c33_without_intermediate_workflow_updates_cleared.ndjson: file 01 with the intermediate workflow item/updated frames removed. It clears.
  • 04_live_main_wonder_8agents_STUCK.ndjson: a live record-muse-msp-replay-fixture.ts run on main with an 8-agent workflow. It shows the same uncommitted clear and the stale turnInput, but it's a confounded trace: the children and the report turn hit a Muse 429, so I'm not offering it as proof on its own.

Happy to turn 01 into a fixture if that's useful.