feat(orchestrator): introduce new orchestrator # 2829

MmacroscopeappBotreviewedJul 27, 2026

MmacroscopeappBotJul 27, 2026

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This is a large production orchestration replacement, not a bounded additive change: it introduces new persistence, provider execution, MCP workflows, checkpoint/rollback behavior, and web/mobile/native integration across 2,027 files. The unresolved review set also identifies material security, lifecycle, data-consistency, and runtime risks, while the PR changes Macroscope configuration and product defaults.

Not approved because:

452 blocking correctness issues found at or above your repo's Minimum Blocking Severity

^{Adjust the Minimum Blocking Severity for this repo — including turning it Off — in Settings. You can add or adjust custom eligibility rules. Learn more.}

MmacroscopeappBotreviewedJul 27, 2026

MmacroscopeappBotreviewedJul 27, 2026

MmacroscopeappBotreviewedJul 27, 2026

MmacroscopeappBotreviewedJul 28, 2026

MmacroscopeappBotreviewedJul 28, 2026

MmacroscopeappBotreviewedJul 28, 2026

MmacroscopeappBotreviewedJul 28, 2026

MmacroscopeappBotreviewedJul 28, 2026

JJulius MarmingeAug 4, 2026

GgoggiJul 29, 2026

Scheduled task reports success while its new-thread launch fails in background

Tested on aec85c546e597dbcf78db81b134e7438d9a5ada3 with a scheduled task configured to create a new worktree. Reproduced twice with fresh threads.

Observed

The scheduled-task record ends with lastRunStatus: succeeded, but the generated thread has ordinal 1 failed before provider start:

text

1	Workspace preparation failed during generate metadata:
2	Text generation failed in generateBranchName: Failed to write temp file
3
4	ENOENT: no such file or directory, open
5	/tmp/t3code-codex-schema-208069-6MZJpS/919af017300b

The first reproduction used a custom TMPDIR; I restarted the desktop/server with TMPDIR=/tmp and confirmed the active server process inherited it. The second reproduction failed identically, so this is not caused by that local temp configuration, permissions, or available space.

Status propagation

ScheduledTaskService.runTask treats threadLaunch.launch(...) returning as completion and immediately derives lastRunStatus from that Exit. ThreadLaunchService.launch returns successfully after schedulePreparation(...) forks prepareInBackground into preparationScope. When generateBranchName later fails, the prepared run/thread is marked failed, but that failure cannot update the already-completed scheduled-task run.

This creates two user-visible problems:

A recoverable branch-name metadata failure prevents the scheduled task from starting at all.
The scheduled task still reports success because it tracks dispatch rather than background preparation outcome.

Local workaround confirmed from the launch conditions: choose Use the project checkout or Use a specific checkout; both bypass the implicit worktree branch-name generation path. New-worktree scheduled tasks remain broken in this environment.

Suggested direction: await a preparation receipt/outcome before completing the scheduled-task run, and either fall back to fallbackBranchName(threadId) when branch-name generation fails or otherwise make metadata failure non-fatal.

@macroscope-app review

MmacroscopeappBotAug 4, 2026

Manual reviews triggered for commit 9fe8044:

Effect Service Conventions

^{All prior checks · these links stay valid even if you push more commits.}

Ccolonelpanic8Aug 9, 2026

@juliusmarminge Based on communications I've seen on twitter and elsewhere, it seems that this branch is, in some sense "the future" of t3code. I'm wondering why, then, it is kept in a constant state of conflict with main. Presumably, the two will have to be reconciled, at some point, right?

As someone who maintains a somewhat substantial set of patches on top of upstream t3code, its a bit difficult to know how to proceed with maintaining these. I don't mind not getting any of my stuff merged ever, but it would be nice to understand roughly where things are headed with this branch, because a lot of the patches that I have touch the surface of what is going on here, and its difficult to know what exactly I should be tracking...

BbsmassenaAug 14, 2026

Hey, just came across an issue with scheduled tasks for remote projects. The UI lets me select a project from another environment, but the task is created and executed in the local environment. Since it only stores the projectId, the local server tries to find the remote project in its own database, and the run fails with Project not found. I think the task should be routed to the selected project’s environment, or the project picker should only show projects from the environment where the task will run.

Ccolonelpanic8Aug 20, 2026

Is this pr actually just a stress test of github's pull request page? love to see +181,657 -85,511. There's really no way any of this could have been done more incrementally?

TTheBitAug 21, 2026

Is this pr actually just a stress test of github's pull request page? love to see +181,657 -85,511. There's really no way any of this could have been done more incrementally?

Julius contributed :) Знімок екрана 2026-08-21 о 13 38 11

MmacroscopeappBotAug 29, 2026

Macroscope skipped reviewing this pull request. Per-review cost limit exceeded (workspace setting).

This review would cost an estimated $34.58, which exceeds your per-review limit of $15.00.

The top 3 files driving up this estimate:

File	Diff Size	Estimate
`packages/contracts/src/orchestrationV2.ts`	127.64KB	$5.06
`apps/server/src/mcp/OrchestratorMcpService.ts`	75.98KB	$3.01
`apps/server/src/orchestration-v2/EventSink.ts`	34.00KB	$1.35

Tip

To get this pull request reviewed, you can:

Comment @macroscope-app on this PR to request a manual review (monthly spend limits still apply).
Exclude the file(s) above from review by adding a pattern to your .macroscope/ignore.md — note that creating this file replaces Macroscope's built-in default ignores rather than extending them.
Raise your cost limit in your workspace billing settings.

^{Turn off this reminder going forward}

CchristophercallejongarciaAug 29, 2026

A key use case for provider switching is quota exhaustion. When a user reaches the usage limit of the active model or provider during a long-running session, T3 Code should allow switching to another authenticated provider without losing the thread context, current worktree, pending plan, or file-change state.

Ideally, the limit error should offer a direct Continue with another provider action. If full context replay is unavailable, T3 Code could generate and transfer a structured handoff automatically.

Stash drawer with four entries

Fixed in 84ef39dce4d. Stash is a separate tab again. The queue and activity share a column that stops 4px before Stash. Opening Stash shows its own drawer and preserves the queue's expanded or collapsed state when closed. Waiting threads keep the Stash tab too.

Before	After
Stash merged into the queue card	Separate Stash tab beside the queue and activity column

Stash open:

Checked the queue, waiting state, expanded tasks, and Stash at desktop and narrow widths. Targeted lint, web typecheck, and all six existing queue/stash tests passed. No new tests or stylesheets.

Before: editing lowers the count and adds a separate row with duplicate cancel controls

Queued-message editing is fixed in a794ced3160. These screenshots replace the earlier edit-row examples.

Editing highlights the original queue row instead of hiding it or adding a row. The count and header remain visible, including with one queued message.
Row numbers are removed. There is one Cancel action, and the save button uses a checkmark.
Edit, Steer, and Remove use the app tooltip component instead of native title tooltips.
The narrow-screen toolbar keeps the same height when switching between Stop and Save.

Verified in the shared preview: typing and Cancel leave the saved text unchanged; Save updates the message in place. Queue and composer dimensions stay unchanged when entering and leaving edit mode at desktop and narrow widths. Targeted lint, web typecheck, and 21 existing focused tests pass. No new tests were added.

This covers the shared web/desktop composer and narrow web layouts. Provider adapters, wire contracts, connection modes, and the native mobile client are unchanged. The composer documentation now describes saving with the checkmark.

Before:

After:

After: original queued message stays visible while its draft is edited, with one Cancel and a checkmark to save

Short edit and cancel demo

Single-message, narrow-screen, and tooltip examples

Editing the only queued message retains its header and count

Single-message editing at a narrow viewport

App-styled edit tooltip

Disabled idle Submit button still shows its tooltip

Updated the composer actions in a41ba601149.

A running turn shows Interrupt only when the composer is empty.
With text or attachments, the button shows a steer arrow. Click or Enter steers.
Holding Cmd/Ctrl changes it to a queue icon. Modifier-click and Cmd/Ctrl+Enter queue.
Editing a queued message keeps the checkmark, even while holding a modifier.
Each state uses an app-styled tooltip, including disabled Submit and busy states.

Verified click and keyboard dispatch, edit save/cancel, modifier release, and the narrow web layout in the mock preview. Targeted lint, the web typecheck, and 30 existing tests passed. No new tests or CSS files.

Steer	Queue with Cmd/Ctrl held
Steer button and keyboard shortcut tooltip	Queue icon while holding the modifier

Interrupt, queued update, idle, and narrow layout

Empty running composer	Updating a queued message
Interrupt tooltip on the empty running composer	Checkmark with Update queued message tooltip

Narrow web composer shows only Steer when it has content

Watch two Mod press/release cycles.

Fixed the tooltip wrapping on Mod in 330c44fd4d7.

Changing the order of the shortcut text triggered the tooltip's content resize. The text now stays Enter to steer, Mod+Enter to queue while the icon and action still switch to Queue.

Verified repeated Mod press/release cycles at desktop and narrow widths. The tooltip stayed 26px high throughout, compared with 42px when the old text wrapped. Targeted lint and the web typecheck passed.

Before fix, Mod held	After fix, Mod held
Wrapped tooltip	Stable single-line tooltip

AastarktcAug 30, 2026

Bug: delegated-task completion wakes are silently starved after two deliveries per cohort — parent waits forever on the third wave

TL;DR. delegate_task (async) promises the parent a wake on every child terminal, and its tool description tells the agent to "end the turn instead of polling." But finalizeDelegatedCompletionDelivery refuses to reserve a follow-up delivery once a cohort has settled two deliveries (settledDeliveryCount < 2). Any child whose completion lands in a third wave stays completionDelivery: "pending" forever: no wake, no queued message, nothing on restart. The orchestrating agent — following the tool contract — sits idle waiting for a completion that already happened. With 4–5+ parallel children whose completions straddle wake turns, this is close to guaranteed.

Observed on the #7211 stack (V2 snapshot of 2026-08-28); everything cited below is apps/server/src/orchestration-v2/ code from this PR's branch (none of it exists on main), and none of it is provider-specific.

The two contracts that collide

OrchestratorMcpService.ts, delegate-task creation:

1	// Async delegations wake the parent on every child terminal; wait
2	// delegations deliver through the blocking tool call, so a wake is
3	// only needed if the parent settled first (timeout, disconnect).
4	completionWake: input.mode === "wait" ? "settled_only" : "always",

Orchestrator.ts, planDelegatedCompletionDelivery:

1	const settledDeliveryCount = cohort?.settledDeliveryCount ?? 0;
2	if (settledDeliveryCount >= 2) {
3	// A cohort permits one initial delivery and one successor. Keep the
4	// result pending and inspectable instead of recursively re-arming the
5	// parent for every child that finishes after that bounded handoff.
6	...state: "pending"..., offer: false

…and the same settledDeliveryCount < 2 guard in finalizeDelegatedCompletionDelivery's canReserveFollowUp, which is the sweep that runs when a wake turn settles. settledDeliveryCount never resets, so this isn't a throttle — it's lifetime starvation per cohort. Nothing ever revisits a "pending" task after the cap engages: handleTerminalRun only sweeps delivery runs, the startup recovery path re-runs the same capped sweep, and finalizeAppOwnedSubagent early-returns on existingResultTransfer. The tool description ("An async child's completion wakes this thread… end the turn instead of polling") makes the failure mode maximally quiet: the agent does exactly what it was told and hangs.

Forensic timeline (real thread, from `orchestration_v2_projection_*`)

Parent thread delegated 5 async children in one turn (run 7), so all 5 share run 7's cohort. Timestamps from the projections DB:

time (UTC)	event	cohort state
01:01:52	run 7 delegates t10, t11, t13, t15, t17	—
01:05:49	t10 completes → delivery 1, wake run 8 starts	`settled=0`, delivery gen 1
01:05:56–01:07:01	t17, t13, t15 complete during run 8 → `"pending"`
01:07:11	run 8 settles → sweep reserves delivery 2 = {t15, t17, t13} → wake run 9 starts 01:07:12	`settled=1`, delivery gen 2
01:07:20	t11 completes, 8s into run 9 → `"pending"`
01:09:12	run 9 settles → sweep: `settledDeliveryCount` → 2, `pendingTaskIds = [t11]`, but `canReserveFollowUp` requires `< 2` → refused	`settled=2`, delivery `null`, t11 pending
01:09–01:29	parent idle; t11's result queryable but no wake exists or ever will
01:29:48	human notices the stall and nudges; that run reads `task_status` → t11 finally `acknowledged`

Final cohort row on run 7: settledDeliveryCount=2, nextGeneration=3, disposition=open, delivery=null — with a terminal, never-delivered task. Children delegated during later wake turns (t12 on run 8, t14 on run 9, t16 on run 10…) each started fresh cohorts with reset counters, so their wakes worked — which is exactly why moderate parallelism masks the bug until one dispatch wave is large enough to stagger across ≥3 waves.

(The queued-delivery merging worked beautifully, for the record — t13/t15/t17 amending the queued wake into one bundled message is the right behavior. The problem is only the lifetime cap.)

Suggested fixes (any of these)

Make the cap a concurrency bound, not a lifetime bound. The recursion the comment fears is "one delivery in flight + one reserved successor" — that's already bounded without a lifetime counter. Reset settledDeliveryCount (or simply drop it) whenever a sweep finds pendingTaskIds.length > 0 after the delivery drains; the wake-per-terminal contract then holds and the parent still never sees more than one queued wake at a time.
Sweep pendings on any parent-run settle. Extend handleTerminalRun to arm a delivery for terminal app-owned tasks left "pending"/unarmed whenever the parent thread goes non-live. The existing generation + alreadyDispatched dedupe in ProviderContinuationService.currentDelegatedCompletionDelivery already guards against the duplicate-wake concern.
At minimum, don't be silent. If the cap stays, surface refused deliveries (system line in the thread: "N delegated results pending — read with task_status") and soften the tool description's "end the turn instead of polling," which is currently a trap.

Related smaller hole in the same family: for mode="wait" children (settled_only), planDelegatedCompletionDelivery returns with the task untouched (not even "pending") when the parent has a live run — that branch precedes the pending-marking branches — and the post-timeout wake-policy upgrade knowingly skips re-planning when the parent settled in between ("a missed wake is cheaper than a duplicate one"). Same silent-starvation class, just harder to hit because the blocking wait usually owns delivery.

Happy to provide the raw projection rows or test this against a patched build — we run this stack daily.

EElliotDrelAug 30, 2026

@juliusmarminge one addition I'd really like here: scheduled messages.

I think this should be separate from normal queued messages. Queued messages mean “send after the current run finishes”; a scheduled message means “send this message at a specific date/time.”

The important architectural requirement is that the schedule must be persisted and executed by the T3 server/environment that actually runs the agent, not by whichever web/mobile/desktop client created it. If I schedule a message from my laptop or phone against an agent running on another machine, I should be able to close/disconnect that client and the message should still fire at the selected time.

In other words, the client should only create/edit/cancel the schedule. The host environment owns the durable record and dispatch timer.

A simple UX could be a “Send later” action from the composer that creates a one-shot scheduled message for the current thread. It should be visible/cancellable/editable before it fires.

I noticed this PR already adds server-persisted scheduled tasks, but those schedules appear recurring (interval / fixed_time). This feels like the same server-side scheduling infrastructure could support a one-shot send_at/timestamp schedule without conflating it with the existing queued-message semantics.

@juliusmarminge one addition I'd really like here: scheduled messages.
I think this should be separate from normal queued messages. Queued messages mean “send after the current run finishes”; a scheduled message means “send this message at a specific date/time.”
The important architectural requirement is that the schedule must be persisted and executed by the T3 server/environment that actually runs the agent, not by whichever web/mobile/desktop client created it. If I schedule a message from my laptop or phone against an agent running on another machine, I should be able to close/disconnect that client and the message should still fire at the selected time.
In other words, the client should only create/edit/cancel the schedule. The host environment owns the durable record and dispatch timer.
A simple UX could be a “Send later” action from the composer that creates a one-shot scheduled message for the current thread. It should be visible/cancellable/editable before it fires.
I noticed this PR already adds server-persisted scheduled tasks, but those schedules appear recurring (interval / fixed_time). This feels like the same server-side scheduling infrastructure could support a one-shot send_at/timestamp schedule without conflating it with the existing queued-message semantics.

Narrow T3 activity summaries

Implemented T3 action summaries in c01ff20757e. The same groups now describe orchestration work instead of reducing it to “used tools”:

Ran 2 commands and sent messages to 3 threads
Sent 5 messages to 2 threads, with repeated message IDs deduplicated
Created 4 threads and delegated 2 tasks, including batch creation

Mixed groups show two specific categories and a count of the remaining actions. Commands, file changes, and orchestration changes take priority over reads and status checks. Failed calls do not inflate successful sends or creations; failure counts appear first so narrow layouts cannot truncate them away. A reported child failure is separate from failure of the status-check tool itself.

Grouping boundaries, expansion, individual call details, and icons are unchanged. Native mobile layout is unchanged. Preview/worktree tools and unknown tools retain the generic fallback.

Verified 53 focused tests across the timeline, T3 counting, and tool presentation, plus scoped lint and web/shared/client-runtime typechecks. Checked the scripted preview at desktop and narrow widths, including expanding back to the original calls and their input/output.

Before	After
Generic tool summaries	T3 action summaries

Narrow layout — failure count stays visible