Sign up
View stargazersSign up

fix(server): honor requested same-model subagent counts#25153

closed2
main
fix/issue-24967
opened3d agobyJJulius Marminge
1 files changed
View on GitHubTwo-way sync enabledPR last imported 2d ago

Problem

A Sol xhigh parent can interpret the injected “One child per named model” sentence as a hard cap. In the real web client, asking for two Sol children in separate worktrees substituted a different model for the second child; explicitly requiring both to use codex/gpt-6.1-sol then produced a refusal citing the cap.

Change

Replace that sentence with an instruction to honor the requested number of children, including several using one model. All other model-selection, settings, and worktree instructions stay intact.

Scope and approval

Fixes pingdotgg/t3code#24967

Triaged bug via the automated issue flow. This focused correction restores explicitly requested same-model delegation through the existing capability.

Verification

  • Reproduced on main 264969317c using the actual web dev server on Linux, Codex CLI 0.163.0, and a Sol xhigh parent. Before runtime transcript.
  • Verified fix 7e92300c26: two same-model children completed; two same-model children completed in separate worktrees; the explicit exact-model correction also ended with two codex/gpt-6.1-sol xhigh children replying OK. The recorded parent protocol contains the replacement instruction. After runtime transcript, exact prompts, results, and workspace identities.
  • The run included an HTTP 503 response despite child completion and a later worktree-preparation timeout. The exact-model correction required a same-model retry for its second successful child; both failures remain in the transcript.
  • vp run check:changed passed format, lint, both knip checks, and typechecks for 21 affected packages. Check output.
  • After merging current main, vp test run apps/server/src/provider/CodexDeveloperInstructions.test.ts apps/server/src/orchestration-v2/Adapters/AcpAdapterV2.test.ts packages/provider-core/src/server/runtimeInstructions.test.ts: 72 passed. Test output. The changed Codex instruction entry remains below its 4,000-byte limit.
  • No model-in-the-loop eval harness exists here; a new substring assertion would check wording rather than the reported refusal. Runtime evidence is the proof. Independent proof audit found the narrow fix sufficient.
  • No render paths, contracts, persisted data, or scheduler limits changed. The shared server instruction reaches every client; other provider/model combinations and native Windows execution were not exercised. Runtime instruction compliance remains model-dependent.

Supplementary real-client captures (1280×800, light theme):

Before: cap-based refusalAfter: exact-model worktree results, with retry disclosed
Before: Sol refuses two children on the same modelAfter: two exact-model children return OK in separate worktrees

Implemented with Claude Opus 5.5 in the Claude harness; orchestrated and verified with GPT-6.1 Sol in the Codex harness.

Checks

22 successful · 12 skipped · 1 neutral
All checks passed
863c93b

Added 2 commits

  • fix(server): orchestration instructions honor requested same-model child counts7e92300
  • Merge remote-tracking branch 'origin/main' into fix/issue-24967863c93b
MmacroscopeappBot3d ago
Synced from GitHub

Approvability

Verdict: Needs Human Approval

Macroscope's review found this PR requires human approval — This updates shared orchestration instructions used by multiple provider integrations, changing default behavior so repeated same-model subagents can be launched. The fix is narrowly scoped, but because it changes a product default, human review is warranted.

You can add or adjust custom eligibility rules. Learn more.

Ggithub-actionsBot3d ago
Synced from GitHub

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ℹ️ The exact PR base did not have a successful artifact. Baseline uses the latest successful main measurement shown below.

ProviderMetricMain baselineThis PRImpactPR ceiling
CodexTotal thread wire5.0 KiB5.0 KiB0 B (0.0%)6.8 KiB✅
CodexThread snapshot wire3.8 KiB3.8 KiB0 B (0.0%)4.9 KiB✅
CodexLive turn WebSocket wire1.2 KiB1.2 KiB0 B (0.0%)2.0 KiB✅
CodexLive turn WebSocket decoded20.9 KiB20.9 KiB0 B (0.0%)29.3 KiB✅
CodexLive turn messages220 (0.0%)8✅
ClaudeTotal thread wire5.0 KiB5.0 KiB+7 B (+0.1%)6.8 KiB✅
ClaudeThread snapshot wire3.8 KiB3.8 KiB0 B (0.0%)4.9 KiB✅
ClaudeLive turn WebSocket wire1.2 KiB1.2 KiB+7 B (+0.6%)2.0 KiB✅
ClaudeLive turn WebSocket decoded21.2 KiB21.2 KiB0 B (0.0%)29.3 KiB✅
ClaudeLive turn messages220 (0.0%)8✅

Baseline: 1a13127 · PR result: 863c93b · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

CcoderabbitaiBot3d ago
Synced from GitHub

Review in Change Stack →

No actionable comments were generated in the recent review.

External image Recent review info
External image Run configuration
External image Commits

Reviewing files that changed from the base of the PR and between 44df3360165ba96a6902496812feca2b60c7fcc9 and 863c93bf5660a3bcabca4ad448eec8db4784adb8.

External image Files selected for processing: 1
  • packages/provider-core/src/server/orchestrationInstructions.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.

Walkthrough

The orchestration instructions now allow the requested number of delegate children, including multiple children that use the same model. The previous one-child-per-named-model restriction was removed.

Changes: Delegate count guidance

Layer / File(s)Summary
Update delegate count instruction
packages/provider-core/src/server/orchestrationInstructions.ts		The instructions now allow multiple delegate children for the same named model when the user requests them.

Priority: External image Normal

Estimated code review effort: 1 (Trivial) | ~5 minutes

Change: Bug fix · Severity of issue fixed: Medium

Possibly related PRs

Suggested labels: bugfix, size:XS

Suggested reviewers: t3dotgg

Merge Risk: External image Minimal · up to 863c9

No actionable merge-blocking risk was identified.

Pre-merge checks: Passed 3 · Failed 1

Failed Failed checks: 1 warning

Check nameStatusExplanationResolution
Linked Issues checkWarningThe instruction change satisfies the main behavior in [#24967]. It now directs the parent to start the requested number of children, including several children with one model. However, [#24967] also r…Add an automated orchestration eval for two distinct successful launches with the same provider and model. Require the requested model for both children. Reject one child, parent-only work, or model substitution.
Passed Passed checks: 3 passed
Check nameStatusExplanation
Title checkPassedThe title clearly and concisely describes the main change: honoring requested counts for same-model subagents.
Description checkPassedThe description covers the problem, change, scope and approval, verification results, limitations, screenshots, and agent disclosure. It provides specific runtime and test evidence.
Out of Scope Changes checkPassedThe PR changes one sentence in packages/provider-core/src/server/orchestrationInstructions.ts. The change directly implements [#24967]. No unrelated source, contract, persistence, scheduler, or rend…
Full details: Out of Scope Changes check

Explanation

The PR changes one sentence in packages/provider-core/src/server/orchestrationInstructions.ts. The change directly implements [#24967]. No unrelated source, contract, persistence, scheduler, or render changes are present in the reviewed diff.

Full details: Linked Issues check

Explanation

The instruction change satisfies the main behavior in [#24967]. It now directs the parent to start the requested number of children, including several children with one model. However, [#24967] also requests an orchestration eval with two distinct successful same-model launches and rejection of parent-only work or model substitution. The PR changes only packages/provider-core/src/server/orchestrationInstructions.ts and adds no such automated eval. Runtime transcripts do not replace the requested eval.

Resolution

Add an automated orchestration eval for two distinct successful launches with the same provider and model. Require the requested model for both children. Reject one child, parent-only work, or model substitution.

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

Ttheo2d ago

Closing as a duplicate. #25359 already removed "One child per named model" on main. Eval cases for this regression are in #24635.