Sign up
View stargazersSign up

feat(cua): attach the managed Cua Driver to every provider session#16528

draft1
t3code/cua/1-host
t3code/cua/2-providers
opened23d agobyJJulius Marminge
15 files changed
View on GitHubTwo-way sync enabledPR last imported 55m ago

With the driver running (previous layer), every provider now gets it as a stdio MCP server named cua-driver:

ProviderHow
ClaudeSDK mcpServers; the query reopens when the set changes
Codexthread mcp_servers, unless the user's merged Codex config already defines cua-driver (read once per directory with config/read); theirs wins
CursorSDK mcpServers
Grok, Antigravity, Devin, ACP registryACP mcpServers, in both stdio and MCP-over-ACP lists
OpenCode 1.xmcp.add; the prompt mentions it only if the add succeeded
OpenCode 2per-thread, per-directory registration with permission rules, like its t3-code server
PiPi has no MCP client, so T3's Pi extension bridges the stdio server and passes screenshots through as images

Sessions with the driver get a short <computer_use> block: keep input in the background, escalate to foreground only when a background action did not land, and leave the desktop as found.

#15006 skipped Codex, OpenCode 2, and Pi. #10899 only attached Codex. This covers all of them.

#15006 also ran a per-thread proxy so T3 could end Cua's session at every turn end. This stack drops it: Stop still ends the provider's turn, and Cua's own idle timeout hides the agent cursor.

Verification

  • Focused tests: Claude and Codex attachment, user-config precedence in Codex, the <computer_use> block, and the existing Claude, Codex, Cursor, ACP, Grok, OpenCode, OpenCode 2, Pi, and session-manager suites (687 tests).
  • Checked by hand: Pi's stdio bridge against the real 0.34 driver lists 58 tools, calls list_apps, and reconnects after the driver exits.

Built with Claude Opus 5.5 in T3 Code, through the Claude Code harness.

🤖 Generated with Claude Code

Checks

20 successful · 8 skipped
All checks passed
65dea46
Ggithub-actionsBot23d ago
Synced from GitHub

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ℹ️ No successful main baseline artifact is available yet. This run establishes the initial measurement.

ProviderMetricMain baselineThis PRImpactPR ceiling
CodexTotal thread wire—5.0 KiB—6.8 KiB✅
CodexThread snapshot wire—3.8 KiB—4.9 KiB✅
CodexLive turn WebSocket wire—1.2 KiB—2.0 KiB✅
CodexLive turn WebSocket decoded—21.6 KiB—24.4 KiB✅
CodexLive turn messages—2—8✅
ClaudeTotal thread wire—5.0 KiB—6.8 KiB✅
ClaudeThread snapshot wire—3.8 KiB—4.9 KiB✅
ClaudeLive turn WebSocket wire—1.3 KiB—2.0 KiB✅
ClaudeLive turn WebSocket decoded—21.9 KiB—24.4 KiB✅
ClaudeLive turn messages—2—8✅

Baseline: unavailable · PR result: 65dea46 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

JJulius Marminge11d ago

Batch other-providers-19: this draft is blocked on its host layer. I compared its full diff against current main (9077e818d8). Main has neither apps/server/src/cua/CuaDriver.ts / CuaDriver.acquire nor the CuaDriverMcpConfiguration contract consumed here. Its actual base remains t3code/cua/1-host.

Porting provider attachment alone would leave an unresolved service import and contract, with no managed executable/socket to attach or verify. Rebuilding the missing host lifecycle, setting, packaging and platform support would take over work outside this batch. I am leaving the maintainer draft and its existing stack intact; no code was pushed and no merge was performed.

Resume after the host layer is available on main, then port the injection paths to the provider packages in that revision and verify real sessions for each supported provider. This Linux environment cannot validate the macOS/Windows host or Electron/iOS/Android surfaces. Existing test counts and manual Pi proof in the body were not rerun for this batch.

Ssaphid7d ago
Synced from GitHub

Source review of 7418a60: OpenCode 2 appears to leave a cross-thread Cua permission gap. In mcpRules, the cua-driver-* deny is added only when the current thread has a Cua descriptor. A full-access session without that descriptor therefore retains wildcard allow, while another thread can register its Cua server in the same directory (lines 3008–3049).

Please make the Cua namespace deny unconditional, then allow only the attached thread’s server. A focused regression case would keep B’s harmless fake Cua server registered while A has no attachment, then check A and its subagents cannot call B’s tools; also cover disabling an attachment.

This is a source-derived finding, not a reproduced runtime failure. The existing host-layer dependency still applies; no contributor code was executed and this is not a validation pass.

Added 1 commit

  • feat(cua): attach the managed Cua Driver to every provider session65dea46