Sign up
View stargazers13Sign up

test(server): run fake provider CLIs through a Node stub on every host#9567

merged1
windows-tests/symlinks
windows-tests/fake-cli
openedSep 4, 2026byJJulius Marminge
12 files changed
View on GitHubTwo-way sync enabledPR last imported 3d ago

Seven suites faked a provider CLI with a #!/bin/sh script made executable by chmod, which Windows cannot run: the shebang is ignored and there is no extension for PATHEXT to match, so the spawn either failed or fell through to a real CLI on PATH. CodexTextGeneration's fake was a 120-line sh argument parser.

Add apps/server/src/testUtils/fakeCli.ts. writeFakeCli writes the CLI's behaviour as a Node stub and a launcher shaped for the host: a shebang script on posix, a .cmd shim on Windows, which resolveSpawnCommand already routes through a shell. execScriptSource covers the common case of handing over to the mock ACP agent after an argv check. ClaudeTextGeneration, which had its own .cmd branch, moves onto the shared helper too.

The tests that assert the agent logged SIGTERM on stop are skipped on Windows, where the child is terminated rather than signalled.

Part of the Windows test-suite stack rooted at #9564; the manual Windows lane comes from #9538.

Model: Claude Fable 5 in Claude Code.

🤖 Generated with Claude Code

Note

Run fake provider CLI fixtures through a shared Node stub on Windows and POSIX hosts

  • Adds a shared writeFakeCli utility in fakeCli.ts that emits a Node stub module plus a platform-appropriate launcher (POSIX shell or Windows .cmd), replacing per-test shell-script generation across provider and text-generation test suites
  • Refactors CursorAdapter, CursorProvider, GrokAdapter, GrokProvider, CodexTextGeneration, CursorTextGeneration, GrokTextGeneration, and ClaudeTextGeneration tests to build their fake agent binaries through the shared writer
  • Adds a Windows .cmd wrapper for the Codex collaboration mock peer and selects it on win32 in the integration test; adds .cmd/.bat CRLF checkout rules in .gitattributes
  • Guards SIGTERM-dependent child-process exit assertions with a win32 skip condition in CursorAdapter, CursorProvider, GrokAdapter, CursorTextGeneration, and CodexCollabRuntime tests, since Windows terminates processes without delivering SIGTERM
  • Risk: the new writeFakeCli writer is the single path for fake CLI creation; any mismatch between the generated Node stub contract and an individual test's expectations (e.g. argument validation in execScriptSource) will surface as fixture failures across all refactored suites

Macroscope summarized b4e082c.

Note

Low Risk Changes are limited to test fixtures, git attributes, and conditional test skips; no production provider or spawn logic is modified.

Overview Provider and text-generation tests no longer build fake CLIs with hand-written #!/bin/sh wrappers (and chmod), which fail or hit the real binary on Windows. They now go through writeFakeCli in apps/server/src/testUtils/fakeCli.ts, which writes a Node stub plus a host-specific launcher (POSIX shell script or .cmd shim), with env applied via a JSON sidecar. execScriptSource covers the common “validate argv, then import the mock ACP agent” pattern; Codex’s large shell-based fake is reimplemented in Node the same way.

Codex collab integration picks codexCollabMockPeer.cmd on win32 (new file); .gitattributes forces CRLF for *.cmd / *.bat.

Tests that assert SIGTERM in an exit log are skipIf on Windows, where children are killed without that signal. Production spawn paths are unchanged—this is test/fixture plumbing for the Windows test suite.

Reviewed by Cursor Bugbot for commit b4e082c2d33a1197543f2c9425f1e0b82ddacbbe. Bugbot is set up for automated code reviews on this repo. Configure here.

apps/server/src/testUtils/fakeCli.tsline 99Outdated · 339ef03Open thread
……@@ -0,0 +1,106 @@
96+ " process.exit(11);",
97+ "}",
98+ `const forwarded = ${options.forwardArgs === false ? "[]" : "args"};`,
99+ `const result = spawnSync(process.execPath, [${JSON.stringify(options.scriptPath)}, ...forwarded], {`,
MmacroscopeappBoton line 99Sep 4, 2026

🟡 Medium testUtils/fakeCli.ts:99

When the ACP session terminates the stub with SIGTERM, the scriptPath process remains running, so tests waiting for the mock agent's SIGTERM log time out and leave a child process behind. Because spawnSync only waits for the child and does not forward termination signals to it, handle SIGTERM/SIGINT and relay them to the spawned script, or use an exec-style handoff for long-lived scripts.

No longer relevant as of e439d44d643ad25f9d69b65812ece4e9807aaaa6

JJulius MarmingeSep 4, 2026

Superseded: the stub now imports the mock in-process rather than spawning it, so there is no intermediate process to relay signals through.

Ggithub-actionsBotSep 4, 2026
Synced from GitHub

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ℹ️ No successful main baseline artifact is available yet. This run establishes the initial measurement.

ProviderMetricMain baselineThis PRImpactPR ceiling
CodexTotal thread wire—13.8 KiB—15.1 KiB✅
CodexThread snapshot wire—7.0 KiB—7.3 KiB✅
CodexLive turn WebSocket wire—6.7 KiB—7.8 KiB✅
CodexLive turn WebSocket decoded—58.5 KiB—66.4 KiB✅
CodexLive turn messages—10—21✅
ClaudeTotal thread wire—13.8 KiB—15.1 KiB✅
ClaudeThread snapshot wire—7.0 KiB—7.3 KiB✅
ClaudeLive turn WebSocket wire—6.7 KiB—7.8 KiB✅
ClaudeLive turn WebSocket decoded—59.3 KiB—66.4 KiB✅
ClaudeLive turn messages—10—21✅

Baseline: unavailable · PR result: b4e082c · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 113.8 KiB
  • Claude decoded thread snapshot: 114.5 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

apps/server/src/testUtils/fakeCli.tsline 7Outdated · 339ef03Open thread
……@@ -0,0 +1,106 @@
4+
5+import { HostProcessPlatform } from "@t3tools/shared/hostProcess";
6+
7+const windowsHost = HostProcessPlatform.defaultValue() === "win32";
MmacroscopeappBoton line 7Sep 4, 2026

This freezes a service-derived value into a module global at import time: HostProcessPlatform is a Context.Reference, so reading defaultValue() here bypasses the environment and any test that provides an override (e.g. the yield* isHostWindows read this helper replaces in ClaudeTextGeneration.test.ts) no longer affects which launcher shape gets written.

Since the helper is intentionally synchronous, consider resolving the platform per call instead of once per module — e.g. an optional platform on FakeCliOptions that defaults to HostProcessPlatform.defaultValue() inside writeFakeCli, letting Effect callers pass the platform they acquired from the environment.

suggestion
1

Posted via Macroscope — Effect Service Conventions

JJulius MarmingeSep 4, 2026

Agreed. writeFakeCli now takes an optional platform and only falls back to HostProcessPlatform.defaultValue() when a plain test omits it; the Effect callers pass what they read from the environment.

apps/server/src/provider/testFixtures/codexCollabMockPeer.cmdlines 5–5Outdated · e439d44Open thread
……@@ -0,0 +1,7 @@
2+rem Wrapper so CodexSessionRuntime can spawn the mock peer on Windows: the
3+rem runtime always passes "app-server" as the first argument; drop it and
4+rem run the .mjs peer with node.
5+shift
MmacroscopeappBoton lines 5–5Sep 4, 2026

🟠 High testFixtures/codexCollabMockPeer.cmd:5

Every Windows CodexCollab spawn fails before the mock starts because %~dp0 resolves from the shifted app-server argument, so Node looks for codexCollabMockPeer.mjs in the working directory (such as /tmp) instead of the fixture directory. shift shifts %1 into %0; use shift /1 so the batch-file path remains available.

suggestion
1shift /1
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
text
1In file @apps/server/src/provider/testFixtures/codexCollabMockPeer.cmd around line 5:
2
3Every Windows CodexCollab spawn fails before the mock starts because `%~dp0` resolves from the shifted `app-server` argument, so Node looks for `codexCollabMockPeer.mjs` in the working directory (such as `/tmp`) instead of the fixture directory. `shift` shifts `%1` into `%0`; use `shift /1` so the batch-file path remains available.
MmacroscopeappBotSep 4, 2026
Synced from GitHub

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — The changes are confined to cross-platform test fixtures and test utilities, with no product-runtime or default-setting impact. Human review is required because the PR adds file-level and line-level directives that suppress static-analysis diagnostics.

No code changes detected at b4e082c. Prior analysis still applies.

You can add or adjust custom eligibility rules. Learn more.

Added 1 commit

  • test(server): run fake provider CLIs through a Node stub on every hostb4e082c