refactor(server): share attachment message intake#10580

merged3
agents/mcp-shared/project-inputs
agents/mcp-shared/attachment-intake
openedSep 7, 2026byJJulius Marminge
View on GitHubTwo-way sync enabledPR last imported 3d ago

Part 3/16 of the shared-core and MCP stack. Based on #10578. Next: #10554.

RPC and MCP separately sequenced pending attachment claims and message execution. This extracts ThreadMessageIntake operations around the existing claim, dispatch/send and ThreadLaunch services, then makes RPC use them. The MCP consumers follow in their own layers.

A launch or dispatch error can follow a durable message commit. Intake releases copies proven unused by the returned durable message/events, including accepted replays, and by typed pre-acceptance failures. It retains copies when acceptance is uncertain; a typed claim failure still cleans its partial batch before dispatch. This removes the RPC blanket error cleanup that could delete accepted message files. It may retain unused copies on uncertain failures. There is no new receipt store, retry journal, lock, provider logic, or service identity. Startup queuing, provenance, analytics and wire mapping stay at the RPC boundary.

Validation: AttachmentClaims, ThreadLaunch and ThreadManagement suites: 38 tests passed; server typecheck passed. One shared real V2 test accepts a launch, loses its result, and proves persisted attachment bytes survive. It also proves launch/command/send replays leave exactly the three accepted files, and nonexistent-project/thread failures add no orphaned copies. Existing partial-claim cleanup coverage remains.

Layer size: 4 files, +379/-150. No MCP tools are introduced in this prerequisite.

Prepared with Codex in the OpenAI agent runtime.

Note

Share attachment intake logic across dispatchCommand, sendToThread, and launchThread handlers

  • Adds shared intake handlers in ThreadMessageIntake.ts that claim pending attachments before processing and release them based on outcome.
  • Introduces dispatchWasNotAccepted classifier to distinguish definite pre-acceptance failures (projection errors, provider adapter errors, rejected commands, ID conflicts) from uncertain failures.
  • releaseUnusedClaims removes only claimed paths absent from the accepted result, preserving referenced copies.
  • WebSocket handlers in ws.ts delegate to the shared intake logic instead of doing attachment claiming inline; launch RPC now uses tag-specific error conversion for AttachmentClaimError, ThreadLaunchError, and ServerRuntimeStartupError.
  • Adds a filesystem-backed integration test in ThreadLaunchService.test.ts covering uncertain launch replay, pre-acceptance failure cleanup, idempotent retries, and attachment byte preservation.
  • Risk: failures previously classified as definite (any observed error released all claims) are now retained when the dispatchWasNotAccepted classifier marks them uncertain; reviewers should verify that unmatched thread-management errors in the classifier do not cause attachment leaks for scenarios previously cleaned up.

Macroscope summarized 4cfe358.

Added 1 commit

  • refactor(server): share attachment message intake43e6ee2
apps/server/src/orchestration-v2/ThreadMessageIntake.tsline 42Outdated Ā· 43e6ee2Open thread
……@@ -0,0 +1,50 @@
39+ message: "Uploaded attachments need a thread id at launch.",
40+ });
41+ }
42+ const claimed = yield* AttachmentClaims.claimPendingAttachments({
MmacroscopeappBoton line 42Sep 7, 2026

🟠 High orchestration-v2/ThreadMessageIntake.ts:42

Retrying an already accepted launch after a lost response copies the pending attachment to a new random thread-scoped path, then launches.launch returns the existing receipt without referencing that copy. The unreferenced file is never released, so each retry permanently leaks an attachment; perform the idempotency check before claiming or release claimed.claimedPaths when the launch is already committed.

šŸš€ Reply "fix it for me" or copy this AI Prompt for your agent:
text
1In file @apps/server/src/orchestration-v2/ThreadMessageIntake.ts around line 42:
2
3Retrying an already accepted launch after a lost response copies the pending attachment to a new random thread-scoped path, then `launches.launch` returns the existing receipt without referencing that copy. The unreferenced file is never released, so each retry permanently leaks an attachment; perform the idempotency check before claiming or release `claimed.claimedPaths` when the launch is already committed.

Fixed in c80aae28752480aca3d0ec0d9a773dca4586e821. Shared intake now removes claims not referenced by the returned durable message/events, including accepted launch, command, and send replays. It also removes copies on typed receipt/planning rejections and launch project/receipt failures before message dispatch. The real V2 regression now proves replays leave only accepted files and missing-project/thread requests leave no orphaned copies. The same regression retains the lost-result-after-acceptance case and verifies the original bytes survive. Generic launch/dispatch errors remain ambiguous, so blanket tapError cleanup would reintroduce data loss; those copies are intentionally retained. No new receipt store, journal, lock, or read-after-failure query was added. Focused suites: 38 passed; server typecheck and targeted lint/format passed.

apps/server/src/orchestration-v2/ThreadMessageIntake.tslines 46–49Outdated Ā· 43e6ee2Open thread
……@@ -0,0 +1,50 @@
46+ return yield* launches.launch({
47+ ...input,
48+ initialMessage: { ...input.initialMessage, attachments: claimed.attachments },
49+ });
MmacroscopeappBoton lines 46–49Sep 7, 2026

🟠 High orchestration-v2/ThreadMessageIntake.ts:46

launchThread leaks the copied attachment whenever launches.launch fails before accepting the initial message. claimPendingAttachments creates a non-pending final path, so the pending sweep never reclaims it and repeated failed launches accumulate orphaned files; release claimed.claimedPaths on launch failure.

diff
1- return yield* launches.launch({
2+ return yield* launches.launch({
3 ...input,
4 initialMessage: { ...input.initialMessage, attachments: claimed.attachments },
5- });
6+ }).pipe(
7+ Effect.tapError(() =>
8+ AttachmentClaims.releaseClaimedAttachments(claimed.claimedPaths),
9+ ),
10+ );
Also found in 2 other location(s)

apps/server/src/ws.ts:1319

ThreadMessageIntake.dispatchCommand claims and copies pending attachments before dispatch, but the new path no longer releases claimedPaths if threads.dispatch fails before accepting the command. For example, dispatching a valid staged upload to a deleted/nonexistent thread copies it under that thread id, then ensureCommandTranscripts/the orchestrator can fail. sweepStalePendingAttachments only deletes ids in the pending segment, so this final-path copy is retained indefinitely despite no message referencing it; repeated failed submissions accumulate orphaned attachment files.

apps/server/src/ws.ts:1437

ThreadMessageIntake.launchThread copies pending attachments before launches.launch, but no longer removes the copy when launch fails before the initial message is committed. A launch using a staged attachment and a nonexistent project fails during ThreadLaunchService.launch's project lookup, leaving the copied final attachment unreferenced. These ids are not in the pending segment, so sweepStalePendingAttachments will never reclaim them; failed launch attempts leak attachment files.

šŸš€ Reply "fix it for me" or copy this AI Prompt for your agent:
text
1In file @apps/server/src/orchestration-v2/ThreadMessageIntake.ts around lines 46-49:
2
3`launchThread` leaks the copied attachment whenever `launches.launch` fails before accepting the initial message. `claimPendingAttachments` creates a non-pending final path, so the pending sweep never reclaims it and repeated failed launches accumulate orphaned files; release `claimed.claimedPaths` on launch failure.
4
5Also found in 2 other location(s):
6- apps/server/src/ws.ts:1319 -- `ThreadMessageIntake.dispatchCommand` claims and copies pending attachments before dispatch, but the new path no longer releases `claimedPaths` if `threads.dispatch` fails before accepting the command. For example, dispatching a valid staged upload to a deleted/nonexistent thread copies it under that thread id, then `ensureCommandTranscripts`/the orchestrator can fail. `sweepStalePendingAttachments` only deletes ids in the pending segment, so this final-path copy is retained indefinitely despite no message referencing it; repeated failed submissions accumulate orphaned attachment files.
7- apps/server/src/ws.ts:1437 -- `ThreadMessageIntake.launchThread` copies pending attachments before `launches.launch`, but no longer removes the copy when launch fails before the initial message is committed. A launch using a staged attachment and a nonexistent project fails during `ThreadLaunchService.launch`'s project lookup, leaving the copied final attachment unreferenced. These ids are not in the pending segment, so `sweepStalePendingAttachments` will never reclaim them; failed launch attempts leak attachment files.

Fixed in c80aae28752480aca3d0ec0d9a773dca4586e821. Shared intake now removes claims not referenced by the returned durable message/events, including accepted launch, command, and send replays. It also removes copies on typed receipt/planning rejections and launch project/receipt failures before message dispatch. The real V2 regression now proves replays leave only accepted files and missing-project/thread requests leave no orphaned copies. The same regression retains the lost-result-after-acceptance case and verifies the original bytes survive. Generic launch/dispatch errors remain ambiguous, so blanket tapError cleanup would reintroduce data loss; those copies are intentionally retained. No new receipt store, journal, lock, or read-after-failure query was added. Focused suites: 38 passed; server typecheck and targeted lint/format passed.

apps/server/src/orchestration-v2/ThreadMessageIntake.tsline 19Outdated Ā· 43e6ee2Open thread
……@@ -0,0 +1,50 @@
16+ threadId: command.threadId,
17+ attachments: command.attachments,
18+ });
19+ return yield* threads.dispatch({ ...command, attachments: claimed.attachments });
MmacroscopeappBoton line 19Sep 7, 2026

🟠 High orchestration-v2/ThreadMessageIntake.ts:19

Failed dispatchCommand and launchThread calls permanently retain the thread-scoped copies created by claimPendingAttachments, so repeated requests for nonexistent threads or projects leave unreferenced upload payloads on disk and can exhaust server storage. These failures occur before a message or thread is accepted, but no cleanup follows threads.dispatch(...) or launches.launch(...), and the pending-file sweep does not remove the copies. Release the claimed attachments whenever dispatch or launch fails before acceptance.

šŸš€ Reply "fix it for me" or copy this AI Prompt for your agent:
text
1In file @apps/server/src/orchestration-v2/ThreadMessageIntake.ts around line 19:
2
3Failed `dispatchCommand` and `launchThread` calls permanently retain the thread-scoped copies created by `claimPendingAttachments`, so repeated requests for nonexistent threads or projects leave unreferenced upload payloads on disk and can exhaust server storage. These failures occur before a message or thread is accepted, but no cleanup follows `threads.dispatch(...)` or `launches.launch(...)`, and the pending-file sweep does not remove the copies. Release the claimed attachments whenever dispatch or launch fails before acceptance.

Fixed in c80aae28752480aca3d0ec0d9a773dca4586e821. Shared intake now removes claims not referenced by the returned durable message/events, including accepted launch, command, and send replays. It also removes copies on typed receipt/planning rejections and launch project/receipt failures before message dispatch. The real V2 regression now proves replays leave only accepted files and missing-project/thread requests leave no orphaned copies. The same regression retains the lost-result-after-acceptance case and verifies the original bytes survive. Generic launch/dispatch errors remain ambiguous, so blanket tapError cleanup would reintroduce data loss; those copies are intentionally retained. No new receipt store, journal, lock, or read-after-failure query was added. Focused suites: 38 passed; server typecheck and targeted lint/format passed.

Synced from GitHub

Thread transfer impact

āš ļø The latest CI run did not produce a thread transfer result for 4cfe358.

This comment will update automatically after the next completed run.

Synced from GitHub

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — This PR changes production WebSocket dispatch and launch behavior by centralizing attachment intake, altering cleanup after uncertain acceptance, and preserving attachments for replay. The added tests cover the primary durability and retry scenarios, but the new error-sensitive runtime logic merits human review.

You can add or adjust custom eligibility rules. Learn more.

Added 1 commit

  • fix(server): release proven unused attachment claimsc80aae2
apps/server/src/ws.tslines 1484–1495Outdated Ā· c80aae2Open thread
……@@ -1454,104 +1432,68 @@ const makeWsRpcLayer = (
1492+ : "Failed to launch thread",
1493+ cause,
1494+ }),
1495+ ),
MmacroscopeappBoton lines 1484–1495Sep 7, 2026

This catch-all mapError now inspects cause._tag and derives the wrapper's message from cause.message. Per the error conventions, a wrapper's message should come from its own attributes, and statically known tagged failures should be handled with Effect.catchTags where their context is known, instead of tag-sniffing a whole multi-step pipeline's error channel.

suggestion
1 Effect.catchTags({
2 AttachmentClaimError: (error) =>
3 new OrchestrationV2ThreadLaunchError({
4 commandId: input.commandId,
5 projectId: input.projectId,
6 message: error.message,
7 cause: error,
8 }),
9 }),
10 Effect.mapError(
11 (cause) =>
12 new OrchestrationV2ThreadLaunchError({
13 commandId: input.commandId,
14 projectId: input.projectId,
15 message: "Failed to launch thread",
16 cause,
17 }),
18 ),

Posted via Macroscope — Effect Service Conventions

Fixed in 4cfe3581d4: the RPC launch boundary now handles AttachmentClaimError, ThreadLaunchError, and ServerRuntimeStartupError explicitly with Effect.catchTags. The attachment claim message remains unchanged because it is an existing, safe caller-facing validation message; launch/startup failures retain the existing fixed message. The suggested catchTags-then-mapError sequence would rewrap the translated claim error and erase that message, so the mapping is exhaustive in one catchTags instead. Canonical server typecheck, targeted type-aware lint, formatter and diff checks pass. No behavior or new test machinery was added.

Added 1 commit

  • refactor(rpc): map launch errors by their typed tags4cfe358