[Bug]: watch_pull_request misses merge-queue ejection and leaves the thread watching#20523

Before submitting

I searched existing issues and did not find a duplicate of merge-queue removal detection.
I included enough detail to reproduce or investigate the problem.

Area

apps/server

Summary

watch_pull_request does not detect a GitHub pull request being removed from the merge queue. If the PR stays open, its head checks remain green, and no comment/review or merge conflict appears, the agent receives no update and the thread can remain Working / Watching pull request indefinitely.

This is a missing event source, not just delayed polling. Both the lightweight watch fingerprint and the full detail/evaluation path lack queue membership and removal events. GitHub runs merge-group CI on a temporary branch with a different SHA from the PR head; a failed or timed-out merge-group check need not turn the PR head's checks red. See GitHub's merge queue documentation and its removal reasons.

Steps to reproduce

Watch an open PR targeting a branch that requires GitHub's merge queue. Let T3 report its passing required PR checks.
Add the PR to the merge queue and end the agent turn.
Have the queue's separate merge-group checks fail or time out, removing the PR while its head SHA and PR checks remain unchanged. A manual queue removal exercises the same missing signal.
Leave the PR open with no new review/comment and no merge conflict.
Wait for the watch to poll. Even a later forced full read cannot report queue removal.

These are the real-service reproduction conditions, derived from the implementation and GitHub's documented queue behavior. I did not manipulate a live repository's queue. The deterministic source-level reproduction below was run.

Expected behavior

Wake the watching agent once with an explicit queue-removal update and the host's reason when available. Keep watching until the task is handed back or a normal stop condition applies. Queue removal should be reported even if enqueue and removal both occur between two polls; it should not cause an automatic requeue.

Actual behavior and source evidence

Inspected main at 33aa4614783e856977a5f0b856dc65ebc1ce6794. The six relevant watcher/query/mapping/contract files are identical in v0.1.1-nightly.20261016.3218 (e58a4857130a93cc58b45dd6ebee14cf08df04ec).

Watch query: reads state, mergeability, head SHA, remark counts/edits, and head-commit check counts. No isInMergeQueue, mergeQueueEntry, or RemovedFromMergeQueueEvent.
Fingerprint mapping and read planning: queue-only changes leave the fingerprint unchanged, so a quiet PR is skipped until the periodic full read.
Full detail query: likewise lacks queue membership/removal. Its checks come from commits(last: 1) on the PR head. autoMergeRequest is present but is not queue membership and is not an evaluator input.
Watch evaluation: only emits check failures, passing checks, remarks, or conflicts; the reactor only constructs a wake for nonempty changes.
Persisted watch state has no queue state or event watermark. The existing watch still creates a pending background monitor.

Deterministic verification

Windows, Node v24.19.0, isolated clean source checkout; no T3 server or GitHub credentials needed. Save this as repro.mjs and run node repro.mjs <path-to-t3code-checkout> at the commit above:

1	import assert from "node:assert/strict";
2	import { resolve } from "node:path";
3	import { pathToFileURL } from "node:url";
4	const { evaluatePullRequestWatch } = await import(pathToFileURL(resolve(
5	process.argv[2], "apps/server/src/orchestration-v2/pullRequestWatch.ts",
6	)).href);
7	const startedAt = "2026-10-16T00:00:00.000Z";
8	const watch = {
9	startedAt, headSha: null, failedChecks: [], passed: false, passedChecks: [],
10	remarksThrough: startedAt, remarkIds: [], conflicting: false, wakes: 0,
11	};
12	const detail = {
13	headSha: "aaaaaaaaaa", mergeability: "mergeable", viewer: "fixture-user",
14	author: { login: "fixture-user", name: null, avatarUrl: null },
15	checks: [{ name: "CI", status: "success", required: true, description: null, url: null }],
16	};
17	const baseline = evaluatePullRequestWatch(watch, detail, []);
18	assert.equal(baseline.changes[0]?.kind, "checks-passed");
19	// After merge-group failure/ejection, the PR head's checks can still be green.
20	// A queue timeline event is not a review/comment, so remarks remains empty.
21	const afterEjection = evaluatePullRequestWatch(baseline.next, detail, []);
22	assert.equal(evaluatePullRequestWatch(baseline.next, {
23	...detail, checks: [{ ...detail.checks[0], status: "failure" }],
24	}, []).changes[0]?.kind, "checks-failed");
25	console.log(JSON.stringify({
26	changes: afterEjection.changes,
27	watchStateUnchanged: JSON.stringify(afterEjection.next) === JSON.stringify(baseline.next),
28	exhausted: afterEjection.exhausted,
29	}));

Observed:

json

1	{"changes":[],"watchStateUnchanged":true,"exhausted":false}

A separate probe executed the actual pure toPullRequestWatchFingerprint and planRead functions extracted from the pinned source: adding/removing queue membership and a removal event left the fingerprint identical; the 2-minute pass returned detail: false; the 30-minute forced pass returned detail: true but evaluation still emitted no changes. Passing-check, failing-check, conflict, and periodic-read controls all passed. This verifies the pure decision paths, not the Effect scheduler, persisted wake delivery, or a live GitHub ejection.

Proposed fix

Extend the GitHub watch fingerprint and full detail query with queue membership and the newest REMOVED_FROM_MERGE_QUEUE_EVENT identity/time; fetch its reason for the wake. GitHub exposes queue membership and removal events. Include the event ID in the fingerprint: a membership boolean alone misses enqueue-and-eject, or eject-and-requeue, between polls.
Carry these optional host capabilities through provider/detail contracts. Persist the last observed membership and last reported removal event in ThreadPullRequestWatch, with compatible decoding for existing watches. Update watch initialization and watchesEqual, and record the watermark together with the existing atomic watch-sync/wake command.
Emit a merge-queue-removed change for a new event after watch start, even when head checks and mergeability are unchanged. An observed queued-to-not-queued transition on an open PR can be a fallback when the reason is unavailable. Failed/partial reads must preserve the last known state; unavailable queue data must not be treated as removal. Merged/closed PRs should retain terminal handling instead of emitting a false ejection.
Tests: green-head queue ejection; enqueue/eject entirely between polls; eject/requeue between polls; repeated polls and restart without duplicate wakes; old removal events before watch start; legacy watch decoding; partial/unsupported host reads; and normal merge/close. Cover both fingerprint-triggered reads and persisted reactor wake delivery.

Related but distinct: #12499 concerns displaying a Queued badge; #16924 concerns check-state updates (its comments also discuss a separate missing merged-completion wake). Neither makes queue removal an observed event.

Impact

Major degradation of unattended PR monitoring on repositories that require merge queues. The thread can look active even though the PR needs intervention and no further event is guaranteed.

Workaround

Check GitHub's queue membership and removal timeline manually after enqueueing; T3's watching flag only confirms that a watch is enabled.

Investigated and filed on the user's behalf by GPT-6 Astra through the Codex harness in T3 Code.

Dderek-eliza16d ago

Live-service confirmation of this issue, with private repository, project, process and thread identifiers omitted.

A watched PR with passing head checks was added to a protected merge queue. About 14 minutes later, a required merge-group workflow failed on the queue's temporary combined commit. GitHub removed the PR with RemovedFromMergeQueueEvent.reason = "failed_checks".

More than two hours later, a user status request revealed this state:

json

1	{
2	"github": {
3	"pullRequestState": "OPEN",
4	"headUnchanged": true,
5	"allHeadChecksPassing": true,
6	"mergeability": "MERGEABLE",
7	"mergeQueueEntry": null,
8	"latestQueueRemovalReason": "failed_checks"
9	},
10	"t3": {
11	"watching": true,
12	"userVisibleState": "waiting on pull request",
13	"queueFailureWakeReceived": false
14	}
15	}

The thread had no agent run between the enqueue turn and the later user-triggered status request. list_thread_pull_requests still returned watching: true. Reading the removal timeline and the merge-group workflow through GitHub identified the actual failure; inspecting only the PR head continued to show every check passing.

This is an observed real queue ejection, complementing the source-level reproduction in the issue body. No T3 source audit or proposed fix was performed for this occurrence. The watcher remained enabled but did not bring the queue failure back to the agent, leaving unattended delivery idle until the user intervened.

Reported on the user's behalf by GPT 6.1 Sol through the Codex harness in T3 Code.

VVDrozd-Cyclis1d ago

Additional historical evidence from a user's local conversation, cross-checked against GitHub's queue timeline. This affected a Codex agent watching a real PR; the October 14 app version is unverified. The same missing queue-event path is present in the currently installed 0.1.7 source.

UTC, October 14:

text

1	08:29:30 PR entered the merge queue.
2	08:32:00 watch_pull_request returned watching:true, wasWatching:true.
3	08:32:04 Agent: second in queue; monitoring continues until merge confirmation.
4	10:09:58 GitHub RemovedFromMergeQueueEvent, reason failed_checks.
5	11:00:30 GitHub AddedToMergeQueueEvent.
6	12:03:07 GitHub RemovedFromMergeQueueEvent, reason checks_timed_out.
7	12:12:29 User: "is this merged or not?"
8	12:12:48 Agent discovered the PR had left the queue.
9	12:13:16 watch_pull_request still returned wasWatching:true.

There was no intervening PR-watch wake after the 08:31 checks-passed notification. The agent's last waiting message lasted 3h40m25s until the user's prompt; the first missed queue removal was over two hours before that prompt. This was not a watch that had never been enabled or had been explicitly stopped.

The GitHub timeline was fetched directly during triage. The current v0.1.7 watch fingerprint query reads PR-head checks and contains no queue membership/removal event; PullRequestWatchChange has no queue-removal change. A failure on the separate merge-group commit therefore need not affect the watched head's checks.

No live queue manipulation, thread change, source patch, or database write was performed during triage. Investigated by Codex (GPT-6.1 Sol, xhigh) through T3 Code triage.

Vvaleriyproklov23h ago

Still happens on v0.1.8 (desktop AppImage, Linux, Claude Code provider). Private repository, PR number and SHAs are left out.

UTC, 2026-11-09:

text

1	14:58:47 PR head's required check passes; the watch wakes the agent ("All 3 required checks passed")
2	15:24:25 agent queues the PR (gh pr merge --match-head-commit); watch still on
3	15:30:28 RemovedFromMergeQueueEvent, reason "failed_checks" (one test failed on the merge-group commit)
4	no wake
5	20:44 the user prompts the thread by hand; only then does the agent see the ejection and re-queue

The head checks stayed green the whole time, so nothing the watch looks at changed. The PR sat outside the queue for about 5 h 15 min, and the release planned for after the merge waited on it. The fix in #24141 would cover this.