[Bug]: PR watch misses final CI completion after an early required-check success#23712

opened1d agobyDderek-eliza

Before submitting

  • I searched existing issues and did not find a duplicate issue.
  • I included enough detail to reproduce or investigate the problem.

Related discussion: #23245 already describes the broader check-state notification limitation. This report raises the observed final-completion gap for bug triage; the relationship is explicit so maintainers can consolidate it if appropriate.

Area

apps/server

Steps to reproduce

Observed sequence; not yet reproduced in a separate controlled installation:

  1. Watch an open GitHub PR using watch_pull_request. Have one required check complete before several other CI jobs on the same head.
  2. Receive the notification: All 1 required check passed.
  3. Have the agent inspect the full check list, confirm that other jobs are still running, leave the watch active, and end its turn as instructed.
  4. Let the remaining jobs finish successfully without another push, comment, review, or merge.
  5. Inspect the thread and the PR's checks.

Expected behavior

The thread should receive a final completion wake when the previously pending CI jobs finish, allowing the agent to verify the complete result and hand the work back to the user.

If waiting for those checks is intentionally unsupported, the watch response should make that limitation explicit. The current instruction that T3 Code "keeps watching and wakes you on the next change" leaves the agent waiting for a completion notification that never arrives.

Actual behavior

The early required-check success notification arrived while other CI jobs were still running. No later completion notification reached the conversation when all 13 checks on that same head had succeeded.

A manual user prompt was needed to resume the agent. At that point:

  • GitHub reported all 13 checks as completed successfully, including the final aggregate CI check.
  • list_thread_pull_requests still reported watching: true and the PR as open.
  • The agent then explicitly stopped watching to return the thread to the inbox.

The PR had not merged or closed, and the watch had not already ended. This distinguishes the incident from silent termination after a merge.

Impact

Minor bug or occasional failure

Unattended CI follow-up stalls after an early success notification. The user must manually inspect the PR and prompt the agent even though monitoring remains active. The tool instructs the agent to rely on wake-ups rather than run a separate polling loop.

Logs or stack traces

Relevant notification text:

text
1All 1 required check passed
2T3 Code keeps watching and wakes you on the next change, so end your turn when you are done.

Subsequent manual inspection found all 13 checks successful and the watch still active. No server polling or delivery logs were collected, so this report does not claim a confirmed runtime root cause.

Current public source in apps/server/src/orchestration-v2/pullRequestWatch.ts is consistent with the symptom: evaluatePullRequestWatch selects required checks as its success gate when any exist. Once that gate has passed, later success of checks outside the gate does not emit another checks-passed change. This is source inspection, not a reproduction against a pinned installed build.

Workaround

Manually prompt the agent to inspect the full CI results, then call unwatch_pull_request when its work is complete.