Sign up
View stargazersSign up

[Bug]: PR watch never reports a rerun that fails again, and on GitHub a failed run can be hidden by another event's run of the same check#32277

opened3d agobyRroyalaid

Before submitting

  • I searched existing issues and did not find a duplicate.
  • I included enough detail to reproduce or investigate the problem.

Area

apps/server

Steps to reproduce

Real-world A (any host):

  1. Watch a PR; check lint fails; the agent is woken with "Checks failed".
  2. The agent reruns the job and ends its turn to wait.
  3. The rerun starts and fails again between two watch reads.

Real-world B (GitHub, workflow on both push and pull_request):

  1. Push to the PR branch. Each check runs twice on one head, once per event.
  2. The push run of lint fails while the pull_request run is still running, then the pull_request run passes.

Deterministic:

  • evaluatePullRequestWatch with a watch that recorded lint failing at run URL .../job/1, then a read where lint is failing at .../job/2: no change is returned.
  • toChecks in packages/source-control-github over a rollup with lint failed (event push) and lint in progress (event pull_request): it returns pending; with the pull_request run passed, it returns success. With both runs failed, it keeps the first-listed run's URL, so a rerun of the other run that fails again changes nothing the watch can see.

Expected behavior

A failed run the agent has not been told about wakes it, including a rerun that failed again. On GitHub, a check reports the worse of its events' runs (failure over running, running over passed, passed over a run cancelled next to it), and between two failed runs shows the one that finished last.

Actual behavior

A: the watch stays silent; the agent waits for a result that already arrived. B: the check reads pending, then passed, and the failed push run is never reported.

Impact

Major degradation or frequent failure

Version or commit

0.1.8-nightly.20261106.4262 (c55db3e9c7); code read at main 6577c6b618, unchanged at 240446e2d9.

Environment

macOS and Linux, T3 server with Claude and Codex providers; GitHub Actions workflows triggered on both push and pull_request.

Logs or stack traces

None kept from a live run; the deterministic cases above fail on main.

Workaround

Ask the agent to poll after a rerun instead of waiting on the watch.

Proposed fix (for the PR, once triaged)

The watch records the failing runs it last read (name and URL) and reports a failing check whose URL changed. The GitHub reader selects workflowRun.event, keeps each event's newest run of a check, and merges by the order above, keeping the required flag. Tests: watch "a rerun that fails again between two reads is news"; GitHub "a failed push run is not hidden by a running pull_request run", "a failed run stays failed after the other event's run passes", "the required flag survives merging the events", "when both events' runs failed, the check shows the run that failed last" (all fail on main), plus guards that pass on both ("runs from both events passing read as passed", "a run cancelled next to a passing run is not a failure", "the same job name in two workflows stays two checks", "reruns within one event keep the newest attempt"). Limit: a host that reports no run URL falls back to the name.

Rroyalaid9h ago
Synced from GitHub

GitLab hits case A too, and the fix in #32319 does not cover it as written.

toChecks in packages/source-control-gitlab/src/server/gitLabMergeRequestJson.ts reports a merge request as one check named Pipeline, with url set to head_pipeline.web_url. Retrying failed jobs (the "Retry" button or glab ci retry) runs them inside the same pipeline, so the pipeline id and web_url stay the same. With #32319, a retry that fails again between two watch reads still matches the told [name, url] pair, and the watch stays silent.

Observed on 0.1.9-nightly.20261110.4449, code read at upstream main c6ca605630:

  1. The pipeline failed and the watch woke the agent with "Checks failed".
  2. The agent retried three failed jobs at 00:36:31 UTC. Each one hit runner_system_failure and finished by 00:36:43, so the pipeline went from running back to failed in about 12 seconds, between two sweeps.
  3. The watch never woke again. The thread showed "Watching pull request" for more than 90 minutes with the result already in.

The MR's head_pipeline already carries a value that changes on each re-finish: finished_at (here 2026-11-10T00:39:53.753Z; updated_at also moves). Folding it into the run identity for the GitLab check, for example a url or run key of web_url#finished_at, or adding finished_at to RawPipelineSchema and keying failedRuns on it, would let #32319's comparison see a retried pipeline that failed again. Reporting per-job checks would also work, but it is a larger change.