[Bug]: Usage-limit auto-resume gives up permanently when the continuation is rejected right after the reset#33708

opened8h agobyBbrazyptian96

Before submitting

  • I searched existing issues and did not find a duplicate. Related but distinct: #27773 / #27787 (Codex failures lose resetAt on a first-request 429), #17808 (auto-resume never fires), #22307 (early restoration).
  • I included enough detail to reproduce or investigate the problem.

Area

apps/server

Summary

Auto-resume fired on time, but the provider rejected the continuation run with usageLimitExceeded a few seconds after the reported reset. That failure was recorded without a usable future resetAt, so UsageLimitRecoveryWorker never re-armed and the thread stayed failed until I continued it by hand about 2 hours later. One rejected continuation ends recovery for good.

This is separate from #27773. Even when the reset time is recovered (for example via the account/rateLimits/read probe in #27787), the probe may return the window that just expired, or null, at this moment. limitRecoveryCommand rejects any resetAt that is not later than the run's completion time, so it would still return null.

Steps to reproduce

  1. Run a Codex (app-server driver) thread on a ChatGPT-plan account until the 5h window is exhausted. The failure carries resetAt from the limitId: "codex" snapshot.
  2. Arm Resume at reset (auto-resume).
  3. Wait for the reset. The worker dispatches Continue where you left off. about 1–2 s after resetAt.
  4. If the backend still rejects that first request (usageLimitExceeded), recovery stops permanently.

Observed once (T3 Code 0.1.7). This is timing-dependent, and I have not reproduced it on demand.

Timeline (UTC, from statev2.sqlite and the provider event log)

TimeEvent
22:00:04account/rateLimits/updated limitId:"codex", primary.usedPercent:100, resetsAt:1794268919 (00:01:59Z). Run fails usage_limit, resetAt:"…T00:01:59.000Z".
22:03:07limit-arm:…:1794268919000, autoResume:true.
00:02:00.579limit-resume:… dispatched; continuation run created (+1.6 s after reset).
00:02:06.775Only rate-limit notification: limitId:"premium", primary:null, secondary:null (ignored by mergeCodexRateLimits).
00:02:06.781error: usageLimitExceeded, "…try again at 7:01 PM." (server-local time = 00:01Z, already past), willRetry:false.
00:02:06.815Failure item: class:"usage_limit", resetAt:null. No further limit-arm / limit-resume.
01:59:19Manual continue runs normally. The next limitId:"codex" snapshot shows primary.usedPercent:0 in a new window.

Relevant code (0.1.7 bundle)

  • UsageLimitRecoveryWorker.limitRecoveryCommand returns null when !thread.usageLimitResetAt, or when resetMs <= latestRunCompletedAt. There is no fallback or retry for a failed continuation.
  • CodexProvider: codexUsageLimitResetAt(rateLimitSnapshot) is the only source of failure.resetAt. The error text's "try again at …" is not used, and here it was already in the past.

Expected behavior

If a run started by limit-resume (usageLimitContinuationOfRunId set) fails again with usage_limit, and there is no reset time later than now, auto-resume should stay armed. It should retry with a bounded backoff (for example +2 min, +5 min, +15 min, then give up and show the existing "retry manually" state), still respecting cancel, settle, snooze, and archive.

A small grace delay after resetAt before dispatching (for example 30–60 s) would also make the first attempt less likely to land before the provider's counters roll over. On its own, though, it doesn't fix the dead end above.

Actual behavior

The thread remains failed with no armed recovery. The UI gives no indication that auto-resume was attempted and abandoned.

Notes

The cause on the provider side (why a request 7 s after the reported reset was still rejected) is inferred, not proven: possibly propagation lag in the backend limiter. The client-side dead end is confirmed from the event store and the bundled source.