fix(server): preserve hub usage readings after provider rate limits#19820

⚙️ Run configuration

• Configuration used: Path: .coderabbit.config.ts
• Review profile: CHILL
• Plan: Advanced
• Run ID: 2ebd3c23-277f-4d0c-a958-b1170e11aced

📥 Commits

Reviewing files that changed from the base of the PR and between 1451e7cb23034d29774338608368a15d8845df36 and ef207fca709b4ddcd9115376c7000e0ae6d56cdf.

📒 Files selected for processing: 3

• apps/server/src/usage/cliproxyApi.test.ts
• apps/server/src/usage/cliproxyApi.ts
• docs/user/usage.md

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 1 remain after this review.

CLIProxyAPI usage reads now use per-account pacing and caching, with provider rate limits reported separately. Credit redemption clears cached readings based on its outcome. Tests and usage documentation cover the updated behavior.

Changes: CLIProxyAPI usage reads

Layer / File(s)	Summary
Throttled usage reads apps/server/src/usage/cliproxyApi.ts, apps/server/src/usage/cliproxyApi.test.ts, docs/user/usage.md	Usage reads are keyed by hub and account identity. Successful reads schedule the next probe after 60 seconds; failed reads use increasing intervals up to eight minutes. The last successful reading remains available for up to fifteen minutes. Tests and documentation cover pacing, retention, identity separation, and provider 429 messages.
Redemption and usage cache apps/server/src/usage/cliproxyApi.ts, apps/server/src/usage/cliproxyApi.test.ts	Redemption clears the affected account’s cached reading on completion or interruption, except when redemption returns the provider rate-limit error. Tests cover concurrent reads and redemption outcomes.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Sequence Diagram(s)

mermaid

1	sequenceDiagram
2	participant UsageConsumer
3	participant CLIProxyAPI
4	participant CLIProxyAPIHub
5	UsageConsumer->>CLIProxyAPI: request account usage
6	alt probe is not scheduled
7	CLIProxyAPI-->>UsageConsumer: return cached reading or unavailable usage
8	else probe is scheduled
9	CLIProxyAPI->>CLIProxyAPIHub: request account usage
10	CLIProxyAPIHub-->>CLIProxyAPI: return usage or provider error
11	CLIProxyAPI-->>UsageConsumer: return reading or unavailable usage
12	end

Suggested reviewers: juliusmarminge

No confirmed issue blocks merging the usage-pacing change; the freshness policy still awaits maintainer approval.

The change affects usage freshness and reset-credit recovery without adding permissions or new external destinations. Account checks and recovery ordering remain intact. Remaining uncertainty concerns long-running state accumulation and provider behavior during rate limiting.

Retained concerns No architecture-level concerns identified.

Security review details

Security Blast Radius

• observed — The new mutable state is shared within the server usage service across configured hubs. Keys separate hub URL, account file ID, provider, email, and workspace, limiting ordinary cache reuse and invalidation to that identity. It stores account usage information, not provider bearer tokens.

Trust Boundaries and Controls

• observed — Each account read still obtains an authenticated account listing from the configured hub before consulting cached usage, and disabled or unsupported accounts are filtered out. Hub-supplied account metadata determines the cache identity and provider auth index, while outbound provider destinations remain fixed by server code. These trust relationships predate the PR.

Resilience and Maintainability Implications

• observed — The recovery design conservatively discards cached readings after uncertain credit outcomes. Cleanup waits for older same-identity reads, preventing them from restoring pre-redemption state after invalidation. Added behavior tests cover pacing, identity isolation, concurrent reads, provider rejection, uncertain failure, and interruption.

Hardening Proposals

• proposed — Consider bounded eviction or pruning of inactive identities when accounts or configured sources disappear. This would bound long-running memory consumption and residual account metadata without treating the current behavior as a demonstrated security finding.

^Failed Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Description check	Warning	The description covers the problem, change, scope, and verification. However, it does not provide the explicit maintainer approval required for the proposed freshness policy; it says that approval is …	Add a link to a maintainer comment that explicitly approves the one-minute refresh, two-to-eight-minute backoff, and fifteen-minute retention policy, or document the maintainer discussion and approval in the description.

^Passed Passed checks: 3 passed

Check name	Status	Explanation
Title check	Passed	The title clearly and concisely describes the main change: preserving hub usage readings after provider rate limits.
Linked Issues check	Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	Passed	Check skipped because no linked issues were found for this pull request.

Full details: Description check

Explanation

The description covers the problem, change, scope, and verification. However, it does not provide the explicit maintainer approval required for the proposed freshness policy; it says that approval is still pending.

Resolution

Add a link to a maintainer comment that explicitly approves the one-minute refresh, two-to-eight-minute backoff, and fifteen-minute retention policy, or document the maintainer discussion and approval in the description.

🧪 Generate unit tests (beta)

• Create a new PR

Full review finished.