[Bug]: Usage prices long-context requests at base rates and ignores LiteLLM's above_272k / above_100k tiers#25414
Before submitting
- I searched existing issues and did not find a duplicate.
- I included enough detail to reproduce or investigate the problem.
Area
apps/server
Summary
Usage prices every request at a model's base rates, even when the request's prompt is above the model's long-context threshold. LiteLLM publishes the higher rates (*_above_272k_tokens for the GPT-5.4+/GPT-6 family, *_above_100k_tokens for claude-haiku-5-5), but parseRateTable never reads them, so cost for long-context requests is understated. Token totals are not affected.
The comment on ModelRate in apps/server/src/usage/usagePricing.ts says these variants are omitted because "transcripts do not record" them. Transcripts don't record a tier, but each record does carry the prompt size, which is all the tier needs:
- Claude Code: per message,
input_tokens + cache_read_input_tokens + cache_creation_input_tokens. - Codex: per turn,
event_msg→token_count→info.last_token_usage.input_tokens(which already includescached_input_tokens).
Steps to reproduce
- Run a Codex turn on a model with a 272K tier (for example
gpt-6-sol) whose prompt is above 272K tokens, or use this minimal fixture line in a rollout file:
| 1 | {"type":"event_msg","payload":{"type":"token_count","info":{"last_token_usage":{"input_tokens":300000,"cached_input_tokens":280000,"output_tokens":1000,"reasoning_output_tokens":0,"total_tokens":301000}}}} |
- Open Usage and compare the cost with the LiteLLM entry for that model.
Expected behavior
For gpt-6-sol, the request above is priced with the above-272K rates: 20,000 × 4e-06 + 280,000 × 4e-07 + 1,000 × 1.5e-05 = $0.207. The same applies to claude-haiku-5-5 above 100K (input_cost_per_token_above_100k_tokens and the matching cache and output fields), and to Fast/priority usage (*_above_272k_tokens_priority).
Actual behavior
It is priced at base rates: 20,000 × 2e-06 + 280,000 × 2e-07 + 1,000 × 1e-05 = $0.106, about half.
readTokenRatesreads onlyinput_cost_per_token,output_cost_per_token,cache_read_input_token_costandcache_creation_input_token_cost(plus_priority/_ultrafast), never an_above_<N>k_tokensfield.- The usage records keep only summed totals, so even with the rates parsed, the cost step can't tell which tokens were above the threshold.
Suggested fix
When a record is scanned, compare its prompt size with the threshold from the entry's _above_<N>k_tokens fields and save a long-context flag on the record (or keep long-context totals separately when aggregating). Then price flagged tokens with the _above_ rates, falling back to the base rate for any field the entry doesn't list, as cacheRate already does. Models with no _above_ fields keep today's behaviour.
Impact
Minor bug or occasional failure
Version or commit
Desktop app 0.1.2 (Beta), checked against main @ cc93cc564
Environment
macOS, Codex and Claude Code transcripts