[Bug]: Codex usage drops the request after an interrupted turn whose last request only has a token_usage_record#29365
Before submitting
- I searched existing issues and did not find a duplicate.
- I included enough detail to reproduce or investigate the problem.
Area
apps/server
Steps to reproduce
Scan this synthetic rollout with the current Codex usage reader. A turn is interrupted after its second model request finishes. That request is only written as a top-level token_usage_record; no token_count follows it.
| 1 | {"type":"session_meta","timestamp":"2026-11-02T10:00:00.000Z","payload":{"id":"aborted-turn-repro","model_provider":"openai"}} |
| 2 | {"type":"turn_context","timestamp":"2026-11-02T10:00:00.000Z","payload":{"turn_id":"turn-1","model":"gpt-6-sol"}} |
| 3 | {"type":"token_usage_record","timestamp":"2026-11-02T10:00:01.000Z","payload":{"turn_id":"turn-1","response_id":"resp_1","usage":{"input_tokens":100,"cached_input_tokens":0,"cache_write_input_tokens":0,"output_tokens":20,"reasoning_output_tokens":0,"total_tokens":120}}} |
| 4 | {"type":"event_msg","timestamp":"2026-11-02T10:00:01.001Z","payload":{"type":"token_count","info":{"last_token_usage":{"input_tokens":100,"cached_input_tokens":0,"cache_write_input_tokens":0,"output_tokens":20,"reasoning_output_tokens":0,"total_tokens":120},"total_token_usage":{"input_tokens":100,"cached_input_tokens":0,"cache_write_input_tokens":0,"output_tokens":20,"reasoning_output_tokens":0,"total_tokens":120}}}} |
| 5 | {"type":"token_usage_record","timestamp":"2026-11-02T10:00:02.000Z","payload":{"turn_id":"turn-1","response_id":"resp_2","usage":{"input_tokens":150,"cached_input_tokens":0,"cache_write_input_tokens":0,"output_tokens":10,"reasoning_output_tokens":0,"total_tokens":160}}} |
| 6 | {"type":"event_msg","timestamp":"2026-11-02T10:00:03.000Z","payload":{"type":"turn_aborted","turn_id":"turn-1","reason":"interrupted"}} |
| 7 | {"type":"turn_context","timestamp":"2026-11-02T10:00:04.000Z","payload":{"turn_id":"turn-2","model":"gpt-6-sol"}} |
| 8 | {"type":"token_usage_record","timestamp":"2026-11-02T10:00:05.000Z","payload":{"turn_id":"turn-2","response_id":"resp_3","usage":{"input_tokens":200,"cached_input_tokens":0,"cache_write_input_tokens":0,"output_tokens":30,"reasoning_output_tokens":0,"total_tokens":230}}} |
| 9 | {"type":"event_msg","timestamp":"2026-11-02T10:00:05.001Z","payload":{"type":"token_count","info":{"last_token_usage":{"input_tokens":200,"cached_input_tokens":0,"cache_write_input_tokens":0,"output_tokens":30,"reasoning_output_tokens":0,"total_tokens":230},"total_token_usage":{"input_tokens":450,"cached_input_tokens":0,"cache_write_input_tokens":0,"output_tokens":60,"reasoning_output_tokens":0,"total_tokens":510}}}} |
Expected behavior
Bill all three requests: 510 processed tokens, which equals the final cumulative counters, with no skipped-record warning.
Actual behavior
Only the first request is billed (120 tokens) and the Usage page reports 1 malformed or inconsistent usage record. The next turn's token_count cumulative includes the interrupted request, so cumulative - previous (390) is the interrupted request (160) plus this request (230). The delta check fails and drops this request. The interrupted request is never seen, because the parser ignores token_usage_record lines.
Evidence and diagnosis
Seen once in a real rollout (macOS, Codex, September 2026, gpt-5.6-sol):
| 1 | token_count total 27,296,136 |
| 2 | token_usage_record 69,953 (request 1 of the interrupted turn) |
| 3 | turn_aborted reason: interrupted |
| 4 | token_usage_record 70,283 |
| 5 | token_count total 27,436,372, last 70,283 |
| 6 | 27,296,136 + 69,953 + 70,283 = 27,436,372 |
A read-only replay of 786 Codex rollouts from three hosts through the current reducer found this pattern once: 163 turn_aborted events, 35,329 cumulative token_counts and 18,384 token_usage_records. So it is rare. Since #22558 and #23729 it is the only remaining cause of a skipped Codex record in that set, apart from the rewound counters reported in #29363.
One detail matters for any fix. token_usage_record is not one-to-one with token_count. In 121 cases, more than one token_usage_record sits between two snapshots whose delta still equals last_token_usage exactly. Its thread_token_usage totals also differ from total_token_usage. So the records can't replace the counters. They can only bridge a gap when their sum explains it exactly.
Possible approach (for maintainers to decide)
Keep the sum of token_usage_record usage since the last cumulative snapshot in the scan state. When a delta fails exact reconciliation but equals that sum exactly, bill last plus the unreported remainder, then reset the sum at every snapshot. This needs a new usage marker and projected fields for token_usage_record, one optional state field, and the reducer able to return two records for one line. That may be more machinery than one rare case justifies. I'm happy to send a PR if you want this handled.
Impact
Minor bug or occasional failure. The interrupted request and the following request are missing from usage, and the partial-source warning stays on the Usage page.
Version or commit
T3 Code 0.1.6 (desktop). Same logic on main at ac8dce0cc3ab (parser blob 20939e54db79946a8f0e0f3badabdde917752b59).
Environment
macOS ARM64 desktop. The synthetic reproduction needs no provider account.
Workaround
None in the app.