perf(server): skip oversized history during handoff selection #23821 · pingdotgg/t3code

Scope and approval

This is a one-line performance fix for proven unnecessary serialization in the existing selector. It adds no setting or behavior choice. Existing budget tests already cover oversized multilingual items, JSON escaping, counter boundaries and recovery, so the PR does not add duplicate fixtures.

Verification

Current-main verification (2026-10-24): reproduced on d53873b6a4799e87afd63c8d288ef0089f5a3748 and compared with exact PR head 2ea956066e6f19ac7f9116ff4cf84b5c3e179135. The 48 focused tests in ContextHandoffBudget.test.ts and testkit/ProviderSwitch.integration.test.ts pass on both revisions. Two independent reviews found no blocking findings.

Live-server runtime transcript: six HTTP requests per revision read 103 persisted history items with 100 command outputs of 32 KiB and a 16,000 budget. Main serialized 103 candidates; this PR serialized 3. Both selected 3, omitted 100, and persisted one handoff per request. A temporary diagnostic endpoint invoked the production handoff service directly. Ordinary provider-switch orchestration and provider delivery were not exercised; these measurements do not establish overall provider-switch latency. This server-only change has no affected render paths.

Reproduction sources: diagnostic route, instrumentation, HTTP request driver. These evidence-only sources are uploaded, not committed.

Same 100 command outputs, 32 KiB each, 16,000-byte budget	Baseline	Patch
Median (min–max), 21 trials	5.292 ms (5.096–5.661)	0.012 ms (0.009–0.018)

Node 26.12.1, Linux x64, Intel i5-14400F; 20 warmups, 21 alternating baseline/patch trials, 10 selections per sample. Both select the same three messages and omit the same 100 command outputs. A broader harness made 6,805 exact-result comparisons, including lone surrogates and escaping. A longer-warmed small-history control measured 17.395 → 17.450 microseconds, within noise.

vp test run apps/server/src/orchestration-v2/ContextHandoffBudget.test.ts apps/server/src/orchestration-v2/ContextHandoffService.test.ts

: 20 passed.

vp run check:changed upstream/main: format, lint, knip and affected-package typechecks passed. The combined local verification also covered the independent terminal fix; commits are separate.

git diff --check passed. Base is current main 22840ab943.

These are warmed synthetic selector measurements. They exclude history loading, historical-message construction, provider calls and overall handoff latency. No client or provider-adapter changes are needed.

Self-contained before/after reproduction

Run from the PR checkout with Node 26.12.1. This loads the exact baseline and patched production functions and checks output equality. No third-party benchmark dependency is needed.

1	node --input-type=module <<'JS'
2	import * as assert from 'node:assert/strict';
3	import * as fs from 'node:fs';
4	import { execFileSync } from 'node:child_process';
5	import { stripTypeScriptTypes } from 'node:module';
6	import { performance } from 'node:perf_hooks';
7
8	const base = '22840ab943b8418744b48223411e46ba4fdcef1a';
9	const file = 'packages/provider-core/src/server/handoffBudget.ts';
10	async function load(source, name) {
11	const start = source.indexOf('export function historicalMessage(');
12	const end = source.indexOf('export function handoffCoverage(', start);
13	assert.ok(start >= 0 && end > start);
14	const js = stripTypeScriptTypes(source.slice(start, end));
15	return import(`data:text/javascript;base64,${Buffer.from(js + `\n//# sourceURL=${name}`).toString('base64')}`);
16	}
17	const baseline = await load(execFileSync('git', ['show', `${base}:${file}`], { encoding: 'utf8' }), 'before');
18	const patched = await load(fs.readFileSync(file, 'utf8'), 'after');
19	const message = (id, role, text) => ({ itemId: `item:${id}`, role, text, kind: `${role}_message`, threadId: 'thread:fixture', runId: `run:${id}`, providerThreadId: 'provider:fixture', status: 'completed' });
20	const logs = Array.from({ length: 100 }, (_, i) => {
21	const line = `test src/component-${i}.test.ts: expected output differs at line ${i + 1}\n`;
22	return baseline.historicalMessage({ id: `item:command:${i}`, threadId: 'thread:fixture', runId: `run:${Math.floor(i / 5)}`, providerThreadId: 'provider:fixture', status: 'completed', type: 'command_execution', input: `vp test run src/component-${i}.test.ts`, output: line.repeat(Math.ceil(32768 / line.length)).slice(0, 32768), exitCode: 1 });
23	});
24	const input = {
25	messages: [message(0, 'user', 'Keep the API compatible and run the focused tests.'), ...logs, message(10000, 'user', 'Continue investigating the test failures.'), message(10001, 'assistant', 'The failing test is reproduced. Check the captured command output.')],
26	coverage: 'Provider handoff; omitted history remains retrievable through t3_thread_read.', budget: 16000,
27	};
28	const expected = baseline.selectHistory(input);
29	assert.deepEqual(patched.selectHistory(input), expected);
30	for (let i = 0; i < 20; i++) { baseline.selectHistory(input); patched.selectHistory(input); }
31	const samples = { baseline: [], patched: [] };
32	let sink = 0;
33	for (let i = 0; i < 21; i++) {
34	const order = i % 2 ? [['patched', patched], ['baseline', baseline]] : [['baseline', baseline], ['patched', patched]];
35	for (const [label, parser] of order) {
36	const start = performance.now();
37	for (let j = 0; j < 10; j++) sink += parser.selectHistory(input).omittedItems;
38	samples[label].push((performance.now() - start) / 10);
39	}
40	}
41	const stats = values => {
42	const sorted = values.toSorted((a, b) => a - b);
43	return { minMs: sorted[0], medianMs: sorted[10], maxMs: sorted.at(-1), samples: values };
44	};
45	console.log(JSON.stringify({ node: process.version, selectedItems: expected.messages.length, omittedItems: expected.omittedItems, baseline: stats(samples.baseline), patched: stats(samples.patched), sink }, null, 2));
46	JS

Implemented and verified with GPT-6.1 Sol (xhigh) and GPT-6 Astra through Codex in T3 Code. Final review: GPT-6 Astra (high), APPROVE with no blocking findings.

MmacroscopeappBotapproved these changes18d ago

MmacroscopeappBot18d ago

Approvability

Verdict: Approved at 2ea9560

Macroscope's review found this PR approvable — This one-line change avoids serializing history entries whose text alone exceeds the remaining budget, while the existing cost check still governs selection. It preserves handoff ordering and omission behavior without changing APIs, defaults, or product capabilities.

^{You can add or adjust custom eligibility rules. Learn more.}

CcoderabbitaiBot18d ago

Review in Change Stack →

No actionable comments were generated in the recent review.

ℹ️ Recent review info

⚙️ Run configuration

Source mode: Auto
Repository: pingdotgg/t3code
Branch: main
File: .coderabbit.config.ts
Commit: 22840ab
Layers: Repository TS config
Review profile: CHILL
Plan: Advanced
Run ID: 05448e36-f979-4eb8-a986-ff9bd4949eff

📥 Commits

Reviewing files that changed from the base of the PR and between 22840ab943b8418744b48223411e46ba4fdcef1a and 2ea956066e6f19ac7f9116ff4cf84b5c3e179135.

📒 Files selected for processing: 1

packages/provider-core/src/server/handoffBudget.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.

Walkthrough

The history selector now rejects a message when its text length exceeds the remaining budget, before calculating serialized cost. The existing serialized-cost check remains in place.

Changes: Handoff history selection

Layer / File(s)	Summary
Reject oversized history messages `packages/provider-core/src/server/handoffBudget.ts`	`tryAdd` checks message text length against the remaining budget before calculating serialized cost. The serialized-cost check remains in place.

Priority: External image Normal

Estimated code review effort: 2 (Simple) | ~5 minutes

Change: Refactor · Severity of issue fixed: Medium

Suggested reviewers: maria-rcks

Merge Risk: External image Minimal · up to 2ea95

The change preserves handoff selection behavior with no identified merge-blocking risk.

Pre-merge checks: ^Passed 4

Check name	Status	Explanation
Title check	Passed	The title clearly and concisely describes the performance change to history selection.
Linked Issues check	Passed	Issue `#23820` requires `selectHistory` to avoid encoding a candidate when its text length exceeds the remaining history budget. The change adds that check before either serialized-cost calculation in `…
Out of Scope Changes check	Passed	The whole-PR diff changes one condition in `packages/provider-core/src/server/handoffBudget.ts`. This change directly implements issue `#23820`. No unrelated changes appear in the diff.
Description check	Passed	The pull request includes all required sections. It clearly describes the problem, the one-line change, issue and approval context, focused verification results, benchmark limits, and agent usage.

Full details: Linked Issues check

Explanation

Issue #23820 requires selectHistory to avoid encoding a candidate when its text length exceeds the remaining history budget. The change adds that check before either serialized-cost calculation in handoffBudget.ts. The existing ContextHandoffBudget.test.ts covers oversized multilingual items, selection, omission counts, and final budget limits. The check uses a safe lower bound because UTF-8 JSON bytes cannot be fewer than the JavaScript UTF-16 code units in the original text.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

perf(server): skip oversized history during handoff selection#23821

Problem

Change

Scope and approval

Verification

Checks

Approvability