A flooding terminal at its 8 MB history cap rewrites the whole history file about 4 times a second#20535

What happened

While a terminal produces continuous output, its persisted history file is rewritten in full roughly every 250 ms. I measured this with the production history class at the 8 MB cap. One join-plus-write cycle takes 7.6 ms and writes 7.8 MB. Sustained output at the cap therefore writes about 30 MB/s to disk indefinitely. Only the newly appended lines changed between writes. History that stays under the 5,000-line cap (short lines) pays proportionally less, about 0.5 ms per cycle at 0.5 MB.

Diagnosis

In apps/server/src/terminal/Manager.ts, persistWorker (a KeyedCoalescingWorker) sleeps for the 250 ms debounce added in #17183, then calls writeFileString(historyPath, request.history.value()). BoundedTerminalHistory.value() re-joins every retained chunk because append() clears the cached string. There is no append or delta path, so each cycle costs O(history size) of string building, encoding, and file rewriting. The worker is a single sequential fiber, which caps the aggregate rate at roughly 4 writes per second across terminals; the cost here is bytes per write, not write count.

I measured this with the real BoundedTerminalHistory class at the production caps (5,000 lines and 8 MB, Manager.ts:91-92). The fixture filled it to 7,800,000 bytes with 2,000-character lines. Then 20 cycles appended 1 KB, called value(), and wrote the result synchronously to a file. The median cycle took 7.59 ms (range 5.26-20.03 ms) on Node v26.12.0. The plain synchronous write stands in for the Effect file write. Both rewrite the whole string, so the join cost dominates either way.

Possible directions (maintainer's choice). Append only the new chunks and rewrite in full only when the head trim drops lines. Or raise the debounce when the history is large and still growing. The head trim means a pure append log needs periodic compaction. That is the main design point.

Limits. Long lines must bind the 8 MB byte cap before the 5,000-line cap, so this is a worst case. Build logs and progress output with long lines reach it. Short-line shells stay near 0.5 MB. I did not measure a live terminal flood end to end. Harness available on request.

Steps to reproduce

Fill a terminal's history toward the 8 MB cap (for example a build emitting long lines) and keep output flowing. Watch the history file's mtime and the server process write bytes. The file is fully rewritten about 4 times per second. The class-level harness in Diagnosis reproduces the join-plus-write cost without a live session.

Version

upstream/main 6cecd22789 (measured at 2577baf8b0; terminal Manager code read at 6cecd22789)

Environment

Linux, Node v26.12.0

Evidence

Related issues

#17183 added the 250 ms batching this builds on. #18428 (accepted) covers the separate unbounded live-event queue in the same manager; this report is only the history file rewrite.

Fix applied or workaround

None applied. I can open a pull request for this once triage confirms the direction. Not written yet; the approach I would measure is append-with-occasional-full-rewrite-on-trim, keeping the existing debounce and caps.

Filed by

Muse Wonder Full 1.0 via Muse Code in T3 Code, on behalf of bompus