boot-service.log is never rotated and grows without limit#24755

opened18d agobyBbompus

What happened

When T3 runs as a boot service (systemd on Linux, launchd on macOS), userdata/logs/boot-service.log only ever grows. Neither manager rotates a file it appends to, and nothing in T3 trims it. A fork author reported it reaching 106 MB after offline spells (relay DNS and GitHub lookups failing and logging stack traces). On a Linux machine running a nightly it is 9.2 MB and still growing.

Diagnosis

The unit appends the service's stdout and stderr to this file (StandardOutput=append: and StandardError=append: in the systemd unit; StandardOutPath and StandardErrorPath in the launchd plist, see renderBootServiceUnit and renderBootServicePlist in apps/server/src/cloud/bootService.ts). The manager keeps that file open, so the rename-based rotation used for T3's other logs would leave the manager writing to the renamed file. I found no code that limits this file's size.

Steps to reproduce

Not deterministic: the size depends on how much the service logs. Install the boot service (t3 service install), let it run, and watch ls -l <T3 home>/userdata/logs/boot-service.log. It grows during any period where the server logs repeatedly, such as being offline.

Version

0.1.3-nightly.20261024.3630

Environment

Ubuntu 26.04, Linux 7.2.12 (x86_64), systemd 259. The macOS 106 MB figure is the fork author's report, not measured here.

Evidence

shell

1	-rw-rw-r-- 1 user user 9242562 Oct 24 22:47 <T3 home>/userdata/logs/boot-service.log

None found for this file. Searched open and closed issues and pull requests for "boot-service.log", "log rotation" and "service log grows". Open issues about T3 Connect connector logs (#16331, #24485) concern a different log.

Fix applied or workaround

None applied. I can open a pull request for this once triage confirms the direction. The branch exists: https://github.com/bompus/t3code/tree/fix/boot-service-log-cap . The unit now tells the service launcher which file the manager writes, and the launcher copies it to boot-service.log.1 and empties the live file in place past 16 MiB, at start and every 10 minutes. I ran it against a real systemd user unit appending to a file; the unit kept writing with no holes. Not tried on launchd.

Filed by

Claude Sonnet 5.5 (Claude Code in T3 Code)

Oonlydole9d ago

The issue only has a secondhand macOS figure, so here are measured numbers. The machine is macOS 27.2 on 0.1.7-nightly.20261102.4093, with the desktop app's background service running under launchd.

boot-service.log was 141.8 MB and 1.43 million lines. It had been appended since Oct 3 and never rotated.
89% of it was two warnings that log a full stack on every sweep: "automatic thread settlement skipped" (75.4 MB, 25,684 entries) and "thread branch pull request lookup failed" (50.2 MB, 17,402 entries).
They ran at about 360 an hour, day and night, from the 0.1.2 through the 0.1.6 nightlies. gh had no credential for one remote host during that time, and they stopped once I fixed that.
Each entry is about 2.9 KB, so that one failure wrote about 1 MB an hour. At the 10 MiB threshold in #24765, the live file plus its one archive would hold about 20 hours of it.

On a Raspberry Pi 5 (Debian 13 arm64, systemd user service) the file is smaller, with 36,797 lines since Oct 18. 1,126 of them are settle warnings for one thread in 19 hours. A queued message there was stuck behind a failed run.

The repeated stacks are the pattern from #19781. #24608 fixes it for quota pauses only.