fix(server): keep large sparse checkouts on the fast checkpoint path#12154

Why

Fixes #12080.

Git already remembers enough about unchanged files to avoid reading them again. #10792 taught checkpoint capture to reuse that information, but two checks kept large sparse checkouts from benefiting:

• The file listing could exceed 16 MiB, which made T3 throw away the copied index and start over.
• Sparse checkout marks files it intentionally leaves off disk with skip-worktree. T3 treated those normal sparse-checkout flags as a reason to start over too.

So a checkpoint could reread a huge repository every turn and hit the 30-second timeout. Even if it finished, staging could fail just because an untracked file existed outside the selected sparse directories. Files left off disk could also look like deletions after the fallback.

What Changed

Keep the useful index information instead of throwing it away. The flag check now reads the complete output as a stream, so the buffer limit no longer decides whether reuse works. It can inspect sparse directory entries without expanding them into a full file listing.

Capture now accepts normal sparse-checkout flags and uses sparse-aware staging. Files intentionally absent stay in the snapshot. Present files, including nonignored additions outside the sparse rules, still get captured. The real index stays untouched, and a nested workspace still captures only its own changes.

The fallback rebuilds a sparse index for cone-mode checkouts. If it cannot safely rebuild a non-cone checkout's index, it fails rather than saving a snapshot with false deletions. The timestamp protections from #10792 stay in place.

Skipped file entries are checked against Git's sparse rules so a manually skipped in-cone deletion still gets captured. Git versions without add --sparse keep the previous capture path.

Results

On a generated 460,000-file repository with 230,000 files checked out, ten edits, and an 87.3 MB full listing:

That's about 4x faster. These are medians from three paired runs per workspace, alternating which version ran first. Every paired snapshot tree matched, and the real index stayed byte-for-byte identical. A separate capture with an out-of-cone untracked file succeeded in 12.03 s. These measurements include the fixes from automated review.

All measurements above use synthetic data on Linux/ext4. The 30-second subprocess timeout is unchanged. Large new or modified files still need to be read and hashed, so this does not fix every timeout covered by #3646.

70 focused tests pass, plus server typecheck, targeted lint, formatting, and diff checks. The regressions cover sparse snapshot contents, two consecutive captures, root and nested workspaces, safe fallback, and flags beyond the output cap. Clean-filter tests prove that truncation no longer causes unchanged files to be reread. Existing split-index and racy-timestamp tests still pass. A synthetic capture with a locally built Git 2.25.0 also passed and preserved the real index.

Checklist

• This PR is small and focused
• I explained what changed and why

Implemented and verified with GPT-6 Astra through OpenCode, model github-copilot/gpt-6-astra.

Summary by CodeRabbit

• New Features

  • Process output can now be streamed in full while buffered output remains capped, improving handling of large command results.
  • Large command results remain available through streamed byte chunks even when buffered output is truncated.

• Bug Fixes

  • Improved checkpoint creation for sparse-checkout repositories, including more reliable index inspection and preservation of modified content.
  • Added compatibility handling for Git versions without sparse-index support.
  • Added clearer handling when checkpoints cannot be rebuilt for non-cone sparse checkouts.