git: Decode non-UTF-8 blobs for project diffs (#60821)
## Summary
Fixes #56449.
Related to #16965.
Zed’s Git panel and Project Diff build UI diffs from `language::Buffer`
diff bases loaded through the Git backend. Git blob loading previously
converted bytes with `String::from_utf8(...).ok()`, so legacy-encoded
blobs were treated as missing and the whole worktree file appeared newly
added.
This follows the same encoding path used for worktree buffers:
- move shared byte decoding and encoding into `language`
- keep Git blob, revision, and index APIs byte-oriented with `Vec<u8>`
- decode diff bases and index contents in `GitStore`, where
`language::Buffer`s are created
- encode index writes using the open buffer’s encoding and BOM so
partial staging does not rewrite the file as UTF-8
- keep worktree loading and saving on the same shared implementation
Regression coverage includes Windows-1251 decoding/encoding,
UTF-8/UTF-16 BOM preservation, raw Windows-1251 Git blob loading, and a
`BufferDiffSnapshot` assertion that a one-line CP1251 edit produces one
modified-line hunk instead of a full-file rewrite.
This does not run Git `textconv` commands. It fixes the reported
legacy-encoding case without executing repository-configured commands or
modifying working files on disk.
## Testing
- `cargo test -p language file_content::tests --locked`
- `cargo test -p git repository::tests::test_load_revisions --locked`
- `CARGO_INCREMENTAL=0 cargo test -p project
git_store::tests::test_decode_git_text_windows_1251_one_line_change
--locked`
- `CARGO_INCREMENTAL=0 cargo test -p project --test integration
test_restaging_hunk_after_optimistic_unstage --locked`
- `CARGO_INCREMENTAL=0 cargo check -p project --tests --locked`
- `CARGO_INCREMENTAL=0 cargo check -p git_ui --tests --locked`
- `cargo fmt --all --check`
- `git diff --check`
## Suggested .rules additions
- N/A
Release Notes:
- Fixed Git panel and Project Diff rendering for legacy-encoded text
files whose Git blobs are not valid UTF-8.
---------
Co-authored-by: Cole Miller <cole@zed.dev>
8968bf7808hvck committed on 8/17/2026, 1:47:11 AM· committed by GitHubparent098e440