Correct UTF-16 saving and add heuristic encoding detection (#45243)

This commit fixes an issue where saving UTF-16 files resulted in UTF-8
bytes due to `encoding_rs` default behavior. It also introduces a
heuristic to detect BOM-less UTF-16 and binary files.

Changes:
- Manually implement UTF-16LE/BE encoding during file save to avoid
implicit UTF-8 conversion.
- Add `analyze_byte_content` to guess UTF-16LE/BE or Binary based on
null byte distribution.
- Prevent loading binary files as text by returning an error when binary
content is detected.

Special thanks to @CrazyboyQCD for pointing out the `encoding_rs`
behavior and providing the fix, and to @ConradIrwin for the suggestion
on the detection heuristic.

Closes #14654

Release Notes:

- (nightly only) Fixed an issue where saving files with UTF-16 encoding
incorrectly wrote them as UTF-8. Also improved detection for binary
files and BOM-less UTF-16.
1bc3fa8154Ichimura Tomoo committed on 12/19/2025, 6:18:20 PM· committed by GitHubparent2291631
3 files changedLine totals unavailable