# Make opening large files use less peak memory \(\#62748\) · gitcafe/zed

[View on GitCafe](https://git.cafe/gitcafe/zed/commit/00c0e96e769062e373203c62830f510fa121db76)

Repository: [gitcafe/zed](https://git.cafe/gitcafe/zed)

Visibility: public

Requested revision: 00c0e96e769062e373203c62830f510fa121db76

Requested commit: 00c0e96e769062e373203c62830f510fa121db76

Commit: 00c0e96e769062e373203c62830f510fa121db76

Tree: 17be764be47e71a319aca12027673b1cf75f1a68

Author: Ali

Committer: GitHub

## Message

```
Make opening large files use less peak memory (#62748)

# Objective
Opening a file laods it twice. `decode_file_text` builds the whole file
as a `String`, and that `String` stays alive alongside the finished rope
while `text::Buffer::new` copies it in. The `Vec` behind it grows by
doubling, so it also commits up to nearly the file's size again in
capacity it never uses.

Partially addresses #27283.

## Solution

Add `decode_file_text_to_rope`, which streams the file in 1 MB blocks
straight into a `Rope`, validating UTF-8 and normalizing line endings as
it goes. The file is never fully held as a `String`.

`LoadedFile::text` becomes a `Rope` carrying the `LineEnding` detected
before normalizing, so `buffer_store` calls `Buffer::new_normalized`.

## Testing

On a 729 MB SQL dump, peak memory fell 25% and CPU fell around 28%.
tested on 5950x, win11.

Release Notes:

- Improved memory use when opening large files, reducing peak memory
during load by roughly the size of the file itself.
```

## Parents

- [2040e0de59db363748e2299c8043beba7c870da5](https://git.cafe/gitcafe/zed/commit/2040e0de59db363748e2299c8043beba7c870da5?format=markdown)

[Source at this commit](https://git.cafe/gitcafe/zed/tree/00c0e96e769062e373203c62830f510fa121db76?format=markdown)