# Standardize on u64 for token counts \(\#32869\) · gitcafe/zed

[View on GitCafe](https://git.cafe/gitcafe/zed/commit/5405c2c2d3d3939c8da20dd1b6f188681b5705c3)

Repository: [gitcafe/zed](https://git.cafe/gitcafe/zed)

Visibility: public

Requested revision: 5405c2c2d3d3939c8da20dd1b6f188681b5705c3

Requested commit: 5405c2c2d3d3939c8da20dd1b6f188681b5705c3

Commit: 5405c2c2d3d3939c8da20dd1b6f188681b5705c3

Tree: 690004ced3f58d350ffc03e51b13f6152f83165f

Author: Richard Feldman

Committer: GitHub

## Message

```
Standardize on u64 for token counts (#32869)

Previously we were using a mix of `u32` and `usize`, e.g. `max_tokens:
usize, max_output_tokens: Option<u32>` in the same `struct`.

Although [tiktoken](https://github.com/openai/tiktoken) uses `usize`,
token counts should be consistent across targets (e.g. the same model
doesn't suddenly get a smaller context window if you're compiling for
wasm32), and these token counts could end up getting serialized using a
binary protocol, so `usize` is not the right choice for token counts.

I chose to standardize on `u64` over `u32` because we don't store many
of them (so the extra size should be insignificant) and future models
may exceed `u32::MAX` tokens.

Release Notes:

- N/A
```

## Parents

- [a391d67366c32c6c82534bcbb9a9dab47edbea26](https://git.cafe/gitcafe/zed/commit/a391d67366c32c6c82534bcbb9a9dab47edbea26?format=markdown)

[Source at this commit](https://git.cafe/gitcafe/zed/tree/5405c2c2d3d3939c8da20dd1b6f188681b5705c3?format=markdown)