# stage2: tokenizer: require null terminated source · gitcafe/zig

[View on GitCafe](https://git.cafe/gitcafe/zig/commit/3f680abbe2c4d2eeefd0eb73b8af25d1768e6ceb)

Repository: [gitcafe/zig](https://git.cafe/gitcafe/zig)

Visibility: public

Requested revision: 3f680abbe2c4d2eeefd0eb73b8af25d1768e6ceb

Requested commit: 3f680abbe2c4d2eeefd0eb73b8af25d1768e6ceb

Commit: 3f680abbe2c4d2eeefd0eb73b8af25d1768e6ceb

Tree: 06d67e5aa40cede2686d492445d4d32c2f69a33c

Author: Andrew Kelley

Committer: Andrew Kelley

## Message

```
stage2: tokenizer: require null terminated source

By requiring the source file to be null-terminated, we avoid extra
branching while simplifying the logic at the same time.

Running ast-check on a large zig source file (udivmodti4_test.zig),
master branch compared to this commit:
 * 4% faster wall clock
 * 7% fewer cache misses
 * 1% fewer branches

```

## Parents

- [8ce880ca753ce95138bf03d956cf363ea2dfde5a](https://git.cafe/gitcafe/zig/commit/8ce880ca753ce95138bf03d956cf363ea2dfde5a?format=markdown)

[Source at this commit](https://git.cafe/gitcafe/zig/tree/3f680abbe2c4d2eeefd0eb73b8af25d1768e6ceb?format=markdown)