# zeta2: Compute diff-aware chrF metric \(\#43485\) · gitcafe/zed

[View on GitCafe](https://git.cafe/gitcafe/zed/commit/99d7b2fa1de8c9d5b6d33db0d4c90ba1571e2f8e)

Repository: [gitcafe/zed](https://git.cafe/gitcafe/zed)

Visibility: public

Requested revision: 99d7b2fa1de8c9d5b6d33db0d4c90ba1571e2f8e

Requested commit: 99d7b2fa1de8c9d5b6d33db0d4c90ba1571e2f8e

Commit: 99d7b2fa1de8c9d5b6d33db0d4c90ba1571e2f8e

Tree: 26c0369858a1cf2271c16558bd90af6783f82158

Author: Oleksiy Syvokon

Committer: GitHub

## Message

```
zeta2: Compute diff-aware chrF metric (#43485)

Zeta evals now include a character n-gram metric adapted for multi-edit diffs (“delta chrF”). It works as follows:

1. Reconstruct the original, golden (expected), and actual texts from unified diffs.
   - "original": the text before any edits
   - "golden": the text after applying the expected edits
   - "actual": the text after applying the actual edits

2. Compute n-gram count deltas between original→golden and original→actual.
   - n-grams are computed as in chrF (max n=6, whitespace ignored).

3. Compare these deltas to assess how well the actual edits match the expected edits.
   - As in standard chrF, classify n-grams as true positives, false positives, and false negatives, and report the F-beta score with beta=2.

Release Notes:

- N/A

```

## Parents

- [91400e74897898eeb5026d5bea5b35666c773c96](https://git.cafe/gitcafe/zed/commit/91400e74897898eeb5026d5bea5b35666c773c96?format=markdown)

[Source at this commit](https://git.cafe/gitcafe/zed/tree/99d7b2fa1de8c9d5b6d33db0d4c90ba1571e2f8e?format=markdown)