# Add new action to run agent eval \(\#29158\) · gitcafe/zed

[View on GitCafe](https://git.cafe/gitcafe/zed/commit/458ffaa1349e7e9e35fe497f2c6db4f123c7fc24)

Repository: [gitcafe/zed](https://git.cafe/gitcafe/zed)

Visibility: public

Requested revision: 458ffaa1349e7e9e35fe497f2c6db4f123c7fc24

Requested commit: 458ffaa1349e7e9e35fe497f2c6db4f123c7fc24

Commit: 458ffaa1349e7e9e35fe497f2c6db4f123c7fc24

Tree: 74d569656ac783111364d9cd1c2f10e2040d630e

Author: Nathan Sobo

Committer: GitHub

## Message

````
Add new action to run agent eval (#29158)

The old one wasn't linking, and
https://github.com/zed-industries/zed/pull/29081 has a bunch of merge
conflicts. Wanted to start simple/small.

## Todo

* [x] Remove low-signal examples
* [x] Make the eval run on a cron, on main, and on any PR with the
`run-eval` label
* [x] Noise in logs about failure to write settings
    ```
[2025-04-21T20:45:04Z ERROR settings] Failed to write settings to file
"/home/runner/.config/zed/settings.json"
    
       Caused by:
No such file or directory (os error 2) at path
"/home/runner/.config/zed/.tmpLewFEs"
    ```
* [x] `Agentic loop stalled`
(https://github.com/zed-industries/zed/actions/runs/14581044243/job/40897622894)
* [x] Make sure that events are recorded in snowflake
* [ ] Change judge criteria to be more explicit about meanings of scores

Release Notes:

- N/A

---------

Co-authored-by: Antonio Scandurra <me@as-cii.com>
Co-authored-by: Agus Zubiaga <hi@aguz.me>
Co-authored-by: Max Brunsfeld <maxbrunsfeld@gmail.com>
Co-authored-by: Thomas Mickley-Doyle <tmickleydoyle@gmail.com>
````

## Parents

- [b14356d1d3980aa4dc5c0d01e163b9581bbad967](https://git.cafe/gitcafe/zed/commit/b14356d1d3980aa4dc5c0d01e163b9581bbad967?format=markdown)

[Source at this commit](https://git.cafe/gitcafe/zed/tree/458ffaa1349e7e9e35fe497f2c6db4f123c7fc24?format=markdown)