# eval_cli: Add remote benchmark orchestration \(\#59802\) · gitcafe/zed

[View on GitCafe](https://git.cafe/gitcafe/zed/commit/10f501d7007c7a104456d8d0750e92eb0d25e328)

Repository: [gitcafe/zed](https://git.cafe/gitcafe/zed)

Visibility: public

Requested revision: 10f501d7007c7a104456d8d0750e92eb0d25e328

Requested commit: 10f501d7007c7a104456d8d0750e92eb0d25e328

Commit: 10f501d7007c7a104456d8d0750e92eb0d25e328

Tree: da91ad25d5cfb4d411782bd628e43025bf834c15

Author: Anant Goel

Committer: GitHub

## Message

```
eval_cli: Add remote benchmark orchestration (#59802)

Summary:

- Add the `zed-eval` Python CLI for Modal/Harbor/Pier benchmark
orchestration, including content-addressed remote builds, run/suite
management, reporting, rejudge, baseline, and cleanup workflows.
- Extend `eval-cli` for remote evals with provider/model overrides and
step/tool-call metrics in `result.json`.
- Add install/source-run helper scripts so `zed-eval` can be installed
or run from the checkout without manually setting `PYTHONPATH`.
- Harden the remote harness wrappers around exit-code preservation,
archive extraction, custom secret wiring, and Harbor/Pier option parity,
with regression coverage.

Testing:

- Using the CLI for two weeks
- `PYTHONPATH=crates/eval_cli python3 -m compileall -q
crates/eval_cli/zed_eval`
- `uv run --project crates/eval_cli/zed_eval python -m unittest discover
-s crates/eval_cli/zed_eval/tests`
- `bash -n crates/eval_cli/script/install-zed-eval
crates/eval_cli/script/zed-eval`
- `cargo check -p eval_cli`
- `cargo fmt --package eval_cli -- --check`
- `cargo test -p eval_cli --no-run`
- `./script/clippy -p eval_cli`

Release Notes:

- N/A
```

## Parents

- [2df74932bc6666e7a00e877d375dafb39a7a9273](https://git.cafe/gitcafe/zed/commit/2df74932bc6666e7a00e877d375dafb39a7a9273?format=markdown)

[Source at this commit](https://git.cafe/gitcafe/zed/tree/10f501d7007c7a104456d8d0750e92eb0d25e328?format=markdown)