This adds support for running self-hosted Sweep Next Edit edit
prediction models through Zed's OpenAI-compatible provider. The target
use case is a local deployment based on
[sweepai/sweep-next-edit-1.5B](https://huggingface.co/sweepai/sweep-next-edit-1.5B),
using the rewrite-window prompting approach described in [OSS Next
Edit](https://blog.sweep.dev/posts/oss-next-edit). Personal user testing
shows impressive quality from this small edit prediction model.
This is valuable because it allows more flexibility for Zed users to
self-host state of the art next edit prediction models!
This PR addresses the self-hosted workflow discussed in
https://github.com/zed-industries/zed/discussions/50929.
Note that the self-hosted model path needed more than just a new
dropdown value. Zed has to build the rewrite-window prompt shape the
model expects, map the rewritten window back into anchored edits, and
make `prompt_format: "infer"` work with the filename-style model
identifiers that llama-server local server reports.
The main changes are:
- add a `Sweep` prompt format for OpenAI-compatible edit prediction
providers and route it to a dedicated `SweepPrompt` model path
- build Sweep rewrite-window prompts from the active cursor window,
recent change history, and related file excerpts
- convert rewrite responses back into anchored edits and suppress no-op
rewrites
- add fixed cursor-window extraction used by the Sweep rewrite prompt
path
- recognize `sweep-next-edit` model names during `infer`, including
filename-style identifiers such as
`sweepai_sweep-next-edit-1.5B_sweep-next-edit-1.5b.q8_0.v2.gguf`
- reject reserved Sweep prompt tokens before sending malformed requests
to a self-hosted server
- update the docs to describe the actual Sweep rewrite-window prompt
format
Local setup used for manual verification:
```sh
llama-server \
--hf-repo sweepai/sweep-next-edit-1.5B \
--hf-file sweep-next-edit-1.5b.q8_0.v2.gguf \
--ctx-size 8192 \
--host 127.0.0.1 \
--port 8080
```
```json
{
"edit_predictions": {
"provider": "open_ai_compatible_api",
"open_ai_compatible_api": {
"api_url": "http://127.0.0.1:8080/v1/completions",
"model": "sweepai_sweep-next-edit-1.5B_sweep-next-edit-1.5b.q8_0.v2.gguf",
"prompt_format": "infer",
"max_output_tokens": 512
}
}
}
```
The request that produced this PR included screenshots of the
OpenAI-compatible provider configuration and the prompt format dropdown
with `Sweep` selected.
Tests added in this branch:
- `test_sweep_prompt_format_routes_to_sweep_prompt_model`
- `test_fixed_line_window_around_cursor_start_middle_and_end`
-
`test_sweep_prompt_request_prediction_diffs_rewritten_window_into_anchored_edits`
-
`test_sweep_prompt_request_prediction_returns_none_for_identical_rewrite`
-
`test_original_window_for_current_window_uses_latest_pre_edit_snapshot`
-
`test_original_window_for_current_window_returns_none_without_matching_history`
-
`test_recent_change_block_from_event_formats_original_and_updated_sections`
Verification run locally:
- `cargo test -p zed sweep_prompt_format_routes`
- `cargo test -p zed subscribe_uses_stale_provider_config`
- `cargo test -p edit_prediction sweep_prompt`
- `cargo test -p edit_prediction
fixed_line_window_around_cursor_start_middle_and_end`
- `cargo test -p edit_prediction
test_sweep_prompt_request_prediction_diffs_rewritten_window_into_anchored_edits`
- `cargo test -p edit_prediction
test_sweep_prompt_request_prediction_returns_none_for_identical_rewrite`
- `./script/clippy -p zed -p edit_prediction -p settings_content`
Release Notes:
- Added support for self-hosted Sweep Next Edit models in
OpenAI-compatible edit predictions, including the `sweep` prompt format
and `infer` detection for `sweep-next-edit` model names.
---------
Co-authored-by: Ben Kunkle <ben@zed.dev>8 files changedLine totals unavailable