# language_model: Make LanguageModel plain data served by its provider \(\#64750\) · gitcafe/zed

[View on GitCafe](https://git.cafe/gitcafe/zed/commit/975845875bb2ddbad8bb6153e87c570396b0fffb)

Repository: [gitcafe/zed](https://git.cafe/gitcafe/zed)

Visibility: public

Requested revision: 975845875bb2ddbad8bb6153e87c570396b0fffb

Requested commit: 975845875bb2ddbad8bb6153e87c570396b0fffb

Commit: 975845875bb2ddbad8bb6153e87c570396b0fffb

Tree: c3f44f6b6a40921d8d47662893519858f8dc8ae2

Author: Mikayla Maki

Committer: GitHub

## Message

```
language_model: Make LanguageModel plain data served by its provider (#64750)

`LanguageModel` used to be a trait object. Each provider implemented it
on a private struct that bundled the model's metadata with the client,
credentials, rate limiter and a snapshot of its config, and consumers
held `Arc<dyn LanguageModel>` or a `ConfiguredModel { provider, model }`
pair.

This makes `LanguageModel` plain data: a `Clone + Debug + PartialEq`
struct with the same getters the trait had, held by value everywhere.
`ConfiguredModel` is gone. Providers serve requests themselves:
`LanguageModelProvider` gains `stream_completion` (plus the text/tool
helpers, `count_input_tokens`, `compact` and `api_key`), each taking
`&LanguageModel`. Senders look the provider up with
`LanguageModelRegistry::provider_for_model` when they send.

Every provider now has one lookup that decides which models it offers
and with what config. `provided_models` and the defaults are built from
it, and each request resolves the model's current config from it by id.
There are no per-model or per-request objects. A model the provider no
longer offers fails with a new, non-retried
`LanguageModelCompletionError::ModelUnavailable` instead of being served
from a stale snapshot.

This prepares for opening provider-side sessions over an append-only
message log. With models as plain data and requests on the provider, a
session can be one generic type rather than one per provider, and
deciding whether a session can be reused becomes a value comparison.

## Suggested reading order

1. `crates/language_model/src/language_model.rs` (the struct and
provider trait), `registry.rs`, and `fake_provider.rs`.
2. One representative provider:
`crates/language_models/src/provider/anthropic.rs`.
3. Providers that deviate: `bedrock.rs`, `llama_cpp.rs`,
`open_router.rs`, `crates/copilot_chat`, `crates/language_models_cloud`,
`crates/openai_subscribed`, `crates/x_ai_subscribed`.
4. Callers: `crates/agent/src/thread.rs`, then `agent_ui`, `git_ui`,
`sidebar`. Most of the remaining diff is tests.

## Behavior changes worth reviewing

- Rate limiting is one concurrency limit per model id, shared across the
app, with a limit of 16. Before, each model instance had its own limit
of 4.
- `Thread::refresh_model` re-selects a thread's model by id when its
provider reports a state change, so capability changes (for example a
llama.cpp model finishing loading) reach the thread.
- Bedrock resolves auth once per request for both Converse and Mantle,
off the foreground thread, with the SDK's default 5s credential load
timeout. Expiring credentials (SSO, STS, credential processes) are
cached until shortly before expiry and dropped on any auth change;
credentials without an expiry are re-read every request, so credential
files rewritten by external tools take effect immediately. Mantle
credential resolution now goes through Zed's HTTP client, like Converse.
- Bedrock honors `AWS_BEARER_TOKEN_BEDROCK`, then `AWS_BEARER_TOKEN`
(even if empty), over AWS credentials unless an API key is configured in
Zed, matching the SDK. This now applies to Mantle too, so users with
IAM/profile auth and one of these variables set must unset it (not empty
it) for Mantle to use IAM/profile.
- A Bedrock Mantle model with the same id as a Converse model replaces
it, so each id is offered and served once.
- OpenRouter and Vercel AI Gateway report default-model metadata from
the same lookup they serve requests from.
- A request for a model that has left a provider's list (for example
after signing out of Copilot or Zed) returns `ModelUnavailable`. Commit
message generation shows this as an error instead of doing nothing.

## Tests

`FakeLanguageModel` is gone. `FakeLanguageModelProvider` serves any
model id and owns every request sent to it; tests answer a request by
naming its model (`send_last_text(&model, ...)`, `send_text(&model,
&request, ...)`), so requests to different models can't be confused.

Tested: `agent` (753 tests), `language_model`, `language_models`,
`language_models_cloud`, `copilot_chat`, `openai_subscribed` and
`x_ai_subscribed`, plus `cargo check --workspace --tests`. The
`agent_ui`, `git_ui` and `sidebar` test binaries type-check but were not
run locally because of a linker issue in my environment; CI will cover
them. Providers have not yet been exercised against live services.

Release Notes:

- N/A
```

## Parents

- [e52ab15eac51e5644da6a3c9e1fac9bb2330b000](https://git.cafe/gitcafe/zed/commit/e52ab15eac51e5644da6a3c9e1fac9bb2330b000?format=markdown)

[Source at this commit](https://git.cafe/gitcafe/zed/tree/975845875bb2ddbad8bb6153e87c570396b0fffb?format=markdown)