# Pass routing headers to ChatGPT subscription API \(\#62556\) · gitcafe/zed

[View on GitCafe](https://git.cafe/gitcafe/zed/commit/a8fafdd7ee36fb3fb98ebbfe5d3be983301d9e74)

Repository: [gitcafe/zed](https://git.cafe/gitcafe/zed)

Visibility: public

Requested revision: a8fafdd7ee36fb3fb98ebbfe5d3be983301d9e74

Requested commit: a8fafdd7ee36fb3fb98ebbfe5d3be983301d9e74

Commit: a8fafdd7ee36fb3fb98ebbfe5d3be983301d9e74

Tree: 23e28496cef965ee46a9d7794c22f89f09786b49

Author: Oleksiy Syvokon

Committer: GitHub

## Message

```
Pass routing headers to ChatGPT subscription API (#62556)

## Objective

Improve cache hit rate of LLM requests sent to ChatGPT subscription API.

## Solution

Set headers that help route request to the correct servers. This is
ChatGPT specific; plain OpenAI API doesn't require it.

## Testing

I run a benchmark which imitates a thread that makes 20 sequential tool
calls. I tried different cache route strategies. Results are below:


| Configuration                                   | Hit rate |
|-------------------------------------------------|---------:|
| No identity headers, no cache key               | 22.2%    |
| `thread-id` only, no cache key                  | 44.4%    |
| `thread-id` + matching cache key                | 42.1%    |
| Cache key only                                  | 53.6%    |
| Turn state + cache key                          | 47.4%    |
| `session-id` + matching cache key               | 100.0%   |
| `session-id` + `thread-id` + matching cache key | 97.4%    |



---

Release Notes:

- Improved ChatGPT subscription caching
```

## Parents

- [c05e34637b4f7f100a688bf6ac71cb70877fc8ad](https://git.cafe/gitcafe/zed/commit/c05e34637b4f7f100a688bf6ac71cb70877fc8ad?format=markdown)

[Source at this commit](https://git.cafe/gitcafe/zed/tree/a8fafdd7ee36fb3fb98ebbfe5d3be983301d9e74?format=markdown)