## Objective

Improve cache hit rate of LLM requests sent to ChatGPT subscription API.

## Solution

Set headers that help route request to the correct servers. This is
ChatGPT specific; plain OpenAI API doesn't require it.

## Testing

I run a benchmark which imitates a thread that makes 20 sequential tool
calls. I tried different cache route strategies. Results are below:


| Configuration                                   | Hit rate |
|-------------------------------------------------|---------:|
| No identity headers, no cache key               | 22.2%    |
| `thread-id` only, no cache key                  | 44.4%    |
| `thread-id` + matching cache key                | 42.1%    |
| Cache key only                                  | 53.6%    |
| Turn state + cache key                          | 47.4%    |
| `session-id` + matching cache key               | 100.0%   |
| `session-id` + `thread-id` + matching cache key | 97.4%    |



---

Release Notes:

- Improved ChatGPT subscription caching