The edit prediction, web search and completions endpoints in Cloud all use tokens called LlmApiToken. These were independently created, cached, and refreshed in three places: the cloud language model provider, the edit prediction store, and the cloud web search provider. Each held its own LlmApiToken instance, meaning three separate requests to get these tokens at startup / login and three redundant refreshes whenever the server signaled a token update was needed. We already had a global singleton reacting to the refresh signals: RefreshLlmTokenListener. It now holds a single LlmApiToken that all three services use, performs the refresh itself, and emits RefreshLlmTokenEvent only after the token is fresh. That event is used by the language model provider to re-fetch models after a refresh. The singleton is accessed only through `LlmApiToken::global()`. I have tested this manually, and it token acquisition and usage appear to be working fine. Edit: I've tested it with a long running session, and refresh seems to be working fine too. Release Notes: - N/A --------- Co-authored-by: Marshall Bowers <git@maxdeviant.com>
7 files changedLine totals unavailable