mirror of
https://github.com/coder/coder.git
synced 2026-06-02 20:48:20 +00:00
docs(docs/ai-coder/ai-gateway): document key failover for AI Gateway (#25893)
Document the automatic key failover feature for AI Gateway, which allows configuring multiple centralized API keys per provider instance (OpenAI and Anthropic only). ## Changes - **`docs/ai-coder/ai-gateway/providers.md`**: Add "Key failover" section covering supported providers, configuration via the API (max 5 keys), and failover behavior (auth errors permanently disable a key until restart/reload, exhausted pool returns `429` or `502`). - **`docs/ai-coder/ai-gateway/auth.md`**: Add note in BYOK section clarifying that key failover is skipped when a user-supplied credential is present. > [!NOTE] > Generated by Coder Agents (by @ssncferreira)
This commit is contained in:
@@ -92,6 +92,10 @@ When no user credential is present, AI Gateway falls back to the admin-configure
|
|||||||
This approach offers centralized keys as a default,
|
This approach offers centralized keys as a default,
|
||||||
while allowing individual users to bring their own key.
|
while allowing individual users to bring their own key.
|
||||||
|
|
||||||
|
> [!NOTE]
|
||||||
|
> When a BYOK credential is present, [key failover](./providers.md#key-failover)
|
||||||
|
> is skipped.
|
||||||
|
|
||||||
Visit individual [client pages](./clients/index.md) for configuration details.
|
Visit individual [client pages](./clients/index.md) for configuration details.
|
||||||
|
|
||||||
### Enable or disable BYOK
|
### Enable or disable BYOK
|
||||||
|
|||||||
@@ -166,6 +166,36 @@ single failure, which may resolve on the next change. See
|
|||||||
[Monitoring](./monitoring.md#provider-metrics) for the full metric list
|
[Monitoring](./monitoring.md#provider-metrics) for the full metric list
|
||||||
and sample alert queries.
|
and sample alert queries.
|
||||||
|
|
||||||
|
## Key failover
|
||||||
|
|
||||||
|
You can configure multiple centralized API keys for a single provider instance
|
||||||
|
so that AI Gateway automatically retries with the next key when one fails. This
|
||||||
|
is transparent to end users, and clients see no difference in behavior or need
|
||||||
|
any configuration changes.
|
||||||
|
|
||||||
|
Key failover is supported for **OpenAI** and **Anthropic** providers. Amazon
|
||||||
|
Bedrock and GitHub Copilot do not support key failover.
|
||||||
|
|
||||||
|
Multiple keys can be added per provider through the
|
||||||
|
[AI Providers API](../../reference/api/aiproviders.md). Each provider supports
|
||||||
|
a maximum of **5 keys**.
|
||||||
|
|
||||||
|
### Failover behavior
|
||||||
|
|
||||||
|
Every request starts with the first key in the list. If a key is rate-limited
|
||||||
|
or returns an authentication error, AI Gateway automatically retries the request
|
||||||
|
with the next available key.
|
||||||
|
|
||||||
|
> [!WARNING]
|
||||||
|
> A key that fails with an authentication error (`401 Unauthorized` or
|
||||||
|
> `403 Forbidden`) is permanently disabled and will not be used again until the
|
||||||
|
> server is restarted or the provider configuration is reloaded.
|
||||||
|
|
||||||
|
If all keys in the pool are exhausted, AI Gateway returns:
|
||||||
|
|
||||||
|
- `429 Too Many Requests` when at least one key is rate-limited, with a `Retry-After` header set to the shortest cooldown across all keys.
|
||||||
|
- `502 Bad Gateway` when every key has failed permanently.
|
||||||
|
|
||||||
## Bring Your Own Key
|
## Bring Your Own Key
|
||||||
|
|
||||||
A provider's configured credentials are the centralized default. When
|
A provider's configured credentials are the centralized default. When
|
||||||
|
|||||||
Reference in New Issue
Block a user