docs(docs/ai-coder/ai-gateway): document key failover for AI Gateway (#25893)

Document the automatic key failover feature for AI Gateway, which allows
configuring multiple centralized API keys per provider instance (OpenAI
and Anthropic only).

## Changes

- **`docs/ai-coder/ai-gateway/providers.md`**: Add "Key failover"
section covering supported providers, configuration via the API (max 5
keys), and failover behavior (auth errors permanently disable a key
until restart/reload, exhausted pool returns `429` or `502`).
- **`docs/ai-coder/ai-gateway/auth.md`**: Add note in BYOK section
clarifying that key failover is skipped when a user-supplied credential
is present.

> [!NOTE]
> Generated by Coder Agents (by @ssncferreira)
This commit is contained in:
Susana Ferreira
2026-06-01 16:04:55 +01:00
committed by GitHub
parent 9fc12afdaa
commit 98c2b60820
2 changed files with 34 additions and 0 deletions
+4
View File
@@ -92,6 +92,10 @@ When no user credential is present, AI Gateway falls back to the admin-configure
This approach offers centralized keys as a default,
while allowing individual users to bring their own key.
> [!NOTE]
> When a BYOK credential is present, [key failover](./providers.md#key-failover)
> is skipped.
Visit individual [client pages](./clients/index.md) for configuration details.
### Enable or disable BYOK
+30
View File
@@ -166,6 +166,36 @@ single failure, which may resolve on the next change. See
[Monitoring](./monitoring.md#provider-metrics) for the full metric list
and sample alert queries.
## Key failover
You can configure multiple centralized API keys for a single provider instance
so that AI Gateway automatically retries with the next key when one fails. This
is transparent to end users, and clients see no difference in behavior or need
any configuration changes.
Key failover is supported for **OpenAI** and **Anthropic** providers. Amazon
Bedrock and GitHub Copilot do not support key failover.
Multiple keys can be added per provider through the
[AI Providers API](../../reference/api/aiproviders.md). Each provider supports
a maximum of **5 keys**.
### Failover behavior
Every request starts with the first key in the list. If a key is rate-limited
or returns an authentication error, AI Gateway automatically retries the request
with the next available key.
> [!WARNING]
> A key that fails with an authentication error (`401 Unauthorized` or
> `403 Forbidden`) is permanently disabled and will not be used again until the
> server is restarted or the provider configuration is reloaded.
If all keys in the pool are exhausted, AI Gateway returns:
- `429 Too Many Requests` when at least one key is rate-limited, with a `Retry-After` header set to the shortest cooldown across all keys.
- `502 Bad Gateway` when every key has failed permanently.
## Bring Your Own Key
A provider's configured credentials are the centralized default. When