From 98c2b60820324afceb4180f10d274aece1362a66 Mon Sep 17 00:00:00 2001 From: Susana Ferreira Date: Mon, 1 Jun 2026 16:04:55 +0100 Subject: [PATCH] docs(docs/ai-coder/ai-gateway): document key failover for AI Gateway (#25893) Document the automatic key failover feature for AI Gateway, which allows configuring multiple centralized API keys per provider instance (OpenAI and Anthropic only). ## Changes - **`docs/ai-coder/ai-gateway/providers.md`**: Add "Key failover" section covering supported providers, configuration via the API (max 5 keys), and failover behavior (auth errors permanently disable a key until restart/reload, exhausted pool returns `429` or `502`). - **`docs/ai-coder/ai-gateway/auth.md`**: Add note in BYOK section clarifying that key failover is skipped when a user-supplied credential is present. > [!NOTE] > Generated by Coder Agents (by @ssncferreira) --- docs/ai-coder/ai-gateway/auth.md | 4 ++++ docs/ai-coder/ai-gateway/providers.md | 30 +++++++++++++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/docs/ai-coder/ai-gateway/auth.md b/docs/ai-coder/ai-gateway/auth.md index d7bb48aa85..e35e4f3d6b 100644 --- a/docs/ai-coder/ai-gateway/auth.md +++ b/docs/ai-coder/ai-gateway/auth.md @@ -92,6 +92,10 @@ When no user credential is present, AI Gateway falls back to the admin-configure This approach offers centralized keys as a default, while allowing individual users to bring their own key. +> [!NOTE] +> When a BYOK credential is present, [key failover](./providers.md#key-failover) +> is skipped. + Visit individual [client pages](./clients/index.md) for configuration details. ### Enable or disable BYOK diff --git a/docs/ai-coder/ai-gateway/providers.md b/docs/ai-coder/ai-gateway/providers.md index fae50e282f..084a3227db 100644 --- a/docs/ai-coder/ai-gateway/providers.md +++ b/docs/ai-coder/ai-gateway/providers.md @@ -166,6 +166,36 @@ single failure, which may resolve on the next change. See [Monitoring](./monitoring.md#provider-metrics) for the full metric list and sample alert queries. +## Key failover + +You can configure multiple centralized API keys for a single provider instance +so that AI Gateway automatically retries with the next key when one fails. This +is transparent to end users, and clients see no difference in behavior or need +any configuration changes. + +Key failover is supported for **OpenAI** and **Anthropic** providers. Amazon +Bedrock and GitHub Copilot do not support key failover. + +Multiple keys can be added per provider through the +[AI Providers API](../../reference/api/aiproviders.md). Each provider supports +a maximum of **5 keys**. + +### Failover behavior + +Every request starts with the first key in the list. If a key is rate-limited +or returns an authentication error, AI Gateway automatically retries the request +with the next available key. + +> [!WARNING] +> A key that fails with an authentication error (`401 Unauthorized` or +> `403 Forbidden`) is permanently disabled and will not be used again until the +> server is restarted or the provider configuration is reloaded. + +If all keys in the pool are exhausted, AI Gateway returns: + +- `429 Too Many Requests` when at least one key is rate-limited, with a `Retry-After` header set to the shortest cooldown across all keys. +- `502 Bad Gateway` when every key has failed permanently. + ## Bring Your Own Key A provider's configured credentials are the centralized default. When