shishantbiswas/coder
1
0
Fork 0
mirror of https://github.com/coder/coder.git synced 2026-06-02 20:48:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: document AI providers seeding mechanism & support for new types (#25855)

Browse Source 
Adds a new **Provider Configuration** reference page (`providers.md`) covering:

- The migration from environment-variable-based provider config to database-backed management introduced in v2.34, including the one-time seeding behavior and deprecation of `CODER_AI_GATEWAY_PROVIDER_<N>_*` and related flags
- All supported provider types (`openai`, `anthropic`, `bedrock`, `copilot`, `azure`, `google`, `openrouter`, `vercel`, `openai-compat`) with setup notes for each
- Provider lifecycle statuses (`enabled`, `disabled`, `error`) and their effect on request handling
- Reload behavior and how configuration changes apply without restarting `coderd`
- Bring Your Own Key (BYOK) and failure mode reference table

Updates **Setup** (`setup.md`) to replace the environment-variable-based provider configuration instructions with dashboard-driven steps (Add provider form, provider list, edit/disable flow), referencing the new `providers.md` page for deeper detail. Screenshots of the provider list, add, and edit forms are included.

Adds a **Provider metrics** section to **Monitoring** (`monitoring.md`) documenting the `coder_aibridged_*` and `coder_aibridgeproxyd_*` Prometheus metrics for provider status and reload timestamps, along with two suggested PromQL alert queries.
This commit is contained in:
Danny Kopping
2026-06-01 15:33:37 +02:00
committed by GitHub
parent d0fa9ff986
commit f9937a8931
7 changed files with 260 additions and 216 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ai-coder/ai-gateway/monitoring.md
+40
View File
@@ -15,6 +15,46 @@ We provide an example Grafana dashboard that you can import as a starting point
These logs and metrics can be used to determine usage patterns, track costs, and evaluate tooling adoption. These logs and metrics can be used to determine usage patterns, track costs, and evaluate tooling adoption.
## Provider metrics
`aibridged` (the in-process daemon) and `aibridgeproxyd` (the external
proxy) each export Prometheus metrics describing the configured
provider pool and its reload loop. See
[Provider Configuration](./providers.md) for the lifecycle these
metrics describe.
| Metric | Type | Labels | Purpose |
|------------------------------------------------------------------------|---------|--------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------|
| `coder_aibridged_provider_info` | gauge | `provider_name`, `provider_type`, `status` | One series per configured provider. Value is always `1`; the `status` label (`enabled`, `disabled`, `error`) carries the alertable signal. |
| `coder_aibridged_providers_last_reload_timestamp_seconds` | gauge | | Unix timestamp of the last reload attempt, success or failure. |
| `coder_aibridged_providers_last_reload_success_timestamp_seconds` | gauge | | Unix timestamp of the last reload that successfully refreshed the pool. |
| `coder_aibridgeproxyd_provider_info` | gauge | `provider_name`, `provider_type`, `status` | Same shape as `aibridged_provider_info` but reported by the external proxy. |
| `coder_aibridgeproxyd_providers_last_reload_timestamp_seconds` | gauge | | Last reload attempt timestamp in `aibridgeproxyd`. |
| `coder_aibridgeproxyd_providers_last_reload_success_timestamp_seconds` | gauge | | Last successful reload timestamp in `aibridgeproxyd`. |
| `coder_aibridgeproxyd_connect_sessions_total` | counter | `type` (`mitm`, `tunneled`) | CONNECT sessions established by the proxy. |
| `coder_aibridgeproxyd_mitm_requests_total` | counter | `provider` | MITM requests handled. |
| `coder_aibridgeproxyd_inflight_mitm_requests` | gauge | `provider` | In-flight MITM requests. |
| `coder_aibridgeproxyd_mitm_responses_total` | counter | `code`, `provider` | MITM responses by HTTP status code. |
### Suggested alerts
Alert on any provider entering a non-`enabled` status:
```promql
sum by (provider_name, status) (coder_aibridged_provider_info{status!="enabled"}) > 0
```
Alert when the reload loop is firing but failing to refresh the pool
for longer than a few minutes:
```promql
(coder_aibridged_providers_last_reload_timestamp_seconds
- coder_aibridged_providers_last_reload_success_timestamp_seconds) > 300
```
Repeat the same query against `coder_aibridgeproxyd_*` if you run the
external proxy.
## Structured Logging ## Structured Logging
AI Bridge can emit structured logs for every interception event to your AI Bridge can emit structured logs for every interception event to your
docs/ai-coder/ai-gateway/providers.md
+184
View File
@@ -0,0 +1,184 @@
# Provider Configuration
> [!NOTE]
> AI Gateway requires the [AI Governance Add-On](../ai-governance.md).
Providers are deployment-scoped and managed from the dashboard or the
[AI Providers API](../../reference/api/aiproviders.md). See
[Setup](./setup.md#configure-providers) for the steps to add, edit, and
disable a provider.
This page covers the provider types AI Gateway supports, the setup
considerations for each, how a provider's lifecycle affects request
handling, and how to monitor providers.
## Database management of providers
> [!NOTE]
> Since v2.34, provider environment variables and flags, including
> `CODER_AI_GATEWAY_PROVIDER_<N>_*`, `CODER_AI_GATEWAY_OPENAI_*`,
> `CODER_AI_GATEWAY_ANTHROPIC_*`, and their `--aibridge/ai-gateway-*`
> equivalents, are deprecated. Provider configuration is now stored in
> the database, and any environment variables set on startup are used to
> seed it.
>
> This is a once-off operation. The environment variables have no effect
> once seeding has completed.
>
> **Any changes to the provider environment variables after seeding will
> cause the server to fail to start, to prevent operators from updating a
> configuration that is ineffectual.**
>
> The environment variables can be safely removed once seeding has
> completed. Visit `https://<your-coder-host>/ai/settings` to see which
> providers have been seeded.
After seeding, manage providers through the dashboard or API. A provider
that has been edited or removed there is not recreated or overwritten
from the environment on the next restart.
## Provider types
AI Gateway speaks two upstream API formats: the **OpenAI** format
(Chat Completions and Responses) and the **Anthropic** format
(Messages). Every provider type maps to one of these.
| Type | API format | Setup notes |
|-----------------|------------|-------------------------------------------------------------------|
| `openai` | OpenAI | Native OpenAI, or any OpenAI-compatible endpoint via the base URL |
| `anthropic` | Anthropic | Native Anthropic, or an Anthropic-compatible broker |
| `bedrock` | Anthropic | Anthropic models hosted on AWS Bedrock; authenticates via AWS |
| `copilot` | OpenAI | GitHub Copilot; authenticates via each user's GitHub OAuth token |
| `azure` | OpenAI | OpenAI-compatible endpoint only |
| `google` | OpenAI | OpenAI-compatible endpoint only |
| `openrouter` | OpenAI | OpenAI-compatible endpoint only |
| `vercel` | OpenAI | OpenAI-compatible endpoint only |
| `openai-compat` | OpenAI | Generic OpenAI-compatible endpoint |
`azure`, `google`, `openrouter`, `vercel`, and `openai-compat` are
supported only as OpenAI-compatible endpoints: AI Gateway sends them
OpenAI-format requests, so each must expose an OpenAI-compatible API at
its base URL. They have no provider-specific integration beyond that.
### OpenAI
Set the base URL to the upstream endpoint and provide an API key. The
default `https://api.openai.com/v1/` targets the native OpenAI service;
point it at any OpenAI-compatible endpoint (for example, a hosted proxy
or LiteLLM deployment) when needed.
If you create an [OpenAI key](https://platform.openai.com/api-keys)
with minimal privileges, this is the minimum required set:
![List Models scope should be set to "Read", Model Capabilities set to "Request"](../../images/aibridge/openai_key_scope.png)
### Anthropic
Set the base URL and provide an API key. The default
`https://api.anthropic.com/` targets Anthropic's public API; override it
for Anthropic-compatible brokers.
Anthropic does not allow [API keys](https://console.anthropic.com/settings/keys)
to have restricted permissions at the time of writing (June 2026).
### Amazon Bedrock
Bedrock providers serve Anthropic models hosted on AWS and authenticate
with AWS credentials rather than a registered API key. Configure:
- A **region** (or a full base URL when routing through a proxy or a
non-standard endpoint that does not follow the
`https://bedrock-runtime.<region>.amazonaws.com` format).
- The **model** and **small fast model** identifiers.
Do not attach API keys to a Bedrock provider.
AI Gateway resolves AWS credentials one of two ways:
- **AWS SDK default credential chain (recommended).** When no explicit
credentials are configured, the AWS SDK resolves them automatically
from the environment: IAM Roles (instance profiles, IRSA, ECS task
roles), shared config files, environment variables, SSO, and more.
Attaching an IAM Role to the compute running Coder follows
[AWS best practices](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html)
for temporary credentials. The role must permit `bedrock:InvokeModel`
and `bedrock:InvokeModelWithResponseStream` for the configured models.
- **Static credentials.** Provide an access key and secret for an IAM
user with the same Bedrock permissions.
### GitHub Copilot
GitHub Copilot offers three plans: Individual, Business, and Enterprise,
each with its own API endpoint. Add one `copilot` provider per plan your
organization uses, setting the base URL accordingly:
| Plan | Base URL |
|------------|--------------------------------------------|
| Individual | `https://api.individual.githubcopilot.com` |
| Business | `https://api.business.githubcopilot.com` |
| Enterprise | `https://api.enterprise.githubcopilot.com` |
Copilot providers authenticate with each user's request-time GitHub
OAuth token, so do not attach API keys. For client-side setup (proxy,
certificates, IDE configuration), see
[GitHub Copilot client configuration](./clients/copilot.md).
### OpenAI-compatible providers
Azure-hosted OpenAI, Google, OpenRouter, Vercel, and any other
OpenAI-compatible service are configured with the matching type (or the
generic `openai-compat`), the provider's OpenAI-compatible base URL, and
an API key.
> [!NOTE]
> See the [Supported APIs](./reference.md#supported-apis) section for
> precise endpoint coverage and interception behavior.
## Provider lifecycle
Every provider carries an explicit status, surfaced through the
[`provider_info`](./monitoring.md#provider-metrics) metric and the API:
| Status | Meaning | Effect on requests |
|------------|-------------------------------------------------------------------------------|--------------------------------------------------|
| `enabled` | Configuration is valid and the provider is serving traffic | Requests are proxied to the upstream |
| `disabled` | The provider exists but has been turned off | Requests are rejected with a non-retryable error |
| `error` | The provider is enabled but cannot be built (missing credentials, bad config) | Requests fail; the error is surfaced in metrics |
Disabling a provider does not delete it, its credentials, or its
historical interception data. Re-enabling restores it to service.
## Monitoring and reloads
Provider configuration changes take effect automatically, without
restarting `coderd`. AI Gateway records the timestamp of each reload
attempt and each successful reload, exposed as Prometheus metrics:
- `coder_aibridged_providers_last_reload_timestamp_seconds`
- `coder_aibridged_providers_last_reload_success_timestamp_seconds`
If you run the [external proxy](./ai-gateway-proxy/index.md), it exposes
the same pair under the `coder_aibridgeproxyd_` prefix.
A growing gap between the attempt and success timestamps means reloads
are firing but failing to apply. Alert on that gap rather than on a
single failure, which may resolve on the next change. See
[Monitoring](./monitoring.md#provider-metrics) for the full metric list
and sample alert queries.
## Bring Your Own Key
A provider's configured credentials are the centralized default. When
Bring Your Own Key (BYOK) is enabled, a user's own credential takes
precedence over the provider's for that user's requests, and AI Gateway
falls back to the provider credentials when the user has none. See
[Authentication](./auth.md#bring-your-own-key-byok) for the BYOK flow
and how to enable or disable it.
## Failure modes
| Symptom | Likely cause | Corrective action |
|------------------------------------------------|------------------------------------------------------------|------------------------------------------|
| Startup fails referencing an existing provider | Env config drifted from a provider already in the database | Remove the provider env vars and restart |
| Provider returns errors with no upstream call | The provider is `disabled` or in `error` status | Consult the server logs for details |
| Configuration changes not taking effect | Reloads are firing but failing to apply | Consult the server logs for details |
docs/ai-coder/ai-gateway/setup.md
+30 -216
View File
@@ -2,20 +2,17 @@
AI Gateway runs inside the Coder control plane (`coderd`), requiring no separate compute to deploy or scale. Once enabled, `coderd` runs the `aibridged` in-memory and brokers traffic to your configured AI providers on behalf of authenticated users. AI Gateway runs inside the Coder control plane (`coderd`), requiring no separate compute to deploy or scale. Once enabled, `coderd` runs the `aibridged` in-memory and brokers traffic to your configured AI providers on behalf of authenticated users.
**Required**:
1. The [AI Governance Add-On](../ai-governance.md) license.
1. Feature must be [enabled](#activation) using the server flag
1. One or more [providers](#configure-providers) API key(s) must be configured
> [!NOTE] > [!NOTE]
> AI Gateway environment variables and CLI flags have migrated to the new > Since v2.34, provider environment variables and flags are deprecated.
> `CODER_AI_GATEWAY_*` and `--ai-gateway-*` naming scheme. The earlier > Provider configuration is now stored in the database, and any
> `CODER_AIBRIDGE_*` and `--aibridge-*` names continue to work as aliases. > environment variables set on startup are used to seed it once. See
> [Database management of providers](./providers.md#database-management-of-providers)
> for details.
## Activation ## Activation
You will need to enable AI Gateway explicitly: AI Gateway must be enabled in deployment config before users can authenticate
to it.
```sh ```sh
export CODER_AI_GATEWAY_ENABLED=true export CODER_AI_GATEWAY_ENABLED=true
@@ -24,224 +21,41 @@ coder server
coder server --ai-gateway-enabled=true coder server --ai-gateway-enabled=true
``` ```
_AI Gateway is enabled by default as of v2.34._
## Configure Providers ## Configure Providers
AI Gateway proxies requests to upstream LLM APIs. Configure at least one provider before exposing AI Gateway to end users. Configure at least one provider before exposing AI Gateway to end users.
<div class="tabs"> Providers are deployment-scoped. Add them from the dashboard or the
[AI Providers API](../../reference/api/aiproviders.md). Changes take effect
without restarting `coderd`.
### OpenAI ### Dashboard
Set the following when routing [OpenAI-compatible](https://coder.com/docs/reference/cli/server#--ai-gateway-openai-key) traffic through AI Gateway: 1. Navigate to **Admin settings** > **AI**
1. Select **Providers**
1. Click **Add provider**
1. Select the provider type
1. Enter a unique lowercase name, the upstream endpoint, and the credentials
1. Save the provider
- `CODER_AI_GATEWAY_OPENAI_KEY` or `--ai-gateway-openai-key` Each provider gets its own AI Gateway route at
- `CODER_AI_GATEWAY_OPENAI_BASE_URL` or `--ai-gateway-openai-base-url` `/api/v2/aibridge/<provider-name>/`.
The default base URL (`https://api.openai.com/v1/`) works for the native OpenAI service. Point the base URL at your preferred OpenAI-compatible endpoint (for example, a hosted proxy or LiteLLM deployment) when needed.
If you'd like to create an [OpenAI key](https://platform.openai.com/api-keys) with minimal privileges, this is the minimum required set:
![List Models scope should be set to "Read", Model Capabilities set to "Request"](../../images/aibridge/openai_key_scope.png)
### Anthropic
Set the following when routing [Anthropic-compatible](https://coder.com/docs/reference/cli/server#--ai-gateway-anthropic-key) traffic through AI Gateway:
- `CODER_AI_GATEWAY_ANTHROPIC_KEY` or `--ai-gateway-anthropic-key`
- `CODER_AI_GATEWAY_ANTHROPIC_BASE_URL` or `--ai-gateway-anthropic-base-url`
The default base URL (`https://api.anthropic.com/`) targets Anthropic's public API. Override it for Anthropic-compatible brokers.
Anthropic does not allow [API keys](https://console.anthropic.com/settings/keys) to have restricted permissions at the time of writing (Nov 2025).
### Amazon Bedrock
Set the following when routing [Amazon Bedrock](https://coder.com/docs/reference/cli/server#--ai-gateway-bedrock-region) traffic through AI Gateway:
**Required:**
- `CODER_AI_GATEWAY_BEDROCK_REGION` or `--ai-gateway-bedrock-region`.
Alternatively, set `CODER_AI_GATEWAY_BEDROCK_BASE_URL` or `--ai-gateway-bedrock-base-url` to a full URL (e.g., when routing through a proxy between AI Gateway and AWS Bedrock or using a non-standard endpoint that doesn't follow the `https://bedrock-runtime.<region>.amazonaws.com` format).
If both are set, `CODER_AI_GATEWAY_BEDROCK_BASE_URL` takes precedence.
- `CODER_AI_GATEWAY_BEDROCK_MODEL` or `--ai-gateway-bedrock-model`
- `CODER_AI_GATEWAY_BEDROCK_SMALL_FAST_MODEL` or `--ai-gateway-bedrock-small-fastmodel`
> [!NOTE] > [!NOTE]
> These Bedrock settings configure AI Gateway only. To configure Bedrock as an > Provider names must be unique and use lowercase, hyphen-separated identifiers
> Agents provider, see [Configuring AWS Bedrock](../agents/models.md#configuring-aws-bedrock). > such as `anthropic-corp` or `azure-openai`. Once deleted, another provider
> may reuse the name.
**Optional:** ![AI Providers list page](../../images/aibridge/providers-list.png)
- `CODER_AI_GATEWAY_BEDROCK_ACCESS_KEY` or `--ai-gateway-bedrock-access-key` ![Add Anthropic provider form](../../images/aibridge/provider-add-anthropic.png)
- `CODER_AI_GATEWAY_BEDROCK_ACCESS_KEY_SECRET` or `--ai-gateway-bedrock-access-key-secret`
#### Authentication Open an existing provider to rotate credentials, update its endpoint, or
disable it without restarting `coderd`.
AI Gateway supports two credential configuration paths: ![Edit Anthropic provider form](../../images/aibridge/provider-edit-anthropic.png)
##### AWS SDK default credential chain (recommended)
When no credentials are set in AI Gateway config, the AWS SDK resolves them automatically from the environment.
This includes IAM Roles (instance profiles, IRSA, ECS task roles), shared config files, environment variables, SSO, and more.
**IAM Roles are the recommended approach** when AI Gateway runs on AWS infrastructure.
Attach an IAM Role with Bedrock permissions to the compute running AI Gateway (EC2 instance, EKS pod via IRSA, or ECS task), no credentials need to be configured in AI Gateway itself.
The IAM Role must have permission to invoke the Bedrock models configured for AI Gateway (`bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream`).
See [Amazon Bedrock identity-based policy examples](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html) for policy examples,
and [AWS IAM role creation](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-service.html) for general guidance on attaching roles to AWS services.
This aligns with [AWS best practices](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html) for using temporary credentials instead of long-lived access keys.
##### Static credentials
For deployments when explicit credentials are preferred, provide an access key and secret for an IAM User:
1. **Choose a region** where you want to use Bedrock.
2. **Generate API keys** in the [AWS Bedrock console](https://us-east-1.console.aws.amazon.com/bedrock/home?region=us-east-1#/api-keys/long-term/create) (replace `us-east-1` in the URL with your chosen region):
- Choose an expiry period for the key.
- Click **Generate**.
- This creates an IAM user with strictly-scoped permissions for Bedrock access.
3. **Create an access key** for the IAM user:
- After generating the API key, click **"You can directly modify permissions for the IAM user associated"**.
- In the IAM user page, navigate to the **Security credentials** tab.
- Under **Access keys**, click **Create access key**.
- Select **"Application running outside AWS"** as the use case.
- Click **Next**.
- Add a description like "Coder AI Gateway token".
- Click **Create access key**.
- Save both the access key ID and secret access key securely.
4. **Configure your Coder deployment** with the credentials:
```sh
export CODER_AI_GATEWAY_BEDROCK_REGION=us-east-1
export CODER_AI_GATEWAY_BEDROCK_ACCESS_KEY=<your-access-key-id>
export CODER_AI_GATEWAY_BEDROCK_ACCESS_KEY_SECRET=<your-secret-access-key>
coder server
```
### GitHub Copilot
GitHub Copilot offers three plans: Individual, Business, and Enterprise,
each with its own API endpoint. Configure one or more `copilot` providers
using the [indexed provider format](#multiple-instances-of-the-same-provider)
depending on which plans your organization uses.
Copilot providers use OAuth app installations for authentication rather than
static API keys.
```sh
# GitHub Copilot (Individual)
export CODER_AI_GATEWAY_PROVIDER_0_TYPE=copilot
export CODER_AI_GATEWAY_PROVIDER_0_NAME=copilot
# GitHub Copilot Business
export CODER_AI_GATEWAY_PROVIDER_1_TYPE=copilot
export CODER_AI_GATEWAY_PROVIDER_1_NAME=copilot-business
export CODER_AI_GATEWAY_PROVIDER_1_BASE_URL=https://api.business.githubcopilot.com
# GitHub Copilot Enterprise
export CODER_AI_GATEWAY_PROVIDER_2_TYPE=copilot
export CODER_AI_GATEWAY_PROVIDER_2_NAME=copilot-enterprise
export CODER_AI_GATEWAY_PROVIDER_2_BASE_URL=https://api.enterprise.githubcopilot.com
```
The default base URL targets the individual Copilot API
(`api.individual.githubcopilot.com`). Override `CODER_AI_GATEWAY_PROVIDER_<N>_BASE_URL`
for Business or Enterprise tiers as shown above.
For client-side setup (proxy, certificates, IDE configuration), see
[GitHub Copilot client configuration](./clients/copilot.md).
### ChatGPT
Configure a ChatGPT provider by creating an `openai`-typed instance with the
ChatGPT Codex base URL:
```sh
export CODER_AI_GATEWAY_PROVIDER_0_TYPE=openai
export CODER_AI_GATEWAY_PROVIDER_0_NAME=chatgpt
export CODER_AI_GATEWAY_PROVIDER_0_BASE_URL=https://chatgpt.com/backend-api/codex
```
</div>
> [!NOTE]
> See the [Supported APIs](./reference.md#supported-apis) section below for precise endpoint coverage and interception behavior.
### Multiple instances of the same provider
You can configure multiple instances of the same provider type, for example, to
route different teams to separate API keys, use different base URLs per region, or
connect to both a direct API and a proxy simultaneously. Use indexed environment
variables following the pattern `CODER_AI_GATEWAY_PROVIDER_<N>_<KEY>`:
```sh
# Anthropic routed through a corporate proxy
export CODER_AI_GATEWAY_PROVIDER_0_TYPE=anthropic
export CODER_AI_GATEWAY_PROVIDER_0_NAME=anthropic-corp
export CODER_AI_GATEWAY_PROVIDER_0_KEY=sk-ant-corp-xxx
export CODER_AI_GATEWAY_PROVIDER_0_BASE_URL=https://llm-proxy.internal.example.com/anthropic
# Anthropic direct (for teams that need direct access)
export CODER_AI_GATEWAY_PROVIDER_1_TYPE=anthropic
export CODER_AI_GATEWAY_PROVIDER_1_NAME=anthropic-direct
export CODER_AI_GATEWAY_PROVIDER_1_KEY=sk-ant-direct-yyy
# Azure-hosted OpenAI deployment
export CODER_AI_GATEWAY_PROVIDER_2_TYPE=openai
export CODER_AI_GATEWAY_PROVIDER_2_NAME=azure-openai
export CODER_AI_GATEWAY_PROVIDER_2_KEY=azure-key-zzz
export CODER_AI_GATEWAY_PROVIDER_2_BASE_URL=https://my-deployment.openai.azure.com/
# Anthropic via AWS Bedrock
export CODER_AI_GATEWAY_PROVIDER_3_TYPE=anthropic
export CODER_AI_GATEWAY_PROVIDER_3_NAME=anthropic-bedrock
export CODER_AI_GATEWAY_PROVIDER_3_BEDROCK_REGION=us-west-2
export CODER_AI_GATEWAY_PROVIDER_3_BEDROCK_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE
export CODER_AI_GATEWAY_PROVIDER_3_BEDROCK_ACCESS_KEY_SECRET=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
coder server
```
Each provider instance gets a unique route based on its `NAME`. Clients send
requests to `/api/v2/aibridge/<NAME>/` to target a specific instance:
| Instance name | Route |
|---------------------|-----------------------------------------------------|
| `anthropic-corp` | `/api/v2/aibridge/anthropic-corp/v1/messages` |
| `anthropic-direct` | `/api/v2/aibridge/anthropic-direct/v1/messages` |
| `azure-openai` | `/api/v2/aibridge/azure-openai/v1/chat/completions` |
| `anthropic-bedrock` | `/api/v2/aibridge/anthropic-bedrock/v1/messages` |
**Supported keys per provider:**
| Key | Required | Description |
|------------|----------|------------------------------------------------------|
| `TYPE` | Yes | Provider type: `openai`, `anthropic`, or `copilot` |
| `NAME` | No | Unique instance name for routing. Defaults to `TYPE` |
| `KEY` | No | API key for upstream authentication (alias: `KEYS`) |
| `BASE_URL` | No | Base URL of the upstream API |
For `anthropic` providers using AWS Bedrock, the following keys are also
available: `BEDROCK_BASE_URL`, `BEDROCK_REGION`,
`BEDROCK_ACCESS_KEY` (alias: `BEDROCK_ACCESS_KEYS`),
`BEDROCK_ACCESS_KEY_SECRET` (alias: `BEDROCK_ACCESS_KEY_SECRETS`),
`BEDROCK_MODEL`, `BEDROCK_SMALL_FAST_MODEL`.
> [!NOTE]
> Indices must be contiguous and start at `0`. Each instance must have a unique
> `NAME`. If two instances of the same `TYPE` omit `NAME`, they will both
> default to the type name and fail with a duplicate name error.
>
> The legacy single-provider environment variables (`CODER_AI_GATEWAY_OPENAI_KEY`,
> `CODER_AI_GATEWAY_ANTHROPIC_KEY`, etc.) continue to work. However, setting
> both a legacy variable and an indexed provider with the same default name
> (e.g. `CODER_AI_GATEWAY_OPENAI_KEY` and an indexed provider named `openai`)
> will produce a startup error. Remove one or the other to resolve the
> conflict.
## API Dumps ## API Dumps
docs/images/aibridge/provider-add-anthropic.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 58 KiB

docs/images/aibridge/provider-edit-anthropic.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 62 KiB

docs/images/aibridge/providers-list.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 80 KiB

docs/manifest.json
+6
View File
@@ -1252,6 +1252,12 @@
} }
] ]
}, },
{
"title": "Provider Configuration",
"description": "How AI Gateway stores, seeds, and reloads provider configuration",
"path": "./ai-coder/ai-gateway/providers.md",
"state": ["ai governance add-on"]
},
{ {
"title": "Auditing AI Sessions", "title": "Auditing AI Sessions",
"description": "How to audit AI sessions", "description": "How to audit AI sessions",