coder/docs/ai-coder/ai-gateway/setup.md

Setup

AI Gateway runs inside the Coder control plane (coderd), requiring no separate compute to deploy or scale. Once enabled, coderd runs the aibridged in-memory and brokers traffic to your configured AI providers on behalf of authenticated users.

Required:

1. A Premium license with the AI Governance Add-On.
2. Feature must be enabled using the server flag
3. One or more providers API key(s) must be configured

Activation

You will need to enable AI Gateway explicitly:

export CODER_AIBRIDGE_ENABLED=true
coder server
# or
coder server --aibridge-enabled=true

Configure Providers

AI Gateway proxies requests to upstream LLM APIs. Configure at least one provider before exposing AI Gateway to end users.

OpenAI

Set the following when routing OpenAI-compatible traffic through AI Gateway:

• CODER_AIBRIDGE_OPENAI_KEY or --aibridge-openai-key
• CODER_AIBRIDGE_OPENAI_BASE_URL or --aibridge-openai-base-url

The default base URL (https://api.openai.com/v1/) works for the native OpenAI service. Point the base URL at your preferred OpenAI-compatible endpoint (for example, a hosted proxy or LiteLLM deployment) when needed.

If you'd like to create an OpenAI key with minimal privileges, this is the minimum required set:

Anthropic

Set the following when routing Anthropic-compatible traffic through AI Gateway:

• CODER_AIBRIDGE_ANTHROPIC_KEY or --aibridge-anthropic-key
• CODER_AIBRIDGE_ANTHROPIC_BASE_URL or --aibridge-anthropic-base-url

The default base URL (https://api.anthropic.com/) targets Anthropic's public API. Override it for Anthropic-compatible brokers.

Anthropic does not allow API keys to have restricted permissions at the time of writing (Nov 2025).

Amazon Bedrock

Set the following when routing Amazon Bedrock traffic through AI Gateway:

• CODER_AIBRIDGE_BEDROCK_REGION or --aibridge-bedrock-region
• CODER_AIBRIDGE_BEDROCK_ACCESS_KEY or --aibridge-bedrock-access-key
• CODER_AIBRIDGE_BEDROCK_ACCESS_KEY_SECRET or --aibridge-bedrock-access-key-secret
• CODER_AIBRIDGE_BEDROCK_MODEL or --aibridge-bedrock-model
• CODER_AIBRIDGE_BEDROCK_SMALL_FAST_MODEL or --aibridge-bedrock-small-fast-model

Note

CODER_AIBRIDGE_BEDROCK_BASE_URL or --aibridge-bedrock-base-url may be used instead of CODER_AIBRIDGE_BEDROCK_REGION/--aibridge-bedrock-region if you would like to specify a URL which does not follow the form of https://bedrock-runtime.<region>.amazonaws.com - for example if using a proxy between AI Gateway and AWS Bedrock.

Obtaining Bedrock credentials

1. Choose a region where you want to use Bedrock.

2. Generate API keys in the AWS Bedrock console (replace us-east-1 in the URL with your chosen region):

   • Choose an expiry period for the key.
   • Click Generate.
   • This creates an IAM user with strictly-scoped permissions for Bedrock access.

3. Create an access key for the IAM user:

   • After generating the API key, click "You can directly modify permissions for the IAM user associated".
   • In the IAM user page, navigate to the Security credentials tab.
   • Under Access keys, click Create access key.
   • Select "Application running outside AWS" as the use case.
   • Click Next.
   • Add a description like "Coder AI Gateway token".
   • Click Create access key.
   • Save both the access key ID and secret access key securely.

4. Configure your Coder deployment with the credentials:

   export CODER_AIBRIDGE_BEDROCK_REGION=us-east-1
   export CODER_AIBRIDGE_BEDROCK_ACCESS_KEY=<your-access-key-id>
   export CODER_AIBRIDGE_BEDROCK_ACCESS_KEY_SECRET=<your-secret-access-key>
   coder server
   

Additional providers and Model Proxies

AI Gateway can relay traffic to other OpenAI- or Anthropic-compatible services or model proxies like LiteLLM by pointing the base URL variables above at the provider you operate. Share feedback or follow along in the aibridge issue tracker as we expand support for additional providers.

Note

See the Supported APIs section below for precise endpoint coverage and interception behavior.

Data Retention

AI Gateway records prompts, token usage, tool invocations, and model reasoning for auditing and monitoring purposes. By default, this data is retained for 60 days.

Configure retention using --aibridge-retention or CODER_AIBRIDGE_RETENTION:

coder server --aibridge-retention=90d

Or in YAML:

aibridge:
  retention: 90d

Set to 0 to retain data indefinitely.

For duration formats, how retention works, and best practices, see the Data Retention documentation.

Structured Logging

AI Gateway can emit structured logs for every interception record, making it straightforward to export data to external SIEM or observability platforms.

Enable with --aibridge-structured-logging or CODER_AIBRIDGE_STRUCTURED_LOGGING:

coder server --aibridge-structured-logging=true

Or in YAML:

aibridge:
  structured_logging: true

These logs are written to the same output stream as all other coderd logs, using the format configured by --log-human (default, writes to stderr) or --log-json. For machine ingestion, set --log-json to a file path or /dev/stderr so that records are emitted as JSON.

Filter for AI Gateway records in your logging pipeline by matching on the "interception log" message. Each log line includes a record_type field that indicates the kind of event captured:

record_type	Description	Key fields
interception_start	A new intercepted request begins.	interception_id, initiator_id, provider, model, client, started_at
interception_end	An intercepted request completes.	interception_id, ended_at
token_usage	Token consumption for a response.	interception_id, input_tokens, output_tokens, created_at
prompt_usage	The last user prompt in a request.	interception_id, prompt, created_at
tool_usage	A tool/function call made by the model.	interception_id, tool, input, server_url, injected, created_at
model_thought	Model reasoning or thinking content.	interception_id, content, created_at