shishantbiswas/coder
1
0
Fork 0
mirror of https://github.com/coder/coder.git synced 2026-06-02 20:48:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor: load AI providers from the database at startup (#25672)

Browse Source 
Replace the env-based `BuildProviders` with a DB-backed loader. The database is now the single source of truth for runtime provider configuration; env config arrives via `SeedAIProvidersFromEnv` (run at boot) and `BuildProviders` reads it back as `aibridge.Provider` instances. `cli/server.go` and `enterprise/cli/server.go` both call the same path, so aibridged and aibridgeproxyd see the same provider set.

Per-provider `DumpDir` is replaced by a top-level `CODER_AI_GATEWAY_DUMP_DIR` base; each provider's effective dump path is `<base>/<provider name>`.
This commit is contained in:
Danny Kopping
2026-05-26 15:57:01 +02:00
committed by GitHub
parent dfd7ca3b98
commit 282ab7de34
19 changed files with 570 additions and 258 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ai-coder/ai-gateway/setup.md
+39 -14
View File
@@ -218,20 +218,12 @@ requests to `/api/v2/aibridge/<NAME>/` to target a specific instance:
**Supported keys per provider:**
| Key | Required | Description |
|------------|----------|-------------------------------------------------------|
| `TYPE` | Yes | Provider type: `openai`, `anthropic`, or `copilot` |
| `NAME` | No | Unique instance name for routing. Defaults to `TYPE` |
| `KEY` | No | API key for upstream authentication (alias: `KEYS`) |
| `BASE_URL` | No | Base URL of the upstream API |
| `DUMP_DIR` | No | Directory for provider API request and response dumps |
> [!WARNING]
> `DUMP_DIR` is not intended for regular use. Setting this option
> results in a high number of writes. Dump files contain raw request and
> response data, which may include proprietary or sensitive information
> (prompts, completions, tool inputs). Enable only briefly for diagnostic
> purposes and protect the target directory.
| Key | Required | Description |
|------------|----------|------------------------------------------------------|
| `TYPE` | Yes | Provider type: `openai`, `anthropic`, or `copilot` |
| `NAME` | No | Unique instance name for routing. Defaults to `TYPE` |
| `KEY` | No | API key for upstream authentication (alias: `KEYS`) |
| `BASE_URL` | No | Base URL of the upstream API |
For `anthropic` providers using AWS Bedrock, the following keys are also
available: `BEDROCK_BASE_URL`, `BEDROCK_REGION`,
@@ -251,6 +243,39 @@ available: `BEDROCK_BASE_URL`, `BEDROCK_REGION`,
> will produce a startup error. Remove one or the other to resolve the
> conflict.
## API Dumps
AI Gateway can dump provider request and response pairs to disk for debugging.
Configure the dump directory with `--ai-gateway-dump-dir` or
`CODER_AI_GATEWAY_DUMP_DIR`:
```sh
coder server --ai-gateway-dump-dir=/var/lib/coder/ai-gateway-dumps
```
Or in YAML:
```yaml
ai_gateway:
api_dump_dir: /var/lib/coder/ai-gateway-dumps
```
This top-level setting replaces the previous per-provider `DUMP_DIR` field.
For each provider, AI Gateway writes dumps under `<base>/<provider_name>`, where
`<base>` is the configured dump directory and `<provider_name>` is the provider
instance name used in the route. For example, a provider named `anthropic-corp`
with `/var/lib/coder/ai-gateway-dumps` configured writes to
`/var/lib/coder/ai-gateway-dumps/anthropic-corp`.
Sensitive headers are redacted before dumps are written. Leave the value empty
to disable dumping.
> [!WARNING]
> API dumps are intended for short diagnostic sessions only. Dump files contain
> raw request and response data, which may include proprietary or sensitive
> information such as prompts, completions, and tool inputs. Protect the target
> directory and disable dumping when diagnostics are complete.
## Data Retention
AI Gateway records prompts, token usage, tool invocations, and model reasoning for auditing and
docs/reference/api/general.md
Generated
+1 -1
View File
@@ -185,6 +185,7 @@ curl -X GET http://coder-server:8080/api/v2/deployment/config \
"base_url": "string",
"key": "string"
},
"api_dump_dir": "string",
"bedrock": {
"access_key": "string",
"access_key_secret": "string",
@@ -213,7 +214,6 @@ curl -X GET http://coder-server:8080/api/v2/deployment/config \
"bedrock_model": "string",
"bedrock_region": "string",
"bedrock_small_fast_model": "string",
"dump_dir": "string",
"name": "string",
"type": "string"
}
docs/reference/api/schemas.md
Generated
+5 -6
View File
@@ -442,6 +442,7 @@
"base_url": "string",
"key": "string"
},
"api_dump_dir": "string",
"bedrock": {
"access_key": "string",
"access_key_secret": "string",
@@ -470,7 +471,6 @@
"bedrock_model": "string",
"bedrock_region": "string",
"bedrock_small_fast_model": "string",
"dump_dir": "string",
"name": "string",
"type": "string"
}
@@ -488,6 +488,7 @@
|-------------------------------------|----------------------------------------------------------------------|----------|--------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `allow_byok` | boolean | false | | |
| `anthropic` | [codersdk.AIBridgeAnthropicConfig](#codersdkaibridgeanthropicconfig) | false | | Deprecated: Use Providers with indexed CODER_AI_GATEWAY_PROVIDER_<N>_* env vars instead. |
| `api_dump_dir` | string | false | | Api dump dir is the base directory under which each provider's request/response dumps are written, in a subdirectory named after the provider. Empty disables dumping. |
| `bedrock` | [codersdk.AIBridgeBedrockConfig](#codersdkaibridgebedrockconfig) | false | | Deprecated: Use Providers with indexed CODER_AI_GATEWAY_PROVIDER_<N>_* env vars instead. |
| `budget_period` | string | false | | |
| `budget_policy` | string | false | | Budget settings for AI Governance cost controls. |
@@ -1245,6 +1246,7 @@
"base_url": "string",
"key": "string"
},
"api_dump_dir": "string",
"bedrock": {
"access_key": "string",
"access_key_secret": "string",
@@ -1273,7 +1275,6 @@
"bedrock_model": "string",
"bedrock_region": "string",
"bedrock_small_fast_model": "string",
"dump_dir": "string",
"name": "string",
"type": "string"
}
@@ -1344,7 +1345,6 @@
"bedrock_model": "string",
"bedrock_region": "string",
"bedrock_small_fast_model": "string",
"dump_dir": "string",
"name": "string",
"type": "string"
}
@@ -1358,7 +1358,6 @@
| `bedrock_model` | string | false | | |
| `bedrock_region` | string | false | | |
| `bedrock_small_fast_model` | string | false | | |
| `dump_dir` | string | false | | Dump dir is the directory path for dumping API requests and responses. |
| `name` | string | false | | Name is the unique instance identifier used for routing. Defaults to Type if not provided. |
| `type` | string | false | | Type is the provider type: "openai", "anthropic", or "copilot". |
@@ -5706,6 +5705,7 @@ CreateWorkspaceRequest provides options for creating a new workspace. Only one o
"base_url": "string",
"key": "string"
},
"api_dump_dir": "string",
"bedrock": {
"access_key": "string",
"access_key_secret": "string",
@@ -5734,7 +5734,6 @@ CreateWorkspaceRequest provides options for creating a new workspace. Only one o
"bedrock_model": "string",
"bedrock_region": "string",
"bedrock_small_fast_model": "string",
"dump_dir": "string",
"name": "string",
"type": "string"
}
@@ -6305,6 +6304,7 @@ CreateWorkspaceRequest provides options for creating a new workspace. Only one o
"base_url": "string",
"key": "string"
},
"api_dump_dir": "string",
"bedrock": {
"access_key": "string",
"access_key_secret": "string",
@@ -6333,7 +6333,6 @@ CreateWorkspaceRequest provides options for creating a new workspace. Only one o
"bedrock_model": "string",
"bedrock_region": "string",
"bedrock_small_fast_model": "string",
"dump_dir": "string",
"name": "string",
"type": "string"
}
docs/reference/cli/server.md
Generated
+10
View File
@@ -1889,6 +1889,16 @@ Emit structured logs for AI Gateway interception records. Use this for exporting
Once enabled, extra headers will be added to upstream requests to identify the user (actor) making requests to AI Gateway. This is only needed if you are using a proxy between AI Gateway and an upstream AI provider. This will send X-Ai-Bridge-Actor-Id (the ID of the user making the request) and X-Ai-Bridge-Actor-Metadata-Username (their username).
### --ai-gateway-dump-dir
| | |
|-------------|-----------------------------------------|
| Type | <code>string</code> |
| Environment | <code>$CODER_AI_GATEWAY_DUMP_DIR</code> |
| YAML | <code>ai_gateway.api_dump_dir</code> |
Base directory for dumping AI Bridge request/response pairs to disk for debugging. When set, each provider writes under a subdirectory named after the provider. Sensitive headers are redacted. Leave empty to disable.
### --ai-gateway-allow-byok
| | |