shishantbiswas/coder
1
0
Fork 0
mirror of https://github.com/coder/coder.git synced 2026-06-02 20:48:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor: load AI providers from the database at startup (#25672)

Browse Source 
Replace the env-based `BuildProviders` with a DB-backed loader. The database is now the single source of truth for runtime provider configuration; env config arrives via `SeedAIProvidersFromEnv` (run at boot) and `BuildProviders` reads it back as `aibridge.Provider` instances. `cli/server.go` and `enterprise/cli/server.go` both call the same path, so aibridged and aibridgeproxyd see the same provider set.

Per-provider `DumpDir` is replaced by a top-level `CODER_AI_GATEWAY_DUMP_DIR` base; each provider's effective dump path is `<base>/<provider name>`.
This commit is contained in:
Danny Kopping
2026-05-26 15:57:01 +02:00
committed by GitHub
parent dfd7ca3b98
commit 282ab7de34
19 changed files with 570 additions and 258 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ai-coder/ai-gateway/setup.md
+39 -14
View File
@@ -218,20 +218,12 @@ requests to `/api/v2/aibridge/<NAME>/` to target a specific instance:
**Supported keys per provider:**
| Key | Required | Description |
|------------|----------|-------------------------------------------------------|
| `TYPE` | Yes | Provider type: `openai`, `anthropic`, or `copilot` |
| `NAME` | No | Unique instance name for routing. Defaults to `TYPE` |
| `KEY` | No | API key for upstream authentication (alias: `KEYS`) |
| `BASE_URL` | No | Base URL of the upstream API |
| `DUMP_DIR` | No | Directory for provider API request and response dumps |
> [!WARNING]
> `DUMP_DIR` is not intended for regular use. Setting this option
> results in a high number of writes. Dump files contain raw request and
> response data, which may include proprietary or sensitive information
> (prompts, completions, tool inputs). Enable only briefly for diagnostic
> purposes and protect the target directory.
| Key | Required | Description |
|------------|----------|------------------------------------------------------|
| `TYPE` | Yes | Provider type: `openai`, `anthropic`, or `copilot` |
| `NAME` | No | Unique instance name for routing. Defaults to `TYPE` |
| `KEY` | No | API key for upstream authentication (alias: `KEYS`) |
| `BASE_URL` | No | Base URL of the upstream API |
For `anthropic` providers using AWS Bedrock, the following keys are also
available: `BEDROCK_BASE_URL`, `BEDROCK_REGION`,
@@ -251,6 +243,39 @@ available: `BEDROCK_BASE_URL`, `BEDROCK_REGION`,
> will produce a startup error. Remove one or the other to resolve the
> conflict.
## API Dumps
AI Gateway can dump provider request and response pairs to disk for debugging.
Configure the dump directory with `--ai-gateway-dump-dir` or
`CODER_AI_GATEWAY_DUMP_DIR`:
```sh
coder server --ai-gateway-dump-dir=/var/lib/coder/ai-gateway-dumps
```
Or in YAML:
```yaml
ai_gateway:
api_dump_dir: /var/lib/coder/ai-gateway-dumps
```
This top-level setting replaces the previous per-provider `DUMP_DIR` field.
For each provider, AI Gateway writes dumps under `<base>/<provider_name>`, where
`<base>` is the configured dump directory and `<provider_name>` is the provider
instance name used in the route. For example, a provider named `anthropic-corp`
with `/var/lib/coder/ai-gateway-dumps` configured writes to
`/var/lib/coder/ai-gateway-dumps/anthropic-corp`.
Sensitive headers are redacted before dumps are written. Leave the value empty
to disable dumping.
> [!WARNING]
> API dumps are intended for short diagnostic sessions only. Dump files contain
> raw request and response data, which may include proprietary or sensitive
> information such as prompts, completions, and tool inputs. Protect the target
> directory and disable dumping when diagnostics are complete.
## Data Retention
AI Gateway records prompts, token usage, tool invocations, and model reasoning for auditing and