mirror of https://github.com/coder/coder.git synced 2026-06-02 20:48:20 +00:00

Files

T

Paweł Banaszewski e00e85765b chore: move aibridge library code into coder repo (#24190 )

This PR merges code from `coder/aibridge` repository into `coder/coder`.
It was split into 4 PRs for easier review but stacked PRs will need to
be merged into this PR so all checks pass.

* https://github.com/coder/coder/pull/24190 -> raw code copy (this PR,
before merging PRs on top of it, it was just 1 commit:
https://github.com/coder/coder/commit/70d33f33200c7e77df910957595715f81f9bec24)
* https://github.com/coder/coder/pull/24570 -> update imports in
`coder/coder` to use copied code
* https://github.com/coder/coder/pull/24586 -> linter fixes and CI
integration (also added README.md)
* https://github.com/coder/coder/pull/24571 -> added exclude to
scripts/check_emdash.sh check

Original PR message (before PR squash):
Moves coder/aibridge code into coder/coder repository.

Omitted files:

- `go.mod`, `go.sum`, `.gitignore`, `.github/workflows/ci.yml,`
`Makefile`, `LICENSE`, `README.md` (modified README.md is added later)
- `.github`, `example`, `buildinfo,` `scripts` directories

Simple verification script (will list omitted files)

```
tmp=$(mktemp -d)
echo "$tmp"
git clone --depth=1 https://github.com/coder/aibridge "$tmp/aibridge"
git clone --depth=1 --branch pb/aibridge-code-move https://github.com/coder/coder "$tmp/coder"
diff -rq --exclude=.git "$tmp/aibridge" "$tmp/coder/aibridge"
# rm -rf "$tmp"
```

2026-04-22 17:01:01 +02:00

6.0 KiB

Raw Blame History

aibridge

aibridge is an HTTP gateway that sits between AI clients and upstream AI providers (Anthropic, OpenAI). It intercepts requests to record token usage, prompts, and tool invocations per user. Optionally supports centralized MCP tool injection with allowlist/denylist filtering.

Architecture

┌─────────────────┐     ┌───────────────────────────────────────────┐
│    AI Client    │     │                    aibridge               │
│  (Claude Code,  │────▶│  ┌─────────────────┐    ┌─────────────┐   │
│   Cursor, etc.) │     │  │  RequestBridge  │───▶│  Providers  │   │
└─────────────────┘     │  │  (http.Handler) │    │  (Anthropic │   │
                        │  └─────────────────┘    │   OpenAI)   │   │
                        │                         └──────┬──────┘   │
                        │                                │          │
                        │                                ▼          │    ┌─────────────┐
                        │  ┌─────────────────┐    ┌─────────────┐   │    │  Upstream   │
                        │  │    Recorder     │◀───│ Interceptor │─── ───▶│    API      │
                        │  │ (tokens, tools, │    │ (streaming/ │   │    │ (Anthropic  │
                        │  │  prompts)       │    │  blocking)  │   │    │   OpenAI)   │
                        │  └────────┬────────┘    └──────┬──────┘   │    └─────────────┘
                        │           │                    │          │
                        │           ▼             ┌──────▼──────┐   │
                        │  ┌ ─ ─ ─ ─ ─ ─ ─ ┐      │  MCP Proxy  │   │
                        │  │    Database   │      │   (tools)   │   │
                        │  └ ─ ─ ─ ─ ─ ─ ─ ┘      └─────────────┘   │
                        └───────────────────────────────────────────┘

Components

RequestBridge: The main http.Handler that routes requests to providers
Provider: Defines bridged routes (intercepted) and passthrough routes (proxied)
Interceptor: Handles request/response processing and streaming
Recorder: Interface for capturing usage data (tokens, prompts, tools)
MCP Proxy (optional): Connects to MCP servers to list tool, inject them into requests, and invoke them in an inner agentic loop

Request Flow

Client sends request to /anthropic/v1/messages or /openai/v1/chat/completions
Actor extraction: Request must have an actor in context (via AsActor()).
Upstream call: Request forwarded to the AI provider
Response relay: Response streamed/sent to client
Recording: Token usage, prompts, and tool invocations recorded

With MCP enabled: Tools from configured MCP servers are centrally defined and injected into requests (prefixed bmcp_). Allowlist/denylist regex patterns control which tools are available. When the model selects an injected tool, the gateway invokes it in an inner agentic loop, and continues the conversation loop until complete.

Passthrough routes (/v1/models, /v1/messages/count_tokens) are reverse-proxied directly.

Observability

Prometheus Metrics

Create metrics with NewMetrics(prometheus.Registerer):

Metric	Type	Description
`interceptions_total`	Counter	Intercepted request count
`interceptions_inflight`	Gauge	Currently processing requests
`interceptions_duration_seconds`	Histogram	Request duration
`tokens_total`	Counter	Token usage (input/output)
`prompts_total`	Counter	User prompt count
`injected_tool_invocations_total`	Counter	MCP tool invocations
`passthrough_total`	Counter	Non-intercepted requests

Recorder Interface

Implement Recorder to persist usage data to your database:

aibridge_interceptions - request metadata (provider, model, initiator, timestamps)
aibridge_token_usages - input/output token counts per response
aibridge_user_prompts - user prompts
aibridge_tool_usages - tool invocations (injected and client-defined)

type Recorder interface {
    RecordInterception(ctx context.Context, req *InterceptionRecord) error
    RecordInterceptionEnded(ctx context.Context, req *InterceptionRecordEnded) error
    RecordTokenUsage(ctx context.Context, req *TokenUsageRecord) error
    RecordPromptUsage(ctx context.Context, req *PromptUsageRecord) error
    RecordToolUsage(ctx context.Context, req *ToolUsageRecord) error
}

Supported Routes

Provider	Route	Type
Anthropic	`/anthropic/v1/messages`	Bridged (intercepted)
Anthropic	`/anthropic/v1/models`	Passthrough
Anthropic	`/anthropic/v1/messages/count_tokens`	Passthrough
OpenAI	`/openai/v1/chat/completions`	Bridged (intercepted)
OpenAI	`/openai/v1/models`	Passthrough

6.0 KiB Raw Blame History