mirror of
https://github.com/coder/coder.git
synced 2026-06-03 13:08:25 +00:00
Susana Ferreira
0766cc3097
## Description Adds automatic key failover for passthrough routes for the Anthropic and OpenAI providers. A new `keyFailoverTransport` wraps the reverse-proxy transport: centralized requests walk the configured key pool and retry with the next key on key-specific failures (401/403/429), reusing the same key-marking semantics as the bridged routes. BYOK passthrough requests run as a single attempt with no failover. ## Changes - New `keypool.KeyFailoverConfig` carrying the `Pool` to walk and the provider-specific closures (`IsBYOK`, `InjectAuthKey`, `MarkKey`, `BuildExhaustedResponse`). - New `keypool.NewKeyFailoverTransport`: wraps an inner `http.RoundTripper`. Returns `inner` unchanged when `Pool` is nil, otherwise produces a transport that buffers the request body once, walks the pool per request, and replays each attempt with the next key. - New `Provider.KeyFailoverConfig(logger)` interface method. Anthropic injects `X-Api-Key`; OpenAI injects `Authorization: Bearer ...`; Copilot returns an empty config. - `passthrough.go` wires `NewKeyFailoverTransport` around the existing apidump middleware, so every retry attempt is recorded. ## Related Issues Related to: https://github.com/coder/internal/issues/1446 Related to: https://linear.app/codercom/issue/AIGOV-197/aibridge-automatic-key-failover-for-bridged-and-passthrough-routes ## Follow-up PRs - Remove dead `Provider.InjectAuthHeader` method now that all auth is applied per-attempt by `KeyFailoverTransport`. - Bedrock multi-key support. - Refactor provider vs interceptor config separation. - Record the actually-used key in the interception credential hint after failover. > [!NOTE] > Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira
35 lines
957 B
Go
35 lines
957 B
Go
package utils
|
|
|
|
import (
|
|
"bytes"
|
|
"fmt"
|
|
"io"
|
|
"math"
|
|
"net/http"
|
|
"strconv"
|
|
"time"
|
|
)
|
|
|
|
// NewJSONErrorResponse builds an *http.Response with a JSON body
|
|
// and optional Retry-After header. Used to synthesize bridge-side
|
|
// error responses (e.g. key-pool exhaustion, marshaling
|
|
// fallbacks). Retry-After is set to whole seconds (rounded up)
|
|
// when retryAfter is positive, and omitted otherwise.
|
|
func NewJSONErrorResponse(status int, retryAfter time.Duration, body []byte) *http.Response {
|
|
h := http.Header{}
|
|
h.Set("Content-Type", "application/json")
|
|
if retryAfter > 0 {
|
|
h.Set("Retry-After", strconv.Itoa(int(math.Ceil(retryAfter.Seconds()))))
|
|
}
|
|
return &http.Response{
|
|
Status: fmt.Sprintf("%d %s", status, http.StatusText(status)),
|
|
StatusCode: status,
|
|
Proto: "HTTP/1.1",
|
|
ProtoMajor: 1,
|
|
ProtoMinor: 1,
|
|
Header: h,
|
|
Body: io.NopCloser(bytes.NewReader(body)),
|
|
ContentLength: int64(len(body)),
|
|
}
|
|
}
|