mirror of
https://github.com/coder/coder.git
synced 2026-06-02 20:48:20 +00:00
Kacper Sawicki
6f86f67754
## Summary This adds configurable overload protection to the AI Bridge daemon to prevent the server from being overwhelmed during periods of high load. Partially addresses coder/internal#1153 (rate limits and concurrency control; circuit breakers are deferred to a follow-up). ## New Configuration Options | Option | Environment Variable | Description | Default | |--------|---------------------|-------------|---------| | `--aibridge-max-concurrency` | `CODER_AIBRIDGE_MAX_CONCURRENCY` | Maximum number of concurrent AI Bridge requests. Set to 0 to disable (unlimited). | `0` | | `--aibridge-rate-limit` | `CODER_AIBRIDGE_RATE_LIMIT` | Maximum number of AI Bridge requests per second. Set to 0 to disable rate limiting. | `0` | ## Behavior When limits are exceeded: - **Concurrency limit**: Returns HTTP `503 Service Unavailable` with message "AI Bridge is currently at capacity. Please try again later." - **Rate limit**: Returns HTTP `429 Too Many Requests` with `Retry-After` header. Both protections are optional and disabled by default (0 values). ## Implementation The overload protection is implemented as reusable middleware in `coderd/httpmw/ratelimit.go`: 1. **`RateLimitByAuthToken`**: Per-user rate limiting that uses `APITokenFromRequest` to extract the authentication token, with fallback to `X-Api-Key` header for AI provider compatibility (e.g., Anthropic). Falls back to IP-based rate limiting if no token is present. Includes `Retry-After` header for backpressure signaling. 2. **`ConcurrencyLimit`**: Uses an atomic counter to track in-flight requests and reject when at capacity. The middleware is applied in `enterprise/coderd/aibridge.go` via `r.Group` in the following order: 1. Concurrency check (faster rejection for load shedding) 2. Rate limit check **Note**: Rate limiting currently applies to all AI Bridge requests, including pass-through requests. Ideally only actual interceptions should count, but this would require changes in the aibridge library. ## Testing Added comprehensive tests for: - Rate limiting by auth token (Bearer token, X-Api-Key, no token fallback to IP) - Different tokens not rate limited against each other - Disabled when limit is zero - Retry-After header is set on 429 responses - Concurrency limiting (allows within limit, rejects over limit, disabled when zero)
21 KiB
Generated
21 KiB
Generated
General
API root handler
Code samples
# Example request using curl
curl -X GET http://coder-server:8080/api/v2/ \
-H 'Accept: application/json'
GET /
Example responses
200 Response
{
"detail": "string",
"message": "string",
"validations": [
{
"detail": "string",
"field": "string"
}
]
}
Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | OK | codersdk.Response |
Build info
Code samples
# Example request using curl
curl -X GET http://coder-server:8080/api/v2/buildinfo \
-H 'Accept: application/json'
GET /buildinfo
Example responses
200 Response
{
"agent_api_version": "string",
"dashboard_url": "string",
"deployment_id": "string",
"external_url": "string",
"provisioner_api_version": "string",
"telemetry": true,
"upgrade_message": "string",
"version": "string",
"webpush_public_key": "string",
"workspace_proxy": true
}
Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | OK | codersdk.BuildInfoResponse |
Report CSP violations
Code samples
# Example request using curl
curl -X POST http://coder-server:8080/api/v2/csp/reports \
-H 'Content-Type: application/json' \
-H 'Coder-Session-Token: API_KEY'
POST /csp/reports
Body parameter
{
"csp-report": {}
}
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
body |
body | coderd.cspViolation | true | Violation report |
Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | OK |
To perform this operation, you must be authenticated. Learn more.
Get deployment config
Code samples
# Example request using curl
curl -X GET http://coder-server:8080/api/v2/deployment/config \
-H 'Accept: application/json' \
-H 'Coder-Session-Token: API_KEY'
GET /deployment/config
Example responses
200 Response
{
"config": {
"access_url": {
"forceQuery": true,
"fragment": "string",
"host": "string",
"omitHost": true,
"opaque": "string",
"path": "string",
"rawFragment": "string",
"rawPath": "string",
"rawQuery": "string",
"scheme": "string",
"user": {}
},
"additional_csp_policy": [
"string"
],
"address": {
"host": "string",
"port": "string"
},
"agent_fallback_troubleshooting_url": {
"forceQuery": true,
"fragment": "string",
"host": "string",
"omitHost": true,
"opaque": "string",
"path": "string",
"rawFragment": "string",
"rawPath": "string",
"rawQuery": "string",
"scheme": "string",
"user": {}
},
"agent_stat_refresh_interval": 0,
"ai": {
"bridge": {
"anthropic": {
"base_url": "string",
"key": "string"
},
"bedrock": {
"access_key": "string",
"access_key_secret": "string",
"model": "string",
"region": "string",
"small_fast_model": "string"
},
"enabled": true,
"inject_coder_mcp_tools": true,
"max_concurrency": 0,
"openai": {
"base_url": "string",
"key": "string"
},
"rate_limit": 0,
"retention": 0
}
},
"allow_workspace_renames": true,
"autobuild_poll_interval": 0,
"browser_only": true,
"cache_directory": "string",
"cli_upgrade_message": "string",
"config": "string",
"config_ssh": {
"deploymentName": "string",
"sshconfigOptions": [
"string"
]
},
"dangerous": {
"allow_all_cors": true,
"allow_path_app_sharing": true,
"allow_path_app_site_owner_access": true
},
"derp": {
"config": {
"block_direct": true,
"force_websockets": true,
"path": "string",
"url": "string"
},
"server": {
"enable": true,
"region_code": "string",
"region_id": 0,
"region_name": "string",
"relay_url": {
"forceQuery": true,
"fragment": "string",
"host": "string",
"omitHost": true,
"opaque": "string",
"path": "string",
"rawFragment": "string",
"rawPath": "string",
"rawQuery": "string",
"scheme": "string",
"user": {}
},
"stun_addresses": [
"string"
]
}
},
"disable_owner_workspace_exec": true,
"disable_password_auth": true,
"disable_path_apps": true,
"disable_workspace_sharing": true,
"docs_url": {
"forceQuery": true,
"fragment": "string",
"host": "string",
"omitHost": true,
"opaque": "string",
"path": "string",
"rawFragment": "string",
"rawPath": "string",
"rawQuery": "string",
"scheme": "string",
"user": {}
},
"enable_authz_recording": true,
"enable_terraform_debug_mode": true,
"ephemeral_deployment": true,
"experiments": [
"string"
],
"external_auth": {
"value": [
{
"app_install_url": "string",
"app_installations_url": "string",
"auth_url": "string",
"client_id": "string",
"device_code_url": "string",
"device_flow": true,
"display_icon": "string",
"display_name": "string",
"id": "string",
"mcp_tool_allow_regex": "string",
"mcp_tool_deny_regex": "string",
"mcp_url": "string",
"no_refresh": true,
"regex": "string",
"revoke_url": "string",
"scopes": [
"string"
],
"token_url": "string",
"type": "string",
"validate_url": "string"
}
]
},
"external_token_encryption_keys": [
"string"
],
"healthcheck": {
"refresh": 0,
"threshold_database": 0
},
"hide_ai_tasks": true,
"http_address": "string",
"http_cookies": {
"same_site": "string",
"secure_auth_cookie": true
},
"job_hang_detector_interval": 0,
"logging": {
"human": "string",
"json": "string",
"log_filter": [
"string"
],
"stackdriver": "string"
},
"metrics_cache_refresh_interval": 0,
"notifications": {
"dispatch_timeout": 0,
"email": {
"auth": {
"identity": "string",
"password": "string",
"password_file": "string",
"username": "string"
},
"force_tls": true,
"from": "string",
"hello": "string",
"smarthost": "string",
"tls": {
"ca_file": "string",
"cert_file": "string",
"insecure_skip_verify": true,
"key_file": "string",
"server_name": "string",
"start_tls": true
}
},
"fetch_interval": 0,
"inbox": {
"enabled": true
},
"lease_count": 0,
"lease_period": 0,
"max_send_attempts": 0,
"method": "string",
"retry_interval": 0,
"sync_buffer_size": 0,
"sync_interval": 0,
"webhook": {
"endpoint": {
"forceQuery": true,
"fragment": "string",
"host": "string",
"omitHost": true,
"opaque": "string",
"path": "string",
"rawFragment": "string",
"rawPath": "string",
"rawQuery": "string",
"scheme": "string",
"user": {}
}
}
},
"oauth2": {
"github": {
"allow_everyone": true,
"allow_signups": true,
"allowed_orgs": [
"string"
],
"allowed_teams": [
"string"
],
"client_id": "string",
"client_secret": "string",
"default_provider_enable": true,
"device_flow": true,
"enterprise_base_url": "string"
}
},
"oidc": {
"allow_signups": true,
"auth_url_params": {},
"client_cert_file": "string",
"client_id": "string",
"client_key_file": "string",
"client_secret": "string",
"email_domain": [
"string"
],
"email_field": "string",
"group_allow_list": [
"string"
],
"group_auto_create": true,
"group_mapping": {},
"group_regex_filter": {},
"groups_field": "string",
"icon_url": {
"forceQuery": true,
"fragment": "string",
"host": "string",
"omitHost": true,
"opaque": "string",
"path": "string",
"rawFragment": "string",
"rawPath": "string",
"rawQuery": "string",
"scheme": "string",
"user": {}
},
"ignore_email_verified": true,
"ignore_user_info": true,
"issuer_url": "string",
"name_field": "string",
"organization_assign_default": true,
"organization_field": "string",
"organization_mapping": {},
"scopes": [
"string"
],
"sign_in_text": "string",
"signups_disabled_text": "string",
"skip_issuer_checks": true,
"source_user_info_from_access_token": true,
"user_role_field": "string",
"user_role_mapping": {},
"user_roles_default": [
"string"
],
"username_field": "string"
},
"pg_auth": "string",
"pg_connection_url": "string",
"pprof": {
"address": {
"host": "string",
"port": "string"
},
"enable": true
},
"prometheus": {
"address": {
"host": "string",
"port": "string"
},
"aggregate_agent_stats_by": [
"string"
],
"collect_agent_stats": true,
"collect_db_metrics": true,
"enable": true
},
"provisioner": {
"daemon_poll_interval": 0,
"daemon_poll_jitter": 0,
"daemon_psk": "string",
"daemon_types": [
"string"
],
"daemons": 0,
"force_cancel_interval": 0
},
"proxy_health_status_interval": 0,
"proxy_trusted_headers": [
"string"
],
"proxy_trusted_origins": [
"string"
],
"rate_limit": {
"api": 0,
"disable_all": true
},
"redirect_to_access_url": true,
"retention": {
"api_keys": 0,
"audit_logs": 0,
"connection_logs": 0,
"workspace_agent_logs": 0
},
"scim_api_key": "string",
"session_lifetime": {
"default_duration": 0,
"default_token_lifetime": 0,
"disable_expiry_refresh": true,
"max_admin_token_lifetime": 0,
"max_token_lifetime": 0,
"refresh_default_duration": 0
},
"ssh_keygen_algorithm": "string",
"strict_transport_security": 0,
"strict_transport_security_options": [
"string"
],
"support": {
"links": {
"value": [
{
"icon": "bug",
"location": "navbar",
"name": "string",
"target": "string"
}
]
}
},
"swagger": {
"enable": true
},
"telemetry": {
"enable": true,
"trace": true,
"url": {
"forceQuery": true,
"fragment": "string",
"host": "string",
"omitHost": true,
"opaque": "string",
"path": "string",
"rawFragment": "string",
"rawPath": "string",
"rawQuery": "string",
"scheme": "string",
"user": {}
}
},
"terms_of_service_url": "string",
"tls": {
"address": {
"host": "string",
"port": "string"
},
"allow_insecure_ciphers": true,
"cert_file": [
"string"
],
"client_auth": "string",
"client_ca_file": "string",
"client_cert_file": "string",
"client_key_file": "string",
"enable": true,
"key_file": [
"string"
],
"min_version": "string",
"redirect_http": true,
"supported_ciphers": [
"string"
]
},
"trace": {
"capture_logs": true,
"data_dog": true,
"enable": true,
"honeycomb_api_key": "string"
},
"update_check": true,
"user_quiet_hours_schedule": {
"allow_user_custom": true,
"default_schedule": "string"
},
"verbose": true,
"web_terminal_renderer": "string",
"wgtunnel_host": "string",
"wildcard_access_url": "string",
"workspace_hostname_suffix": "string",
"workspace_prebuilds": {
"failure_hard_limit": 0,
"reconciliation_backoff_interval": 0,
"reconciliation_backoff_lookback": 0,
"reconciliation_interval": 0
},
"write_config": true
},
"options": [
{
"annotations": {
"property1": "string",
"property2": "string"
},
"default": "string",
"description": "string",
"env": "string",
"flag": "string",
"flag_shorthand": "string",
"group": {
"description": "string",
"name": "string",
"parent": {
"description": "string",
"name": "string",
"parent": {},
"yaml": "string"
},
"yaml": "string"
},
"hidden": true,
"name": "string",
"required": true,
"use_instead": [
{}
],
"value": null,
"value_source": "",
"yaml": "string"
}
]
}
Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | OK | codersdk.DeploymentConfig |
To perform this operation, you must be authenticated. Learn more.
SSH Config
Code samples
# Example request using curl
curl -X GET http://coder-server:8080/api/v2/deployment/ssh \
-H 'Accept: application/json' \
-H 'Coder-Session-Token: API_KEY'
GET /deployment/ssh
Example responses
200 Response
{
"hostname_prefix": "string",
"hostname_suffix": "string",
"ssh_config_options": {
"property1": "string",
"property2": "string"
}
}
Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | OK | codersdk.SSHConfigResponse |
To perform this operation, you must be authenticated. Learn more.
Get deployment stats
Code samples
# Example request using curl
curl -X GET http://coder-server:8080/api/v2/deployment/stats \
-H 'Accept: application/json' \
-H 'Coder-Session-Token: API_KEY'
GET /deployment/stats
Example responses
200 Response
{
"aggregated_from": "2019-08-24T14:15:22Z",
"collected_at": "2019-08-24T14:15:22Z",
"next_update_at": "2019-08-24T14:15:22Z",
"session_count": {
"jetbrains": 0,
"reconnecting_pty": 0,
"ssh": 0,
"vscode": 0
},
"workspaces": {
"building": 0,
"connection_latency_ms": {
"p50": 0,
"p95": 0
},
"failed": 0,
"pending": 0,
"running": 0,
"rx_bytes": 0,
"stopped": 0,
"tx_bytes": 0
}
}
Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | OK | codersdk.DeploymentStats |
To perform this operation, you must be authenticated. Learn more.
Get enabled experiments
Code samples
# Example request using curl
curl -X GET http://coder-server:8080/api/v2/experiments \
-H 'Accept: application/json' \
-H 'Coder-Session-Token: API_KEY'
GET /experiments
Example responses
200 Response
[
"example"
]
Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | OK | array of codersdk.Experiment |
Response Schema
Status Code 200
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
[array item] |
array | false |
To perform this operation, you must be authenticated. Learn more.
Get safe experiments
Code samples
# Example request using curl
curl -X GET http://coder-server:8080/api/v2/experiments/available \
-H 'Accept: application/json' \
-H 'Coder-Session-Token: API_KEY'
GET /experiments/available
Example responses
200 Response
[
"example"
]
Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | OK | array of codersdk.Experiment |
Response Schema
Status Code 200
| Name | Type | Required | Restrictions | Description |
|---|---|---|---|---|
[array item] |
array | false |
To perform this operation, you must be authenticated. Learn more.
Update check
Code samples
# Example request using curl
curl -X GET http://coder-server:8080/api/v2/updatecheck \
-H 'Accept: application/json'
GET /updatecheck
Example responses
200 Response
{
"current": true,
"url": "string",
"version": "string"
}
Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | OK | codersdk.UpdateCheckResponse |
Get token config
Code samples
# Example request using curl
curl -X GET http://coder-server:8080/api/v2/users/{user}/keys/tokens/tokenconfig \
-H 'Accept: application/json' \
-H 'Coder-Session-Token: API_KEY'
GET /users/{user}/keys/tokens/tokenconfig
Parameters
| Name | In | Type | Required | Description |
|---|---|---|---|---|
user |
path | string | true | User ID, name, or me |
Example responses
200 Response
{
"max_token_lifetime": 0
}
Responses
| Status | Meaning | Description | Schema |
|---|---|---|---|
| 200 | OK | OK | codersdk.TokenConfig |
To perform this operation, you must be authenticated. Learn more.