# ai-gateway start Run a standalone AI Gateway server ## Usage ```console coder ai-gateway start [flags] ``` ## Description ```console Runs a standalone replica of the AI Gateway. Standalone replicas serve LLM client traffic on a dedicated HTTP listener and connect to coderd using the Coder deployment URL and an AI Gateway key. Set --url or CODER_URL to the Coder deployment address, and set --key (CODER_AI_GATEWAY_KEY) or --key-file (CODER_AI_GATEWAY_KEY_FILE). A user login or session token is not required. ``` ## Options ### --key | | | |-------------|------------------------------------| | Type | string | | Environment | $CODER_AI_GATEWAY_KEY | The AI Gateway key used to authenticate to coderd. ### --key-file | | | |-------------|-----------------------------------------| | Type | string | | Environment | $CODER_AI_GATEWAY_KEY_FILE | Path to a file containing the AI Gateway key used to authenticate to coderd. ### --http-address | | | |-------------|---------------------------------------------| | Type | string | | Environment | $CODER_AI_GATEWAY_HTTP_ADDRESS | | Default | 127.0.0.1:4001 | The bind address to serve incoming AI Gateway client traffic. ### --tls-cert-file | | | |-------------|----------------------------------------------| | Type | string | | Environment | $CODER_AI_GATEWAY_TLS_CERT_FILE | Path to a PEM-encoded TLS certificate. Enables TLS termination when set together with --tls-key-file. ### --tls-key-file | | | |-------------|---------------------------------------------| | Type | string | | Environment | $CODER_AI_GATEWAY_TLS_KEY_FILE | Path to a PEM-encoded TLS private key. Enables TLS termination when set together with --tls-cert-file. ### --prometheus-enable | | | |-------------|----------------------------------------------| | Type | bool | | Environment | $CODER_PROMETHEUS_ENABLE | | YAML | introspection.prometheus.enable | Serve prometheus metrics on the address defined by prometheus address. ### --prometheus-address | | | |-------------|-----------------------------------------------| | Type | host:port | | Environment | $CODER_PROMETHEUS_ADDRESS | | YAML | introspection.prometheus.address | | Default | 127.0.0.1:2112 | The bind address to serve prometheus metrics. ### --trace | | | |-------------|-------------------------------------------| | Type | bool | | Environment | $CODER_TRACE_ENABLE | | YAML | introspection.tracing.enable | Whether application tracing data is collected. It exports to a backend configured by environment variables. See: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/exporter.md. ### --trace-honeycomb-api-key | | | |-------------|---------------------------------------------| | Type | string | | Environment | $CODER_TRACE_HONEYCOMB_API_KEY | Enables trace exporting to Honeycomb.io using the provided API Key. ### --trace-logs | | | |-------------|------------------------------------------------| | Type | bool | | Environment | $CODER_TRACE_LOGS | | YAML | introspection.tracing.captureLogs | Enables capturing of logs as events in traces. This is useful for debugging, but may result in a very large amount of events being sent to the tracing backend which may incur significant costs. ### -l, --log-filter | | | |-------------|-------------------------------------------| | Type | string-array | | Environment | $CODER_LOG_FILTER | | YAML | introspection.logging.filter | Filter debug logs by matching against a given regex. Use .* to match all debug logs. ### --log-human | | | |-------------|----------------------------------------------| | Type | string | | Environment | $CODER_LOGGING_HUMAN | | YAML | introspection.logging.humanPath | | Default | /dev/stderr | Output human-readable logs to a given file. ### --log-json | | | |-------------|---------------------------------------------| | Type | string | | Environment | $CODER_LOGGING_JSON | | YAML | introspection.logging.jsonPath | Output JSON logs to a given file. ### --log-stackdriver | | | |-------------|----------------------------------------------------| | Type | string | | Environment | $CODER_LOGGING_STACKDRIVER | | YAML | introspection.logging.stackdriverPath | Output Stackdriver compatible logs to a given file. ### --ai-gateway-max-concurrency | | | |-------------|------------------------------------------------| | Type | int | | Environment | $CODER_AI_GATEWAY_MAX_CONCURRENCY | | YAML | ai_gateway.max_concurrency | | Default | 0 | Maximum number of concurrent AI Gateway requests per replica. Set to 0 to disable (unlimited). ### --ai-gateway-rate-limit | | | |-------------|-------------------------------------------| | Type | int | | Environment | $CODER_AI_GATEWAY_RATE_LIMIT | | YAML | ai_gateway.rate_limit | | Default | 0 | Maximum number of AI Gateway requests per second per replica. Set to 0 to disable (unlimited). ### --ai-gateway-send-actor-headers | | | |-------------|---------------------------------------------------| | Type | bool | | Environment | $CODER_AI_GATEWAY_SEND_ACTOR_HEADERS | | YAML | ai_gateway.send_actor_headers | | Default | false | Once enabled, extra headers will be added to upstream requests to identify the user (actor) making requests to AI Gateway. This is only needed if you are using a proxy between AI Gateway and an upstream AI provider. This will send X-Ai-Bridge-Actor-Id (the ID of the user making the request) and X-Ai-Bridge-Actor-Metadata-Username (their username). ### --ai-gateway-dump-dir | | | |-------------|-----------------------------------------| | Type | string | | Environment | $CODER_AI_GATEWAY_DUMP_DIR | | YAML | ai_gateway.api_dump_dir | Base directory for dumping AI Gateway request/response pairs to disk for debugging. When set, each provider writes under a subdirectory named after the provider. Sensitive headers are redacted. Leave empty to disable. ### --ai-gateway-allow-byok | | | |-------------|-------------------------------------------| | Type | bool | | Environment | $CODER_AI_GATEWAY_ALLOW_BYOK | | YAML | ai_gateway.allow_byok | | Default | true | Allow users to provide their own LLM API keys or subscriptions. When disabled, only centralized key authentication is permitted. ### --ai-gateway-circuit-breaker-enabled | | | |-------------|--------------------------------------------------------| | Type | bool | | Environment | $CODER_AI_GATEWAY_CIRCUIT_BREAKER_ENABLED | | YAML | ai_gateway.circuit_breaker_enabled | | Default | false | Enable the circuit breaker to protect against cascading failures from upstream AI provider overload (503, 529).