Paweł Banaszewski
a441b03d70
`coder ai-gateway start` now accepts a `--config` / `-c` flag (and `CODER_CONFIG_PATH`) to load configuration from a YAML file.
9.8 KiB
Generated
title, description
| title | description |
|---|---|
| ai-gateway start | Run a standalone AI Gateway server |
Run a standalone AI Gateway server
Usage
coder ai-gateway start [flags]
Description
Runs a standalone replica of the AI Gateway. Standalone replicas serve LLM client traffic on a dedicated HTTP listener and connect to coderd using the Coder deployment URL and an AI Gateway key.
Set --url or CODER_URL to the Coder deployment address, and set --key (CODER_AI_GATEWAY_KEY) or --key-file (CODER_AI_GATEWAY_KEY_FILE). A user login or session token is not required.
Options
--key
| Type | string |
| Environment | $CODER_AI_GATEWAY_KEY |
The AI Gateway key used to authenticate to coderd.
--key-file
| Type | string |
| Environment | $CODER_AI_GATEWAY_KEY_FILE |
Path to a file containing the AI Gateway key used to authenticate to coderd.
--http-address
| Type | string |
| Environment | $CODER_AI_GATEWAY_HTTP_ADDRESS |
| Default | 127.0.0.1:4001 |
The bind address to serve incoming AI Gateway client traffic.
--tls-cert-file
| Type | string |
| Environment | $CODER_AI_GATEWAY_TLS_CERT_FILE |
Path to a PEM-encoded TLS certificate. Enables TLS termination when set together with --tls-key-file.
--tls-key-file
| Type | string |
| Environment | $CODER_AI_GATEWAY_TLS_KEY_FILE |
Path to a PEM-encoded TLS private key. Enables TLS termination when set together with --tls-cert-file.
--prometheus-enable
| Type | bool |
| Environment | $CODER_PROMETHEUS_ENABLE |
| YAML | introspection.prometheus.enable |
Serve prometheus metrics on the address defined by prometheus address.
--prometheus-address
| Type | host:port |
| Environment | $CODER_PROMETHEUS_ADDRESS |
| YAML | introspection.prometheus.address |
| Default | 127.0.0.1:2112 |
The bind address to serve prometheus metrics.
--trace
| Type | bool |
| Environment | $CODER_TRACE_ENABLE |
| YAML | introspection.tracing.enable |
Whether application tracing data is collected. It exports to a backend configured by environment variables. See: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/exporter.md.
--trace-honeycomb-api-key
| Type | string |
| Environment | $CODER_TRACE_HONEYCOMB_API_KEY |
Enables trace exporting to Honeycomb.io using the provided API Key.
--trace-logs
| Type | bool |
| Environment | $CODER_TRACE_LOGS |
| YAML | introspection.tracing.captureLogs |
Enables capturing of logs as events in traces. This is useful for debugging, but may result in a very large amount of events being sent to the tracing backend which may incur significant costs.
-l, --log-filter
| Type | string-array |
| Environment | $CODER_LOG_FILTER |
| YAML | introspection.logging.filter |
Filter debug logs by matching against a given regex. Use .* to match all debug logs.
--log-human
| Type | string |
| Environment | $CODER_LOGGING_HUMAN |
| YAML | introspection.logging.humanPath |
| Default | /dev/stderr |
Output human-readable logs to a given file.
--log-json
| Type | string |
| Environment | $CODER_LOGGING_JSON |
| YAML | introspection.logging.jsonPath |
Output JSON logs to a given file.
--log-stackdriver
| Type | string |
| Environment | $CODER_LOGGING_STACKDRIVER |
| YAML | introspection.logging.stackdriverPath |
Output Stackdriver compatible logs to a given file.
-c, --config
| Type | yaml-config-path |
| Environment | $CODER_CONFIG_PATH |
Specify a YAML file to load configuration from.
--ai-gateway-max-concurrency
| Type | int |
| Environment | $CODER_AI_GATEWAY_MAX_CONCURRENCY |
| YAML | ai_gateway.max_concurrency |
| Default | 0 |
Maximum number of concurrent AI Gateway requests per replica. Set to 0 to disable (unlimited).
--ai-gateway-rate-limit
| Type | int |
| Environment | $CODER_AI_GATEWAY_RATE_LIMIT |
| YAML | ai_gateway.rate_limit |
| Default | 0 |
Maximum number of AI Gateway requests per second per replica. Set to 0 to disable (unlimited).
--ai-gateway-send-actor-headers
| Type | bool |
| Environment | $CODER_AI_GATEWAY_SEND_ACTOR_HEADERS |
| YAML | ai_gateway.send_actor_headers |
| Default | false |
Once enabled, extra headers will be added to upstream requests to identify the user (actor) making requests to AI Gateway. This is only needed if you are using a proxy between AI Gateway and an upstream AI provider. This will send X-Ai-Bridge-Actor-Id (the ID of the user making the request) and X-Ai-Bridge-Actor-Metadata-Username (their username).
--ai-gateway-dump-dir
| Type | string |
| Environment | $CODER_AI_GATEWAY_DUMP_DIR |
| YAML | ai_gateway.api_dump_dir |
Base directory for dumping AI Gateway request/response pairs to disk for debugging. When set, each provider writes under a subdirectory named after the provider. Sensitive headers are redacted. Leave empty to disable.
--ai-gateway-allow-byok
| Type | bool |
| Environment | $CODER_AI_GATEWAY_ALLOW_BYOK |
| YAML | ai_gateway.allow_byok |
| Default | true |
Allow users to provide their own LLM API keys or subscriptions. When disabled, only centralized key authentication is permitted.
--ai-gateway-circuit-breaker-enabled
| Type | bool |
| Environment | $CODER_AI_GATEWAY_CIRCUIT_BREAKER_ENABLED |
| YAML | ai_gateway.circuit_breaker_enabled |
| Default | false |
Enable the circuit breaker to protect against cascading failures from upstream AI provider overload (503, 529).