docs(cli): add chunkTimeout to provider options and propagate to streamText

Add chunkTimeout propagation to Vercel AI SDK streamText timeout.chunkMs
for silent stream dropout detection. The config option already existed in
the provider schema and worked at the fetch wrapper level, but was not
passed through to the streamText call. This adds dual-layer protection
at both the fetch layer and the SDK stream consumer layer.

Also updates docs to document chunkTimeout with failure scenario and
recommended values (15-30s).
This commit is contained in:
Fatty911
2026-05-03 19:54:10 +08:00
committed by Catriel Müller
parent b0e7f4ac48
commit c3e15d1486
2 changed files with 16 additions and 0 deletions
@@ -375,6 +375,7 @@ You can also set options that apply to all models from a provider:
| `apiKey` | `string` | API key (supports `{env:VAR}` syntax) |
| `baseURL` | `string` | Override the provider's base API URL |
| `timeout` | `number \| false` | Request timeout in milliseconds. Defaults to `300000` (5 minutes); set to `false` to disable |
| `chunkTimeout` | `number` | Timeout in milliseconds between streamed response chunks. If no chunk arrives within this window, the request is aborted and retried. This catches silent provider dropouts where the TCP connection stays open but SSE streaming stops. Recommended: `15000``30000` (1530 seconds) for providers with unreliable streaming. |
## Filtering Available Models
+15
View File
@@ -336,6 +336,18 @@ const live: Layer.Layer<
)
// Default runtime path: AI SDK owns provider execution and tool dispatch;
// LLMAISDK.toLLMEvents below normalizes fullStream parts for the processor.
// Pass chunkTimeout from provider/model options to streamText for AI SDK-level
// chunk idle timeout detection. This catches silenced stream dropouts where
// the TCP connection stays open but no SSE chunks arrive. // kilocode_change
const chunkTimeout =
typeof prepared.params.options["chunkTimeout"] === "number"
? prepared.params.options["chunkTimeout"]
: undefined // kilocode_change
if (chunkTimeout) {
// kilocode_change
l.debug("chunk idle timeout configured", { chunkTimeout }) // kilocode_change
} // kilocode_change
const result = streamText({
// kilocode_change
onError(error) {
@@ -373,6 +385,9 @@ const live: Layer.Layer<
toolChoice: input.toolChoice,
maxOutputTokens: prepared.params.maxOutputTokens,
abortSignal: input.abort,
// Stream chunk idle timeout: aborts if no SSE chunk arrives within the window.
// Catches silent provider dropouts that keep the TCP connection alive but stop sending data. // kilocode_change
...(chunkTimeout ? { timeout: { chunkMs: chunkTimeout } } : {}), // kilocode_change
headers: prepared.headers,
maxRetries: input.retries ?? 0,
messages: prepared.messages,