mirror of
https://github.com/Kilo-Org/kilocode.git
synced 2026-08-28 19:11:03 +08:00
docs(cli): add chunkTimeout to provider options and propagate to streamText
Add chunkTimeout propagation to Vercel AI SDK streamText timeout.chunkMs for silent stream dropout detection. The config option already existed in the provider schema and worked at the fetch wrapper level, but was not passed through to the streamText call. This adds dual-layer protection at both the fetch layer and the SDK stream consumer layer. Also updates docs to document chunkTimeout with failure scenario and recommended values (15-30s).
This commit is contained in:
@@ -375,6 +375,7 @@ You can also set options that apply to all models from a provider:
|
||||
| `apiKey` | `string` | API key (supports `{env:VAR}` syntax) |
|
||||
| `baseURL` | `string` | Override the provider's base API URL |
|
||||
| `timeout` | `number \| false` | Request timeout in milliseconds. Defaults to `300000` (5 minutes); set to `false` to disable |
|
||||
| `chunkTimeout` | `number` | Timeout in milliseconds between streamed response chunks. If no chunk arrives within this window, the request is aborted and retried. This catches silent provider dropouts where the TCP connection stays open but SSE streaming stops. Recommended: `15000`–`30000` (15–30 seconds) for providers with unreliable streaming. |
|
||||
|
||||
## Filtering Available Models
|
||||
|
||||
|
||||
@@ -336,6 +336,18 @@ const live: Layer.Layer<
|
||||
)
|
||||
// Default runtime path: AI SDK owns provider execution and tool dispatch;
|
||||
// LLMAISDK.toLLMEvents below normalizes fullStream parts for the processor.
|
||||
// Pass chunkTimeout from provider/model options to streamText for AI SDK-level
|
||||
// chunk idle timeout detection. This catches silenced stream dropouts where
|
||||
// the TCP connection stays open but no SSE chunks arrive. // kilocode_change
|
||||
const chunkTimeout =
|
||||
typeof prepared.params.options["chunkTimeout"] === "number"
|
||||
? prepared.params.options["chunkTimeout"]
|
||||
: undefined // kilocode_change
|
||||
if (chunkTimeout) {
|
||||
// kilocode_change
|
||||
l.debug("chunk idle timeout configured", { chunkTimeout }) // kilocode_change
|
||||
} // kilocode_change
|
||||
|
||||
const result = streamText({
|
||||
// kilocode_change
|
||||
onError(error) {
|
||||
@@ -373,6 +385,9 @@ const live: Layer.Layer<
|
||||
toolChoice: input.toolChoice,
|
||||
maxOutputTokens: prepared.params.maxOutputTokens,
|
||||
abortSignal: input.abort,
|
||||
// Stream chunk idle timeout: aborts if no SSE chunk arrives within the window.
|
||||
// Catches silent provider dropouts that keep the TCP connection alive but stop sending data. // kilocode_change
|
||||
...(chunkTimeout ? { timeout: { chunkMs: chunkTimeout } } : {}), // kilocode_change
|
||||
headers: prepared.headers,
|
||||
maxRetries: input.retries ?? 0,
|
||||
messages: prepared.messages,
|
||||
|
||||
Reference in New Issue
Block a user