mirror of
https://github.com/coder/coder.git
synced 2026-10-16 14:48:56 +08:00
Ethan
77479408b4
The `chatloop` stream silence guard cancels and retries a model attempt after `10m` of silence, and it was hard-coded. It's armed before the request opens, so it also bounds time to first token, and slow local models (e.g. `llama-server` on a large prompt) can spend longer than that on prompt processing alone, meaning every attempt just gets cancelled and retried. This PR adds `--chat-stream-silence-timeout` / `CODER_CHAT_STREAM_SILENCE_TIMEOUT` (default `10m`) and passes it to both the main loop and the advisor. Setting it to `0` disables the guard entirely, and anything above `24h` is rejected, which also keeps the watchdog arithmetic below from overflowing. Whilst doing this I ran into the runner's `15m` task watchdog, which kept counting during an open stream. That silently cancelled any healthy stream longer than `15m`, and would've stopped a silence timeout above `15m` from ever firing. To fix, the silence guard now kicks the watchdog out to `max(15m, silence + 5m)` whenever it arms or sees a part. Silent streams still fail through the guard, healthy streams can run for as long as they like, and hang detection is unchanged everywhere else. With the guard disabled there's nothing to kick against, so arming it just stops the watchdog for the rest of the attempt. That does mean the DB write after the stream isn't bounded either when `0` is set; I think that's fine for an opt-in setting, and a user can still stop the chat from the UI if it wedges there. I did consider removing the guard entirely, but the provider HTTP clients have no timeout, so a provider that holds the connection open would only ever be caught as a generic task hang, which is what #25782 was fixing.