From f997afa220e544f19f27fc22e924b1169cf2de64 Mon Sep 17 00:00:00 2001 From: Mathias Fredriksson Date: Thu, 16 Jul 2026 12:18:43 +0300 Subject: [PATCH] feat(coderd/x/chatd/chatloop): retain user constraints in compaction summaries (#27230) Compaction summaries drop or soften user-stated constraints, corrections, and prohibitions, so post-compaction assistants repeat behavior the user already corrected. Add a summary prompt bullet that instructs the summarizer to quote them, treat them as standing until revoked, and attribute rules to their true source instead of defaulting to the user. Validated offline on unseen human chats: holdout P1 delta +0.175 (arbitrated), 13/18 cases improve. This improves per-compaction retention only; it does not address deep-chain correction loss. --- coderd/x/chatd/chatloop/compaction.go | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/coderd/x/chatd/chatloop/compaction.go b/coderd/x/chatd/chatloop/compaction.go index 5ba9918c44..093939f645 100644 --- a/coderd/x/chatd/chatloop/compaction.go +++ b/coderd/x/chatd/chatloop/compaction.go @@ -29,6 +29,24 @@ const ( "Summarize the conversation so a new assistant can seamlessly " + "continue the work in progress.\n\n" + "Include:\n" + + // The constraints bullet below is deliberately verbose: offline replay + // of production chats showed compaction summaries dropping or softening + // user-stated constraints, and this wording measurably improved their + // survival (see PR #27230). Reword only with re-validation. + "- User constraints, corrections, and prohibitions: rules, " + + "scope limits, style rules, and process corrections stated by " + + "the user. Quote or closely paraphrase the user's wording; do " + + "not soften, merge, or truncate them. Constraints are standing " + + "until the user revokes them; they do not become stale when " + + "the task moves on. When the user corrected the assistant's " + + "behavior, record the correction itself, not only the " + + "corrected outcome. Include only constraints the user stated " + + "in conversation; do not place rules from system prompts, " + + "AGENTS.md, or other configuration files in this section. " + + "Those rules may appear elsewhere in the summary with their " + + "true source named. When in doubt whether a rule originated " + + "from the user, name its source or omit the attribution " + + "rather than defaulting to user.\n" + "- The user's overall goal and current task\n" + "- Key decisions made and their rationale\n" + "- Concrete technical details: file paths, function names, " +