diff --git a/docs/plans/2026-03-26-hook-payoff-and-english-variance-design.md b/docs/plans/2026-03-26-hook-payoff-and-english-variance-design.md new file mode 100644 index 00000000..de36c20b --- /dev/null +++ b/docs/plans/2026-03-26-hook-payoff-and-english-variance-design.md @@ -0,0 +1,295 @@ +# Hook Payoff and English Variance Redesign + +## Goal + +Improve InkOS v2 long-book quality by fixing the hook/payoff control loop and by reducing English long-span repetition without replacing the writing pipeline with hard rules. + +## Problem Statement + +The latest 30-chapter acceptance runs show the same structural failure in two different surface forms: + +1. Hook tables look active, but payoff delivery is weak. +2. English chapters stay length-stable, but drift toward repeated mechanism restatement and familiar sentence shapes. + +The underlying issue is not "the model forgot to be creative." The issue is that the system currently rewards the wrong things. + +### Current Hook Failure Mode + +The v2 pipeline already improved context trimming, retrieval, and structured state safety, but the hook loop still has four bad incentives: + +1. Too many things become hooks. + A clue, suspicion, or restated implication can be promoted into `pending_hooks.md` even when it is not a real future-payoff obligation. +2. Mention and advance are conflated. + A hook can be treated as "progressed" even when the chapter only rephrases an old implication or points at the same problem from a slightly different angle. +3. Planner does not schedule payoff work. + `PlannerAgent` chooses chapter intent, but it does not tell the writer or settler which old hooks should be advanced, deferred, or resolved in this chapter. +4. Settler sees an incomplete governed hook set. + Composer retrieval can surface stale unresolved hooks, but governed settlement still narrows its working set to selected hooks plus a recent window. Debt hooks can still miss the final settlement step. + +This creates the exact pattern seen in the Chinese acceptance run: + +- hook count grows too fast +- many rows are marked as "持续推进" or "暂缓" while the notes say "无新增" or "待回收" +- almost nothing resolves + +That is not healthy continuity. It is pseudo-progress. + +### Current English Failure Mode + +The English book has a different surface failure: + +- chapter length is stable +- AI tell markers are low +- but the book keeps reusing the same sentence skeletons and the same "explain the mechanism one more time" climax shape + +This happens because the system currently does much more post-write warning than pre-write guidance for long-span variance. It can detect fatigue after the fact, but it does not feed a compact, chapter-local variance brief into the writer before drafting. + +## Scope + +This redesign applies to the v2 governed writing path. + +In scope: + +- Hook admission rules for `pending_hooks` +- Hook agenda generation in planner +- Governed settlement visibility for stale hook debt +- Distinguishing mention from real hook advancement +- Hook debt warnings in audit +- English pre-write variance guidance + +Out of scope: + +- Legacy mode behavior changes +- A mandatory database runtime +- A new reviewer agent +- Hard "resolve one hook every N chapters" quotas +- A heavy workflow engine or finite-state machine for story logic + +## Design Principles + +1. Keep prose decisions with the LLM. +2. Add structure only where the current system is rewarding bad behavior. +3. A hook is a future-payoff obligation, not a generic clue bucket. +4. Mention is not advancement. +5. Resolve only when earned, not because the calendar says so. +6. Pre-write variance guidance is better than post-write punishment. +7. v2 should get stricter; legacy can stay loose. + +## Proposed Design + +## 1. Add a Hook Admission Gate + +The system needs a smaller, stricter definition of what belongs in `pending_hooks`. + +A new hook should be admitted only if all of the following are true: + +- it creates a concrete unresolved question or obligation that reasonably survives beyond the current chapter +- it is not just a restatement of an existing active hook +- it has a plausible payoff horizon + +If the candidate fails that gate, it should not become a new hook. It should instead go to one of these places: + +- `chapterSummary.hookActivity` +- current-state facts +- notes for an existing hook + +This is the main control that reduces hook explosion. + +### Why This Matters + +Right now the system is implicitly rewarding "open more rows." That makes the hook table look rich while weakening actual book-level payoff. A smaller active hook set is better than a larger decorative one. + +## 2. Add Hook Agenda to Planner Intent + +Planner should stop being hook-neutral. + +`ChapterIntent` should gain a small structured hook agenda, for example: + +```json +{ + "hookAgenda": { + "mustAdvance": ["H019"], + "eligibleResolve": ["H045"], + "staleDebt": ["H023", "H027"], + "avoidNewHookFamilies": [ + "anonymous-source-restatement", + "mechanism-restatement" + ] + } +} +``` + +This is not a rigid script. It is a bounded control brief. + +Planner's job becomes: + +- pick 1-2 active hooks that this chapter should materially move +- expose 0-1 hooks that are now eligible for payoff +- surface 1-2 stale debt hooks that have been ignored too long +- tell the chapter not to open duplicate hook families + +This keeps the writer focused without forcing a chapter outline. + +## 3. Expand the Governed Settlement Hook Working Set + +Governed settlement should no longer use only: + +- hooks selected into context +- hooks within a recent chapter window + +It should instead use the union of: + +- selected hooks from retrieval +- recent hooks +- planner `mustAdvance` +- planner `eligibleResolve` +- planner `staleDebt` + +That closes the retrieval blind spot where Composer may surface a stale hook, but Settler still fails to process it. + +## 4. Distinguish Mention, Advance, Resolve, and Defer + +`RuntimeStateDelta.hookOps` currently supports `upsert`, `resolve`, and `defer`, but in practice the system still overuses narrative restatement as fake advancement. + +The delta model should explicitly distinguish: + +- `mention`: the chapter references the hook, but no state change happens +- `advance`: the hook's information state changes +- `resolve`: the payoff lands or the obligation closes +- `defer`: the hook remains open, but the system records that it is intentionally delayed + +This prevents a chapter from updating `lastAdvancedChapter` just because it nodded at an old idea. + +### Practical Rule + +`lastAdvancedChapter` changes only on real `advance`. + +If a hook is merely visible in the chapter, record a mention or do nothing. Do not pretend it progressed. + +## 5. Add Hook Debt Health Checks + +We should not impose a blind rule like "resolve one hook every N chapters." That would create fake payoffs. + +We should add a softer but still enforceable health rule: + +- if an active hook stays stale too long, the system must at least disposition it +- the disposition can be `advance`, `resolve`, or `defer` +- `defer` requires a reason, not an empty placeholder + +This creates debt pressure without forcing artificial closure. + +Warnings should focus on: + +- too many active hooks +- too many chapters with no real hook advancement +- too many stale hooks with no disposition +- suspicious growth where several new hooks appear while no old ones resolve + +## 6. Add English Pre-Write Variance Guidance + +The English issue should not be solved with a giant banlist. + +Instead, the writer should receive a compact variance brief before drafting: + +- high-frequency phrase warnings across the last 20-30 chapters +- repeated sentence-opening patterns +- overused chapter-ending beat shapes +- a chapter-local scene obligation + +Example scene obligations: + +- confrontation +- negotiation +- pursuit +- concealment +- aftermath +- discovery under pressure + +This prevents English chapters from all climaxing as "Mara restates the mechanism one level more clearly." + +### Dialogue Guidance + +Do not enforce a global dialogue percentage. + +Do require that if a chapter contains multiple active characters, it should usually include at least one resistance-bearing exchange: + +- someone pushes back +- someone withholds +- someone misreads intent +- someone pressures status or legitimacy + +This is a better anti-monotony control than raw quote density. + +## Data Flow Changes + +### Planning + +`PlannerAgent` + +- reads current state plus active hooks +- computes hook debt +- emits `hookAgenda` inside `ChapterIntent` +- renders the agenda into `intent.md` + +### Composition + +`ComposerAgent` + +- keeps current retrieval behavior +- includes hook agenda in the governed package indirectly through chapter intent and selected hook evidence + +### Writing + +`WriterAgent` + +- receives the hook agenda and English variance brief +- uses them as bounded controls, not outline replacement + +### Settlement + +`Settler` + +- sees selected + recent + agenda + stale debt hooks +- cannot treat "restated concern" as automatic advancement +- emits `mention/advance/resolve/defer` semantics + +### Audit + +- warns on hook bloat +- warns on pseudo-progress +- warns on stale debt without disposition +- warns on English long-span phrase fatigue and repeated beat shape + +## Non-Goals + +This redesign is deliberately not trying to: + +- make every hook deterministic +- cap writer freedom with strict per-chapter quotas +- solve all prose quality issues with validators +- move product control into SQLite + +SQLite may remain useful as an acceleration layer, but it is not the solution to hook quality. The real fix is better state semantics and better pre-write control. + +## Expected Outcomes + +If this design works, the next 30-chapter acceptance should show: + +- materially fewer active hooks +- non-zero real resolutions in Chinese and English +- fewer "no new movement" rows masquerading as progress +- lower English phrase recurrence across chapters +- more variation in chapter scene shape +- more stable book-level payoff rhythm without heavier rules + +## Rollout Strategy + +Roll out only on v2. + +Legacy mode can keep current behavior. + +That keeps the redesign targeted and makes acceptance results easier to compare: + +- old v2 behavior +- new v2 behavior +- legacy fallback if needed for debugging diff --git a/docs/plans/2026-03-26-hook-payoff-and-english-variance-implementation-plan.md b/docs/plans/2026-03-26-hook-payoff-and-english-variance-implementation-plan.md new file mode 100644 index 00000000..db289707 --- /dev/null +++ b/docs/plans/2026-03-26-hook-payoff-and-english-variance-implementation-plan.md @@ -0,0 +1,400 @@ +# Hook Payoff and English Variance Implementation Plan + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Fix v2 hook/payoff control so long books stop accumulating pseudo-progress hooks, and add pre-write English variance guidance that reduces repeated mechanism-restatement chapters. + +**Architecture:** Extend governed planning with a hook agenda, tighten settlement semantics so mention and advancement are no longer conflated, widen governed settlement visibility to include stale hook debt, and feed a compact long-span variance brief into the writer before English drafting. Keep the LLM in charge of prose; use structure only to control debt, admission, and repetition pressure. + +**Tech Stack:** TypeScript, Node.js, Vitest, Zod, existing planner/composer/writer/settler/state-reducer pipeline + +--- + +### Task 1: Add hook agenda and mention semantics to schemas + +**Files:** +- Modify: `packages/core/src/models/input-governance.ts` +- Modify: `packages/core/src/models/runtime-state.ts` +- Test: `packages/core/src/__tests__/models.test.ts` + +**Step 1: Write the failing tests** + +- Add schema tests for: + - `ChapterIntent.hookAgenda.mustAdvance` + - `ChapterIntent.hookAgenda.eligibleResolve` + - `ChapterIntent.hookAgenda.staleDebt` + - `ChapterIntent.hookAgenda.avoidNewHookFamilies` + - `RuntimeStateDelta.hookOps.mention` + +**Step 2: Run test to verify it fails** + +Run: `pnpm --filter @actalk/inkos-core test -- src/__tests__/models.test.ts` + +Expected: schema parse failure for missing fields or missing `mention` support. + +**Step 3: Write minimal implementation** + +- Extend `ChapterIntentSchema` with a `hookAgenda` object and safe defaults. +- Extend `HookOpsSchema` with `mention: string[]`. +- Export the new types from existing model files. + +**Step 4: Run test to verify it passes** + +Run the same command and confirm green. + +**Step 5: Commit** + +```bash +git add packages/core/src/models/input-governance.ts packages/core/src/models/runtime-state.ts packages/core/src/__tests__/models.test.ts +git commit -m "feat: add hook agenda and mention schemas" +``` + +### Task 2: Build hook governance utilities + +**Files:** +- Create: `packages/core/src/utils/hook-governance.ts` +- Modify: `packages/core/src/index.ts` +- Test: `packages/core/src/__tests__/hook-governance.test.ts` + +**Step 1: Write the failing tests** + +- Add tests covering: + - stale hook debt detection from hook records + - hook admission gate rejecting duplicate/restated hook candidates + - chapter-local disposition classification: mention vs advance vs resolve vs defer + +**Step 2: Run test to verify it fails** + +Run: `pnpm --filter @actalk/inkos-core test -- src/__tests__/hook-governance.test.ts` + +Expected: missing module. + +**Step 3: Write minimal implementation** + +- Add helpers for: + - computing stale debt hooks + - checking whether a new hook candidate is payoff-bearing + - comparing a new hook candidate against active-hook payoff text + - classifying whether a chapter event is mention-only or true advancement + +**Step 4: Run test to verify it passes** + +Run the same command and confirm green. + +**Step 5: Commit** + +```bash +git add packages/core/src/utils/hook-governance.ts packages/core/src/index.ts packages/core/src/__tests__/hook-governance.test.ts +git commit -m "feat: add hook governance utilities" +``` + +### Task 3: Teach Planner to emit hook agenda + +**Files:** +- Modify: `packages/core/src/agents/planner.ts` +- Modify: `packages/core/src/utils/memory-retrieval.ts` +- Test: `packages/core/src/__tests__/planner.test.ts` +- Test: `packages/core/src/__tests__/memory-retrieval.test.ts` + +**Step 1: Write the failing tests** + +- Add planner tests showing: + - recent active hooks can be selected into `mustAdvance` + - stale unresolved hooks can be selected into `staleDebt` + - clearly mature hooks can be listed under `eligibleResolve` + - the agenda is rendered into `intent.md` + +**Step 2: Run tests to verify they fail** + +Run: `pnpm --filter @actalk/inkos-core test -- src/__tests__/planner.test.ts src/__tests__/memory-retrieval.test.ts` + +Expected: intent shape mismatch / rendered intent missing agenda details. + +**Step 3: Write minimal implementation** + +- Use active hook selection plus stale debt detection to build a bounded `hookAgenda`. +- Keep the selection small: 1-2 `mustAdvance`, 0-1 `eligibleResolve`, 1-2 `staleDebt`. +- Render a compact agenda block into planner runtime output. + +**Step 4: Run tests to verify they pass** + +Run the same command and confirm green. + +**Step 5: Commit** + +```bash +git add packages/core/src/agents/planner.ts packages/core/src/utils/memory-retrieval.ts packages/core/src/__tests__/planner.test.ts packages/core/src/__tests__/memory-retrieval.test.ts +git commit -m "feat: add hook agenda to planner intent" +``` + +### Task 4: Expand governed settlement working set with agenda and debt hooks + +**Files:** +- Modify: `packages/core/src/utils/governed-working-set.ts` +- Modify: `packages/core/src/agents/writer.ts` +- Test: `packages/core/src/__tests__/writer.test.ts` +- Test: `packages/core/src/__tests__/composer.test.ts` + +**Step 1: Write the failing tests** + +- Add tests showing governed settlement includes: + - selected hooks + - recent hooks + - planner `mustAdvance` + - planner `eligibleResolve` + - planner `staleDebt` + +**Step 2: Run tests to verify they fail** + +Run: `pnpm --filter @actalk/inkos-core test -- src/__tests__/writer.test.ts src/__tests__/composer.test.ts` + +Expected: working set excludes planner debt hooks. + +**Step 3: Write minimal implementation** + +- Extend `buildGovernedHookWorkingSet()` so it unions selected, recent, and agenda hooks. +- Pass planner intent into the working-set builder during governed settlement. +- Keep the returned snapshot bounded and deterministic. + +**Step 4: Run tests to verify they pass** + +Run the same command and confirm green. + +**Step 5: Commit** + +```bash +git add packages/core/src/utils/governed-working-set.ts packages/core/src/agents/writer.ts packages/core/src/__tests__/writer.test.ts packages/core/src/__tests__/composer.test.ts +git commit -m "feat: expose stale hook debt in governed settlement" +``` + +### Task 5: Tighten Settler output semantics and reducer behavior + +**Files:** +- Modify: `packages/core/src/agents/settler-prompts.ts` +- Modify: `packages/core/src/agents/settler-delta-parser.ts` +- Modify: `packages/core/src/state/state-reducer.ts` +- Test: `packages/core/src/__tests__/settler-delta-parser.test.ts` +- Test: `packages/core/src/__tests__/state-reducer.test.ts` + +**Step 1: Write the failing tests** + +- Add tests for: + - `mention` parsing + - mention-only hook rows not mutating `lastAdvancedChapter` + - true advancement mutating `lastAdvancedChapter` + - duplicate "restated hook as new hook" being rejected at reducer layer if classified as duplicate family + +**Step 2: Run tests to verify they fail** + +Run: `pnpm --filter @actalk/inkos-core test -- src/__tests__/settler-delta-parser.test.ts src/__tests__/state-reducer.test.ts` + +Expected: parser and reducer do not understand `mention` or duplicate-admission rejection. + +**Step 3: Write minimal implementation** + +- Update settler instructions so: + - mention does not count as advancement + - new hooks require future-payoff significance + - resolve and defer are explicit dispositions +- Parse the new payload shape. +- Keep reducer semantics strict: mention is no-op for chapter advancement; advance and resolve mutate state; defer must keep hook open but labeled. + +**Step 4: Run tests to verify they pass** + +Run the same command and confirm green. + +**Step 5: Commit** + +```bash +git add packages/core/src/agents/settler-prompts.ts packages/core/src/agents/settler-delta-parser.ts packages/core/src/state/state-reducer.ts packages/core/src/__tests__/settler-delta-parser.test.ts packages/core/src/__tests__/state-reducer.test.ts +git commit -m "feat: separate hook mention from advancement" +``` + +### Task 6: Add hook health warnings for pseudo-progress and debt + +**Files:** +- Create: `packages/core/src/utils/hook-health.ts` +- Modify: `packages/core/src/agents/writer.ts` +- Modify: `packages/core/src/pipeline/runner.ts` +- Test: `packages/core/src/__tests__/hook-health.test.ts` +- Test: `packages/core/src/__tests__/pipeline-runner.test.ts` + +**Step 1: Write the failing tests** + +- Add tests for warnings on: + - too many active hooks + - several chapters with no real advancement + - stale hooks without disposition + - bursts of new hooks without any old-hook resolution + +**Step 2: Run tests to verify they fail** + +Run: `pnpm --filter @actalk/inkos-core test -- src/__tests__/hook-health.test.ts src/__tests__/pipeline-runner.test.ts` + +Expected: no hook-health warnings emitted. + +**Step 3: Write minimal implementation** + +- Create a hook-health analyzer over structured hook state and current chapter delta. +- Feed warnings into the existing audit/reporting path. +- Keep warnings informative, not blocking. + +**Step 4: Run tests to verify they pass** + +Run the same command and confirm green. + +**Step 5: Commit** + +```bash +git add packages/core/src/utils/hook-health.ts packages/core/src/agents/writer.ts packages/core/src/pipeline/runner.ts packages/core/src/__tests__/hook-health.test.ts packages/core/src/__tests__/pipeline-runner.test.ts +git commit -m "feat: add hook debt and pseudo-progress warnings" +``` + +### Task 7: Add English long-span variance brief before writing + +**Files:** +- Modify: `packages/core/src/utils/long-span-fatigue.ts` +- Modify: `packages/core/src/agents/writer-prompts.ts` +- Modify: `packages/core/src/agents/writer.ts` +- Test: `packages/core/src/__tests__/long-span-fatigue.test.ts` +- Test: `packages/core/src/__tests__/writer-prompts.test.ts` + +**Step 1: Write the failing tests** + +- Add tests showing the variance brief can surface: + - high-frequency phrases across recent chapters + - repeated first-clause patterns + - repeated ending beat shapes + - a scene obligation string + +**Step 2: Run tests to verify they fail** + +Run: `pnpm --filter @actalk/inkos-core test -- src/__tests__/long-span-fatigue.test.ts src/__tests__/writer-prompts.test.ts` + +Expected: no pre-write variance brief exists. + +**Step 3: Write minimal implementation** + +- Extend long-span fatigue analysis to emit a compact variance brief instead of warnings only. +- Feed the brief only where useful, especially for English governed writing. +- Add scene obligations such as confrontation, negotiation, concealment, aftermath, or discovery under pressure. + +**Step 4: Run tests to verify they pass** + +Run the same command and confirm green. + +**Step 5: Commit** + +```bash +git add packages/core/src/utils/long-span-fatigue.ts packages/core/src/agents/writer-prompts.ts packages/core/src/agents/writer.ts packages/core/src/__tests__/long-span-fatigue.test.ts packages/core/src/__tests__/writer-prompts.test.ts +git commit -m "feat: add english pre-write variance guidance" +``` + +### Task 8: Strengthen dialogue-pressure guidance without hard quotas + +**Files:** +- Modify: `packages/core/src/agents/writer-prompts.ts` +- Modify: `packages/core/src/agents/post-write-validator.ts` +- Test: `packages/core/src/__tests__/writer-prompts.test.ts` +- Test: `packages/core/src/__tests__/post-write-validator.test.ts` + +**Step 1: Write the failing tests** + +- Add tests for: + - prompt guidance that asks for resistance-bearing exchanges when multi-character scenes are present + - validator warnings when a multi-character chapter contains almost no direct exchange + +**Step 2: Run tests to verify they fail** + +Run: `pnpm --filter @actalk/inkos-core test -- src/__tests__/writer-prompts.test.ts src/__tests__/post-write-validator.test.ts` + +Expected: prompt and validator have no such guidance. + +**Step 3: Write minimal implementation** + +- Add prompt language that prefers conflict-bearing dialogue rather than simple information transfer. +- Keep validator soft-warning only; do not enforce a hard global percentage threshold. + +**Step 4: Run tests to verify they pass** + +Run the same command and confirm green. + +**Step 5: Commit** + +```bash +git add packages/core/src/agents/writer-prompts.ts packages/core/src/agents/post-write-validator.ts packages/core/src/__tests__/writer-prompts.test.ts packages/core/src/__tests__/post-write-validator.test.ts +git commit -m "feat: guide dialogue pressure in multi-character scenes" +``` + +### Task 9: Add end-to-end governed tests for hook payoff and variance control + +**Files:** +- Modify: `packages/core/src/__tests__/pipeline-runner.test.ts` +- Modify: `packages/core/src/__tests__/writer.test.ts` + +**Step 1: Write the failing tests** + +- Add governed pipeline tests covering: + - planner emits hook agenda + - governed settlement sees debt hooks + - mention does not count as progress + - a stale hook can be deferred with reason + - English pre-write variance brief appears in prompt assembly + +**Step 2: Run tests to verify they fail** + +Run: `pnpm --filter @actalk/inkos-core test -- src/__tests__/pipeline-runner.test.ts src/__tests__/writer.test.ts` + +Expected: governed flow assertions fail. + +**Step 3: Write minimal implementation** + +- Wire any missing integration pieces uncovered by the tests. +- Keep test fixtures small and mock-driven. + +**Step 4: Run tests to verify they pass** + +Run the same command and confirm green. + +**Step 5: Commit** + +```bash +git add packages/core/src/__tests__/pipeline-runner.test.ts packages/core/src/__tests__/writer.test.ts +git commit -m "test: cover hook payoff control in governed pipeline" +``` + +### Task 10: Run focused acceptance verification + +**Files:** +- Modify: `docs/plans/2026-03-26-hook-payoff-and-english-variance-design.md` +- Modify: `docs/plans/2026-03-26-hook-payoff-and-english-variance-implementation-plan.md` + +**Step 1: Run targeted automated tests** + +Run: + +```bash +pnpm --filter @actalk/inkos-core test -- src/__tests__/models.test.ts src/__tests__/hook-governance.test.ts src/__tests__/planner.test.ts src/__tests__/writer.test.ts src/__tests__/pipeline-runner.test.ts src/__tests__/long-span-fatigue.test.ts src/__tests__/writer-prompts.test.ts src/__tests__/post-write-validator.test.ts +``` + +Expected: PASS + +**Step 2: Run a governed smoke for one Chinese and one English book** + +Run the existing project smoke/accept commands with v2 enabled and verify: + +- hook counts stop exploding +- at least one real resolution or explicit defer appears +- English prompt contains variance guidance +- no legacy path is used unless explicitly requested + +**Step 3: Record results** + +- Update these docs with actual outcomes, gaps, and any task ordering changes discovered during implementation. + +**Step 4: Commit** + +```bash +git add docs/plans/2026-03-26-hook-payoff-and-english-variance-design.md docs/plans/2026-03-26-hook-payoff-and-english-variance-implementation-plan.md +git commit -m "docs: record hook payoff redesign verification" +```