Compare commits

...
Author SHA1 Message Date
Robin NewhouseandClaude Sonnet 4.6 e8fd299c45 fix(sdk): harden compaction PRs 2183/2185/2191 per review
- basic-compaction: remove dead lastTurnStartIndex < 0 guard in
  splitLatestTurn (findLastTurnStartIndex returns 0 as sentinel,
  never -1). Add comment explaining the sentinel detection pattern.
- basic-compaction: add comment on removeCandidatesByPredicate
  documenting why the prefix path does not need the closure-safety
  check that removeTailCandidatesByPredicate has — prefix predicates
  are role-monotone so closures never straddle removable and
  non-removable candidates.
- message-builder: add comment in collectTruncationCandidates
  explaining that file blocks may have been pre-truncated by
  transformBlock; Layer A is a potential second cut when aggregate
  budget is still tight.
- message-builder.test: add test covering Layer A truncation of the
  typed-prompt user text block (CLINE-2191) — intentional behaviour
  that was previously untested.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-14 11:53:39 -07:00
Robin Newhouse 5325f3b9b7 fix(sdk): enforce hard provider request byte budget (CLINE-2192)
Draft Layer B hard-guarantee implementation. After Layer A truncation, MessageBuilder now counts the serialized provider request and applies emergency truncation when it still exceeds maxInputTokens * CHARS_PER_TOKEN.

Adds structured truncation for tool_use.input string leaves while preserving tool_use id/name/call_id and JSON shape, emits a user-visible status notice plus task.emergency_truncation telemetry when emergency truncation fires, and threads buildForApi options through SessionRuntime.

Current verification: typecheck:smoke passes; message-builder.test.ts passes (20 tests); compaction.test.ts passes (32 tests). Full @cline/core unit suite still has an unrelated hook-file-hooks.test.ts failure observed before this commit.
2026-05-14 11:34:43 -07:00
Robin Newhouse 5ee594eb3c fix(sdk): widen MessageBuilder truncation candidate set and tie budget to maxInputTokens (CLINE-2191)
CLINE-2191 Layer A. After PR #10739 (allowlist) and PR #10740 (protected-tail trim) the preservation set inside the in-flight turn still included the typed user prompt, the last assistant message, and any in-flight tool_use verbatim. Each can individually exceed any context window. This PR closes the common case.

Changes to MessageBuilder: (1) collectTruncationCandidates also collects user text, assistant text, thinking blocks, and top-level file blocks. tool_use input bodies remain untouched here — a JSON-aware structural truncator that avoids corrupting tool_use_id or breaking input JSON shape is Layer B (CLINE-2192). (2) buildForApi accepts an optional maxInputTokens; when present the aggregate budget becomes maxInputTokens * CHARS_PER_TOKEN, otherwise it falls back to the hardcoded 6 MB default so legacy callers keep their behavior. (3) The largest-first sort gains an insertion-order tiebreaker so the output is deterministic across runs (Layer B will rely on this).

Orchestrator: createRuntimeHooks and createRuntimePrepareTurn now thread modelInfo.maxInputTokens through to prepareProviderMessagesForApi -> buildForApi so the cap reflects the model that is actually being called.

Shared package: CHARS_PER_TOKEN was previously private inside @cline/shared/llms/tokens.ts. Exported alongside the existing estimateTokens.

Tests: 7 new cases in message-builder.test.ts. 913 -> 920 @cline/core unit tests pass. typecheck:smoke clean.

This is the preventive layer. It does NOT yet provide the hard guarantee "the outbound request will never exceed the context window." That guarantee lives in CLINE-2192 (Layer B) which stacks on top of this PR.
2026-05-13 22:11:43 -07:00
Robin Newhouse d94c054843 fix(sdk): trim oldest completed tool pairs from protected tail in basic compaction (CLINE-2185)
CLINE-2136 introduced a deliberate invariant: both runBasicCompaction and runAgenticCompaction refuse to touch the latest turn. That fix prevented orphaned tool_use/tool_result pairs, but it also left the in-flight turn unbounded. A turn whose tool calls accumulated large outputs (editor patches, fetch_web_content, etc.) could push a single request past any context window, which is what CLINE-2183 surfaced on Opus 4.7 against OpenRouter.

PR #10739 widened MessageBuilder.buildForApi's 6 MB allowlist as a triage fix. This PR is the structural follow-up: when the protected tail alone exceeds the compaction target, runBasicCompaction now runs an atomic-pair removal pass on the tail, dropping oldest completed pairs first.

Preservation set inside the tail: (a) the typed user prompt (turn-start), (b) the most recent assistant message, (c) any assistant carrying a tool_use whose tool_result has not yet arrived (in-flight pair). The atomic-pair closure (collectAtomicRemovalIndexes) is reused unchanged.

Implementation notes (deviations from plan.md): (1) Added a new removeTailCandidatesByPredicate that aborts a seed when its atomic closure crosses the preservation set, rather than generic-izing the existing function. Prefix codepath stays byte-identical. (2) Removed the compactable.length === 0 early bail so a transcript that is entirely one in-progress turn still runs the tail trim.

Tests: 5 new compaction.test.ts cases. Existing 27 compaction tests pass unchanged. 913 @cline/core unit tests pass. typecheck:smoke clean.
2026-05-13 18:37:37 -07:00
Robin Newhouse d3acbbc6e9 fix(sdk): truncate editor/apply_patch/fetch_web_content/skills tool results to bound provider payload
CLINE-2183: with the post-CLINE-2136 "protect the latest turn from compaction" invariant in place, a single turn whose tool calls accumulate large outputs from editor, apply_patch, fetch_web_content, or skills could push the outbound request past the model context window (3.8M tokens reported against Opus 4.7's 1M window on OpenRouter) because MessageBuilder.buildForApi only counted tool_results from read_files / search_codebase / run_commands toward its per-block truncation and 6 MB aggregate budget.

Add editor, apply_patch, fetch_web_content, and skills to TARGET_TOOL_NAMES so both safety nets cover the realistic large-output default tools. Bounded-output tools (ask_question, submit_and_exit) stay excluded. MCP tool truncation is dynamic and deferred.

Tests: two new cases in message-builder.test.ts exercising the per-block and aggregate paths for the newly covered tools (6 MB input -> < 1 MB output). Existing 9 builder tests, 908 core unit tests, typecheck:smoke all pass.
2026-05-13 17:23:21 -07:00
9 changed files with 1543 additions and 39 deletions
@@ -15,15 +15,37 @@ import {
truncateToolResultContentForCompaction,
} from "./compaction-shared";
interface BasicCompactionCandidate {
index: number;
/**
* Minimum candidate shape that the atomic-pair removal helpers below need.
* Both the prefix-compaction candidates (BasicCompactionCandidate) and the
* tail-compaction candidates (TailCandidate) extend this; the closure logic
* is identical for both.
*/
interface MinimalCandidate {
message: MessageWithMetadata;
estimatedTokens: number;
}
interface BasicCompactionCandidate extends MinimalCandidate {
index: number;
isFirstUser: boolean;
isLastUser: boolean;
isLastAssistant: boolean;
}
/**
* Tail-compaction candidate. Lives inside the post-CLINE-2136 protected
* tail (everything from the latest typed user prompt onward). The flags
* here drive a stricter preservation predicate than the prefix candidates
* use; see runBasicCompaction / trimProtectedTail.
*/
interface TailCandidate extends MinimalCandidate {
index: number;
isTurnStart: boolean;
isLastAssistant: boolean;
hasInFlightToolUse: boolean;
}
function sanitizeMessageForBasic(
message: MessageWithMetadata,
): MessageWithMetadata | undefined {
@@ -154,15 +176,17 @@ function collectToolResultIds(message: MessageWithMetadata): Set<string> {
return ids;
}
function collectToolPairIds(candidate: BasicCompactionCandidate): Set<string> {
function collectToolPairIds<C extends MinimalCandidate>(
candidate: C,
): Set<string> {
return new Set([
...collectToolUseIds(candidate.message),
...collectToolResultIds(candidate.message),
]);
}
function buildToolPairCandidateIndex(
candidates: BasicCompactionCandidate[],
function buildToolPairCandidateIndex<C extends MinimalCandidate>(
candidates: C[],
): Map<string, Set<number>> {
const indexByToolUseId = new Map<string, Set<number>>();
for (let index = 0; index < candidates.length; index += 1) {
@@ -178,8 +202,8 @@ function buildToolPairCandidateIndex(
return indexByToolUseId;
}
function collectAtomicRemovalIndexes(
candidates: BasicCompactionCandidate[],
function collectAtomicRemovalIndexes<C extends MinimalCandidate>(
candidates: C[],
startIndex: number,
): Set<number> {
const pairIndex = buildToolPairCandidateIndex(candidates);
@@ -204,9 +228,19 @@ function collectAtomicRemovalIndexes(
return removalIndexes;
}
function removeCandidatesByPredicate(
candidates: BasicCompactionCandidate[],
predicate: (candidate: BasicCompactionCandidate) => boolean,
// Unlike removeTailCandidatesByPredicate, this function does NOT check
// whether every member of an atomic closure satisfies the predicate before
// removing. That is safe on the prefix path because the four successive
// predicate calls are role-monotone: each call's predicate either accepts
// ALL members of any possible closure (because closures only span
// same-role candidates on the prefix) or accepts NONE. A tool_use in an
// assistant message is always paired with a tool_result in a user message;
// the first pass removes only assistant non-last messages and the second
// removes only non-last-non-first user messages, so a closure can never
// straddle a protected and an unprotected candidate on the prefix path.
function removeCandidatesByPredicate<C extends MinimalCandidate>(
candidates: C[],
predicate: (candidate: C) => boolean,
targetTokens: number,
estimateMessageTokens: EstimateMessageTokens,
): void {
@@ -312,10 +346,11 @@ function splitLatestTurn(messages: MessageWithMetadata[]): {
protectedTail: MessageWithMetadata[];
} {
const lastTurnStartIndex = findLastTurnStartIndex(messages);
if (
lastTurnStartIndex < 0 ||
(lastTurnStartIndex === 0 && !isTurnStartMessage(messages[0]))
) {
// findLastTurnStartIndex returns 0 as its "not found" sentinel (never -1),
// so we detect the sentinel by checking whether messages[0] is actually a
// turn-start message. When it is not, there is no typed user prompt in
// history and we treat the entire array as compactable with no tail.
if (lastTurnStartIndex === 0 && !isTurnStartMessage(messages[0])) {
return { compactable: messages, protectedTail: [] };
}
return {
@@ -324,6 +359,168 @@ function splitLatestTurn(messages: MessageWithMetadata[]): {
};
}
/**
* Collect tool_use ids inside the tail that do NOT have a matching
* tool_result anywhere in the same tail. These are "in-flight" tool
* calls — the model has emitted them but the runtime hasn't recorded
* a result yet. We must NEVER fold them into a summary or drop them,
* or the provider will reject the next request with "No tool call
* found for function call output with call_id ..." (the inverse of
* the orphaned-tool_result failure mode CLINE-2136 fixed for the
* historical prefix).
*
* The tail snapshot we receive is the one the agent runtime passes
* into prepareTurn BEFORE the next model request. By construction
* any tool_result that has already arrived is in `state.messages`,
* so a missing result reliably means "in-flight" — not "lost".
*/
function findInFlightToolUseIdsInTail(
tail: readonly MessageWithMetadata[],
): Set<string> {
const uses = new Set<string>();
const results = new Set<string>();
for (const message of tail) {
if (!Array.isArray(message.content)) {
continue;
}
for (const block of message.content) {
if (block.type === "tool_use") {
uses.add(block.id);
}
if (block.type === "tool_result") {
results.add(block.tool_use_id);
}
}
}
for (const id of results) {
uses.delete(id);
}
return uses;
}
function buildTailCandidates(
tail: MessageWithMetadata[],
estimateMessageTokens: EstimateMessageTokens,
inFlightToolUseIds: Set<string>,
): TailCandidate[] {
const lastAssistantIndex = findLastAssistantIndex(tail);
const candidates: TailCandidate[] = [];
for (let index = 0; index < tail.length; index += 1) {
const sanitized = sanitizeMessageForBasic(tail[index]) ?? tail[index];
let hasInFlightToolUse = false;
if (Array.isArray(sanitized.content)) {
for (const block of sanitized.content) {
if (block.type === "tool_use" && inFlightToolUseIds.has(block.id)) {
hasInFlightToolUse = true;
break;
}
}
}
candidates.push({
index,
message: sanitized,
estimatedTokens: estimateMessageTokens(sanitized),
// By construction the tail starts at findLastTurnStartIndex,
// so the typed user prompt is always at index 0.
isTurnStart: index === 0 && isTurnStartMessage(tail[0]),
isLastAssistant: index === lastAssistantIndex,
hasInFlightToolUse,
});
}
return candidates;
}
/**
* Drop the oldest completed tool_use/tool_result pairs inside the
* post-CLINE-2136 protected tail when the tail alone exceeds the
* compaction target. Preserves the typed user prompt, the latest
* assistant message, and any assistant message carrying an in-flight
* tool_use (see findInFlightToolUseIdsInTail).
*
* Reuses the existing atomic-pair closure (collectAtomicRemovalIndexes)
* via removeCandidatesByPredicate, so a removal expands across the
* full tool_use_id graph and we never orphan a half-pair.
*/
/**
* Same shape as `removeCandidatesByPredicate` but with one key
* difference: if the atomic-pair closure of a seed candidate touches
* ANY candidate that the predicate marks as not-removable, we abort
* that seed and move on. This is what makes tail trimming safe:
* dropping a tool_result whose matching tool_use is the last
* assistant message would otherwise drag the last assistant into
* the removal set via the closure, violating the preservation
* contract.
*
* On the prefix path the existing `removeCandidatesByPredicate`
* is still used; its closures only group same-role candidates
* (e.g. a non-last assistant with its non-last user tool_result),
* so the two predicates collapse there and behavior is unchanged.
*/
function removeTailCandidatesByPredicate<C extends MinimalCandidate>(
candidates: C[],
predicate: (candidate: C) => boolean,
targetTokens: number,
estimateMessageTokens: EstimateMessageTokens,
): void {
let totalTokens = getTotalTokens(
candidates.map((candidate) => candidate.message),
estimateMessageTokens,
);
for (
let index = 0;
index < candidates.length && totalTokens > targetTokens;
) {
if (!predicate(candidates[index])) {
index += 1;
continue;
}
const removalIndexes = collectAtomicRemovalIndexes(candidates, index);
let closureRemovable = true;
for (const linked of removalIndexes) {
if (!predicate(candidates[linked])) {
closureRemovable = false;
break;
}
}
if (!closureRemovable) {
index += 1;
continue;
}
totalTokens -= Array.from(removalIndexes).reduce(
(total, removalIndex) => total + candidates[removalIndex].estimatedTokens,
0,
);
for (const removalIndex of Array.from(removalIndexes).sort(
(left, right) => right - left,
)) {
candidates.splice(removalIndex, 1);
}
}
}
function trimProtectedTail(
tail: MessageWithMetadata[],
targetTokens: number,
estimateMessageTokens: EstimateMessageTokens,
): MessageWithMetadata[] {
if (tail.length <= 1) {
return tail;
}
const inFlightToolUseIds = findInFlightToolUseIdsInTail(tail);
const candidates = buildTailCandidates(
tail,
estimateMessageTokens,
inFlightToolUseIds,
);
removeTailCandidatesByPredicate(
candidates,
(c) => !c.isTurnStart && !c.isLastAssistant && !c.hasInFlightToolUse,
targetTokens,
estimateMessageTokens,
);
return candidates.map((c) => c.message);
}
export function runBasicCompaction(options: {
context: CoreCompactionContext;
estimateMessageTokens: EstimateMessageTokens;
@@ -336,16 +533,15 @@ export function runBasicCompaction(options: {
const { compactable, protectedTail } = splitLatestTurn(
options.context.messages,
);
if (compactable.length === 0) {
return undefined;
}
// CLINE-2185: previously this function bailed out when there was
// no historical prefix to compact. The tail-trim path below still
// needs to run in that case, so we no longer return undefined here.
// Instead we build candidates over whatever prefix we have (which
// may be empty) and let the prefix passes be no-ops.
const candidates = buildBasicCandidates(
compactable,
options.estimateMessageTokens,
);
if (candidates.length === 0) {
return undefined;
}
removeCandidatesByPredicate(
candidates,
@@ -386,9 +582,30 @@ export function runBasicCompaction(options: {
options.estimateMessageTokens,
);
// CLINE-2185: the protected tail (the in-flight turn after the user's
// latest typed prompt) can itself exceed the compaction target when
// the agent has produced many large tool results in a single turn.
// Trim oldest completed tool pairs out of the tail while preserving
// the typed prompt, the latest assistant message, and any in-flight
// tool_use (whose result has not yet arrived). Atomic-pair closure
// guarantees no orphaned tool_use/tool_result blocks.
const tailTokensBefore = getTotalTokens(
protectedTail,
options.estimateMessageTokens,
);
const tailMessagesBefore = protectedTail.length;
let finalTail = protectedTail;
if (tailTokensBefore > targetTokens) {
finalTail = trimProtectedTail(
protectedTail,
targetTokens,
options.estimateMessageTokens,
);
}
const nextMessages = [
...candidates.map((candidate) => candidate.message),
...protectedTail,
...finalTail,
];
if (!haveMessagesChanged(options.context.messages, nextMessages)) {
return undefined;
@@ -413,6 +630,9 @@ export function runBasicCompaction(options: {
tokensAfter: afterTokens,
targetTokens,
maxInputTokens: options.context.maxInputTokens,
tailTokensBefore,
tailMessagesBefore,
tailMessagesAfter: finalTail.length,
});
return { messages: nextMessages };
@@ -372,6 +372,124 @@ describe("createContextCompactionPrepareTurn", () => {
expect(compacted).toBe(messages);
});
// CLINE-2185: protected-tail trim. The in-flight turn (everything
// from the latest typed user prompt forward) used to be returned
// verbatim by runBasicCompaction. That meant a single turn whose
// tool calls accumulated large outputs could push the request past
// any context window. These tests cover the new tail trim.
it("trims completed tool pairs inside the protected tail when the tail alone exceeds the trigger (CLINE-2185)", () => {
const messages: LlmsProviders.Message[] = [
{ role: "user", content: "Read three files" },
assistantToolUseMessage("tail-a"),
toolResultMessage("tail-a", "a".repeat(2_000)),
assistantToolUseMessage("tail-b"),
toolResultMessage("tail-b", "b".repeat(2_000)),
assistantToolUseMessage("tail-c"),
toolResultMessage("tail-c", "c".repeat(2_000)),
];
// Force a target that fits only the typed prompt + last
// completed pair plus a little slack.
const compacted = runForcedBasicCompaction(messages, 3_000);
expectNoOrphanedToolPairs(compacted);
const pairs = collectToolPairPresence(compacted);
// Oldest two pairs are removed; the most recent pair survives
// because the last assistant + its tool_result are preserved.
expect(pairs.get("tail-a")).toBeUndefined();
expect(pairs.get("tail-b")).toBeUndefined();
expect(pairs.get("tail-c")).toEqual({ hasResult: true, hasUse: true });
// The typed prompt is always preserved as the turn-start.
expect(compacted[0]).toEqual({
role: "user",
content: "Read three files",
});
});
it("preserves an in-flight tool_use whose result has not yet arrived (CLINE-2185)", () => {
const messages: LlmsProviders.Message[] = [
{ role: "user", content: "Do the work" },
assistantToolUseMessage("done-a"),
toolResultMessage("done-a", "a".repeat(2_000)),
// In-flight: model just emitted this; tool_result not yet
// recorded in state.messages.
assistantToolUseMessage("inflight-b"),
];
const compacted = runForcedBasicCompaction(messages, 1);
const pairs = collectToolPairPresence(compacted);
// Older completed pair is removed.
expect(pairs.get("done-a")).toBeUndefined();
// In-flight tool_use is preserved (its tool_result will land
// before the next request — dropping it would synthesize the
// "Tool execution was interrupted before a result was produced"
// failure mode CLINE-2136 fixed for the historical prefix).
// We do NOT call expectNoOrphanedToolPairs here because an
// in-flight tool_use legitimately has no matching tool_result
// at this snapshot in time.
expect(pairs.get("inflight-b")).toEqual({
hasResult: false,
hasUse: true,
});
});
it("preserves the typed prompt and the last assistant message under aggressive tail compaction (CLINE-2185)", () => {
const messages: LlmsProviders.Message[] = [
{ role: "user", content: "Latest typed prompt" },
assistantToolUseMessage("only-pair"),
toolResultMessage("only-pair", "result body"),
];
const compacted = runForcedBasicCompaction(messages, 1);
expectNoOrphanedToolPairs(compacted);
// Everything in this transcript is in the preservation set:
// turn-start user, last assistant, and the matching
// tool_result that the closure pulls in with it.
expect(compacted).toEqual(messages);
});
it("returns the tail unchanged when the entire tail is in the preservation set (CLINE-2185)", () => {
const messages: LlmsProviders.Message[] = [
{ role: "user", content: "Begin" },
// in-flight: no tool_result yet
assistantToolUseMessage("inflight-x"),
];
const compacted = runForcedBasicCompaction(messages, 1);
// runBasicCompaction returns the original messages array
// unchanged when haveMessagesChanged detects no change.
expect(compacted).toBe(messages);
});
it("still compacts the historical prefix when the tail is small (CLINE-2185)", () => {
const oldAnswer = "Old answer ".repeat(50);
const messages: LlmsProviders.Message[] = [
{ role: "user", content: "Old request" },
{ role: "assistant", content: oldAnswer },
{ role: "user", content: "Read the latest file" },
assistantToolUseMessage("tail-a"),
toolResultMessage("tail-a", "latest result"),
];
// Pick a budget below the prefix size but well above the
// tiny tail, so the prefix pass fires and the tail trim does
// not need to do anything.
const targetTokens =
totalJsonTokens(messages) - estimateJsonTokens(messages[1]) + 10;
const compacted = runForcedBasicCompaction(messages, targetTokens);
// Prefix "Old answer..." is gone; tail intact.
expect(compacted).toEqual([
{ role: "user", content: "Old request" },
{ role: "user", content: "Read the latest file" },
assistantToolUseMessage("tail-a"),
toolResultMessage("tail-a", "latest result"),
]);
});
it("does not add unsupported max output tokens to Codex OAuth summarizer requests", () => {
const codexConfig = resolveSummarizerConfig({
activeProviderConfig: {
@@ -754,7 +754,7 @@ export class SessionRuntime {
modelInfo?.capabilities?.includes("images") ?? true,
...this.config.toolContextMetadata,
},
hooks: this.createRuntimeHooks(),
hooks: this.createRuntimeHooks(modelInfo),
prepareTurn: this.createRuntimePrepareTurn(modelInfo, tools),
initialMessages,
systemPrompt,
@@ -863,7 +863,9 @@ export class SessionRuntime {
this.extensionsInitialized = true;
}
private createRuntimeHooks(): Partial<AgentRuntimeHooks> {
private createRuntimeHooks(
modelInfo: ModelInfo | undefined,
): Partial<AgentRuntimeHooks> {
const hooks = mergeRuntimeHooks([
this.config.hooks,
...this.contributionRegistry
@@ -878,8 +880,10 @@ export class SessionRuntime {
return control;
}
const messages = control?.messages ?? ctx.request.messages;
const preparedMessages =
await this.prepareMessagesForModelRequest(messages);
const preparedMessages = await this.prepareMessagesForModelRequest(
messages,
modelInfo?.maxInputTokens,
);
return {
...control,
messages: preparedMessages,
@@ -907,7 +911,26 @@ export class SessionRuntime {
return async (context) => {
const messages = agentMessagesToMessagesWithMetadata(context.messages);
const apiMessages = await this.prepareProviderMessagesForApi(messages);
const buildForApiOptions: Parameters<
typeof this.messageBuilder.buildForApi
>[1] = {
maxInputTokens: modelInfo?.maxInputTokens,
emitStatusNotice: context.emitStatusNotice,
telemetry: this.telemetry,
sessionId: this.config.sessionId,
provider: this.config.providerId,
modelId: this.config.modelId,
agentIdentity: {
agentId: context.agentId,
conversationId:
context.conversationId ?? this.conversation.getConversationId(),
parentAgentId: context.parentAgentId ?? undefined,
},
};
const apiMessages = await this.prepareProviderMessagesForApi(
messages,
buildForApiOptions,
);
const result = await prepareTurn({
agentId: context.agentId,
conversationId:
@@ -942,15 +965,18 @@ export class SessionRuntime {
private async prepareMessagesForModelRequest(
messages: readonly AgentMessage[],
maxInputTokens?: number,
): Promise<AgentMessage[]> {
const providerMessages = await this.prepareProviderMessagesForApi(
agentMessagesToMessages(messages),
{ maxInputTokens },
);
return messagesToAgentMessages(providerMessages);
}
private async prepareProviderMessagesForApi(
messages: MessageWithMetadata[],
options: Parameters<typeof this.messageBuilder.buildForApi>[1] = {},
): Promise<MessageWithMetadata[]> {
let providerMessages = messages;
const messageBuilders =
@@ -958,7 +984,7 @@ export class SessionRuntime {
for (const builder of messageBuilders) {
providerMessages = await builder.build(providerMessages);
}
return this.messageBuilder.buildForApi(providerMessages);
return this.messageBuilder.buildForApi(providerMessages, options);
}
private handleRuntimeEvent(event: AgentRuntimeEvent): void {
@@ -60,6 +60,7 @@ export const CORE_TELEMETRY_EVENTS = {
SUBAGENT_COMPLETED: "task.subagent_completed",
COMPACTION_EXECUTED: "task.compaction_executed",
COMPACTION_SKIPPED: "task.compaction_skipped",
EMERGENCY_TRUNCATION: "task.emergency_truncation",
},
HOOKS: {
DISCOVERY_COMPLETED: "hooks.discovery_completed",
@@ -621,3 +622,38 @@ export function captureCompactionSkipped(
timestamp: new Date().toISOString(),
});
}
/**
* CLINE-2192 Layer B: emitted when `MessageBuilder.buildForApi` had to
* apply emergency truncation to force the outbound request below
* `maxInputTokens * CHARS_PER_TOKEN` bytes. This fires only when
* Layer A's largest-first heuristic was insufficient (adversarial
* inputs, oversized tool_use.input bodies, exotic block layouts).
*
* Treat any occurrence as a signal that production is operating in
* degraded mode — the model is seeing truncated content and the
* agent may produce wrong output as a result. The user-visible
* `emitStatusNotice("compacted to fit context window", ...)` fires
* alongside this event so the operator sees it in the TUI/webview.
*/
export interface CaptureEmergencyTruncationProperties {
ulid: string;
bytesBefore: number;
bytesAfter: number;
maxInputTokens: number;
truncatedBlocks: number;
droppedBlocks: number;
provider?: string;
modelId?: string;
}
export function captureEmergencyTruncation(
telemetry: ITelemetryService | undefined,
properties: CaptureEmergencyTruncationProperties &
Partial<TelemetryAgentIdentityProperties>,
): void {
emit(telemetry, CORE_TELEMETRY_EVENTS.TASK.EMERGENCY_TRUNCATION, {
...properties,
timestamp: new Date().toISOString(),
});
}
@@ -1,5 +1,5 @@
import type { Message } from "@cline/shared";
import { describe, expect, it } from "vitest";
import { describe, expect, it, vi } from "vitest";
import { MessageBuilder } from "./message-builder";
describe("MessageBuilder", () => {
@@ -363,4 +363,453 @@ describe("MessageBuilder", () => {
}),
]);
});
// CLINE-2183: editor / apply_patch / fetch_web_content / skills were not
// in the original TARGET_TOOL_NAMES allowlist, so their tool_results
// bypassed both per-block truncation and the aggregate text budget.
// A coding-heavy turn could push the outbound request past a model's
// context window even with compaction enabled.
it("truncates editor tool results above the per-block limit (CLINE-2183)", () => {
const builder = new MessageBuilder(100);
const messages: Message[] = [
{
role: "assistant",
content: [
{
type: "tool_use",
id: "tool_1",
name: "editor",
input: { path: "/tmp/example.ts", new_text: "x" },
},
],
},
{
role: "user",
content: [
{
type: "tool_result",
tool_use_id: "tool_1",
content: "z".repeat(250),
},
],
},
];
const result = builder.buildForApi(messages);
const content = result[1].content;
expect(Array.isArray(content)).toBe(true);
const block = Array.isArray(content) ? content[0] : undefined;
expect(block?.type).toBe("tool_result");
if (block?.type !== "tool_result") {
throw new Error("expected tool_result");
}
expect(
typeof block.content === "string" ? block.content.length : 0,
).toBeLessThanOrEqual(100);
expect(block.content).toContain("...[truncated");
});
it("applies the aggregate text budget across editor / apply_patch / fetch_web_content / skills tool results (CLINE-2183)", () => {
// 1 MB total budget against ~6 MB of input across the four newly
// covered tools. Use the default TARGET_TOOL_NAMES (omit the
// second constructor arg) to exercise the constant change itself.
const builder = new MessageBuilder(50_000, undefined, 1_000_000);
const messages: Message[] = [
{
role: "assistant",
content: [
{
type: "tool_use",
id: "tool_editor",
name: "editor",
input: { path: "/tmp/a.ts" },
},
{
type: "tool_use",
id: "tool_patch",
name: "apply_patch",
input: { input: "*** Begin Patch" },
},
{
type: "tool_use",
id: "tool_fetch",
name: "fetch_web_content",
input: { requests: [] },
},
{
type: "tool_use",
id: "tool_skills",
name: "skills",
input: { skill: "noop" },
},
],
},
{
role: "user",
content: [
{
type: "tool_result",
tool_use_id: "tool_editor",
content: "e".repeat(1_500_000),
},
{
type: "tool_result",
tool_use_id: "tool_patch",
content: "p".repeat(1_500_000),
},
{
type: "tool_result",
tool_use_id: "tool_fetch",
content: "f".repeat(1_500_000),
},
{
type: "tool_result",
tool_use_id: "tool_skills",
content: "s".repeat(1_500_000),
},
],
},
];
const result = builder.buildForApi(messages);
const totalBytes = result.reduce((sum, message) => {
if (typeof message.content === "string") {
return sum + Buffer.byteLength(message.content, "utf8");
}
return (
sum +
message.content.reduce((inner, block) => {
if (block.type !== "tool_result") {
return inner;
}
return (
inner +
(typeof block.content === "string"
? Buffer.byteLength(block.content, "utf8")
: 0)
);
}, 0)
);
}, 0);
// Either path is acceptable: the per-block truncator
// (`...[truncated N chars]...`) or the aggregate-budget
// truncator (`...[truncated N chars to fit provider request
// budget]...`) must have fired and pulled the request well
// below the configured 1 MB budget. Before CLINE-2183 neither
// fired for these tools and totalBytes was ~6 MB.
expect(totalBytes).toBeLessThanOrEqual(1_000_000);
expect(JSON.stringify(result)).toContain("...[truncated");
});
// CLINE-2191 (Layer A): widen MessageBuilder.collectTruncationCandidates
// beyond tool_result content so user text, assistant text, thinking
// blocks, and top-level file blocks also participate in the aggregate
// budget. Also tie the budget to the model's actual maxInputTokens.
it("truncates user text blocks under the aggregate budget (CLINE-2191)", () => {
const builder = new MessageBuilder(50_000, undefined, 500_000);
const messages: Message[] = [
{
role: "user",
content: [{ type: "text", text: "x".repeat(5_000_000) }],
},
];
const result = builder.buildForApi(messages);
const block = Array.isArray(result[0].content)
? result[0].content[0]
: undefined;
expect(block?.type).toBe("text");
if (block?.type !== "text") throw new Error("expected text");
expect(Buffer.byteLength(block.text, "utf8")).toBeLessThanOrEqual(500_000);
expect(block.text).toContain("provider request budget");
});
it("truncates assistant text and thinking blocks under the aggregate budget (CLINE-2191)", () => {
const builder = new MessageBuilder(50_000, undefined, 250_000);
const messages: Message[] = [
{
role: "assistant",
content: [
{ type: "thinking", thinking: "t".repeat(1_000_000) },
{ type: "text", text: "a".repeat(2_000_000) },
],
},
];
const result = builder.buildForApi(messages);
const serialized = JSON.stringify(result);
expect(serialized.length).toBeLessThan(1_000_000);
expect(serialized).toContain("provider request budget");
// Both blocks got reduced; neither is the full 2 MB / 1 MB original.
const content = result[0].content as Array<{ type: string }>;
const thinking = content.find((b) => b.type === "thinking") as unknown as
| { thinking: string }
| undefined;
const text = content.find((b) => b.type === "text") as unknown as
| { text: string }
| undefined;
if (!thinking || !text) throw new Error("expected both blocks present");
expect(thinking.thinking.length).toBeLessThan(1_000_000);
expect(text.text.length).toBeLessThan(2_000_000);
});
it("truncates top-level file blocks under the aggregate budget (CLINE-2191)", () => {
const builder = new MessageBuilder(50_000, undefined, 200_000);
const messages: Message[] = [
{
role: "user",
content: [
{
type: "file",
path: "/tmp/large.ts",
content: "y".repeat(4_000_000),
},
],
},
];
const result = builder.buildForApi(messages);
const block = Array.isArray(result[0].content)
? result[0].content[0]
: undefined;
expect(block?.type).toBe("file");
if (block?.type !== "file") throw new Error("expected file");
// Per-block max (50_000) brings it well under the 200_000 cap.
expect(Buffer.byteLength(block.content, "utf8")).toBeLessThanOrEqual(
50_000,
);
});
it("skips tool_use input bodies (CLINE-2191, deferred to Layer B)", () => {
// Document the intentional deferral: tool_use.input is structured
// JSON; Layer B will own the structural truncator that avoids
// corrupting tool_use_id or breaking the JSON shape. Layer A
// leaves tool_use blocks untouched even when they exceed budget.
const huge = "z".repeat(4_000_000);
const builder = new MessageBuilder(50_000, undefined, 100_000);
const messages: Message[] = [
{
role: "assistant",
content: [
{
type: "tool_use",
id: "tool_huge",
name: "editor",
input: { body: huge },
},
],
},
];
const result = builder.buildForApi(messages);
const block = Array.isArray(result[0].content)
? result[0].content[0]
: undefined;
expect(block?.type).toBe("tool_use");
if (block?.type !== "tool_use") throw new Error("expected tool_use");
// Input body is unchanged. Layer B will own this.
expect((block.input as { body: string }).body).toBe(huge);
expect(block.id).toBe("tool_huge");
});
it("derives the aggregate budget from maxInputTokens when provided (CLINE-2191)", () => {
// 100_000 tokens * 3 chars/token = 300_000 byte budget.
const builder = new MessageBuilder();
const messages: Message[] = [
{
role: "user",
content: [{ type: "text", text: "p".repeat(2_000_000) }],
},
];
const result = builder.buildForApi(messages, { maxInputTokens: 100_000 });
const block = Array.isArray(result[0].content)
? result[0].content[0]
: undefined;
if (block?.type !== "text") throw new Error("expected text");
expect(Buffer.byteLength(block.text, "utf8")).toBeLessThanOrEqual(300_000);
expect(block.text).toContain("...[truncated");
});
it("falls back to the constructor default budget when maxInputTokens is absent (CLINE-2191)", () => {
const builder = new MessageBuilder(50_000, undefined, 250_000);
const messages: Message[] = [
{
role: "user",
content: [{ type: "text", text: "q".repeat(4_000_000) }],
},
];
// No maxInputTokens passed → ctor's 250_000 byte budget applies.
const result = builder.buildForApi(messages);
const block = Array.isArray(result[0].content)
? result[0].content[0]
: undefined;
if (block?.type !== "text") throw new Error("expected text");
expect(Buffer.byteLength(block.text, "utf8")).toBeLessThanOrEqual(250_000);
});
it("produces deterministic output for equal-byte-length candidates (CLINE-2191)", () => {
// Two candidates of identical byte length. The sort tiebreaker
// (insertion order) ensures the same input always truncates the
// same one first, so the output is byte-identical across runs.
const longA = "a".repeat(500_000);
const longB = "b".repeat(500_000);
const build = () => {
const builder = new MessageBuilder(50_000, undefined, 200_000);
return builder.buildForApi([
{
role: "user",
content: [
{ type: "text", text: longA },
{ type: "text", text: longB },
],
},
]);
};
const first = JSON.stringify(build());
const second = JSON.stringify(build());
expect(first).toBe(second);
});
it("truncates the current typed-prompt user text when the aggregate budget is tight (CLINE-2191)", () => {
// Layer A is unconditional — even the most-recent user message (the
// typed prompt) participates in the aggregate budget. This is intentional:
// the guarantee is that the request fits, not that any particular block
// is preserved. A gigantic typed prompt is unusual but possible.
const builder = new MessageBuilder(50_000, undefined, 100_000);
const messages: Message[] = [
{
role: "user",
content: [{ type: "text", text: "u".repeat(4_000_000) }],
},
];
const result = builder.buildForApi(messages);
const block = Array.isArray(result[0].content)
? result[0].content[0]
: undefined;
if (block?.type !== "text") throw new Error("expected text");
expect(Buffer.byteLength(block.text, "utf8")).toBeLessThanOrEqual(100_000);
expect(block.text).toContain("provider request budget");
});
// CLINE-2192 (Layer B): absolute hard guarantee. These tests use
// maxInputTokens so MessageBuilder must enforce maxInputTokens * 3
// bytes after Layer A.
it("truncates tool_use.input string values without corrupting JSON or tool_use_id (CLINE-2192)", () => {
const notice = vi.fn();
const telemetry = { capture: vi.fn() };
const builder = new MessageBuilder();
const messages: Message[] = [
{
role: "assistant",
content: [
{
type: "tool_use",
id: "tool_1",
name: "editor",
input: { body: "x".repeat(1_000_000), nested: { keep: true } },
},
],
},
];
const result = builder.buildForApi(messages, {
maxInputTokens: 50_000,
emitStatusNotice: notice,
telemetry: telemetry as never,
sessionId: "session-1",
provider: "openrouter",
modelId: "anthropic/claude-opus-4.7",
});
const block = Array.isArray(result[0].content)
? result[0].content[0]
: undefined;
if (block?.type !== "tool_use") throw new Error("expected tool_use");
expect(block.id).toBe("tool_1");
expect(block.name).toBe("editor");
expect((block.input.nested as { keep: boolean }).keep).toBe(true);
expect((block.input.body as string).length).toBeLessThan(1_000_000);
expect(
Buffer.byteLength(JSON.stringify(result), "utf8"),
).toBeLessThanOrEqual(150_000);
expect(JSON.parse(JSON.stringify(block.input))).toEqual(block.input);
expect(notice).toHaveBeenCalledWith(
"compacted to fit context window",
expect.objectContaining({ kind: "emergency_truncation" }),
);
expect(telemetry.capture).toHaveBeenCalledWith(
expect.objectContaining({
event: "task.emergency_truncation",
properties: expect.objectContaining({ ulid: "session-1" }),
}),
);
});
it("enforces the hard byte budget when Layer A's floor would otherwise keep too many small blocks (CLINE-2192)", () => {
const builder = new MessageBuilder();
const content = Array.from({ length: 200 }, (_, i) => ({
type: "text" as const,
text: `${i}:` + "x".repeat(300),
}));
const result = builder.buildForApi(
[{ role: "user", content }],
{ maxInputTokens: 2_000 }, // 6 KB budget
);
const serializedBytes = Buffer.byteLength(JSON.stringify(result), "utf8");
const payloadBytes = (result[0].content as typeof content).reduce(
(total, block) => total + Buffer.byteLength(block.text, "utf8"),
0,
);
expect(serializedBytes).toBeLessThanOrEqual(6_000);
expect(payloadBytes).toBeLessThanOrEqual(6_000);
});
it("produces deterministic Layer B output for adversarial inputs (CLINE-2192)", () => {
const make = () =>
new MessageBuilder().buildForApi(
[
{
role: "assistant",
content: [
{
type: "tool_use",
id: "deterministic",
name: "editor",
input: { a: "a".repeat(100_000), b: "b".repeat(100_000) },
},
],
},
],
{ maxInputTokens: 5_000 },
);
expect(JSON.stringify(make())).toBe(JSON.stringify(make()));
});
it("does not emit emergency_truncation when Layer A alone suffices (CLINE-2192)", () => {
const notice = vi.fn();
const telemetry = { capture: vi.fn() };
new MessageBuilder().buildForApi(
[
{
role: "user",
content: [{ type: "text", text: "x".repeat(20_000) }],
},
],
{
maxInputTokens: 10_000,
emitStatusNotice: notice,
telemetry: telemetry as never,
},
);
expect(notice).not.toHaveBeenCalled();
expect(telemetry.capture).not.toHaveBeenCalled();
});
});
@@ -11,15 +11,35 @@
import {
type ContentBlock,
type ITelemetryService,
type Message,
normalizeUserInput,
type TextContent,
type ToolResultContent,
} from "@cline/shared";
import {
captureEmergencyTruncation,
type TelemetryAgentIdentityProperties,
} from "../../services/telemetry/core-events";
const DEFAULT_MAX_TOOL_RESULT_CHARS = 50_000;
const DEFAULT_MAX_TOTAL_TEXT_BYTES = 6_000_000;
const MIN_TOTAL_BUDGET_TOOL_RESULT_BYTES = 8_000;
const MESSAGE_BUILDER_CHARS_PER_TOKEN = 3;
// CLINE-2192 Layer B: when Layer A can't bring the request under the
// budget (adversarial inputs, oversized tool_use.input bodies, etc.)
// we drop the floor to this much smaller value and aggressively
// middle-truncate every string-bearing block until we fit. Small
// enough to free real budget; large enough that the block still
// carries some signal.
const EMERGENCY_FLOOR_BYTES = 256;
const TRUNCATE_MARKER_EMERGENCY = (n: number) =>
`\n\n...[truncated ${n} chars to fit context window]...\n\n`;
// Tools whose results are large enough to need provider-payload truncation
// (per-block at maxToolResultChars and in aggregate at maxTotalTextBytes).
// Bounded-output tools (ask_question, submit_and_exit) are intentionally
// excluded. MCP tool names are dynamic and not covered here; see
// CLINE-2183 for the broader follow-up.
const TARGET_TOOL_NAMES = new Set([
"read",
"read_files",
@@ -27,6 +47,10 @@ const TARGET_TOOL_NAMES = new Set([
"search_codebase",
"bash",
"run_commands",
"editor",
"apply_patch",
"fetch_web_content",
"skills",
]);
const READ_TOOL_NAMES = new Set(["read", "read_files"]);
const OUTDATED_FILE_CONTENT = "[outdated - see the latest file content]";
@@ -49,6 +73,30 @@ interface TruncationCandidate {
set(value: string): void;
}
/**
* Options for `MessageBuilder.buildForApi`. The Layer A budget knob
* (`maxInputTokens`) plus the Layer B observability surface for the
* brick-wall byte-budget guarantee (CLINE-2192).
*/
export interface BuildForApiOptions {
maxInputTokens?: number;
/**
* Per-turn status-notice channel. When Layer B's emergency
* truncation fires we emit `"compacted to fit context window"`
* so the TUI/webview surfaces a visible signal to the user.
*/
emitStatusNotice?: (
message: string,
metadata?: Record<string, unknown>,
) => void;
/** Per-session telemetry sink for `task.emergency_truncation`. */
telemetry?: ITelemetryService;
sessionId?: string;
provider?: string;
modelId?: string;
agentIdentity?: Partial<TelemetryAgentIdentityProperties>;
}
/**
* Builds an API-safe message copy without mutating original conversation history.
*/
@@ -73,7 +121,10 @@ export class MessageBuilder {
private readonly maxTotalTextBytes = DEFAULT_MAX_TOTAL_TEXT_BYTES,
) {}
buildForApi(messages: Message[]): Message[] {
buildForApi(
messages: Message[],
options: BuildForApiOptions = {},
): Message[] {
this.reindex(messages);
const repairedMessages = this.addMissingToolResults(messages);
@@ -100,7 +151,23 @@ export class MessageBuilder {
return changed ? { ...message, content } : message;
});
return this.truncateToTotalTextBudget(prepared);
const afterLayerA = this.truncateToTotalTextBudget(
prepared,
options.maxInputTokens,
);
// CLINE-2192 Layer B: hard guarantee. If Layer A's largest-first
// candidate-set heuristic couldn't bring the request under the
// budget (adversarial inputs, oversized tool_use.input bodies,
// many small blocks below Layer A's floor), drop to the
// emergency floor and brick-wall the bytes. Always degrades,
// never throws.
if (
typeof options.maxInputTokens === "number" &&
options.maxInputTokens > 0
) {
return this.enforceHardByteBudget(afterLayerA, options);
}
return afterLayerA;
}
private transformBlock(
@@ -780,13 +847,24 @@ export class MessageBuilder {
);
}
private truncateToTotalTextBudget(messages: Message[]): Message[] {
if (this.maxTotalTextBytes <= 0) {
private truncateToTotalTextBudget(
messages: Message[],
maxInputTokens?: number,
): Message[] {
// CLINE-2191: when the orchestrator threads the model's actual
// maxInputTokens, derive the aggregate cap from it. Otherwise
// fall back to the historical 6 MB default so legacy callers
// (existing tests, direct constructors) behave identically.
const effectiveBudget =
typeof maxInputTokens === "number" && maxInputTokens > 0
? maxInputTokens * MESSAGE_BUILDER_CHARS_PER_TOKEN
: this.maxTotalTextBytes;
if (effectiveBudget <= 0) {
return messages;
}
let totalBytes = this.countMessageTextBytes(messages);
if (totalBytes <= this.maxTotalTextBytes) {
if (totalBytes <= effectiveBudget) {
return messages;
}
@@ -804,14 +882,14 @@ export class MessageBuilder {
const candidates = this.collectTruncationCandidates(next);
for (const candidate of candidates) {
if (totalBytes <= this.maxTotalTextBytes) {
if (totalBytes <= effectiveBudget) {
break;
}
const currentBytes = candidate.byteLength;
if (currentBytes <= MIN_TOTAL_BUDGET_TOOL_RESULT_BYTES) {
continue;
}
const overflow = totalBytes - this.maxTotalTextBytes;
const overflow = totalBytes - effectiveBudget;
const targetBytes = Math.max(
MIN_TOTAL_BUDGET_TOOL_RESULT_BYTES,
currentBytes - overflow,
@@ -869,6 +947,52 @@ export class MessageBuilder {
continue;
}
for (const block of message.content) {
// CLINE-2191: also collect candidates for the block types
// that previously bypassed the aggregate budget. These
// share the existing `MIN_TOTAL_BUDGET_TOOL_RESULT_BYTES`
// floor; truncating below that loses too much signal.
//
// `redacted_thinking` is intentionally skipped — its
// content is a fixed placeholder. `tool_use` and its
// `input` body are skipped here too; a JSON-aware
// structural truncator that can drill into values
// without corrupting `tool_use_id`s or breaking JSON
// shape is the responsibility of Layer B (CLINE-2192).
if (block.type === "text") {
candidates.push({
byteLength: utf8ByteLength(block.text),
get: () => block.text,
set: (value) => {
block.text = value;
},
});
continue;
}
if (block.type === "thinking") {
candidates.push({
byteLength: utf8ByteLength(block.thinking),
get: () => block.thinking,
set: (value) => {
block.thinking = value;
},
});
continue;
}
if (block.type === "file") {
// Note: transformBlock already applied per-block
// truncation to file content (via truncateMiddle at
// maxToolResultChars). The candidate here allows Layer A
// to apply a second, tighter cut when the aggregate
// budget is still exceeded after per-block truncation.
candidates.push({
byteLength: utf8ByteLength(block.content),
get: () => block.content,
set: (value) => {
block.content = value;
},
});
continue;
}
if (block.type !== "tool_result") {
continue;
}
@@ -907,7 +1031,124 @@ export class MessageBuilder {
}
}
}
return candidates.sort((l, r) => r.byteLength - l.byteLength);
// CLINE-2191: deterministic tiebreaker on insertion order so
// the same input always produces the same truncation output.
// Layer B (CLINE-2192) will rely on this same invariant.
const indexed = candidates.map((candidate, originalIndex) => ({
candidate,
originalIndex,
}));
indexed.sort(
(l, r) =>
r.candidate.byteLength - l.candidate.byteLength ||
l.originalIndex - r.originalIndex,
);
return indexed.map(({ candidate }) => candidate);
}
/**
* CLINE-2192 Layer B: the brick-wall byte-budget pass. Runs after
* Layer A. If the input is already under budget this is a no-op
* and the input array is returned unchanged.
*
* If still over budget, two passes:
*
* 1. Aggressive middle-truncation of EVERY string-bearing block
* (including `tool_use.input` string leaves), dropping the
* per-block floor to `EMERGENCY_FLOOR_BYTES`. `tool_use_id`,
* `id`, `call_id`, `name` strings are excluded.
* 2. If pass 1 didn't fit (extremely unlikely, but possible if
* the preservation set itself exceeds the budget), drop
* non-essential blocks (oldest assistant text/thinking
* blocks first, then oldest tool pairs atomically). The
* typed prompt (turn-start user) and the last assistant are
* last in the drop order.
*
* On any non-zero work performed: emits `task.emergency_truncation`
* telemetry and a status notice so the operator sees the degraded
* state in the TUI/webview.
*/
private enforceHardByteBudget(
messages: Message[],
options: BuildForApiOptions,
): Message[] {
const budgetBytes =
(options.maxInputTokens ?? 0) * MESSAGE_BUILDER_CHARS_PER_TOKEN;
if (budgetBytes <= 0) {
return messages;
}
const bytesBefore = countProviderRequestBytes(messages);
if (bytesBefore <= budgetBytes) {
return messages;
}
// Deep-clone so we don't mutate the input.
const next = messages.map((message) => {
if (!Array.isArray(message.content)) {
return { ...message };
}
return {
...message,
content: message.content.map((block) =>
cloneContentBlockForMutation(block),
),
};
});
// Pass 1: aggressive middle-truncation, EMERGENCY_FLOOR_BYTES floor.
const candidates = collectEmergencyCandidates(next);
let totalBytes = countProviderRequestBytes(next);
let truncatedBlocks = 0;
for (const candidate of candidates) {
if (totalBytes <= budgetBytes) {
break;
}
const currentBytes = candidate.byteLength;
if (currentBytes <= EMERGENCY_FLOOR_BYTES) {
continue;
}
const overflow = totalBytes - budgetBytes;
const targetBytes = Math.max(
EMERGENCY_FLOOR_BYTES,
currentBytes - overflow,
);
const truncated = truncateMiddleToBytes(
candidate.get(),
targetBytes,
TRUNCATE_MARKER_EMERGENCY,
);
candidate.set(truncated);
totalBytes = countProviderRequestBytes(next);
truncatedBlocks += 1;
}
// Pass 2: if even floor-truncation didn't fit, drop blocks.
const droppedBlocks = dropOldestUntilFits(next, budgetBytes);
const bytesAfter = countProviderRequestBytes(next);
if (truncatedBlocks > 0 || droppedBlocks > 0) {
options.emitStatusNotice?.("compacted to fit context window", {
kind: "emergency_truncation",
maxInputTokens: options.maxInputTokens,
bytesBefore,
bytesAfter,
truncatedBlocks,
droppedBlocks,
});
captureEmergencyTruncation(options.telemetry, {
ulid: options.sessionId ?? options.agentIdentity?.conversationId ?? "",
bytesBefore,
bytesAfter,
maxInputTokens: options.maxInputTokens ?? 0,
truncatedBlocks,
droppedBlocks,
provider: options.provider,
modelId: options.modelId,
...options.agentIdentity,
});
}
return next;
}
}
@@ -965,6 +1206,12 @@ function truncateMiddleToBytes(
}
function cloneContentBlockForMutation(block: ContentBlock): ContentBlock {
if (block.type === "tool_use") {
return {
...block,
input: cloneJsonLike(block.input) as Record<string, unknown>,
};
}
if (block.type !== "tool_result" || typeof block.content === "string") {
return { ...block };
}
@@ -973,3 +1220,411 @@ function cloneContentBlockForMutation(block: ContentBlock): ContentBlock {
content: block.content.map((entry) => ({ ...entry })),
};
}
function cloneJsonLike(value: unknown): unknown {
if (Array.isArray(value)) {
return value.map((entry) => cloneJsonLike(entry));
}
if (value && typeof value === "object") {
const result: Record<string, unknown> = {};
for (const [key, entry] of Object.entries(value)) {
result[key] = cloneJsonLike(entry);
}
return result;
}
return value;
}
function countProviderRequestBytes(messages: Message[]): number {
// CLINE-2192: provider payloads include JSON framing, tool names,
// tool ids, object keys and scalar arguments — not just string
// leaf contents. Counting the JSON-serialized message list is a
// conservative byte-budget proxy and is the metric Layer B must
// force under `maxInputTokens * MESSAGE_BUILDER_CHARS_PER_TOKEN`.
try {
return utf8ByteLength(JSON.stringify(messages));
} catch {
return messages.reduce(
(total, message) => total + utf8ByteLength(String(message)),
0,
);
}
}
/**
* CLINE-2192 Layer B: collect every string-bearing block location the
* brick-wall byte-budget pass may middle-truncate. Strictly wider
* than Layer A's set — it also includes the string leaves inside
* `tool_use.input`. Keys, numbers, booleans, null, and the reserved
* identifier fields (`id`, `tool_use_id`, `call_id`, `name`) are
* excluded.
*
* Order is deterministic: walk message-by-message, block-by-block;
* within each `tool_use.input`, walk the JSON tree depth-first with
* lexicographic key order. The final sort is largest-first with
* insertion-order tiebreaker (same property Layer A uses).
*/
function collectEmergencyCandidates(
messages: Message[],
): TruncationCandidate[] {
const candidates: TruncationCandidate[] = [];
for (const message of messages) {
if (typeof message.content === "string") {
candidates.push({
byteLength: utf8ByteLength(message.content),
get: () => message.content as string,
set: (value) => {
message.content = value;
},
});
continue;
}
if (!Array.isArray(message.content)) {
continue;
}
for (const block of message.content) {
if (block.type === "text") {
candidates.push({
byteLength: utf8ByteLength(block.text),
get: () => block.text,
set: (value) => {
block.text = value;
},
});
continue;
}
if (block.type === "thinking") {
candidates.push({
byteLength: utf8ByteLength(block.thinking),
get: () => block.thinking,
set: (value) => {
block.thinking = value;
},
});
continue;
}
if (block.type === "redacted_thinking") {
candidates.push({
byteLength: utf8ByteLength(block.data),
get: () => block.data,
set: (value) => {
block.data = value;
},
});
continue;
}
if (block.type === "file") {
candidates.push({
byteLength: utf8ByteLength(block.content),
get: () => block.content,
set: (value) => {
block.content = value;
},
});
continue;
}
if (block.type === "image") {
candidates.push({
byteLength: utf8ByteLength(block.data),
get: () => block.data,
set: (value) => {
block.data = value;
},
});
continue;
}
if (block.type === "tool_result") {
if (typeof block.content === "string") {
candidates.push({
byteLength: utf8ByteLength(block.content),
get: () => block.content as string,
set: (value) => {
block.content = value;
},
});
} else {
for (const entry of block.content) {
if (entry.type === "text") {
candidates.push({
byteLength: utf8ByteLength(entry.text),
get: () => entry.text,
set: (value) => {
entry.text = value;
},
});
} else if (entry.type === "file") {
candidates.push({
byteLength: utf8ByteLength(entry.content),
get: () => entry.content,
set: (value) => {
entry.content = value;
},
});
} else if (entry.type === "image") {
candidates.push({
byteLength: utf8ByteLength(entry.data),
get: () => entry.data,
set: (value) => {
entry.data = value;
},
});
}
}
}
continue;
}
if (block.type === "tool_use") {
collectStringLeaves(block.input, (get, set) => {
candidates.push({
byteLength: utf8ByteLength(get()),
get,
set,
});
});
continue;
}
}
}
return candidates
.map((candidate, originalIndex) => ({ candidate, originalIndex }))
.sort(
(l, r) =>
r.candidate.byteLength - l.candidate.byteLength ||
l.originalIndex - r.originalIndex,
)
.map(({ candidate }) => candidate);
}
/**
* Walks `tool_use.input` depth-first with lexicographic key order
* and invokes the visitor for every string leaf with a `get`/`set`
* pair that reads/writes the leaf in place. Non-string leaves
* (numbers, booleans, null) are not visited. Arrays-of-strings ARE
* visited per-index.
*/
function collectStringLeaves(
node: unknown,
visit: (get: () => string, set: (value: string) => void) => void,
): void {
if (Array.isArray(node)) {
for (let i = 0; i < node.length; i += 1) {
const value = node[i];
if (typeof value === "string") {
visit(
() => node[i] as string,
(next) => {
node[i] = next;
},
);
} else {
collectStringLeaves(value, visit);
}
}
return;
}
if (node && typeof node === "object") {
const obj = node as Record<string, unknown>;
for (const key of Object.keys(obj).sort()) {
const value = obj[key];
if (typeof value === "string") {
visit(
() => obj[key] as string,
(next) => {
obj[key] = next;
},
);
} else {
collectStringLeaves(value, visit);
}
}
}
}
/**
* CLINE-2192 Layer B pass 2: when even floor-truncating every
* string leaf still leaves the request over budget, blank out
* remaining string-bearing payloads until it fits. This is intentionally
* brutal but structured: ids, tool names, object keys, booleans,
* numbers, nulls, arrays and object shapes are preserved. A tool call
* may fail because an argument string became empty, but the provider
* request will fit and the agent can recover in the next turn.
*/
function dropOldestUntilFits(messages: Message[], budgetBytes: number): number {
let droppedBlocks = 0;
const candidates = collectEmergencyCandidates(messages);
for (const candidate of candidates) {
if (countProviderRequestBytes(messages) <= budgetBytes) {
break;
}
if (candidate.get().length === 0) {
continue;
}
candidate.set("");
droppedBlocks += 1;
}
if (countProviderRequestBytes(messages) <= budgetBytes) {
return droppedBlocks;
}
for (const removal of collectBlockRemovalCandidates(messages)) {
if (countProviderRequestBytes(messages) <= budgetBytes) {
break;
}
if (removeBlocks(messages, removal.blocks)) {
droppedBlocks += removal.blocks.length;
}
}
return droppedBlocks;
}
interface BlockRef {
messageIndex: number;
blockIndex: number;
}
interface BlockRemovalCandidate {
priority: number;
messageIndex: number;
blockIndex: number;
blocks: BlockRef[];
}
function collectBlockRemovalCandidates(
messages: Message[],
): BlockRemovalCandidate[] {
const toolRefs = new Map<string, BlockRef[]>();
const candidates: BlockRemovalCandidate[] = [];
const lastAssistantIndex = findLastAssistantMessageIndex(messages);
for (
let messageIndex = 0;
messageIndex < messages.length;
messageIndex += 1
) {
const message = messages[messageIndex];
if (!Array.isArray(message.content)) {
continue;
}
for (
let blockIndex = 0;
blockIndex < message.content.length;
blockIndex += 1
) {
const block = message.content[blockIndex];
if (block.type === "tool_use") {
const refs = toolRefs.get(block.id) ?? [];
refs.push({ messageIndex, blockIndex });
toolRefs.set(block.id, refs);
} else if (block.type === "tool_result") {
const refs = toolRefs.get(block.tool_use_id) ?? [];
refs.push({ messageIndex, blockIndex });
toolRefs.set(block.tool_use_id, refs);
}
}
}
for (
let messageIndex = 0;
messageIndex < messages.length;
messageIndex += 1
) {
const message = messages[messageIndex];
if (!Array.isArray(message.content)) {
continue;
}
for (
let blockIndex = 0;
blockIndex < message.content.length;
blockIndex += 1
) {
const block = message.content[blockIndex];
const priority = getDropPriority(
message,
messageIndex,
lastAssistantIndex,
);
if (block.type === "tool_use") {
candidates.push({
priority,
messageIndex,
blockIndex,
blocks: toolRefs.get(block.id) ?? [{ messageIndex, blockIndex }],
});
continue;
}
if (block.type === "tool_result") {
candidates.push({
priority,
messageIndex,
blockIndex,
blocks: toolRefs.get(block.tool_use_id) ?? [
{ messageIndex, blockIndex },
],
});
continue;
}
candidates.push({
priority,
messageIndex,
blockIndex,
blocks: [{ messageIndex, blockIndex }],
});
}
}
return candidates.sort(
(a, b) =>
a.priority - b.priority ||
a.messageIndex - b.messageIndex ||
a.blockIndex - b.blockIndex,
);
}
function getDropPriority(
message: Message,
messageIndex: number,
lastAssistantIndex: number,
): number {
if (message.role === "assistant" && messageIndex !== lastAssistantIndex) {
return 0;
}
if (message.role === "user" && messageIndex !== 0) {
return 1;
}
if (message.role === "assistant") {
return 2;
}
return 3;
}
function findLastAssistantMessageIndex(messages: Message[]): number {
for (let index = messages.length - 1; index >= 0; index -= 1) {
if (messages[index].role === "assistant") {
return index;
}
}
return -1;
}
function removeBlocks(messages: Message[], refs: BlockRef[]): boolean {
let didRemove = false;
const refsByMessage = new Map<number, number[]>();
for (const ref of refs) {
const indexes = refsByMessage.get(ref.messageIndex) ?? [];
indexes.push(ref.blockIndex);
refsByMessage.set(ref.messageIndex, indexes);
}
for (const [messageIndex, blockIndexes] of refsByMessage) {
const message = messages[messageIndex];
if (!message || !Array.isArray(message.content)) {
continue;
}
for (const blockIndex of [...new Set(blockIndexes)].sort((a, b) => b - a)) {
if (blockIndex >= 0 && blockIndex < message.content.length) {
message.content.splice(blockIndex, 1);
didRemove = true;
}
}
}
return didRemove;
}
+1 -1
View File
@@ -127,7 +127,7 @@ export {
resolveReasoningEffortRatio,
} from "./llms/reasoning-effort";
export { DEFAULT_REQUEST_HEADERS, serializeAbortReason } from "./llms/requests";
export { estimateTokens } from "./llms/tokens";
export { CHARS_PER_TOKEN, estimateTokens } from "./llms/tokens";
export type {
ToolApprovalRequest,
ToolApprovalResult,
+1 -1
View File
@@ -141,7 +141,7 @@ export {
resolveReasoningEffortRatio,
} from "./llms/reasoning-effort";
export { DEFAULT_REQUEST_HEADERS, serializeAbortReason } from "./llms/requests";
export { estimateTokens } from "./llms/tokens";
export { CHARS_PER_TOKEN, estimateTokens } from "./llms/tokens";
export type {
ToolApprovalRequest,
ToolApprovalResult,
+1 -1
View File
@@ -5,7 +5,7 @@
* rather than after.
*/
const CHARS_PER_TOKEN = 3;
export const CHARS_PER_TOKEN = 3;
export function estimateTokens(chars: number): number {
return Math.max(1, Math.ceil(chars / CHARS_PER_TOKEN));