test(vscode): exercise full SDK structured edit flow in file-edit e2e (#11442)

* test(vscode): exercise full SDK structured edit flow in diff.test.ts e2e (ENG-2042)

The SDK runtime executes structured (OpenAI-format) tool calls instead of parsing XML-style tool syntax out of assistant text. Teach the e2e mock server to stream an editor tool call for edit_request (arguments split across deltas to exercise fragment reassembly), answer the SDK's follow-up tool-result request (role:'tool' message) with turn-ending completion text, and remove the classic XML-era EDIT_REQUEST/REPLACE_REQUEST responses.

diff.test.ts now covers the full approval flow: approval ask row -> Save -> editor tool writes the file -> completion text, verifying the edit on disk and restoring the git-tracked fixture afterwards. The old 'test.ts: Original <-> Cline's Changes' diff-tab assertions are unreachable under the SDK executor architecture (the editor executor writes via Node fs and does not route through DiffViewProvider); this behavioral difference is documented in the test file.

* test(vscode): address review feedback on diff.test.ts e2e

- Scope the mock server's tool-result follow-up detection to edit_request conversations so tool results from other (future) scenarios don't mis-route to EDIT_REQUEST_COMPLETE.

- Move the fixture readFileSync inside the try block and guard the finally restore, so a failed read doesn't bypass cleanup attribution or write undefined back to the fixture.

* docs(vscode): rephrase diff e2e comments to describe current behavior

Comments described historical behavior (XML-style tool-call parsing that predates the SDK runtime), which is confusing to readers of the current code. Rephrase them to describe the code as it exists now.

* test(vscode): rename diff.test.ts to file-edit.test.ts and drop duplicated preamble

The test no longer touches a diff editor (the SDK editor executor writes files directly after approval), so the 'Diff Editor' name was misleading. Rename the file and describe block to match what it asserts: the file-edit approval flow.

Drop the first half of the test (send hello, wait, New Task, check history), which duplicated chat.test.ts, and the mock server's 500ms delay that existed only to support an 'API Request...' visibility assertion that no longer exists.
This commit is contained in:
Dominic Cooney
2026-06-18 22:21:43 -04:00
committed by Cline Agent
parent 7e2bc6488f
commit 48a4016077
4 changed files with 169 additions and 167 deletions
-46
View File
@@ -1,46 +0,0 @@
import { expect } from "@playwright/test"
import { E2E_WORKSPACE_TYPES, e2e } from "./utils/helpers"
e2e.describe("Diff Editor", () => {
E2E_WORKSPACE_TYPES.forEach(({ title, workspaceType }) => {
e2e.extend({
workspaceType,
})(title, async ({ helper, sidebar }) => {
await helper.signin(sidebar)
const inputbox = sidebar.getByTestId("chat-input")
await expect(inputbox).toBeVisible()
await inputbox.fill("[diff.test.ts] Hello, Cline!")
await expect(inputbox).toHaveValue("[diff.test.ts] Hello, Cline!")
await sidebar.getByTestId("send-button").click()
await expect(inputbox).toHaveValue("")
// Wait for the (mock) agent turn to finish before navigating away —
// the task is persisted to SDK session history when the turn completes,
// and the mock server delays this response by 500ms.
await expect(sidebar.getByText("mock Cline API response")).toBeVisible()
// Back to home page with history. The turn ends in "awaiting_followup"
// (the mock response has no attempt_completion), so the footer shows no
// "Start New Task" button — use the header "New Task" button instead,
// same as chat.test.ts.
await sidebar.getByRole("button", { name: "New Task", exact: true }).first().click()
await expect(sidebar.getByText("Recent")).toBeVisible()
await expect(sidebar.getByText("Hello, Cline!")).toBeVisible() // History with the previous sent message
// Submit a file edit request
await sidebar.getByTestId("chat-input").click()
await sidebar.getByTestId("chat-input").fill("edit_request")
await sidebar.getByTestId("send-button").click({ delay: 50 })
// Wait for the sidebar to load the file edit request
await sidebar.waitForSelector('span:has-text("Cline wants to edit this file:")')
// The SDK-backed path renders a pending edit approval before the user saves it.
await expect(sidebar.getByText(/\/test\.ts/)).toBeVisible()
await expect(sidebar.getByRole("button", { name: "Save" })).toBeVisible()
await expect(sidebar.getByRole("button", { name: "Reject" })).toBeVisible()
})
})
})
@@ -0,0 +1,60 @@
import { readFileSync, writeFileSync } from "node:fs"
import * as path from "node:path"
import { expect } from "@playwright/test"
import { E2E_WORKSPACE_TYPES, e2e } from "./utils/helpers"
// File edits are performed by the SDK's `editor` tool executor, which writes
// the file directly (Node fs) after the user approves the tool call — it does
// not stream the edit through DiffViewProvider, so no diff editor tab (e.g.
// "test.ts: Original ↔ Cline's Changes") opens. This test asserts the
// approval flow: approval ask row → Save → file modified on disk →
// turn-ending completion text.
e2e.describe("File Edit Approval", () => {
E2E_WORKSPACE_TYPES.forEach(({ title, workspaceType }) => {
e2e.extend({
workspaceType,
})(title, async ({ helper, sidebar, workspaceDir }) => {
// The mock editor tool call targets "test.ts" relative to the session
// cwd, which is the first workspace folder in both single-root and
// multi-root workspaces (fixtures/workspace). The fixture file is
// checked into git, so restore it after the (real) edit.
const editedFilePath = path.join(workspaceDir, "test.ts")
let originalFileContent: string | undefined
try {
originalFileContent = readFileSync(editedFilePath, "utf-8")
await helper.signin(sidebar)
// Submit a file edit request. The mock server responds with a
// structured `editor` tool call (path: test.ts, old/new text).
const inputbox = sidebar.getByTestId("chat-input")
await expect(inputbox).toBeVisible()
await inputbox.fill("edit_request")
await sidebar.getByTestId("send-button").click({ delay: 50 })
// The edit tool requires approval (edit tools are never auto-approved
// by default) — the ask row appears with the file path and diff.
await sidebar.waitForSelector('span:has-text("Cline wants to edit this file:")')
await expect(sidebar.getByText("test.ts").first()).toBeVisible()
await expect(sidebar.getByRole("button", { name: "Reject" })).toBeVisible()
// Approve the edit ("Save" is the primary button for file-edit asks).
await sidebar.getByRole("button", { name: "Save", exact: true }).click({ delay: 50 })
// The SDK executes the editor tool and sends the tool result back to
// the (mock) model, which replies with turn-ending completion text.
await expect(sidebar.getByText("I successfully replaced")).toBeVisible({ timeout: 30_000 })
// The edit was actually applied to the file on disk.
expect(readFileSync(editedFilePath, "utf-8")).toContain('export const name = "cline"')
} finally {
// Skip the restore when the initial read failed — there is
// nothing to restore and the read error is the real failure.
if (originalFileContent !== undefined) {
writeFileSync(editedFilePath, originalFileContent, "utf-8")
}
}
})
})
})
+27 -44
View File
@@ -29,58 +29,41 @@ export const E2E_REGISTERED_MOCK_ENDPOINTS = {
},
}
const replace_in_file = `I successfully replaced "john" with "cline" in the test.ts file. The change has been completed and the file now contains:
/**
* Structured `editor` tool call streamed in response to the `edit_request`
* prompt. The mock server streams it as OpenAI-format
* `choices[].delta.tool_calls[]` deltas followed by
* `finish_reason: "tool_calls"`, which is the only tool-call syntax the SDK
* runtime executes.
*
* The `path` is workspace-relative; the SDK editor executor resolves relative
* paths against the session cwd, which is the first workspace folder in both
* the single-root and multi-root e2e workspaces (`fixtures/workspace`).
*/
export const E2E_MOCK_EDITOR_TOOL_CALL = {
id: "call_e2e_edit_1",
name: "editor",
arguments: {
path: "test.ts",
old_text: 'export const name = "john"',
new_text: 'export const name = "cline"',
},
}
const edit_request_complete = `I successfully replaced "john" with "cline" in the test.ts file. The change has been completed and the file now contains:
\`\`\`typescript
export const name = "cline"
\`\`\`
The TypeScript errors shown in the output are unrelated to this change - they appear to be existing issues in the broader codebase related to missing type definitions and dependencies. The specific task of updating the name in test.ts has been completed successfully.
<attempt_completion>
<result>
I have successfully replaced the name "john" with "cline" in the test.ts file. The file now exports:
\`\`\`typescript
export const name = "cline"
\`\`\`
The change has been applied and saved to the file.
</result>
</attempt_completion>`
const edit_request = `<thinking>
The user wants me to replace the name "john" with "cline" in the test.ts file. I can see the file content provided:
\`\`\`typescript
export const name = "john"
\`\`\`
I need to change "john" to "cline". This is a simple targeted edit, so I should use the replace_in_file tool rather than write_to_file since I'm only changing one small part of the file.
I need to:
1. Use replace_in_file to change "john" to "cline" in the test.ts file
2. The SEARCH block should match the exact content: \`export const name = "john"\`
3. The REPLACE block should be: \`export const name = "cline"\`
</thinking>
I'll replace "john" with "cline" in the test.ts file.
<replace_in_file>
<path>test.ts</path>
<diff>
------- SEARCH
export const name = "john"
=======
export const name = "cline"
+++++++ REPLACE
</diff>
</replace_in_file>`
The change has been applied and saved to the file.`
export const E2E_MOCK_API_RESPONSES = {
DEFAULT: "Hello! I'm a mock Cline API response.",
REPLACE_REQUEST: replace_in_file,
EDIT_REQUEST: edit_request,
/** Assistant text streamed before the structured editor tool call. */
EDIT_REQUEST_LEAD_IN: `I'll replace "john" with "cline" in the test.ts file.`,
/** Turn-ending text streamed after the SDK reports the editor tool result. */
EDIT_REQUEST_COMPLETE: edit_request_complete,
}
export const E2E_MOCK_CLINE_RECOMMENDED_MODELS = {
@@ -6,6 +6,7 @@ import {
E2E_MOCK_API_RESPONSES,
E2E_MOCK_CLINE_MODELS,
E2E_MOCK_CLINE_RECOMMENDED_MODELS,
E2E_MOCK_EDITOR_TOOL_CALL,
E2E_REGISTERED_MOCK_ENDPOINTS,
} from "./api"
import { ClineDataMock } from "./data"
@@ -472,26 +473,35 @@ export class ClineApiServerMock {
const body = await readBody()
const parsed = JSON.parse(body)
const { _messages, model = "claude-3-5-sonnet-20241022", stream = true } = parsed
let responseText = E2E_MOCK_API_RESPONSES.DEFAULT
const isEditRequest = body.includes("edit_request")
log("Chat completion mock selection:", {
isEditRequest,
isReplaceResult: body.includes("[replace_in_file for 'test.ts'] Result:"),
})
if (body.includes("[replace_in_file for 'test.ts'] Result:")) {
responseText = E2E_MOCK_API_RESPONSES.REPLACE_REQUEST
}
if (isEditRequest) {
responseText = E2E_MOCK_API_RESPONSES.EDIT_REQUEST
}
if (body.includes("[diff.test.ts] Hello, Cline!")) {
// The playwright test in diff.test.ts needs the "API Request..." text
// to be on the screen long enough to detect it. This worked at 100ms
// too, but setting to 500ms to cover slower CI boxes.
await new Promise((resolve) => setTimeout(resolve, 500))
}
const { messages, model = "claude-3-5-sonnet-20241022", stream = true } = parsed
// The SDK runtime executes structured tool calls and then sends a
// follow-up /chat/completions request containing the tool result as
// a `role: "tool"` message. Detect that follow-up first — the
// original "edit_request" user prompt is still present in the
// conversation history of the follow-up request, so order matters.
// Scoped to edit_request conversations so tool results from other
// (future) scenarios don't mis-route to EDIT_REQUEST_COMPLETE.
const hasToolResult =
body.includes("edit_request") &&
Array.isArray(messages) &&
messages.some((m: { role?: string }) => m?.role === "tool")
let responseText = E2E_MOCK_API_RESPONSES.DEFAULT
let includeEditorToolCall = false
log("Chat completion mock selection:", {
isEditRequest: body.includes("edit_request"),
hasToolResult,
})
if (hasToolResult) {
responseText = E2E_MOCK_API_RESPONSES.EDIT_REQUEST_COMPLETE
} else if (body.includes("edit_request")) {
// Stream lead-in text followed by a structured `editor` tool
// call (OpenAI tool_calls deltas) — the only tool-call syntax
// the SDK runtime executes.
responseText = E2E_MOCK_API_RESPONSES.EDIT_REQUEST_LEAD_IN
includeEditorToolCall = true
}
const generationId = `gen_${++controller.generationCounter}_${Date.now()}`
if (stream) {
@@ -503,68 +513,44 @@ export class ClineApiServerMock {
const randomUUID = uuidv4()
if (isEditRequest) {
const toolCallId = `call_${randomUUID}`
const toolCallChunk = {
id: generationId,
object: "chat.completion.chunk",
created: Math.floor(Date.now() / 1000),
model,
choices: [
{
index: 0,
delta: {
tool_calls: [
{
index: 0,
id: toolCallId,
type: "function",
function: {
name: "editor",
arguments: JSON.stringify({
path: "test.ts",
old_text: 'export const name = "john"',
new_text: 'export const name = "cline"',
}),
},
},
],
},
finish_reason: null,
},
],
}
const finalChunk = {
id: generationId,
object: "chat.completion.chunk",
created: Math.floor(Date.now() / 1000),
model,
choices: [
{
index: 0,
delta: {},
finish_reason: "tool_calls",
},
],
usage: {
prompt_tokens: 140,
completion_tokens: responseText.length,
total_tokens: 140 + responseText.length,
cost: (140 + responseText.length) * 0.00015,
},
}
res.write(`data: ${JSON.stringify(toolCallChunk)}\n\n`)
res.write(`data: ${JSON.stringify(finalChunk)}\n\n`)
res.write("data: [DONE]\n\n")
res.end()
return
}
responseText += `\n\nGenerated UUID: ${randomUUID}`
const chunks = responseText.split(" ")
let chunkIndex = 0
// OpenAI-format streamed tool call deltas, matching what the
// AI SDK's openai-compatible client expects: the first delta
// for a tool_calls index must carry `id` + `function.name`;
// `function.arguments` accumulates as string fragments. Split
// the arguments JSON to exercise fragment reassembly.
const argumentsJson = JSON.stringify(E2E_MOCK_EDITOR_TOOL_CALL.arguments)
const argsSplitAt = Math.floor(argumentsJson.length / 2)
const toolCallDeltas = includeEditorToolCall
? [
[
{
index: 0,
id: E2E_MOCK_EDITOR_TOOL_CALL.id,
type: "function",
function: { name: E2E_MOCK_EDITOR_TOOL_CALL.name, arguments: "" },
},
],
[
{
index: 0,
function: { arguments: argumentsJson.slice(0, argsSplitAt) },
},
],
[
{
index: 0,
function: { arguments: argumentsJson.slice(argsSplitAt) },
},
],
]
: []
let toolCallDeltaIndex = 0
const sendChunk = () => {
if (chunkIndex < chunks.length) {
const chunk = {
@@ -585,6 +571,25 @@ export class ClineApiServerMock {
res.write(`data: ${JSON.stringify(chunk)}\n\n`)
chunkIndex++
setTimeout(sendChunk, 10)
} else if (toolCallDeltaIndex < toolCallDeltas.length) {
const chunk = {
id: generationId,
object: "chat.completion.chunk",
created: Math.floor(Date.now() / 1000),
model,
choices: [
{
index: 0,
delta: {
tool_calls: toolCallDeltas[toolCallDeltaIndex],
},
finish_reason: null,
},
],
}
res.write(`data: ${JSON.stringify(chunk)}\n\n`)
toolCallDeltaIndex++
setTimeout(sendChunk, 10)
} else {
const finalChunk = {
id: generationId,
@@ -595,7 +600,7 @@ export class ClineApiServerMock {
{
index: 0,
delta: {},
finish_reason: "stop",
finish_reason: includeEditorToolCall ? "tool_calls" : "stop",
},
],
usage: {