Update version

CHANGELOG.md

@@ -4,6 +4,19 @@ All notable changes to the "claude-dev" extension will be documented in this fil
 
 <!-- Check [Keep a Changelog](http://keepachangelog.com/) for recommendations on how to structure this file. -->
 
+## [1.1.1]
+
+- Adds option to choose other Claude models (+ GPT-4o, DeepSeek, and Mistral if you use OpenRouter)
+- Adds option to add custom instructions to the end of the system prompt
+
+## [1.1.0]
+
+- Paste images in chat to use Claude's vision capabilities and turn mockups into fully functional applications or fix bugs with screenshots
+
+## [1.0.9]
+
+- Add support for OpenRouter and AWS Bedrock
+
 ## [1.0.8]
 
 - Shows diff view of new or edited files right in the editor

README.md

@@ -8,35 +8,34 @@
   <a href="https://marketplace.visualstudio.com/items?itemName=saoudrizwan.claude-dev"><strong>Download VSCode Extension</strong></a>
 </p>
 
-Thanks to [Claude 3.5 Sonnet's agentic coding capabilities](https://www-cdn.anthropic.com/fed9cc193a14b84131812372d8d5857f8f304c52/Model_Card_Claude_3_Addendum.pdf) Claude Dev can handle complex software development tasks step-by-step. With tools that let him read & write files, analyze project source code, and execute terminal commands (after you grant permission), he can assist you in ways that go beyond simple code completion or tech support. From building software projects to running system operations, Claude Dev is only limited by your imagination.
-
-While autonomous AI scripts traditionally run in sandboxed environments, Claude Dev provides a human-in-the-loop GUI to supervise every file changed and command executed, providing a safe and accessible way to explore the potential of agentic AI.
+Thanks to [Claude 3.5 Sonnet's agentic coding capabilities](https://www-cdn.anthropic.com/fed9cc193a14b84131812372d8d5857f8f304c52/Model_Card_Claude_3_Addendum.pdf) Claude Dev can handle complex software development tasks step-by-step. With tools that let him create & edit files, explore complex projects, and execute terminal commands (after you grant permission), he can assist you in ways that go beyond simple code completion or tech support. While autonomous AI scripts traditionally run in sandboxed environments, Claude Dev provides a human-in-the-loop GUI to supervise every file changed and command executed, providing a safe and accessible way to explore the potential of agentic AI.
 
+-   Paste images in chat to use Claude's vision capabilities and turn mockups into fully functional applications or fix bugs with screenshots
 -   Inspect diffs of every change Claude makes right in the editor, and keep track with syntax highlighted previews in chat
--   Runs CLI commands directly in chat, so you never have to open a terminal yourself (+ respond to interactive commands by sending a message)
--   Presents permission buttons (i.e. 'Approve CLI command') before tool use or sending information to the API
+-   Runs terminal commands directly in chat, so you never have to open a terminal yourself (+ respond to interactive commands by sending a message)
+-   Presents permission buttons (i.e. 'Approve terminal command') before tool use or sending information to the API
 -   Keep track of total tokens and API usage cost for the entire task loop and individual requests
 -   Set a maximum # of API requests allowed for a task before being prompted for permission to proceed
--   When a task is completed, Claude determines if he can present the result to you with a CLI command like `open -a "Google Chrome" index.html`, which you run with a click of a button
+-   When a task is completed, Claude determines if he can present the result to you with a terminal command like `open -a "Google Chrome" index.html`, which you run with a click of a button
 
 _**Pro tip**: Use the `Cmd + Shift + P` shortcut to open the command palette and type `Claude Dev: Open In New Tab` to start a new task right in your editor._
 
 ## How it works
 
-Claude Dev uses an agentic loop style implementation with chain-of-thought prompting and access to powerful tools that give him the ability to accomplish nearly any task. Start by providing a task and the agentic loop fires off, where it might use certain tools (with your permission) to accomplish each step in its thought process.
+Claude Dev uses an autonomous task execution loop with chain-of-thought prompting and access to powerful tools that give him the ability to accomplish nearly any task. Start by providing a task and the loop fires off, where it might use certain tools (with your permission) to accomplish each step in its thought process.
 
 ### Tools
 
 Claude Dev has access to the following capabilities:
 
-1. **`execute_command`**: Execute CLI commands on the system (only with your permission, output is streamed into the chat and you can respond to stdin or exit long-running processes when you're ready)
+1. **`execute_command`**: Execute terminal commands on the system (only with your permission, output is streamed into the chat and you can respond to stdin or exit long-running processes when you're ready)
 2. **`list_files_top_level`**: List all paths for files at the top level of the specified directory (useful for generic file operations like retrieving a file from your Desktop)
 3. **`list_files_recursive`**: List all paths for files in the specified directory and nested subdirectories (excludes files in .gitignore)
 4. **`view_source_code_definitions_top_level`**: Parses all source code files at the top level of the specified directory to extract names of key elements like classes and functions (see more below)
 5. **`read_file`**: Read the contents of a file at the specified path
 6. **`write_to_file`**: Write content to a file at the specified path, automatically creating any necessary directories
 7. **`ask_followup_question`**: Ask the user a question to gather additional information needed to complete a task (due to the autonomous nature of the program, this isn't a typical chatbot–Claude Dev must explicitly interrupt his task loop to ask for more information)
-8. **`attempt_completion`**: Present the result to the user after completing a task, potentially with a CLI command to kickoff a demonstration
+8. **`attempt_completion`**: Present the result to the user after completing a task, potentially with a terminal command to kickoff a demonstration
 
 ### Working in Existing Projects
 
@@ -60,7 +59,8 @@ Claude always asks for your permission first before any tools are executed or in
 
 ## Contribution
 
-Feel free to contribute to this project by submitting issues and pull requests. Contributions are welcome and appreciated!
+Paul Graham said it best, "if you build something now that barely works with AI, the next models will make it _really_ work." I've built this project with the assumption that scaling laws will continue to improve the quality (and cost) of AI models, and what might be difficult for Claude 3.5 Sonnet today will be effortless for future generations. That is the design philosophy I'd like to develop this project with, so it will always be updated with the best models, tools, and capabilities available–without wasting effort on implementing stopgaps like cheaper agents. With that said, I'm always open to suggestions and feedback, so please feel free to contribute to this project by submitting issues and pull requests. Contributions are welcome and appreciated!
+
 To build Claude Dev locally, follow these steps:
 
 1. Clone the repository:
@@ -80,10 +80,17 @@ To build Claude Dev locally, follow these steps:
 ## Reviews
 
 -   ["Claude Sonnet 3.5 Artifacts in VSCode With This Extension"](https://www.youtube.com/watch?v=5FbZ8ALfSTs) by [CoderOne](https://www.youtube.com/@CoderOne)
--   ["Meet Claude Dev — An Open-Source AI Programmer In VS Code"](https://generativeai.pub/meet-claude-dev-an-open-source-autonomous-ai-programmer-in-vs-code-f457f9821b7b) by [Jim Clyde Monge](https://jimclydemonge.medium.com/)
+-   ["ClaudeDev: NEW Coding Agent Can Generate Applications within VS Code!"](https://www.youtube.com/watch?v=UNsQHosbIoE) by [WorldofAI](https://www.youtube.com/@intheworldofai)
+-   ["ClaudeDev: This Mind-Blowing Coding Agent Can Build SaaS Apps in Minutes!"](https://www.youtube.com/watch?v=Ki0nuOeUpT0) by [AI for Devs](https://www.youtube.com/@ai-for-devs)
+-   ["Claude Dev: This Coding Agent can Generate Applications Within VS Code!"](https://www.youtube.com/watch?v=ufq6sHGe0zs) by [AICodeKing](https://www.youtube.com/@AICodeKing)
+-   ["ClaudeDev + NextJS + Supabase: Generate FULL-STACK Apps with Claude 3.5 Sonnet"](https://www.youtube.com/watch?v=GeZBfO1kxA4) by [AICodeKing](https://www.youtube.com/@AICodeKing)
+-   ["Meet Claude Dev — An Open-Source AI Programmer In VS Code"](https://generativeai.pub/meet-claude-dev-an-open-source-autonomous-ai-programmer-in-vs-code-f457f9821b7b) and ["Build games with zero code using Claude Dev in VS Code"](https://www.youtube.com/watch?v=VT-JYVi81ew) by [Jim Clyde Monge](https://jimclydemonge.medium.com/)
+-   ["Claude Dev Builds NextJS App! Aider, Continue Dev & GitHub Copilot Open-Source Alternative"](https://www.youtube.com/watch?v=Rv0wJZRpnCQ) by [Josh Pocock](https://www.youtube.com/@joshfpocock)
+-   ["Claude Dev Is Mindblowing. The Best Coding Assistant tool?"](https://www.youtube.com/watch?v=Vp1Z3VGZroA) by [Yaron Been](https://www.youtube.com/@ecomxfactor-YaronBeen)
+-   ["ClaudeDev: The Ultimate Coding Agent for VS Code"](https://www.youtube.com/watch?v=aq0yw_DtphQ) by [Blas](https://www.youtube.com/@blascerecer)
 -   ["AI Development with Claude Dev"](https://www.linkedin.com/pulse/ai-development-claude-dev-shannon-lal-3ql3e/) by Shannon Lal
 -   ["Code Smarter with Claude Dev: An AI Programmer for Your Projects"](https://www.linkedin.com/pulse/code-smarter-claude-dev-ai-programmer-your-projects-iana-detochka-jiqpe) by Iana D.
--   [Claude Dev also hit top 10 posts of all time on r/ClaudeAI (thank you for all the lovely comments)](https://www.reddit.com/r/ClaudeAI/comments/1e3h0f1/my_submission_to_anthropics_build_with_claude/)
+-   [Claude Dev also hit top 10 posts of all time on r/ClaudeAI](https://www.reddit.com/r/ClaudeAI/comments/1e3h0f1/my_submission_to_anthropics_build_with_claude/)
 
 ## License
 

package.json

@@ -2,7 +2,7 @@
   "name": "claude-dev",
   "displayName": "Claude Dev",
   "description": "Autonomous software engineer right in your IDE, capable of creating/editing files, executing commands, and more with your permission every step of the way.",
-  "version": "1.0.84",
+  "version": "1.1.14",
   "icon": "icon.png",
   "engines": {
     "vscode": "^1.84.0"
@@ -122,16 +122,20 @@
     "typescript": "^5.4.5"
   },
   "dependencies": {
+    "@anthropic-ai/bedrock-sdk": "^0.10.2",
     "@anthropic-ai/sdk": "^0.24.3",
     "@vscode/codicons": "^0.0.36",
     "default-shell": "^2.2.0",
+    "delay": "^6.0.0",
     "diff": "^5.2.0",
     "execa": "^9.3.0",
     "globby": "^14.0.2",
+    "openai": "^4.54.0",
     "os-name": "^6.0.0",
     "p-wait-for": "^5.0.2",
     "serialize-error": "^11.0.3",
+    "tree-kill": "^1.2.2",
     "tree-sitter-wasms": "^0.1.11",
     "web-tree-sitter": "^0.22.6"
   }
-}
+}

src/ClaudeDev.ts

@@ -1,21 +1,25 @@
 import { Anthropic } from "@anthropic-ai/sdk"
 import defaultShell from "default-shell"
 import * as diff from "diff"
-import { execa, ExecaError } from "execa"
+import { execa, ExecaError, ResultPromise } from "execa"
 import fs from "fs/promises"
 import os from "os"
 import osName from "os-name"
 import pWaitFor from "p-wait-for"
 import * as path from "path"
 import { serializeError } from "serialize-error"
+import treeKill from "tree-kill"
 import * as vscode from "vscode"
+import { ApiHandler, buildApiHandler } from "./api"
 import { listFiles, parseSourceCodeForDefinitionsTopLevel } from "./parse-source-code"
 import { ClaudeDevProvider } from "./providers/ClaudeDevProvider"
+import { ApiConfiguration } from "./shared/api"
 import { ClaudeRequestResult } from "./shared/ClaudeRequestResult"
 import { DEFAULT_MAX_REQUESTS_PER_TASK } from "./shared/Constants"
-import { ClaudeAsk, ClaudeSay, ClaudeSayTool } from "./shared/ExtensionMessage"
+import { ClaudeAsk, ClaudeMessage, ClaudeSay, ClaudeSayTool } from "./shared/ExtensionMessage"
 import { Tool, ToolName } from "./shared/Tool"
 import { ClaudeAskResponse } from "./shared/WebviewMessage"
+import delay from "delay"
 
 const SYSTEM_PROMPT =
 	() => `You are Claude Dev, a highly skilled software developer with extensive knowledge in many programming languages, frameworks, design patterns, and best practices.
@@ -36,24 +40,23 @@ CAPABILITIES
 
 RULES
 
-- Unless otherwise specified by the user, you MUST accomplish your task within the following directory: ${
-		vscode.workspace.workspaceFolders?.map((folder) => folder.uri.fsPath).at(0) ??
-		path.join(os.homedir(), "Desktop")
-	}
-- Your current working directory is '${process.cwd()}', and you cannot \`cd\` into a different directory to complete a task. You are stuck operating from '${process.cwd()}', so be sure to pass in the appropriate 'path' parameter when using tools that require a path.
-- If you do not know the contents of an existing file you need to edit, use the read_file tool to help you make informed changes. However if you have already read or written to this file before, you can assume its contents have not changed since then so you would not need to call the read_file tool beforehand.
-- When editing files, always provide the complete file content in your response, regardless of the extent of changes. The system handles diff generation automatically.
-- Before using the execute_command tool, you must first think about the SYSTEM INFORMATION context provided to understand the user's environment and tailor your commands to ensure they are compatible with their system.
-- When creating a new project (such as an app, website, or any software project), unless the user specifies otherwise, organize all new files within a dedicated project directory. Use appropriate file paths when writing files, as the write_to_file tool will automatically create any necessary directories. Structure the project logically, adhering to best practices for the specific type of project being created. Unless otherwise specified, new projects should be easily run without additional setup, for example most projects can be built in HTML, CSS, and JavaScript - which you can open in a browser.
+- Your current working directory is: ${cwd}
+- You cannot \`cd\` into a different directory to complete a task. You are stuck operating from '${cwd}', so be sure to pass in the correct 'path' parameter when using tools that require a path.
+- Do not use the ~ character or $HOME to refer to the home directory.
+- Before using the execute_command tool, you must first think about the SYSTEM INFORMATION context provided to understand the user's environment and tailor your commands to ensure they are compatible with their system. You must also consider if the command you need to run should be executed in a specific directory outside of the current working directory '${cwd}', and if so prepend with \`cd\`'ing into that directory && then executing the command (as one command since you are stuck operating from '${cwd}'). For example, if you needed to run \`npm install\` in a project outside of '${cwd}', you would need to prepend with a \`cd\` i.e. pseudocode for this would be \`cd (path to project) && (command, in this case npm install)\`.
+- When editing files, always provide the complete file content in your response, regardless of the extent of changes. DO NOT use placeholder comments like '//rest of code unchanged' or '//code remains the same'. You MUST include all parts of the file, even if they haven't been modified.
+- If you need to read or edit a file you have already read or edited, you can assume its contents have not changed since then (unless specified otherwise by the user) and skip using the read_file tool before proceeding.
+- When creating a new project (such as an app, website, or any software project), organize all new files within a dedicated project directory unless the user specifies otherwise. Use appropriate file paths when writing files, as the write_to_file tool will automatically create any necessary directories. Structure the project logically, adhering to best practices for the specific type of project being created. Unless otherwise specified, new projects should be easily run without additional setup, for example most projects can be built in HTML, CSS, and JavaScript - which you can open in a browser.
 - You must try to use multiple tools in one request when possible. For example if you were to create a website, you would use the write_to_file tool to create the necessary files with their appropriate contents all at once. Or if you wanted to analyze a project, you could use the read_file tool multiple times to look at several key files. This will help you accomplish the user's task more efficiently.
 - Be sure to consider the type of project (e.g. Python, JavaScript, web application) when determining the appropriate structure and files to include. Also consider what files may be most relevant to accomplishing the task, for example looking at a project's manifest file would help you understand the project's dependencies, which you could incorporate into any code you write.
 - When making changes to code, always consider the context in which the code is being used. Ensure that your changes are compatible with the existing codebase and that they follow the project's coding standards and best practices.
 - Do not ask for more information than necessary. Use the tools provided to accomplish the user's request efficiently and effectively. When you've completed your task, you must use the attempt_completion tool to present the result to the user. The user may provide feedback, which you can use to make improvements and try again.
-- You are only allowed to ask the user questions using the ask_followup_question tool. Use this tool only when you need additional details to complete a task, and be sure to use a clear and concise question that will help you move forward with the task.
+- You are only allowed to ask the user questions using the ask_followup_question tool. Use this tool only when you need additional details to complete a task, and be sure to use a clear and concise question that will help you move forward with the task. However if you can use the available tools to avoid having to ask the user questions, you should do so. For example if you need to know the name of a file, you can use the list files tool to get the name yourself. If the user refers to something vague, you can use the list_files_recursive tool to get a better understanding of the project to see if that helps you clear up any confusion.
 - Your goal is to try to accomplish the user's task, NOT engage in a back and forth conversation.
 - NEVER end completion_attempt with a question or request to engage in further conversation! Formulate the end of your result in a way that is final and does not require further input from the user. 
 - NEVER start your responses with affirmations like "Certaintly", "Okay", "Sure", "Great", etc. You should NOT be conversational in your responses, but rather direct and to the point.
 - Feel free to use markdown as much as you'd like in your responses. When using code blocks, always include a language specifier.
+- When presented with images, utilize your vision capabilities to thoroughly examine them and extract meaningful information. Incorporate these insights into your thought process as you accomplish the user's task.
 
 ====
 
@@ -62,7 +65,7 @@ OBJECTIVE
 You accomplish a given task iteratively, breaking it down into clear steps and working through them methodically.
 
 1. Analyze the user's task and set clear, achievable goals to accomplish it. Prioritize these goals in a logical order.
-2. Work through these goals sequentially, utilizing available tools as necessary. Each goal should correspond to a distinct step in your problem-solving process.
+2. Work through these goals sequentially, utilizing available tools as necessary. Each goal should correspond to a distinct step in your problem-solving process. It is okay for certain steps to take multiple iterations, i.e. if you need to create many files but are limited by your max output limitations, it's okay to create a few files at a time as each subsequent iteration will keep you informed on the work completed and what's remaining. 
 3. Remember, you have extensive capabilities with access to a wide range of tools that can be used in powerful and clever ways as necessary to accomplish each goal. Before calling a tool, do some analysis within <thinking></thinking> tags. First, think about which of the provided tools is the relevant tool to answer the user's request. Second, go through each of the required parameters of the relevant tool and determine if the user has directly provided or given enough information to infer a value. When deciding if the parameter can be inferred, carefully consider all the context to see if it supports a specific value. If all of the required parameters are present or can be reasonably inferred, close the thinking tag and proceed with the tool call. BUT, if one of the values for a required parameter is missing, DO NOT invoke the function (not even with fillers for the missing params) and instead, ask the user to provide the missing parameters using the ask_followup_question tool. DO NOT ask for more information on optional parameters if it is not provided.
 4. Once you've completed the user's task, you must use the attempt_completion tool to present the result of the task to the user. You may also provide a CLI command to showcase the result of your task; this can be particularly useful for web development tasks, where you can run e.g. \`open index.html\` to show the website you've built.
 5. The user may provide feedback, which you can use to make improvements and try again. But DO NOT continue in pointless back and forth conversations, i.e. don't end your responses with questions or offers for further assistance.
@@ -73,6 +76,8 @@ SYSTEM INFORMATION
 
 Operating System: ${osName()}
 Default Shell: ${defaultShell}
+Home Directory: ${os.homedir()}
+Current Working Directory: ${cwd}
 VSCode Visible Files: ${
 		vscode.window.visibleTextEditors
 			?.map((editor) => editor.document?.uri?.fsPath)
@@ -88,11 +93,13 @@ VSCode Opened Tabs: ${
 	}
 `
 
+const cwd =
+	vscode.workspace.workspaceFolders?.map((folder) => folder.uri.fsPath).at(0) ?? path.join(os.homedir(), "Desktop")
+
 const tools: Tool[] = [
 	{
 		name: "execute_command",
-		description:
-			"Execute a CLI command on the system. Use this when you need to perform system operations or run specific commands to accomplish any step in the user's task. You must tailor your command to the user's system and provide a clear explanation of what the command does. Prefer to execute complex CLI commands over creating executable scripts, as they are more flexible and easier to run.",
+		description: `Execute a CLI command on the system. Use this when you need to perform system operations or run specific commands to accomplish any step in the user's task. You must tailor your command to the user's system and provide a clear explanation of what the command does. Prefer to execute complex CLI commands over creating executable scripts, as they are more flexible and easier to run. Commands will be executed in the current working directory: ${cwd}`,
 		input_schema: {
 			type: "object",
 			properties: {
@@ -114,7 +121,7 @@ const tools: Tool[] = [
 			properties: {
 				path: {
 					type: "string",
-					description: "The path of the directory to list contents for.",
+					description: `The path of the directory to list contents for (relative to the current working directory ${cwd})`,
 				},
 			},
 			required: ["path"],
@@ -129,7 +136,7 @@ const tools: Tool[] = [
 			properties: {
 				path: {
 					type: "string",
-					description: "The path of the directory to recursively list contents for.",
+					description: `The path of the directory to recursively list contents for (relative to the current working directory ${cwd})`,
 				},
 			},
 			required: ["path"],
@@ -144,8 +151,7 @@ const tools: Tool[] = [
 			properties: {
 				path: {
 					type: "string",
-					description:
-						"The path of the directory to parse top level source code files for to view their definitions.",
+					description: `The path of the directory (relative to the current working directory ${cwd}) to parse top level source code files for to view their definitions`,
 				},
 			},
 			required: ["path"],
@@ -160,7 +166,7 @@ const tools: Tool[] = [
 			properties: {
 				path: {
 					type: "string",
-					description: "The path of the file to read.",
+					description: `The path of the file to read (relative to the current working directory ${cwd})`,
 				},
 			},
 			required: ["path"],
@@ -169,13 +175,13 @@ const tools: Tool[] = [
 	{
 		name: "write_to_file",
 		description:
-			"Write content to a file at the specified path. If the file exists, only the necessary changes will be applied. If the file doesn't exist, it will be created. Always provide the full intended content of the file. This tool will automatically create any directories needed to write the file.",
+			"Write content to a file at the specified path. If the file exists, only the necessary changes will be applied. If the file doesn't exist, it will be created. Always provide the full intended content of the file, without any truncation. This tool will automatically create any directories needed to write the file.",
 		input_schema: {
 			type: "object",
 			properties: {
 				path: {
 					type: "string",
-					description: "The path of the file to write to.",
+					description: `The path of the file to write to (relative to the current working directory ${cwd})`,
 				},
 				content: {
 					type: "string",
@@ -211,7 +217,7 @@ const tools: Tool[] = [
 				command: {
 					type: "string",
 					description:
-						"The CLI command to execute to show a live demo of the result to the user. For example, use 'open -a \"Google Chrome\" index.html' to display a created website. Avoid commands that run indefinitely (like servers) that don't terminate on their own. Instead, if such a command is needed, include instructions for the user to run it in the 'result' parameter.",
+						"The CLI command to execute to show a live demo of the result to the user. For example, use 'open index.html' to display a created website. This should be valid for the current operating system. Ensure the command is properly formatted and does not contain any harmful instructions.",
 				},
 				result: {
 					type: "string",
@@ -224,88 +230,140 @@ const tools: Tool[] = [
 	},
 ]
 
+type ToolResponse = string | Array<Anthropic.TextBlockParam | Anthropic.ImageBlockParam>
+
 export class ClaudeDev {
-	private client: Anthropic
+	private api: ApiHandler
 	private maxRequestsPerTask: number
+	private customInstructions?: string
 	private requestCount = 0
+	apiConversationHistory: Anthropic.MessageParam[] = []
+	claudeMessages: ClaudeMessage[] = []
 	private askResponse?: ClaudeAskResponse
 	private askResponseText?: string
+	private askResponseImages?: string[]
 	private lastMessageTs?: number
+	private executeCommandRunningProcess?: ResultPromise
 	private providerRef: WeakRef<ClaudeDevProvider>
-	abort: boolean = false
+	private abort: boolean = false
 
-	constructor(provider: ClaudeDevProvider, task: string, apiKey: string, maxRequestsPerTask?: number) {
+	constructor(
+		provider: ClaudeDevProvider,
+		apiConfiguration: ApiConfiguration,
+		maxRequestsPerTask?: number,
+		customInstructions?: string,
+		task?: string,
+		images?: string[]
+	) {
 		this.providerRef = new WeakRef(provider)
-		this.client = new Anthropic({ apiKey })
+		this.api = buildApiHandler(apiConfiguration)
 		this.maxRequestsPerTask = maxRequestsPerTask ?? DEFAULT_MAX_REQUESTS_PER_TASK
+		this.customInstructions = customInstructions
 
-		this.startTask(task)
+		this.startTask(task, images)
 	}
 
-	updateApiKey(apiKey: string) {
-		this.client = new Anthropic({ apiKey })
+	updateApi(apiConfiguration: ApiConfiguration) {
+		this.api = buildApiHandler(apiConfiguration)
 	}
 
 	updateMaxRequestsPerTask(maxRequestsPerTask: number | undefined) {
 		this.maxRequestsPerTask = maxRequestsPerTask ?? DEFAULT_MAX_REQUESTS_PER_TASK
 	}
 
-	async handleWebviewAskResponse(askResponse: ClaudeAskResponse, text?: string) {
-		this.askResponse = askResponse
-		this.askResponseText = text
+	updateCustomInstructions(customInstructions: string | undefined) {
+		this.customInstructions = customInstructions
 	}
 
-	async ask(type: ClaudeAsk, question: string): Promise<{ response: ClaudeAskResponse; text?: string }> {
+	async handleWebviewAskResponse(askResponse: ClaudeAskResponse, text?: string, images?: string[]) {
+		this.askResponse = askResponse
+		this.askResponseText = text
+		this.askResponseImages = images
+	}
+
+	async ask(
+		type: ClaudeAsk,
+		question: string
+	): Promise<{ response: ClaudeAskResponse; text?: string; images?: string[] }> {
 		// If this ClaudeDev instance was aborted by the provider, then the only thing keeping us alive is a promise still running in the background, in which case we don't want to send its result to the webview as it is attached to a new instance of ClaudeDev now. So we can safely ignore the result of any active promises, and this class will be deallocated. (Although we set claudeDev = undefined in provider, that simply removes the reference to this instance, but the instance is still alive until this promise resolves or rejects.)
 		if (this.abort) {
 			throw new Error("ClaudeDev instance aborted")
 		}
 		this.askResponse = undefined
 		this.askResponseText = undefined
+		this.askResponseImages = undefined
 		const askTs = Date.now()
 		this.lastMessageTs = askTs
-		await this.providerRef.deref()?.addClaudeMessage({ ts: askTs, type: "ask", ask: type, text: question })
+		this.claudeMessages.push({ ts: askTs, type: "ask", ask: type, text: question })
 		await this.providerRef.deref()?.postStateToWebview()
 		await pWaitFor(() => this.askResponse !== undefined || this.lastMessageTs !== askTs, { interval: 100 })
 		if (this.lastMessageTs !== askTs) {
 			throw new Error("Current ask promise was ignored") // could happen if we send multiple asks in a row i.e. with command_output. It's important that when we know an ask could fail, it is handled gracefully
 		}
-		const result = { response: this.askResponse!, text: this.askResponseText }
+		const result = { response: this.askResponse!, text: this.askResponseText, images: this.askResponseImages }
 		this.askResponse = undefined
 		this.askResponseText = undefined
+		this.askResponseImages = undefined
 		return result
 	}
 
-	async say(type: ClaudeSay, text?: string): Promise<undefined> {
+	async say(type: ClaudeSay, text?: string, images?: string[]): Promise<undefined> {
 		if (this.abort) {
 			throw new Error("ClaudeDev instance aborted")
 		}
 		const sayTs = Date.now()
 		this.lastMessageTs = sayTs
-		await this.providerRef.deref()?.addClaudeMessage({ ts: sayTs, type: "say", say: type, text: text })
+		this.claudeMessages.push({ ts: sayTs, type: "say", say: type, text: text, images })
 		await this.providerRef.deref()?.postStateToWebview()
 	}
 
-	private async startTask(task: string): Promise<void> {
+	private formatImagesIntoBlocks(images?: string[]): Anthropic.ImageBlockParam[] {
+		return images
+			? images.map((dataUrl) => {
+					// data:image/png;base64,base64string
+					const [rest, base64] = dataUrl.split(",")
+					const mimeType = rest.split(":")[1].split(";")[0]
+					return {
+						type: "image",
+						source: { type: "base64", media_type: mimeType, data: base64 },
+					} as Anthropic.ImageBlockParam
+			  })
+			: []
+	}
+
+	private formatIntoToolResponse(text: string, images?: string[]): ToolResponse {
+		if (images && images.length > 0) {
+			const textBlock: Anthropic.TextBlockParam = { type: "text", text }
+			const imageBlocks: Anthropic.ImageBlockParam[] = this.formatImagesIntoBlocks(images)
+			// Placing images after text leads to better results
+			return [textBlock, ...imageBlocks]
+		} else {
+			return text
+		}
+	}
+
+	private async startTask(task?: string, images?: string[]): Promise<void> {
 		// conversationHistory (for API) and claudeMessages (for webview) need to be in sync
 		// if the extension process were killed, then on restart the claudeMessages might not be empty, so we need to set it to [] when we create a new ClaudeDev client (otherwise webview would show stale messages from previous session)
-		await this.providerRef.deref()?.setClaudeMessages(undefined)
+		this.claudeMessages = []
+		this.apiConversationHistory = []
 		await this.providerRef.deref()?.postStateToWebview()
 
-		// This first message kicks off a task, it is not included in every subsequent message.
-		let userPrompt = `Task: \"${task}\"`
+		let textBlock: Anthropic.TextBlockParam = { type: "text", text: `Task: \"${task}\"` }
+		let imageBlocks: Anthropic.ImageBlockParam[] = this.formatImagesIntoBlocks(images)
 
 		// TODO: create tools that let Claude interact with VSCode (e.g. open a file, list open files, etc.)
 		//const openFiles = vscode.window.visibleTextEditors?.map((editor) => editor.document.uri.fsPath).join("\n")
 
-		await this.say("text", task)
+		await this.say("text", task, images)
 
 		let totalInputTokens = 0
 		let totalOutputTokens = 0
 
-		while (this.requestCount < this.maxRequestsPerTask) {
+		while (!this.abort) {
 			const { didEndLoop, inputTokens, outputTokens } = await this.recursivelyMakeClaudeRequests([
-				{ type: "text", text: userPrompt },
+				textBlock,
+				...imageBlocks,
 			])
 			totalInputTokens += inputTokens
 			totalOutputTokens += outputTokens
@@ -315,6 +373,7 @@ export class ClaudeDev {
 
 			//const totalCost = this.calculateApiCost(totalInputTokens, totalOutputTokens)
 			if (didEndLoop) {
+				// For now a task never 'completes'. This will only happen if the user hits max requests and denies resetting the count.
 				//this.say("task_completed", `Task completed. Total API usage cost: ${totalCost}`)
 				break
 			} else {
@@ -322,13 +381,24 @@ export class ClaudeDev {
 				// 	"tool",
 				// 	"Claude responded with only text blocks but has not called attempt_completion yet. Forcing him to continue with task..."
 				// )
-				userPrompt =
-					"Ask yourself if you have completed the user's task. If you have, use the attempt_completion tool, otherwise proceed to the next step. (This is an automated message, so do not respond to it conversationally. Just proceed with the task.)"
+				textBlock = {
+					type: "text",
+					text: "If you have completed the user's task, use the attempt_completion tool. If you require additional information from the user, use the ask_followup_question tool. Otherwise, if you have not completed the task and do not need additional information, then proceed with the next step of the task. (This is an automated message, so do not respond to it conversationally.)",
+				}
+				imageBlocks = []
 			}
 		}
 	}
 
-	async executeTool(toolName: ToolName, toolInput: any, isLastWriteToFile: boolean = false): Promise<string> {
+	abortTask() {
+		this.abort = true // will stop any autonomously running promises
+		const runningProcessId = this.executeCommandRunningProcess?.pid
+		if (runningProcessId) {
+			treeKill(runningProcessId, "SIGTERM")
+		}
+	}
+
+	async executeTool(toolName: ToolName, toolInput: any, isLastWriteToFile: boolean = false): Promise<ToolResponse> {
 		switch (toolName) {
 			case "write_to_file":
 				return this.writeToFile(toolInput.path, toolInput.content, isLastWriteToFile)
@@ -351,30 +421,46 @@ export class ClaudeDev {
 		}
 	}
 
-	// Calculates cost of a Claude 3.5 Sonnet API request
 	calculateApiCost(inputTokens: number, outputTokens: number): number {
-		const INPUT_COST_PER_MILLION = 3.0 // $3 per million input tokens
-		const OUTPUT_COST_PER_MILLION = 15.0 // $15 per million output tokens
-		const inputCost = (inputTokens / 1_000_000) * INPUT_COST_PER_MILLION
-		const outputCost = (outputTokens / 1_000_000) * OUTPUT_COST_PER_MILLION
+		const inputCost = (this.api.getModel().info.inputPrice / 1_000_000) * inputTokens
+		const outputCost = (this.api.getModel().info.outputPrice / 1_000_000) * outputTokens
 		const totalCost = inputCost + outputCost
 		return totalCost
 	}
 
-	async writeToFile(filePath: string, newContent: string, isLast: boolean): Promise<string> {
+	async writeToFile(relPath?: string, newContent?: string, isLast: boolean = true): Promise<ToolResponse> {
+		if (relPath === undefined) {
+			this.say(
+				"error",
+				"Claude tried to use write_to_file without value for required parameter 'path'. Retrying..."
+			)
+			return "Error: Missing value for required parameter 'path'. Please retry with complete response."
+		}
+
+		if (newContent === undefined) {
+			// Special message for this case since this tends to happen the most
+			this.say(
+				"error",
+				`Claude tried to use write_to_file for '${relPath}' without value for required parameter 'content'. This is likely due to output token limits. Retrying...`
+			)
+			return "Error: Missing value for required parameter 'content'. Please retry with complete response."
+		}
+
 		try {
+			const absolutePath = path.resolve(cwd, relPath)
 			const fileExists = await fs
-				.access(filePath)
+				.access(absolutePath)
 				.then(() => true)
 				.catch(() => false)
+
 			if (fileExists) {
-				const originalContent = await fs.readFile(filePath, "utf-8")
+				const originalContent = await fs.readFile(absolutePath, "utf-8")
 				// fix issue where claude always removes newline from the file
 				if (originalContent.endsWith("\n") && !newContent.endsWith("\n")) {
 					newContent += "\n"
 				}
 				// condensed patch to return to claude
-				const diffResult = diff.createPatch(filePath, originalContent, newContent)
+				const diffResult = diff.createPatch(absolutePath, originalContent, newContent)
 				// full diff representation for webview
 				const diffRepresentation = diff
 					.diffLines(originalContent, newContent)
@@ -388,10 +474,10 @@ export class ClaudeDev {
 					.join("")
 
 				// Create virtual document with new file, then open diff editor
-				const fileName = path.basename(filePath)
+				const fileName = path.basename(absolutePath)
 				vscode.commands.executeCommand(
 					"vscode.diff",
-					vscode.Uri.file(filePath),
+					vscode.Uri.file(absolutePath),
 					// to create a virtual doc we use a uri scheme registered in extension.ts, which then converts this base64 content into a text document
 					// (providing file name with extension in the uri lets vscode know the language of the file and apply syntax highlighting)
 					vscode.Uri.parse(`claude-dev-diff:${fileName}`).with({
@@ -400,11 +486,11 @@ export class ClaudeDev {
 					`${fileName}: Original ↔ Suggested Changes`
 				)
 
-				const { response, text } = await this.ask(
+				const { response, text, images } = await this.ask(
 					"tool",
 					JSON.stringify({
 						tool: "editedExistingFile",
-						path: filePath,
+						path: this.getReadablePath(relPath),
 						diff: diffRepresentation,
 					} as ClaudeSayTool)
 				)
@@ -412,21 +498,24 @@ export class ClaudeDev {
 					if (isLast) {
 						await this.closeDiffViews()
 					}
-					if (response === "textResponse" && text) {
-						await this.say("user_feedback", text)
-						return `The user denied this operation and provided the following feedback:\n\"${text}\"`
+					if (response === "messageResponse") {
+						await this.say("user_feedback", text, images)
+						return this.formatIntoToolResponse(
+							`The user denied this operation and provided the following feedback:\n\"${text}\"`,
+							images
+						)
 					}
 					return "The user denied this operation."
 				}
-				await fs.writeFile(filePath, newContent)
+				await fs.writeFile(absolutePath, newContent)
 				// Finish by opening the edited file in the editor
-				await vscode.window.showTextDocument(vscode.Uri.file(filePath), { preview: false })
+				await vscode.window.showTextDocument(vscode.Uri.file(absolutePath), { preview: false })
 				if (isLast) {
 					await this.closeDiffViews()
 				}
-				return `Changes applied to ${filePath}:\n${diffResult}`
+				return `Changes applied to ${relPath}:\n${diffResult}`
 			} else {
-				const fileName = path.basename(filePath)
+				const fileName = path.basename(absolutePath)
 				vscode.commands.executeCommand(
 					"vscode.diff",
 					vscode.Uri.parse(`claude-dev-diff:${fileName}`).with({
@@ -437,27 +526,34 @@ export class ClaudeDev {
 					}),
 					`${fileName}: New File`
 				)
-				const { response, text } = await this.ask(
+				const { response, text, images } = await this.ask(
 					"tool",
-					JSON.stringify({ tool: "newFileCreated", path: filePath, content: newContent } as ClaudeSayTool)
+					JSON.stringify({
+						tool: "newFileCreated",
+						path: this.getReadablePath(relPath),
+						content: newContent,
+					} as ClaudeSayTool)
 				)
 				if (response !== "yesButtonTapped") {
 					if (isLast) {
 						await this.closeDiffViews()
 					}
-					if (response === "textResponse" && text) {
-						await this.say("user_feedback", text)
-						return `The user denied this operation and provided the following feedback:\n\"${text}\"`
+					if (response === "messageResponse") {
+						await this.say("user_feedback", text, images)
+						return this.formatIntoToolResponse(
+							`The user denied this operation and provided the following feedback:\n\"${text}\"`,
+							images
+						)
 					}
 					return "The user denied this operation."
 				}
-				await fs.mkdir(path.dirname(filePath), { recursive: true })
-				await fs.writeFile(filePath, newContent)
-				await vscode.window.showTextDocument(vscode.Uri.file(filePath), { preview: false })
+				await fs.mkdir(path.dirname(absolutePath), { recursive: true })
+				await fs.writeFile(absolutePath, newContent)
+				await vscode.window.showTextDocument(vscode.Uri.file(absolutePath), { preview: false })
 				if (isLast) {
 					await this.closeDiffViews()
 				}
-				return `New file created and content written to ${filePath}`
+				return `New file created and content written to ${relPath}`
 			}
 		} catch (error) {
 			const errorString = `Error writing file: ${JSON.stringify(serializeError(error))}`
@@ -479,17 +575,25 @@ export class ClaudeDev {
 		}
 	}
 
-	async readFile(filePath: string): Promise<string> {
+	async readFile(relPath?: string): Promise<ToolResponse> {
+		if (relPath === undefined) {
+			this.say("error", "Claude tried to use read_file without value for required parameter 'path'. Retrying...")
+			return "Error: Missing value for required parameter 'path'. Please retry with complete response."
+		}
 		try {
-			const content = await fs.readFile(filePath, "utf-8")
-			const { response, text } = await this.ask(
+			const absolutePath = path.resolve(cwd, relPath)
+			const content = await fs.readFile(absolutePath, "utf-8")
+			const { response, text, images } = await this.ask(
 				"tool",
-				JSON.stringify({ tool: "readFile", path: filePath, content } as ClaudeSayTool)
+				JSON.stringify({ tool: "readFile", path: this.getReadablePath(relPath), content } as ClaudeSayTool)
 			)
 			if (response !== "yesButtonTapped") {
-				if (response === "textResponse" && text) {
-					await this.say("user_feedback", text)
-					return `The user denied this operation and provided the following feedback:\n\"${text}\"`
+				if (response === "messageResponse") {
+					await this.say("user_feedback", text, images)
+					return this.formatIntoToolResponse(
+						`The user denied this operation and provided the following feedback:\n\"${text}\"`,
+						images
+					)
 				}
 				return "The user denied this operation."
 			}
@@ -501,31 +605,33 @@ export class ClaudeDev {
 		}
 	}
 
-	async listFilesTopLevel(dirPath: string): Promise<string> {
+	async listFilesTopLevel(relDirPath?: string): Promise<ToolResponse> {
+		if (relDirPath === undefined) {
+			this.say(
+				"error",
+				"Claude tried to use list_files_top_level without value for required parameter 'path'. Retrying..."
+			)
+			return "Error: Missing value for required parameter 'path'. Please retry with complete response."
+		}
 		try {
-			const files = await listFiles(dirPath, false)
-			const result = files
-				.map((file) => {
-					const relativePath = path.relative(dirPath, file)
-					return file.endsWith("/") ? relativePath + "/" : relativePath
-				})
-				.sort((a, b) => {
-					const aIsDir = a.endsWith("/")
-					const bIsDir = b.endsWith("/")
-					if (aIsDir !== bIsDir) {
-						return aIsDir ? -1 : 1
-					}
-					return a.localeCompare(b, undefined, { numeric: true, sensitivity: "base" })
-				})
-				.join("\n")
-			const { response, text } = await this.ask(
+			const absolutePath = path.resolve(cwd, relDirPath)
+			const files = await listFiles(absolutePath, false)
+			const result = this.formatFilesList(absolutePath, files)
+			const { response, text, images } = await this.ask(
 				"tool",
-				JSON.stringify({ tool: "listFilesTopLevel", path: dirPath, content: result } as ClaudeSayTool)
+				JSON.stringify({
+					tool: "listFilesTopLevel",
+					path: this.getReadablePath(relDirPath),
+					content: result,
+				} as ClaudeSayTool)
 			)
 			if (response !== "yesButtonTapped") {
-				if (response === "textResponse" && text) {
-					await this.say("user_feedback", text)
-					return `The user denied this operation and provided the following feedback:\n\"${text}\"`
+				if (response === "messageResponse") {
+					await this.say("user_feedback", text, images)
+					return this.formatIntoToolResponse(
+						`The user denied this operation and provided the following feedback:\n\"${text}\"`,
+						images
+					)
 				}
 				return "The user denied this operation."
 			}
@@ -542,18 +648,33 @@ export class ClaudeDev {
 		}
 	}
 
-	async listFilesRecursive(dirPath: string): Promise<string> {
+	async listFilesRecursive(relDirPath?: string): Promise<ToolResponse> {
+		if (relDirPath === undefined) {
+			this.say(
+				"error",
+				"Claude tried to use list_files_recursive without value for required parameter 'path'. Retrying..."
+			)
+			return "Error: Missing value for required parameter 'path'. Please retry with complete response."
+		}
 		try {
-			const files = await listFiles(dirPath, true)
-			const result = files.map((file) => path.relative(dirPath, file)).join("\n")
-			const { response, text } = await this.ask(
+			const absolutePath = path.resolve(cwd, relDirPath)
+			const files = await listFiles(absolutePath, true)
+			const result = this.formatFilesList(absolutePath, files)
+			const { response, text, images } = await this.ask(
 				"tool",
-				JSON.stringify({ tool: "listFilesRecursive", path: dirPath, content: result } as ClaudeSayTool)
+				JSON.stringify({
+					tool: "listFilesRecursive",
+					path: this.getReadablePath(relDirPath),
+					content: result,
+				} as ClaudeSayTool)
 			)
 			if (response !== "yesButtonTapped") {
-				if (response === "textResponse" && text) {
-					await this.say("user_feedback", text)
-					return `The user denied this operation and provided the following feedback:\n\"${text}\"`
+				if (response === "messageResponse") {
+					await this.say("user_feedback", text, images)
+					return this.formatIntoToolResponse(
+						`The user denied this operation and provided the following feedback:\n\"${text}\"`,
+						images
+					)
 				}
 				return "The user denied this operation."
 			}
@@ -568,21 +689,81 @@ export class ClaudeDev {
 		}
 	}
 
-	async viewSourceCodeDefinitionsTopLevel(dirPath: string): Promise<string> {
+	getReadablePath(relPath: string): string {
+		// path.resolve is flexible in that it will resolve relative paths like '../../' to the cwd and even ignore the cwd if the relPath is actually an absolute path
+		const absolutePath = path.resolve(cwd, relPath)
+		if (cwd === path.join(os.homedir(), "Desktop")) {
+			// User opened vscode without a workspace, so cwd is the Desktop. Show the full absolute path to keep the user aware of where files are being created
+			return absolutePath
+		}
+		if (path.normalize(absolutePath) === path.normalize(cwd)) {
+			return path.basename(absolutePath)
+		} else {
+			// show the relative path to the cwd
+			const normalizedRelPath = path.relative(cwd, absolutePath)
+			if (absolutePath.includes(cwd)) {
+				return normalizedRelPath
+			} else {
+				// we are outside the cwd, so show the absolute path (useful for when claude passes in '../../' for example)
+				return absolutePath
+			}
+		}
+	}
+
+	formatFilesList(absolutePath: string, files: string[]): string {
+		const sorted = files
+			.map((file) => {
+				// convert absolute path to relative path
+				const relativePath = path.relative(absolutePath, file)
+				return file.endsWith("/") ? relativePath + "/" : relativePath
+			})
+			.sort((a, b) => {
+				// sort directories before files
+				const aIsDir = a.endsWith("/")
+				const bIsDir = b.endsWith("/")
+				if (aIsDir !== bIsDir) {
+					return aIsDir ? -1 : 1
+				}
+				return a.localeCompare(b, undefined, { numeric: true, sensitivity: "base" })
+			})
+
+		if (sorted.length > 1000) {
+			const truncatedList = sorted.slice(0, 1000).join("\n")
+			const remainingCount = sorted.length - 1000
+			return `${truncatedList}\n\n(${remainingCount} files not listed due to automatic truncation. Try listing files in subdirectories if you need to explore further.)`
+		} else if (sorted.length === 0 || (sorted.length === 1 && sorted[0] === "")) {
+			return "No files found or you do not have permission to view this directory."
+		} else {
+			return sorted.join("\n")
+		}
+	}
+
+	async viewSourceCodeDefinitionsTopLevel(relDirPath?: string): Promise<ToolResponse> {
+		if (relDirPath === undefined) {
+			this.say(
+				"error",
+				"Claude tried to use view_source_code_definitions_top_level without value for required parameter 'path'. Retrying..."
+			)
+			return "Error: Missing value for required parameter 'path'. Please retry with complete response."
+		}
 		try {
-			const result = await parseSourceCodeForDefinitionsTopLevel(dirPath)
-			const { response, text } = await this.ask(
+			const absolutePath = path.resolve(cwd, relDirPath)
+			const result = await parseSourceCodeForDefinitionsTopLevel(absolutePath)
+			const { response, text, images } = await this.ask(
 				"tool",
 				JSON.stringify({
 					tool: "viewSourceCodeDefinitionsTopLevel",
-					path: dirPath,
+					path: this.getReadablePath(relDirPath),
 					content: result,
 				} as ClaudeSayTool)
 			)
 			if (response !== "yesButtonTapped") {
-				if (response === "textResponse" && text) {
-					await this.say("user_feedback", text)
-					return `The user denied this operation and provided the following feedback:\n\"${text}\"`
+				if (response === "messageResponse") {
+					await this.say("user_feedback", text, images)
+					return this.formatIntoToolResponse(
+						`The user denied this operation and provided the following feedback:\n\"${text}\"`,
+						images
+					)
 				}
 				return "The user denied this operation."
 			}
@@ -599,53 +780,95 @@ export class ClaudeDev {
 		}
 	}
 
-	async executeCommand(command: string, returnEmptyStringOnSuccess: boolean = false): Promise<string> {
-		const { response, text } = await this.ask("command", command)
+	async executeCommand(command?: string, returnEmptyStringOnSuccess: boolean = false): Promise<ToolResponse> {
+		if (command === undefined) {
+			this.say(
+				"error",
+				"Claude tried to use execute_command without value for required parameter 'command'. Retrying..."
+			)
+			return "Error: Missing value for required parameter 'command'. Please retry with complete response."
+		}
+		const { response, text, images } = await this.ask("command", command)
 		if (response !== "yesButtonTapped") {
-			if (response === "textResponse" && text) {
-				await this.say("user_feedback", text)
-				return `The user denied this operation and provided the following feedback:\n\"${text}\"`
+			if (response === "messageResponse") {
+				await this.say("user_feedback", text, images)
+				return this.formatIntoToolResponse(
+					`The user denied this operation and provided the following feedback:\n\"${text}\"`,
+					images
+				)
 			}
 			return "The user denied this operation."
 		}
+
+		const sendCommandOutput = async (subprocess: ResultPromise, line: string): Promise<void> => {
+			try {
+				const { response, text } = await this.ask("command_output", line)
+				// if this ask promise is not ignored, that means the user responded to it somehow either by clicking primary button or by typing text
+				if (response === "yesButtonTapped") {
+					// SIGINT is typically what's sent when a user interrupts a process (like pressing Ctrl+C)
+					/*
+					.kill sends SIGINT by default. However by not passing any options into .kill(), execa internally sends a SIGKILL after a grace period if the SIGINT failed.
+					however it turns out that even this isn't enough for certain processes like npm starting servers. therefore we use the tree-kill package to kill all processes in the process tree, including the root process.
+					- Sends signal to all children processes of the process with pid pid, including pid. Signal defaults to SIGTERM.
+					*/
+					if (subprocess.pid) {
+						//subprocess.kill("SIGINT") // will result in for loop throwing error
+						treeKill(subprocess.pid, "SIGINT")
+					}
+				} else {
+					// if the user sent some input, we send it to the command stdin
+					// add newline as cli programs expect a newline after each input
+					// (stdin needs to be set to `pipe` to send input to the command, execa does this by default when using template literals - other options are inherit (from parent process stdin) or null (no stdin))
+					subprocess.stdin?.write(text + "\n")
+					// Recurse with an empty string to continue listening for more input
+					sendCommandOutput(subprocess, "") // empty strings are effectively ignored by the webview, this is done solely to relinquish control over the exit command button
+				}
+			} catch {
+				// This can only happen if this ask promise was ignored, so ignore this error
+			}
+		}
+
 		try {
 			let result = ""
 			// execa by default tries to convert bash into javascript, so need to specify `shell: true` to use sh on unix or cmd.exe on windows
 			// also worth noting that execa`input` and the execa(command) have nuanced differences like the template literal version handles escaping for you, while with the function call, you need to be more careful about how arguments are passed, especially when using shell: true.
 			// execa returns a promise-like object that is both a promise and a Subprocess that has properties like stdin
-			const subprocess = execa({ shell: true })`${command}`
+			const subprocess = execa({ shell: true, cwd: cwd })`${command}`
+			this.executeCommandRunningProcess = subprocess
+
+			subprocess.stdout?.on("data", (data) => {
+				if (data) {
+					const output = data.toString()
+					// stream output to user in realtime
+					// do not await since it's sent as an ask and we are not waiting for a response
+					sendCommandOutput(subprocess, output)
+					result += output
+				}
+			})
 
 			try {
-				for await (const chunk of subprocess) {
-					const line = chunk.toString()
-					// stream output to user in realtime
-					// do not await as we are not waiting for a response
-					this.ask("command_output", line)
-						.then(({ response, text }) => {
-							// if this ask promise is not ignored, that means the user responded to it somehow either by clicking primary button or by typing text
-							if (response === "yesButtonTapped") {
-								// SIGINT is typically what's sent when a user interrupts a process (like pressing Ctrl+C)
-								subprocess.kill("SIGINT") // will result in for loop throwing error
-							} else {
-								// if the user sent some input, we send it to the command stdin
-								// add newline as cli programs expect a newline after each input
-								subprocess.stdin?.write(text + "\n")
-							}
-						})
-						.catch(() => {
-							// this can only happen if this ask promise was ignored, so ignore this error
-						})
-					result += `${line}\n`
-				}
+				await subprocess
+				// NOTE: using for await to stream execa output does not return lines that expect user input, so we use listen to the stdout stream and handle data directly, allowing us to process output as soon as it's available even before a full line is complete.
+				// for await (const chunk of subprocess) {
+				// 	const line = chunk.toString()
+				// 	sendCommandOutput(subprocess, line)
+				// 	result += `${line}\n`
+				// }
 			} catch (e) {
 				if ((e as ExecaError).signal === "SIGINT") {
-					const line = `\nUser exited command...`
-					await this.say("command_output", line)
-					result += line
+					await this.say("command_output", `\nUser exited command...`)
+					result += `\n====\nUser terminated command process via SIGINT. This is not an error. Please continue with your task, but keep in mind that the command is no longer running. For example, if this command was used to start a server for a react app, the server is no longer running and you cannot open a browser to view it anymore.`
 				} else {
 					throw e // if the command was not terminated by user, let outer catch handle it as a real error
 				}
 			}
+			// Wait for a short delay to ensure all messages are sent to the webview
+			// This delay allows time for non-awaited promises to be created and
+			// for their associated messages to be sent to the webview, maintaining
+			// the correct order of messages (although the webview is smart about
+			// grouping command_output messages despite any gaps anyways)
+			await delay(100)
+			this.executeCommandRunningProcess = undefined
 			// for attemptCompletion, we don't want to return the command output
 			if (returnEmptyStringOnSuccess) {
 				return ""
@@ -656,17 +879,33 @@ export class ClaudeDev {
 			let errorMessage = error.message || JSON.stringify(serializeError(error), null, 2)
 			const errorString = `Error executing command:\n${errorMessage}`
 			this.say("error", `Error executing command:\n${errorMessage}`) // TODO: in webview show code block for command errors
+			this.executeCommandRunningProcess = undefined
 			return errorString
 		}
 	}
 
-	async askFollowupQuestion(question: string): Promise<string> {
-		const { text } = await this.ask("followup", question)
-		await this.say("user_feedback", text ?? "")
-		return `User's response:\n\"${text}\"`
+	async askFollowupQuestion(question?: string): Promise<ToolResponse> {
+		if (question === undefined) {
+			this.say(
+				"error",
+				"Claude tried to use ask_followup_question without value for required parameter 'question'. Retrying..."
+			)
+			return "Error: Missing value for required parameter 'question'. Please retry with complete response."
+		}
+		const { text, images } = await this.ask("followup", question)
+		await this.say("user_feedback", text ?? "", images)
+		return this.formatIntoToolResponse(`User's response:\n\"${text}\"`, images)
 	}
 
-	async attemptCompletion(result: string, command?: string): Promise<string> {
+	async attemptCompletion(result?: string, command?: string): Promise<ToolResponse> {
+		// result is required, command is optional
+		if (result === undefined) {
+			this.say(
+				"error",
+				"Claude tried to use attempt_completion without value for required parameter 'result'. Retrying..."
+			)
+			return "Error: Missing value for required parameter 'result'. Please retry with complete response."
+		}
 		let resultToSend = result
 		if (command) {
 			await this.say("completion_result", resultToSend)
@@ -678,32 +917,32 @@ export class ClaudeDev {
 			}
 			resultToSend = ""
 		}
-		const { response, text } = await this.ask("completion_result", resultToSend) // this prompts webview to show 'new task' button, and enable text input (which would be the 'text' here)
+		const { response, text, images } = await this.ask("completion_result", resultToSend) // this prompts webview to show 'new task' button, and enable text input (which would be the 'text' here)
 		if (response === "yesButtonTapped") {
-			return ""
+			return "" // signals to recursive loop to stop (for now this never happens since yesButtonTapped will trigger a new task)
 		}
-		await this.say("user_feedback", text ?? "")
-		return `The user is not pleased with the results. Use the feedback they provided to successfully complete the task, and then attempt completion again.\nUser's feedback:\n\"${text}\"`
+		await this.say("user_feedback", text ?? "", images)
+		return this.formatIntoToolResponse(
+			`The user is not pleased with the results. Use the feedback they provided to successfully complete the task, and then attempt completion again.\nUser's feedback:\n\"${text}\"`,
+			images
+		)
 	}
 
 	async attemptApiRequest(): Promise<Anthropic.Messages.Message> {
 		try {
-			const response = await this.client.messages.create(
-				{
-					model: "claude-3-5-sonnet-20240620", // https://docs.anthropic.com/en/docs/about-claude/models
-					// beta max tokens
-					max_tokens: 8192,
-					system: SYSTEM_PROMPT(),
-					messages: (await this.providerRef.deref()?.getApiConversationHistory()) || [],
-					tools: tools,
-					tool_choice: { type: "auto" },
-				},
-				{
-					// https://github.com/anthropics/anthropic-sdk-typescript?tab=readme-ov-file#default-headers
-					headers: { "anthropic-beta": "max-tokens-3-5-sonnet-2024-07-15" },
-				}
-			)
-			return response
+			let systemPrompt = SYSTEM_PROMPT()
+			if (this.customInstructions && this.customInstructions.trim()) {
+				systemPrompt += `
+====
+
+USER'S CUSTOM INSTRUCTIONS
+
+The following additional instructions are provided by the user. They should be followed and given precedence in case of conflicts with previous instructions.
+
+${this.customInstructions.trim()}
+`
+			}
+			return await this.api.createMessage(systemPrompt, this.apiConversationHistory, tools)
 		} catch (error) {
 			const { response } = await this.ask(
 				"api_req_failed",
@@ -730,7 +969,7 @@ export class ClaudeDev {
 			throw new Error("ClaudeDev instance aborted")
 		}
 
-		await this.providerRef.deref()?.addMessageToApiConversationHistory({ role: "user", content: userContent })
+		this.apiConversationHistory.push({ role: "user", content: userContent })
 		if (this.requestCount >= this.maxRequestsPerTask) {
 			const { response } = await this.ask(
 				"request_limit_reached",
@@ -740,7 +979,7 @@ export class ClaudeDev {
 			if (response === "yesButtonTapped") {
 				this.requestCount = 0
 			} else {
-				await this.providerRef.deref()?.addMessageToApiConversationHistory({
+				this.apiConversationHistory.push({
 					role: "assistant",
 					content: [
 						{
@@ -757,14 +996,7 @@ export class ClaudeDev {
 		await this.say(
 			"api_req_started",
 			JSON.stringify({
-				request: {
-					model: "claude-3-5-sonnet-20240620",
-					max_tokens: 8192,
-					system: "(see SYSTEM_PROMPT in https://github.com/saoudrizwan/claude-dev/blob/main/src/ClaudeDev.ts)",
-					messages: [{ conversation_history: "..." }, { role: "user", content: userContent }],
-					tools: "(see tools in https://github.com/saoudrizwan/claude-dev/blob/main/src/ClaudeDev.ts)",
-					tool_choice: { type: "auto" },
-				},
+				request: this.api.createUserReadableRequest(userContent),
 			})
 		)
 		try {
@@ -809,6 +1041,7 @@ export class ClaudeDev {
 						if (toolName === "write_to_file") {
 							currentWriteToFile++
 						}
+						// NOTE: while anthropic sdk accepts string or array of string/image, openai sdk (openrouter) only accepts a string
 						const result = await this.executeTool(
 							toolName,
 							toolInput,
@@ -824,13 +1057,11 @@ export class ClaudeDev {
 			}
 
 			if (assistantResponses.length > 0) {
-				await this.providerRef
-					.deref()
-					?.addMessageToApiConversationHistory({ role: "assistant", content: assistantResponses })
+				this.apiConversationHistory.push({ role: "assistant", content: assistantResponses })
 			} else {
 				// this should never happen! it there's no assistant_responses, that means we got no text or tool_use content blocks from API which we should assume is an error
 				this.say("error", "Unexpected Error: No assistant messages were found in the API response")
-				await this.providerRef.deref()?.addMessageToApiConversationHistory({
+				this.apiConversationHistory.push({
 					role: "assistant",
 					content: [{ type: "text", text: "Failure: I did not have a response to provide." }],
 				})
@@ -860,10 +1091,8 @@ export class ClaudeDev {
 
 			if (toolResults.length > 0) {
 				if (didEndLoop) {
-					await this.providerRef
-						.deref()
-						?.addMessageToApiConversationHistory({ role: "user", content: toolResults })
-					await this.providerRef.deref()?.addMessageToApiConversationHistory({
+					this.apiConversationHistory.push({ role: "user", content: toolResults })
+					this.apiConversationHistory.push({
 						role: "assistant",
 						content: [
 							{

src/api/anthropic.ts

@@ -0,0 +1,64 @@
+import { Anthropic } from "@anthropic-ai/sdk"
+import { ApiHandler, withoutImageData } from "."
+import { anthropicDefaultModelId, AnthropicModelId, anthropicModels, ApiHandlerOptions, ModelInfo } from "../shared/api"
+
+export class AnthropicHandler implements ApiHandler {
+	private options: ApiHandlerOptions
+	private client: Anthropic
+
+	constructor(options: ApiHandlerOptions) {
+		this.options = options
+		this.client = new Anthropic({ apiKey: this.options.apiKey })
+	}
+
+	async createMessage(
+		systemPrompt: string,
+		messages: Anthropic.Messages.MessageParam[],
+		tools: Anthropic.Messages.Tool[]
+	): Promise<Anthropic.Messages.Message> {
+		return await this.client.messages.create(
+			{
+				model: this.getModel().id,
+				max_tokens: this.getModel().info.maxTokens,
+				system: systemPrompt,
+				messages,
+				tools,
+				tool_choice: { type: "auto" },
+			},
+			// https://x.com/alexalbert__/status/1812921642143900036
+			// https://github.com/anthropics/anthropic-sdk-typescript?tab=readme-ov-file#default-headers
+			this.getModel().id === "claude-3-5-sonnet-20240620"
+				? {
+						headers: { "anthropic-beta": "max-tokens-3-5-sonnet-2024-07-15" },
+				  }
+				: undefined
+		)
+	}
+
+	createUserReadableRequest(
+		userContent: Array<
+			| Anthropic.TextBlockParam
+			| Anthropic.ImageBlockParam
+			| Anthropic.ToolUseBlockParam
+			| Anthropic.ToolResultBlockParam
+		>
+	): any {
+		return {
+			model: this.getModel().id,
+			max_tokens: this.getModel().info.maxTokens,
+			system: "(see SYSTEM_PROMPT in src/ClaudeDev.ts)",
+			messages: [{ conversation_history: "..." }, { role: "user", content: withoutImageData(userContent) }],
+			tools: "(see tools in src/ClaudeDev.ts)",
+			tool_choice: { type: "auto" },
+		}
+	}
+
+	getModel(): { id: AnthropicModelId; info: ModelInfo } {
+		const modelId = this.options.apiModelId
+		if (modelId && modelId in anthropicModels) {
+			const id = modelId as AnthropicModelId
+			return { id, info: anthropicModels[id] }
+		}
+		return { id: anthropicDefaultModelId, info: anthropicModels[anthropicDefaultModelId] }
+	}
+}

src/api/bedrock.ts

@@ -0,0 +1,66 @@
+import AnthropicBedrock from "@anthropic-ai/bedrock-sdk"
+import { Anthropic } from "@anthropic-ai/sdk"
+import { ApiHandler, withoutImageData } from "."
+import { ApiHandlerOptions, bedrockDefaultModelId, BedrockModelId, bedrockModels, ModelInfo } from "../shared/api"
+
+// https://docs.anthropic.com/en/api/claude-on-amazon-bedrock
+export class AwsBedrockHandler implements ApiHandler {
+	private options: ApiHandlerOptions
+	private client: AnthropicBedrock
+
+	constructor(options: ApiHandlerOptions) {
+		this.options = options
+		this.client = new AnthropicBedrock({
+			// Authenticate by either providing the keys below or use the default AWS credential providers, such as
+			// using ~/.aws/credentials or the "AWS_SECRET_ACCESS_KEY" and "AWS_ACCESS_KEY_ID" environment variables.
+			awsAccessKey: this.options.awsAccessKey,
+			awsSecretKey: this.options.awsSecretKey,
+
+			// awsRegion changes the aws region to which the request is made. By default, we read AWS_REGION,
+			// and if that's not present, we default to us-east-1. Note that we do not read ~/.aws/config for the region.
+			awsRegion: this.options.awsRegion,
+		})
+	}
+
+	async createMessage(
+		systemPrompt: string,
+		messages: Anthropic.Messages.MessageParam[],
+		tools: Anthropic.Messages.Tool[]
+	): Promise<Anthropic.Messages.Message> {
+		return await this.client.messages.create({
+			model: this.getModel().id,
+			max_tokens: this.getModel().info.maxTokens,
+			system: systemPrompt,
+			messages,
+			tools,
+			tool_choice: { type: "auto" },
+		})
+	}
+
+	createUserReadableRequest(
+		userContent: Array<
+			| Anthropic.TextBlockParam
+			| Anthropic.ImageBlockParam
+			| Anthropic.ToolUseBlockParam
+			| Anthropic.ToolResultBlockParam
+		>
+	): any {
+		return {
+			model: this.getModel().id,
+			max_tokens: this.getModel().info.maxTokens,
+			system: "(see SYSTEM_PROMPT in src/ClaudeDev.ts)",
+			messages: [{ conversation_history: "..." }, { role: "user", content: withoutImageData(userContent) }],
+			tools: "(see tools in src/ClaudeDev.ts)",
+			tool_choice: { type: "auto" },
+		}
+	}
+
+	getModel(): { id: BedrockModelId; info: ModelInfo } {
+		const modelId = this.options.apiModelId
+		if (modelId && modelId in bedrockModels) {
+			const id = modelId as BedrockModelId
+			return { id, info: bedrockModels[id] }
+		}
+		return { id: bedrockDefaultModelId, info: bedrockModels[bedrockDefaultModelId] }
+	}
+}

src/api/index.ts

@@ -0,0 +1,66 @@
+import { Anthropic } from "@anthropic-ai/sdk"
+import { ApiConfiguration, ApiModelId, ModelInfo } from "../shared/api"
+import { AnthropicHandler } from "./anthropic"
+import { AwsBedrockHandler } from "./bedrock"
+import { OpenRouterHandler } from "./openrouter"
+
+export interface ApiHandler {
+	createMessage(
+		systemPrompt: string,
+		messages: Anthropic.Messages.MessageParam[],
+		tools: Anthropic.Messages.Tool[]
+	): Promise<Anthropic.Messages.Message>
+
+	createUserReadableRequest(
+		userContent: Array<
+			| Anthropic.TextBlockParam
+			| Anthropic.ImageBlockParam
+			| Anthropic.ToolUseBlockParam
+			| Anthropic.ToolResultBlockParam
+		>
+	): any
+
+	getModel(): { id: ApiModelId; info: ModelInfo }
+}
+
+export function buildApiHandler(configuration: ApiConfiguration): ApiHandler {
+	const { apiProvider, ...options } = configuration
+	switch (apiProvider) {
+		case "anthropic":
+			return new AnthropicHandler(options)
+		case "openrouter":
+			return new OpenRouterHandler(options)
+		case "bedrock":
+			return new AwsBedrockHandler(options)
+		default:
+			return new AnthropicHandler(options)
+	}
+}
+
+export function withoutImageData(
+	userContent: Array<
+		| Anthropic.TextBlockParam
+		| Anthropic.ImageBlockParam
+		| Anthropic.ToolUseBlockParam
+		| Anthropic.ToolResultBlockParam
+	>
+): Array<
+	Anthropic.TextBlockParam | Anthropic.ImageBlockParam | Anthropic.ToolUseBlockParam | Anthropic.ToolResultBlockParam
+> {
+	return userContent.map((part) => {
+		if (part.type === "image") {
+			return { ...part, source: { ...part.source, data: "..." } }
+		} else if (part.type === "tool_result" && typeof part.content !== "string") {
+			return {
+				...part,
+				content: part.content?.map((contentPart) => {
+					if (contentPart.type === "image") {
+						return { ...contentPart, source: { ...contentPart.source, data: "..." } }
+					}
+					return contentPart
+				}),
+			}
+		}
+		return part
+	})
+}

src/api/openrouter.ts

@@ -0,0 +1,396 @@
+import { Anthropic } from "@anthropic-ai/sdk"
+import OpenAI from "openai"
+import { ApiHandler, withoutImageData } from "."
+import {
+	ApiHandlerOptions,
+	ModelInfo,
+	openRouterDefaultModelId,
+	OpenRouterModelId,
+	openRouterModels,
+} from "../shared/api"
+
+export class OpenRouterHandler implements ApiHandler {
+	private options: ApiHandlerOptions
+	private client: OpenAI
+
+	constructor(options: ApiHandlerOptions) {
+		this.options = options
+		this.client = new OpenAI({
+			baseURL: "https://openrouter.ai/api/v1",
+			apiKey: this.options.openRouterApiKey,
+			defaultHeaders: {
+				"HTTP-Referer": "https://github.com/saoudrizwan/claude-dev", // Optional, for including your app on openrouter.ai rankings.
+				"X-Title": "claude-dev", // Optional. Shows in rankings on openrouter.ai.
+			},
+		})
+	}
+
+	async createMessage(
+		systemPrompt: string,
+		messages: Anthropic.Messages.MessageParam[],
+		tools: Anthropic.Messages.Tool[]
+	): Promise<Anthropic.Messages.Message> {
+		// Convert Anthropic messages to OpenAI format
+		const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
+			{ role: "system", content: systemPrompt },
+			...this.convertToOpenAiMessages(messages),
+		]
+
+		// Convert Anthropic tools to OpenAI tools
+		const openAiTools: OpenAI.Chat.ChatCompletionTool[] = tools.map((tool) => ({
+			type: "function",
+			function: {
+				name: tool.name,
+				description: tool.description,
+				parameters: tool.input_schema, // matches anthropic tool input schema (see https://platform.openai.com/docs/guides/function-calling)
+			},
+		}))
+
+		const createParams: OpenAI.Chat.Completions.ChatCompletionCreateParamsNonStreaming = {
+			model: this.getModel().id,
+			max_tokens: this.getModel().info.maxTokens,
+			messages: openAiMessages,
+			tools: openAiTools,
+			tool_choice: "auto",
+		}
+
+		let completion: OpenAI.Chat.Completions.ChatCompletion
+		try {
+			completion = await this.client.chat.completions.create(createParams)
+		} catch (error) {
+			console.error("Error creating message from normal request. Using streaming fallback...", error)
+			completion = await this.streamCompletion(createParams)
+		}
+
+		const errorMessage = (completion as any).error?.message // openrouter returns an error object instead of the openai sdk throwing an error
+		if (errorMessage) {
+			throw new Error(errorMessage)
+		}
+
+		// Convert OpenAI response to Anthropic format
+		const openAiMessage = completion.choices[0].message
+		const anthropicMessage: Anthropic.Messages.Message = {
+			id: completion.id,
+			type: "message",
+			role: openAiMessage.role, // always "assistant"
+			content: [
+				{
+					type: "text",
+					text: openAiMessage.content || "",
+				},
+			],
+			model: completion.model,
+			stop_reason: (() => {
+				switch (completion.choices[0].finish_reason) {
+					case "stop":
+						return "end_turn"
+					case "length":
+						return "max_tokens"
+					case "tool_calls":
+						return "tool_use"
+					case "content_filter": // Anthropic doesn't have an exact equivalent
+					default:
+						return null
+				}
+			})(),
+			stop_sequence: null, // which custom stop_sequence was generated, if any (not applicable if you don't use stop_sequence)
+			usage: {
+				input_tokens: completion.usage?.prompt_tokens || 0,
+				output_tokens: completion.usage?.completion_tokens || 0,
+			},
+		}
+
+		if (openAiMessage.tool_calls && openAiMessage.tool_calls.length > 0) {
+			anthropicMessage.content.push(
+				...openAiMessage.tool_calls.map((toolCall): Anthropic.ToolUseBlock => {
+					let parsedInput = {}
+					try {
+						parsedInput = JSON.parse(toolCall.function.arguments || "{}")
+					} catch (error) {
+						console.error("Failed to parse tool arguments:", error)
+					}
+					return {
+						type: "tool_use",
+						id: toolCall.id,
+						name: toolCall.function.name,
+						input: parsedInput,
+					}
+				})
+			)
+		}
+
+		return anthropicMessage
+	}
+
+	convertToOpenAiMessages(
+		anthropicMessages: Anthropic.Messages.MessageParam[]
+	): OpenAI.Chat.ChatCompletionMessageParam[] {
+		const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = []
+
+		for (const anthropicMessage of anthropicMessages) {
+			if (typeof anthropicMessage.content === "string") {
+				openAiMessages.push({ role: anthropicMessage.role, content: anthropicMessage.content })
+			} else {
+				// image_url.url is base64 encoded image data
+				// ensure it contains the content-type of the image: data:image/png;base64,
+				/*
+			{ role: "user", content: "" | { type: "text", text: string } | { type: "image_url", image_url: { url: string } } },
+			 // content required unless tool_calls is present
+			{ role: "assistant", content?: "" | null, tool_calls?: [{ id: "", function: { name: "", arguments: "" }, type: "function" }] },
+			{ role: "tool", tool_call_id: "", content: ""}
+			 */
+				if (anthropicMessage.role === "user") {
+					const { nonToolMessages, toolMessages } = anthropicMessage.content.reduce<{
+						nonToolMessages: (Anthropic.TextBlockParam | Anthropic.ImageBlockParam)[]
+						toolMessages: Anthropic.ToolResultBlockParam[]
+					}>(
+						(acc, part) => {
+							if (part.type === "tool_result") {
+								acc.toolMessages.push(part)
+							} else if (part.type === "text" || part.type === "image") {
+								acc.nonToolMessages.push(part)
+							} // user cannot send tool_use messages
+							return acc
+						},
+						{ nonToolMessages: [], toolMessages: [] }
+					)
+
+					// Process non-tool messages
+					if (nonToolMessages.length > 0) {
+						openAiMessages.push({
+							role: "user",
+							content: nonToolMessages.map((part) => {
+								if (part.type === "image") {
+									return {
+										type: "image_url",
+										image_url: { url: `data:${part.source.media_type};base64,${part.source.data}` },
+									}
+								}
+								return { type: "text", text: part.text }
+							}),
+						})
+					}
+
+					// Process tool result messages
+					let toolResultImages: Anthropic.Messages.ImageBlockParam[] = []
+					toolMessages.forEach((toolMessage) => {
+						// The Anthropic SDK allows tool results to be a string or an array of text and image blocks, enabling rich and structured content. In contrast, the OpenAI SDK only supports tool results as a single string, so we map the Anthropic tool result parts into one concatenated string to maintain compatibility.
+						let content: string
+
+						if (typeof toolMessage.content === "string") {
+							content = toolMessage.content
+						} else {
+							content =
+								toolMessage.content
+									?.map((part) => {
+										if (part.type === "image") {
+											toolResultImages.push(part)
+											return "(see following user message for image)"
+										}
+										return part.text
+									})
+									.join("\n") ?? ""
+						}
+						openAiMessages.push({
+							role: "tool",
+							tool_call_id: toolMessage.tool_use_id,
+							content: content,
+						})
+					})
+
+					// If tool results contain images, send as a separate user message
+					// I ran into an issue where if I gave feedback for one of many tool uses, the request would fail.
+					// "Messages following `tool_use` blocks must begin with a matching number of `tool_result` blocks."
+					// Therefore we need to send these images after the tool result messages
+					if (toolResultImages.length > 0) {
+						openAiMessages.push({
+							role: "user",
+							content: toolResultImages.map((part) => ({
+								type: "image_url",
+								image_url: { url: `data:${part.source.media_type};base64,${part.source.data}` },
+							})),
+						})
+					}
+				} else if (anthropicMessage.role === "assistant") {
+					const { nonToolMessages, toolMessages } = anthropicMessage.content.reduce<{
+						nonToolMessages: (Anthropic.TextBlockParam | Anthropic.ImageBlockParam)[]
+						toolMessages: Anthropic.ToolUseBlockParam[]
+					}>(
+						(acc, part) => {
+							if (part.type === "tool_use") {
+								acc.toolMessages.push(part)
+							} else if (part.type === "text" || part.type === "image") {
+								acc.nonToolMessages.push(part)
+							} // assistant cannot send tool_result messages
+							return acc
+						},
+						{ nonToolMessages: [], toolMessages: [] }
+					)
+
+					// Process non-tool messages
+					let content: string | undefined
+					if (nonToolMessages.length > 0) {
+						content = nonToolMessages
+							.map((part) => {
+								if (part.type === "image") {
+									return "" // impossible as the assistant cannot send images
+								}
+								return part.text
+							})
+							.join("\n")
+					}
+
+					// Process tool use messages
+					let tool_calls: OpenAI.Chat.ChatCompletionMessageToolCall[] = toolMessages.map((toolMessage) => ({
+						id: toolMessage.id,
+						type: "function",
+						function: {
+							name: toolMessage.name,
+							// json string
+							arguments: JSON.stringify(toolMessage.input),
+						},
+					}))
+
+					openAiMessages.push({
+						role: "assistant",
+						content,
+						// Cannot be an empty array. API expects an array with minimum length 1, and will respond with an error if it's empty
+						tool_calls: tool_calls.length > 0 ? tool_calls : undefined,
+					})
+				}
+			}
+		}
+
+		return openAiMessages
+	}
+
+	/*
+	Streaming the completion is a fallback behavior for when a normal request responds with an invalid JSON object ("Unexpected end of JSON input"). This would usually happen in cases where the model makes tool calls with large arguments. After talking with OpenRouter folks, streaming mitigates this issue for now until they fix the underlying problem ("some weird data from anthropic got decoded wrongly and crashed the buffer")
+	*/
+	async streamCompletion(
+		createParams: OpenAI.Chat.Completions.ChatCompletionCreateParamsNonStreaming
+	): Promise<OpenAI.Chat.Completions.ChatCompletion> {
+		const stream = await this.client.chat.completions.create({
+			...createParams,
+			stream: true,
+		})
+
+		let textContent: string = ""
+		let toolCalls: OpenAI.Chat.ChatCompletionMessageToolCall[] = []
+
+		try {
+			let currentToolCall: (OpenAI.Chat.ChatCompletionMessageToolCall & { index?: number }) | null = null
+			for await (const chunk of stream) {
+				const delta = chunk.choices[0]?.delta
+				if (delta?.content) {
+					textContent += delta.content
+				}
+				if (delta?.tool_calls) {
+					for (const toolCallDelta of delta.tool_calls) {
+						if (toolCallDelta.index === undefined) {
+							continue
+						}
+						if (!currentToolCall || currentToolCall.index !== toolCallDelta.index) {
+							// new index means new tool call, so add the previous one to the list
+							if (currentToolCall) {
+								toolCalls.push(currentToolCall)
+							}
+							currentToolCall = {
+								index: toolCallDelta.index,
+								id: toolCallDelta.id || "",
+								type: "function",
+								function: { name: "", arguments: "" },
+							}
+						}
+						if (toolCallDelta.id) {
+							currentToolCall.id = toolCallDelta.id
+						}
+						if (toolCallDelta.type) {
+							currentToolCall.type = toolCallDelta.type
+						}
+						if (toolCallDelta.function) {
+							if (toolCallDelta.function.name) {
+								currentToolCall.function.name = toolCallDelta.function.name
+							}
+							if (toolCallDelta.function.arguments) {
+								currentToolCall.function.arguments =
+									(currentToolCall.function.arguments || "") + toolCallDelta.function.arguments
+							}
+						}
+					}
+				}
+			}
+			if (currentToolCall) {
+				toolCalls.push(currentToolCall)
+			}
+		} catch (error) {
+			console.error("Error streaming completion:", error)
+			throw error
+		}
+
+		// Usage information is not available in streaming responses, so we need to estimate token counts
+		function approximateTokenCount(text: string): number {
+			return Math.ceil(new TextEncoder().encode(text).length / 4)
+		}
+		const promptTokens = approximateTokenCount(
+			createParams.messages
+				.map((m) => (typeof m.content === "string" ? m.content : JSON.stringify(m.content)))
+				.join(" ")
+		)
+		const completionTokens = approximateTokenCount(
+			textContent + toolCalls.map((toolCall) => toolCall.function.arguments || "").join(" ")
+		)
+
+		const completion: OpenAI.Chat.Completions.ChatCompletion = {
+			created: Date.now(),
+			object: "chat.completion",
+			id: `openrouter-${Date.now()}-${Math.random().toString(36).slice(2, 11)}`, // this ID won't be traceable back to OpenRouter's systems if you need to debug issues
+			choices: [
+				{
+					message: {
+						role: "assistant",
+						content: textContent,
+						tool_calls: toolCalls.length > 0 ? toolCalls : undefined,
+					},
+					finish_reason: toolCalls.length > 0 ? "tool_calls" : "stop",
+					index: 0,
+					logprobs: null,
+				},
+			],
+			model: this.getModel().id,
+			usage: {
+				prompt_tokens: promptTokens,
+				completion_tokens: completionTokens,
+				total_tokens: promptTokens + completionTokens,
+			},
+		}
+
+		return completion
+	}
+
+	createUserReadableRequest(
+		userContent: Array<
+			| Anthropic.TextBlockParam
+			| Anthropic.ImageBlockParam
+			| Anthropic.ToolUseBlockParam
+			| Anthropic.ToolResultBlockParam
+		>
+	): any {
+		return {
+			model: this.getModel().id,
+			max_tokens: this.getModel().info.maxTokens,
+			messages: [{ conversation_history: "..." }, { role: "user", content: withoutImageData(userContent) }],
+			tools: "(see tools in src/ClaudeDev.ts)",
+			tool_choice: "auto",
+		}
+	}
+
+	getModel(): { id: OpenRouterModelId; info: ModelInfo } {
+		const modelId = this.options.apiModelId
+		if (modelId && modelId in openRouterModels) {
+			const id = modelId as OpenRouterModelId
+			return { id, info: openRouterModels[id] }
+		}
+		return { id: openRouterDefaultModelId, info: openRouterModels[openRouterDefaultModelId] }
+	}
+}

src/parse-source-code/index.ts

@@ -1,10 +1,20 @@
 import * as fs from "fs/promises"
 import { globby } from "globby"
+import os from "os"
 import * as path from "path"
 import { LanguageParser, loadRequiredLanguageParsers } from "./languageParser"
 
 // TODO: implement caching behavior to avoid having to keep analyzing project for new tasks.
 export async function parseSourceCodeForDefinitionsTopLevel(dirPath: string): Promise<string> {
+	// check if the path exists
+	const dirExists = await fs
+		.access(path.resolve(dirPath))
+		.then(() => true)
+		.catch(() => false)
+	if (!dirExists) {
+		return "This directory does not exist or you do not have permission to access it."
+	}
+
 	// Get all files at top level (not gitignored)
 	const allFiles = await listFiles(dirPath, false)
 
@@ -45,11 +55,17 @@ export async function parseSourceCodeForDefinitionsTopLevel(dirPath: string): Pr
 
 export async function listFiles(dirPath: string, recursive: boolean): Promise<string[]> {
 	const absolutePath = path.resolve(dirPath)
+	// Do not allow listing files in root or home directory, which Claude tends to want to do when the user's prompt is vague.
 	const root = process.platform === "win32" ? path.parse(absolutePath).root : "/"
 	const isRoot = absolutePath === root
 	if (isRoot) {
 		return [root]
 	}
+	const homeDir = os.homedir()
+	const isHomeDir = absolutePath === homeDir
+	if (isHomeDir) {
+		return [homeDir]
+	}
 
 	const dirsToIgnore = [
 		"node_modules",
@@ -77,7 +93,7 @@ export async function listFiles(dirPath: string, recursive: boolean): Promise<st
 		markDirectories: true, // Append a / on any directories matched
 		gitignore: recursive, // globby ignores any files that are gitignored
 		ignore: recursive ? dirsToIgnore : undefined, // just in case there is no gitignore, we ignore sensible defaults
-		onlyFiles: recursive, // true by default, false means it will list directories on their own too
+		onlyFiles: false, // true by default, false means it will list directories on their own too
 	}
 	// * globs all files in one dir, ** globs files in nested directories
 	const files = await globby(recursive ? "**" : "*", options)

src/providers/ClaudeDevProvider.ts

@@ -1,27 +1,31 @@
-import { Uri, Webview } from "vscode"
-//import * as weather from "weather-js"
 import * as vscode from "vscode"
 import { ClaudeDev } from "../ClaudeDev"
-import { ClaudeMessage, ExtensionMessage } from "../shared/ExtensionMessage"
+import { ApiModelId, ApiProvider } from "../shared/api"
+import { ExtensionMessage } from "../shared/ExtensionMessage"
 import { WebviewMessage } from "../shared/WebviewMessage"
-import { Anthropic } from "@anthropic-ai/sdk"
-import * as path from "path"
-import os from "os"
-
+import { downloadTask, getNonce, getUri, selectImages } from "../utils"
 /*
 https://github.com/microsoft/vscode-webview-ui-toolkit-samples/blob/main/default/weather-webview/src/providers/WeatherViewProvider.ts
 
 https://github.com/KumarVariable/vscode-extension-sidebar-html/blob/master/src/customSidebarViewProvider.ts
 */
 
+type SecretKey = "apiKey" | "openRouterApiKey" | "awsAccessKey" | "awsSecretKey"
+type GlobalStateKey =
+	| "apiProvider"
+	| "apiModelId"
+	| "awsRegion"
+	| "maxRequestsPerTask"
+	| "lastShownAnnouncementId"
+	| "customInstructions"
+
 export class ClaudeDevProvider implements vscode.WebviewViewProvider {
 	public static readonly sideBarId = "claude-dev.SidebarProvider" // used in package.json as the view's id. This value cannot be changed due to how vscode caches views based on their id, and updating the id would break existing instances of the extension.
 	public static readonly tabPanelId = "claude-dev.TabPanelProvider"
 	private disposables: vscode.Disposable[] = []
 	private view?: vscode.WebviewView | vscode.WebviewPanel
-	private providerInstanceIdentifier = Date.now()
 	private claudeDev?: ClaudeDev
-	private latestAnnouncementId = "jul-29-2024" // update to some unique identifier when we add a new announcement
+	private latestAnnouncementId = "aug-11-2024" // update to some unique identifier when we add a new announcement
 
 	constructor(
 		private readonly context: vscode.ExtensionContext,
@@ -132,15 +136,10 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
 		this.outputChannel.appendLine("Webview view resolved")
 	}
 
-	async tryToInitClaudeDevWithTask(task: string) {
+	async initClaudeDevWithTask(task?: string, images?: string[]) {
 		await this.clearTask() // ensures that an exising task doesn't exist before starting a new one, although this shouldn't be possible since user must clear task before starting a new one
-		const [apiKey, maxRequestsPerTask] = await Promise.all([
-			this.getSecret("apiKey") as Promise<string | undefined>,
-			this.getGlobalState("maxRequestsPerTask") as Promise<number | undefined>,
-		])
-		if (this.view && apiKey) {
-			this.claudeDev = new ClaudeDev(this, task, apiKey, maxRequestsPerTask)
-		}
+		const { maxRequestsPerTask, apiConfiguration, customInstructions } = await this.getState()
+		this.claudeDev = new ClaudeDev(this, apiConfiguration, maxRequestsPerTask, customInstructions, task, images)
 	}
 
 	// Send any JSON serializable data to the react app
@@ -200,7 +199,8 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
         create a content security policy meta tag so that only loading scripts with a nonce is allowed
         As your extension grows you will likely want to add custom styles, fonts, and/or images to your webview. If you do, you will need to update the content security policy meta tag to explicity allow for these resources. E.g.
                 <meta http-equiv="Content-Security-Policy" content="default-src 'none'; style-src ${webview.cspSource}; font-src ${webview.cspSource}; img-src ${webview.cspSource} https:; script-src 'nonce-${nonce}';">
-
+		- 'unsafe-inline' is required for styles due to vscode-webview-toolkit's dynamic style injection
+		- since we pass base64 images to the webview, we need to specify img-src ${webview.cspSource} data:;
 
         in meta tag we add nonce attribute: A cryptographic nonce (only used once) to allow scripts. The server must generate a unique nonce value each time it transmits a policy. It is critical to provide a nonce that cannot be guessed as bypassing a resource's policy is otherwise trivial.
         */
@@ -214,7 +214,7 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
             <meta charset="utf-8">
             <meta name="viewport" content="width=device-width,initial-scale=1,shrink-to-fit=no">
             <meta name="theme-color" content="#000000">
-            <meta http-equiv="Content-Security-Policy" content="default-src 'none'; font-src ${webview.cspSource}; style-src ${webview.cspSource}; script-src 'nonce-${nonce}';">
+            <meta http-equiv="Content-Security-Policy" content="default-src 'none'; font-src ${webview.cspSource}; style-src ${webview.cspSource} 'unsafe-inline'; img-src ${webview.cspSource} data:; script-src 'nonce-${nonce}';">
             <link rel="stylesheet" type="text/css" href="${stylesUri}">
 			<link href="${codiconsUri}" rel="stylesheet" />
             <title>Claude Dev</title>
@@ -250,11 +250,28 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
 						// Could also do this in extension .ts
 						//this.postMessageToWebview({ type: "text", text: `Extension: ${Date.now()}` })
 						// initializing new instance of ClaudeDev will make sure that any agentically running promises in old instance don't affect our new task. this essentially creates a fresh slate for the new task
-						await this.tryToInitClaudeDevWithTask(message.text!)
+						await this.initClaudeDevWithTask(message.text, message.images)
 						break
-					case "apiKey":
-						await this.storeSecret("apiKey", message.text!)
-						this.claudeDev?.updateApiKey(message.text!)
+					case "apiConfiguration":
+						if (message.apiConfiguration) {
+							const {
+								apiProvider,
+								apiModelId,
+								apiKey,
+								openRouterApiKey,
+								awsAccessKey,
+								awsSecretKey,
+								awsRegion,
+							} = message.apiConfiguration
+							await this.updateGlobalState("apiProvider", apiProvider)
+							await this.updateGlobalState("apiModelId", apiModelId)
+							await this.storeSecret("apiKey", apiKey)
+							await this.storeSecret("openRouterApiKey", openRouterApiKey)
+							await this.storeSecret("awsAccessKey", awsAccessKey)
+							await this.storeSecret("awsSecretKey", awsSecretKey)
+							await this.updateGlobalState("awsRegion", awsRegion)
+							this.claudeDev?.updateApi(message.apiConfiguration)
+						}
 						await this.postStateToWebview()
 						break
 					case "maxRequestsPerTask":
@@ -269,8 +286,14 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
 						this.claudeDev?.updateMaxRequestsPerTask(result)
 						await this.postStateToWebview()
 						break
+					case "customInstructions":
+						// User may be clearing the field
+						await this.updateGlobalState("customInstructions", message.text || undefined)
+						this.claudeDev?.updateCustomInstructions(message.text || undefined)
+						await this.postStateToWebview()
+						break
 					case "askResponse":
-						this.claudeDev?.handleWebviewAskResponse(message.askResponse!, message.text)
+						this.claudeDev?.handleWebviewAskResponse(message.askResponse!, message.text, message.images)
 						break
 					case "clearTask":
 						// newTask will start a new task with a given task text, while clear task resets the current session and allows for a new task to be started
@@ -282,7 +305,11 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
 						await this.postStateToWebview()
 						break
 					case "downloadTask":
-						this.downloadTask()
+						downloadTask(this.claudeDev?.apiConversationHistory ?? [])
+						break
+					case "selectImages":
+						const images = await selectImages()
+						await this.postMessageToWebview({ type: "selectedImages", images })
 						break
 					// Add more switch case statements here as more webview message commands
 					// are created within the webview context (i.e. inside media/main.js)
@@ -293,114 +320,32 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
 		)
 	}
 
-	async downloadTask() {
-		// File name
-		const date = new Date()
-		const month = date.toLocaleString("en-US", { month: "short" }).toLowerCase()
-		const day = date.getDate()
-		const year = date.getFullYear()
-		let hours = date.getHours()
-		const minutes = date.getMinutes().toString().padStart(2, "0")
-		const ampm = hours >= 12 ? "pm" : "am"
-		hours = hours % 12
-		hours = hours ? hours : 12 // the hour '0' should be '12'
-		const fileName = `claude_dev_task_${month}-${day}-${year}_${hours}-${minutes}-${ampm}.md`
-
-		// Generate markdown
-		const conversationHistory = await this.getApiConversationHistory()
-		const markdownContent = conversationHistory
-			.map((message) => {
-				const role = message.role === "user" ? "**User:**" : "**Assistant:**"
-				const content = Array.isArray(message.content)
-					? message.content.map(this.formatContentBlockToMarkdown).join("\n")
-					: message.content
-
-				return `${role}\n\n${content}\n\n`
-			})
-			.join("---\n\n")
-
-		// Prompt user for save location
-		const saveUri = await vscode.window.showSaveDialog({
-			filters: { Markdown: ["md"] },
-			defaultUri: vscode.Uri.file(path.join(os.homedir(), "Downloads", fileName)),
-		})
-
-		if (saveUri) {
-			// Write content to the selected location
-			await vscode.workspace.fs.writeFile(saveUri, Buffer.from(markdownContent))
-			vscode.window.showTextDocument(saveUri, { preview: true })
-		}
-	}
-
-	private formatContentBlockToMarkdown(
-		block:
-			| Anthropic.TextBlockParam
-			| Anthropic.ImageBlockParam
-			| Anthropic.ToolUseBlockParam
-			| Anthropic.ToolResultBlockParam
-	): string {
-		switch (block.type) {
-			case "text":
-				return block.text
-			case "image":
-				return `[Image: ${block.source.media_type}]`
-			case "tool_use":
-				let input: string
-				if (typeof block.input === "object" && block.input !== null) {
-					input = Object.entries(block.input)
-						.map(([key, value]) => `${key.charAt(0).toUpperCase() + key.slice(1)}: ${value}`)
-						.join("\n")
-				} else {
-					input = String(block.input)
-				}
-				return `[Tool Use: ${block.name}]\n${input}`
-			case "tool_result":
-				if (typeof block.content === "string") {
-					return `[Tool Result${block.is_error ? " (Error)" : ""}]\n${block.content}`
-				} else if (Array.isArray(block.content)) {
-					return `[Tool Result${block.is_error ? " (Error)" : ""}]\n${block.content
-						.map(this.formatContentBlockToMarkdown)
-						.join("\n")}`
-				} else {
-					return `[Tool Result${block.is_error ? " (Error)" : ""}]`
-				}
-			default:
-				return "[Unexpected content type]"
-		}
-	}
-
 	async postStateToWebview() {
-		const [apiKey, maxRequestsPerTask, claudeMessages, lastShownAnnouncementId] = await Promise.all([
-			this.getSecret("apiKey") as Promise<string | undefined>,
-			this.getGlobalState("maxRequestsPerTask") as Promise<number | undefined>,
-			this.getClaudeMessages(),
-			this.getGlobalState("lastShownAnnouncementId") as Promise<string | undefined>,
-		])
+		const { apiConfiguration, maxRequestsPerTask, lastShownAnnouncementId, customInstructions } =
+			await this.getState()
 		this.postMessageToWebview({
 			type: "state",
 			state: {
-				apiKey,
+				version: this.context.extension?.packageJSON?.version ?? "",
+				apiConfiguration,
 				maxRequestsPerTask,
+				customInstructions,
 				themeName: vscode.workspace.getConfiguration("workbench").get<string>("colorTheme"),
-				claudeMessages,
+				claudeMessages: this.claudeDev?.claudeMessages || [],
 				shouldShowAnnouncement: lastShownAnnouncementId !== this.latestAnnouncementId,
 			},
 		})
 	}
 
 	async clearTask() {
-		if (this.claudeDev) {
-			this.claudeDev.abort = true // will stop any agentically running promises
-			this.claudeDev = undefined // removes reference to it, so once promises end it will be garbage collected
-		}
-		await this.setApiConversationHistory(undefined)
-		await this.setClaudeMessages(undefined)
+		this.claudeDev?.abortTask()
+		this.claudeDev = undefined // removes reference to it, so once promises end it will be garbage collected
 	}
 
 	// Caching mechanism to keep track of webview messages + API conversation history per provider instance
 
 	/*
-	Now that we use retainContextWhenHidden, we don't have to store a cache of claude messages in the user's state, but we do to reduce memory footprint in long conversations.
+	Now that we use retainContextWhenHidden, we don't have to store a cache of claude messages in the user's state, but we could to reduce memory footprint in long conversations.
 
 	- We have to be careful of what state is shared between ClaudeDevProvider instances since there could be multiple instances of the extension running at once. For example when we cached claude messages using the same key, two instances of the extension could end up using the same key and overwriting each other's messages.
 	- Some state does need to be shared between the instances, i.e. the API key--however there doesn't seem to be a good way to notfy the other instances that the API key has changed.
@@ -411,52 +356,66 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
 	However in the future when we implement task history, we'll need to use a unique identifier for each task. As well as manage a data structure that keeps track of task history with their associated identifiers and the task message itself, to present in a 'Task History' view.
 	Task history is a significant undertaking as it would require refactoring how we wait for ask responses--it would need to be a hidden claudeMessage, so that user's can resume tasks that ended with an ask.
 	*/
+	// private providerInstanceIdentifier = Date.now()
+	// getClaudeMessagesStateKey() {
+	// 	return `claudeMessages-${this.providerInstanceIdentifier}`
+	// }
 
-	getClaudeMessagesStateKey() {
-		return `claudeMessages-${this.providerInstanceIdentifier}`
-	}
-
-	getApiConversationHistoryStateKey() {
-		return `apiConversationHistory-${this.providerInstanceIdentifier}`
-	}
+	// getApiConversationHistoryStateKey() {
+	// 	return `apiConversationHistory-${this.providerInstanceIdentifier}`
+	// }
 
 	// claude messages to present in the webview
 
-	async getClaudeMessages(): Promise<ClaudeMessage[]> {
-		const messages = (await this.getGlobalState(this.getClaudeMessagesStateKey())) as ClaudeMessage[]
-		return messages || []
-	}
+	// getClaudeMessages(): ClaudeMessage[] {
+	// 	// const messages = (await this.getGlobalState(this.getClaudeMessagesStateKey())) as ClaudeMessage[]
+	// 	// return messages || []
+	// 	return this.claudeMessages
+	// }
 
-	async setClaudeMessages(messages: ClaudeMessage[] | undefined) {
-		await this.updateGlobalState(this.getClaudeMessagesStateKey(), messages)
-	}
+	// setClaudeMessages(messages: ClaudeMessage[] | undefined) {
+	// 	// await this.updateGlobalState(this.getClaudeMessagesStateKey(), messages)
+	// 	this.claudeMessages = messages || []
+	// }
 
-	async addClaudeMessage(message: ClaudeMessage): Promise<ClaudeMessage[]> {
-		const messages = await this.getClaudeMessages()
-		messages.push(message)
-		await this.setClaudeMessages(messages)
-		return messages
-	}
+	// addClaudeMessage(message: ClaudeMessage): ClaudeMessage[] {
+	// 	// const messages = await this.getClaudeMessages()
+	// 	// messages.push(message)
+	// 	// await this.setClaudeMessages(messages)
+	// 	// return messages
+	// 	this.claudeMessages.push(message)
+	// 	return this.claudeMessages
+	// }
 
 	// conversation history to send in API requests
 
-	async getApiConversationHistory(): Promise<Anthropic.MessageParam[]> {
-		const history = (await this.getGlobalState(
-			this.getApiConversationHistoryStateKey()
-		)) as Anthropic.MessageParam[]
-		return history || []
-	}
+	/*
+	It seems that some API messages do not comply with vscode state requirements. Either the Anthropic library is manipulating these values somehow in the backend in a way thats creating cyclic references, or the API returns a function or a Symbol as part of the message content.
+	VSCode docs about state: "The value must be JSON-stringifyable ... value — A value. MUST not contain cyclic references."
+	For now we'll store the conversation history in memory, and if we need to store in state directly we'd need to do a manual conversion to ensure proper json stringification.
+	*/
 
-	async setApiConversationHistory(history: Anthropic.MessageParam[] | undefined) {
-		await this.updateGlobalState(this.getApiConversationHistoryStateKey(), history)
-	}
+	// getApiConversationHistory(): Anthropic.MessageParam[] {
+	// 	// const history = (await this.getGlobalState(
+	// 	// 	this.getApiConversationHistoryStateKey()
+	// 	// )) as Anthropic.MessageParam[]
+	// 	// return history || []
+	// 	return this.apiConversationHistory
+	// }
 
-	async addMessageToApiConversationHistory(message: Anthropic.MessageParam): Promise<Anthropic.MessageParam[]> {
-		const history = await this.getApiConversationHistory()
-		history.push(message)
-		await this.setApiConversationHistory(history)
-		return history
-	}
+	// setApiConversationHistory(history: Anthropic.MessageParam[] | undefined) {
+	// 	// await this.updateGlobalState(this.getApiConversationHistoryStateKey(), history)
+	// 	this.apiConversationHistory = history || []
+	// }
+
+	// addMessageToApiConversationHistory(message: Anthropic.MessageParam): Anthropic.MessageParam[] {
+	// 	// const history = await this.getApiConversationHistory()
+	// 	// history.push(message)
+	// 	// await this.setApiConversationHistory(history)
+	// 	// return history
+	// 	this.apiConversationHistory.push(message)
+	// 	return this.apiConversationHistory
+	// }
 
 	/*
 	Storage
@@ -464,13 +423,68 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
 	https://www.eliostruyf.com/devhack-code-extension-storage-options/
 	*/
 
+	async getState() {
+		const [
+			storedApiProvider,
+			apiModelId,
+			apiKey,
+			openRouterApiKey,
+			awsAccessKey,
+			awsSecretKey,
+			awsRegion,
+			maxRequestsPerTask,
+			lastShownAnnouncementId,
+			customInstructions,
+		] = await Promise.all([
+			this.getGlobalState("apiProvider") as Promise<ApiProvider | undefined>,
+			this.getGlobalState("apiModelId") as Promise<ApiModelId | undefined>,
+			this.getSecret("apiKey") as Promise<string | undefined>,
+			this.getSecret("openRouterApiKey") as Promise<string | undefined>,
+			this.getSecret("awsAccessKey") as Promise<string | undefined>,
+			this.getSecret("awsSecretKey") as Promise<string | undefined>,
+			this.getGlobalState("awsRegion") as Promise<string | undefined>,
+			this.getGlobalState("maxRequestsPerTask") as Promise<number | undefined>,
+			this.getGlobalState("lastShownAnnouncementId") as Promise<string | undefined>,
+			this.getGlobalState("customInstructions") as Promise<string | undefined>,
+		])
+
+		let apiProvider: ApiProvider
+		if (storedApiProvider) {
+			apiProvider = storedApiProvider
+		} else {
+			// Either new user or legacy user that doesn't have the apiProvider stored in state
+			// (If they're using OpenRouter or Bedrock, then apiProvider state will exist)
+			if (apiKey) {
+				apiProvider = "anthropic"
+			} else {
+				// New users should default to openrouter (better rate limits and wider model selection)
+				apiProvider = "openrouter"
+			}
+		}
+
+		return {
+			apiConfiguration: {
+				apiProvider,
+				apiModelId,
+				apiKey,
+				openRouterApiKey,
+				awsAccessKey,
+				awsSecretKey,
+				awsRegion,
+			},
+			maxRequestsPerTask,
+			lastShownAnnouncementId,
+			customInstructions,
+		}
+	}
+
 	// global
 
-	private async updateGlobalState(key: string, value: any) {
+	private async updateGlobalState(key: GlobalStateKey, value: any) {
 		await this.context.globalState.update(key, value)
 	}
 
-	private async getGlobalState(key: string) {
+	private async getGlobalState(key: GlobalStateKey) {
 		return await this.context.globalState.get(key)
 	}
 
@@ -496,43 +510,15 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
 
 	// secrets
 
-	private async storeSecret(key: string, value: any) {
-		await this.context.secrets.store(key, value)
+	private async storeSecret(key: SecretKey, value?: string) {
+		if (value) {
+			await this.context.secrets.store(key, value)
+		} else {
+			await this.context.secrets.delete(key)
+		}
 	}
 
-	private async getSecret(key: string) {
+	private async getSecret(key: SecretKey) {
 		return await this.context.secrets.get(key)
 	}
 }
-
-/**
- * A helper function that returns a unique alphanumeric identifier called a nonce.
- *
- * @remarks This function is primarily used to help enforce content security
- * policies for resources/scripts being executed in a webview context.
- *
- * @returns A nonce
- */
-export function getNonce() {
-	let text = ""
-	const possible = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
-	for (let i = 0; i < 32; i++) {
-		text += possible.charAt(Math.floor(Math.random() * possible.length))
-	}
-	return text
-}
-
-/**
- * A helper function which will get the webview URI of a given file or resource.
- *
- * @remarks This URI can be used within a webview's HTML as a link to the
- * given file/resource.
- *
- * @param webview A reference to the extension webview
- * @param extensionUri The URI of the directory containing the extension
- * @param pathList An array of strings representing the path to a file/resource
- * @returns A URI pointing to the file/resource
- */
-export function getUri(webview: Webview, extensionUri: Uri, pathList: string[]) {
-	return webview.asWebviewUri(Uri.joinPath(extensionUri, ...pathList))
-}

src/shared/ExtensionMessage.ts

@@ -1,16 +1,21 @@
 // type that represents json data that is sent from extension to webview, called ExtensionMessage and has 'type' enum which can be 'plusButtonTapped' or 'settingsButtonTapped' or 'hello'
 
+import { ApiConfiguration } from "./api"
+
 // webview will hold state
 export interface ExtensionMessage {
-	type: "action" | "state"
+	type: "action" | "state" | "selectedImages"
 	text?: string
 	action?: "plusButtonTapped" | "settingsButtonTapped" | "didBecomeVisible"
 	state?: ExtensionState
+	images?: string[]
 }
 
 export interface ExtensionState {
-	apiKey?: string
+	version: string
+	apiConfiguration?: ApiConfiguration
 	maxRequestsPerTask?: number
+	customInstructions?: string
 	themeName?: string
 	claudeMessages: ClaudeMessage[]
 	shouldShowAnnouncement: boolean
@@ -22,6 +27,7 @@ export interface ClaudeMessage {
 	ask?: ClaudeAsk
 	say?: ClaudeSay
 	text?: string
+	images?: string[]
 }
 
 export type ClaudeAsk =

src/shared/WebviewMessage.ts

@@ -1,15 +1,21 @@
+import { ApiConfiguration, ApiProvider } from "./api"
+
 export interface WebviewMessage {
 	type:
-		| "apiKey"
+		| "apiConfiguration"
 		| "maxRequestsPerTask"
+		| "customInstructions"
 		| "webviewDidLaunch"
 		| "newTask"
 		| "askResponse"
 		| "clearTask"
 		| "didShowAnnouncement"
 		| "downloadTask"
+		| "selectImages"
 	text?: string
 	askResponse?: ClaudeAskResponse
+	apiConfiguration?: ApiConfiguration
+	images?: string[]
 }
 
-export type ClaudeAskResponse = "yesButtonTapped" | "noButtonTapped" | "textResponse"
+export type ClaudeAskResponse = "yesButtonTapped" | "noButtonTapped" | "messageResponse"

src/shared/api.ts

@@ -0,0 +1,208 @@
+export type ApiProvider = "anthropic" | "openrouter" | "bedrock"
+
+export interface ApiHandlerOptions {
+	apiModelId?: ApiModelId
+	apiKey?: string // anthropic
+	openRouterApiKey?: string
+	awsAccessKey?: string
+	awsSecretKey?: string
+	awsRegion?: string
+}
+
+export type ApiConfiguration = ApiHandlerOptions & {
+	apiProvider?: ApiProvider
+}
+
+// Models
+
+export interface ModelInfo {
+	maxTokens: number
+	supportsImages: boolean
+	inputPrice: number
+	outputPrice: number
+}
+
+export type ApiModelId = AnthropicModelId | OpenRouterModelId | BedrockModelId
+
+// Anthropic
+// https://docs.anthropic.com/en/docs/about-claude/models
+export type AnthropicModelId = keyof typeof anthropicModels
+export const anthropicDefaultModelId: AnthropicModelId = "claude-3-5-sonnet-20240620"
+export const anthropicModels = {
+	"claude-3-5-sonnet-20240620": {
+		maxTokens: 8192,
+		supportsImages: true,
+		inputPrice: 3.0, // $3 per million input tokens
+		outputPrice: 15.0, // $15 per million output tokens
+	},
+	"claude-3-opus-20240229": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 15.0,
+		outputPrice: 75.0,
+	},
+	"claude-3-sonnet-20240229": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 3.0,
+		outputPrice: 15.0,
+	},
+	"claude-3-haiku-20240307": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 0.25,
+		outputPrice: 1.25,
+	},
+} as const satisfies Record<string, ModelInfo> // as const assertion makes the object deeply readonly
+
+// AWS Bedrock
+// https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html
+export type BedrockModelId = keyof typeof bedrockModels
+export const bedrockDefaultModelId: BedrockModelId = "anthropic.claude-3-5-sonnet-20240620-v1:0"
+export const bedrockModels = {
+	"anthropic.claude-3-5-sonnet-20240620-v1:0": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 3.0,
+		outputPrice: 15.0,
+	},
+	"anthropic.claude-3-opus-20240229-v1:0": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 15.0,
+		outputPrice: 75.0,
+	},
+	"anthropic.claude-3-sonnet-20240229-v1:0": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 3.0,
+		outputPrice: 15.0,
+	},
+	"anthropic.claude-3-haiku-20240307-v1:0": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 0.25,
+		outputPrice: 1.25,
+	},
+} as const satisfies Record<string, ModelInfo>
+
+// OpenRouter
+// https://openrouter.ai/models?order=newest&supported_parameters=tools
+export type OpenRouterModelId = keyof typeof openRouterModels
+export const openRouterDefaultModelId: OpenRouterModelId = "anthropic/claude-3.5-sonnet:beta"
+export const openRouterModels = {
+	"anthropic/claude-3.5-sonnet:beta": {
+		maxTokens: 8192,
+		supportsImages: true,
+		inputPrice: 3.0,
+		outputPrice: 15.0,
+	},
+	"anthropic/claude-3-opus:beta": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 15,
+		outputPrice: 75,
+	},
+	"anthropic/claude-3-sonnet:beta": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 3,
+		outputPrice: 15,
+	},
+	"anthropic/claude-3-haiku:beta": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 0.25,
+		outputPrice: 1.25,
+	},
+	"openai/gpt-4o-2024-08-06": {
+		maxTokens: 16384,
+		supportsImages: true,
+		inputPrice: 2.5,
+		outputPrice: 10,
+	},
+	"openai/gpt-4o-mini-2024-07-18": {
+		maxTokens: 16384,
+		supportsImages: true,
+		inputPrice: 0.15,
+		outputPrice: 0.6,
+	},
+	"openai/gpt-4-turbo": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 10,
+		outputPrice: 30,
+	},
+	// llama 3.1 models cannot use tools yet
+	// "meta-llama/llama-3.1-405b-instruct": {
+	// 	maxTokens: 2048,
+	// 	supportsImages: false,
+	// 	inputPrice: 2.7,
+	// 	outputPrice: 2.7,
+	// },
+	// "meta-llama/llama-3.1-70b-instruct": {
+	// 	maxTokens: 2048,
+	// 	supportsImages: false,
+	// 	inputPrice: 0.52,
+	// 	outputPrice: 0.75,
+	// },
+	// "meta-llama/llama-3.1-8b-instruct": {
+	// 	maxTokens: 2048,
+	// 	supportsImages: false,
+	// 	inputPrice: 0.06,
+	// 	outputPrice: 0.06,
+	// },
+	// OpenRouter needs to fix mapping gemini 1.5 responses for tool calls properly, they return content with line breaks formatted wrong (too many escapes), and throw errors for being in the wrong order when they're not. They also cannot handle feedback given to a request with multiple tools. Giving feedback to one tool use requests works fine. ("Please ensure that function response turn comes immediately after a function call turn. And the number of function response parts should be equal to number of function call parts of the function call turn.")
+	// "google/gemini-pro-1.5": {
+	// 	maxTokens: 8192,
+	// 	supportsImages: false, // "Function Calling is not supported with non-text input"
+	// 	inputPrice: 2.5,
+	// 	outputPrice: 7.5,
+	// },
+	// "google/gemini-flash-1.5": {
+	// 	maxTokens: 8192,
+	// 	supportsImages: false, // "Function Calling is not supported with non-text input"
+	// 	inputPrice: 0.25,
+	// 	outputPrice: 0.75,
+	// },
+	// "google/gemini-pro": {
+	// 	maxTokens: 8192,
+	// 	supportsImages: false, // "Function Calling is not supported with non-text input"
+	// 	inputPrice: 0.125,
+	// 	outputPrice: 0.375,
+	// },
+	// while deepseek coder can use tools, it may sometimes send tool invocation as a text block
+	"deepseek/deepseek-coder": {
+		maxTokens: 4096,
+		supportsImages: false,
+		inputPrice: 0.14,
+		outputPrice: 0.28,
+	},
+	// mistral models can use tools but aren't great at going step-by-step and proceeding to the next step
+	"mistralai/mistral-large": {
+		maxTokens: 8192,
+		supportsImages: false,
+		inputPrice: 3,
+		outputPrice: 9,
+	},
+	// This model is not capable of complex system/tool prompts
+	// "mistralai/mistral-7b-instruct-v0.1": {
+	// 	maxTokens: 4096,
+	// 	supportsImages: false,
+	// 	inputPrice: 0.06,
+	// 	outputPrice: 0.06,
+	// },
+	// cohere models are not capable of complex system/tool prompts
+	// "cohere/command-r-plus": {
+	// 	maxTokens: 4000,
+	// 	supportsImages: false,
+	// 	inputPrice: 3,
+	// 	outputPrice: 15,
+	// },
+	// "cohere/command-r": {
+	// 	maxTokens: 4000,
+	// 	supportsImages: false,
+	// 	inputPrice: 0.5,
+	// 	outputPrice: 1.5,
+	// },
+} as const satisfies Record<string, ModelInfo>

src/utils/export-markdown.ts

@@ -0,0 +1,79 @@
+import { Anthropic } from "@anthropic-ai/sdk"
+import os from "os"
+import * as path from "path"
+import * as vscode from "vscode"
+
+export async function downloadTask(conversationHistory: Anthropic.MessageParam[]) {
+	// File name
+	const date = new Date()
+	const month = date.toLocaleString("en-US", { month: "short" }).toLowerCase()
+	const day = date.getDate()
+	const year = date.getFullYear()
+	let hours = date.getHours()
+	const minutes = date.getMinutes().toString().padStart(2, "0")
+	const ampm = hours >= 12 ? "pm" : "am"
+	hours = hours % 12
+	hours = hours ? hours : 12 // the hour '0' should be '12'
+	const fileName = `claude_dev_task_${month}-${day}-${year}_${hours}-${minutes}-${ampm}.md`
+
+	// Generate markdown
+	const markdownContent = conversationHistory
+		.map((message) => {
+			const role = message.role === "user" ? "**User:**" : "**Assistant:**"
+			const content = Array.isArray(message.content)
+				? message.content.map(formatContentBlockToMarkdown).join("\n")
+				: message.content
+
+			return `${role}\n\n${content}\n\n`
+		})
+		.join("---\n\n")
+
+	// Prompt user for save location
+	const saveUri = await vscode.window.showSaveDialog({
+		filters: { Markdown: ["md"] },
+		defaultUri: vscode.Uri.file(path.join(os.homedir(), "Downloads", fileName)),
+	})
+
+	if (saveUri) {
+		// Write content to the selected location
+		await vscode.workspace.fs.writeFile(saveUri, Buffer.from(markdownContent))
+		vscode.window.showTextDocument(saveUri, { preview: true })
+	}
+}
+
+function formatContentBlockToMarkdown(
+	block:
+		| Anthropic.TextBlockParam
+		| Anthropic.ImageBlockParam
+		| Anthropic.ToolUseBlockParam
+		| Anthropic.ToolResultBlockParam
+): string {
+	switch (block.type) {
+		case "text":
+			return block.text
+		case "image":
+			return `[Image]`
+		case "tool_use":
+			let input: string
+			if (typeof block.input === "object" && block.input !== null) {
+				input = Object.entries(block.input)
+					.map(([key, value]) => `${key.charAt(0).toUpperCase() + key.slice(1)}: ${value}`)
+					.join("\n")
+			} else {
+				input = String(block.input)
+			}
+			return `[Tool Use: ${block.name}]\n${input}`
+		case "tool_result":
+			if (typeof block.content === "string") {
+				return `[Tool Result${block.is_error ? " (Error)" : ""}]\n${block.content}`
+			} else if (Array.isArray(block.content)) {
+				return `[Tool Result${block.is_error ? " (Error)" : ""}]\n${block.content
+					.map(formatContentBlockToMarkdown)
+					.join("\n")}`
+			} else {
+				return `[Tool Result${block.is_error ? " (Error)" : ""}]`
+			}
+		default:
+			return "[Unexpected content type]"
+	}
+}

src/utils/getNonce.ts

@@ -0,0 +1,16 @@
+/**
+ * A helper function that returns a unique alphanumeric identifier called a nonce.
+ *
+ * @remarks This function is primarily used to help enforce content security
+ * policies for resources/scripts being executed in a webview context.
+ *
+ * @returns A nonce
+ */
+export function getNonce() {
+	let text = ""
+	const possible = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
+	for (let i = 0; i < 32; i++) {
+		text += possible.charAt(Math.floor(Math.random() * possible.length))
+	}
+	return text
+}

src/utils/getUri.ts

@@ -0,0 +1,15 @@
+import { Uri, Webview } from "vscode"
+/**
+ * A helper function which will get the webview URI of a given file or resource.
+ *
+ * @remarks This URI can be used within a webview's HTML as a link to the
+ * given file/resource.
+ *
+ * @param webview A reference to the extension webview
+ * @param extensionUri The URI of the directory containing the extension
+ * @param pathList An array of strings representing the path to a file/resource
+ * @returns A URI pointing to the file/resource
+ */
+export function getUri(webview: Webview, extensionUri: Uri, pathList: string[]) {
+	return webview.asWebviewUri(Uri.joinPath(extensionUri, ...pathList))
+}

src/utils/index.ts

@@ -0,0 +1,4 @@
+export * from "./getNonce"
+export * from "./getUri"
+export * from "./process-images"
+export * from "./export-markdown"

src/utils/process-images.ts

@@ -0,0 +1,45 @@
+import * as vscode from "vscode"
+import fs from "fs/promises"
+import * as path from "path"
+
+export async function selectImages(): Promise<string[]> {
+	const options: vscode.OpenDialogOptions = {
+		canSelectMany: true,
+		openLabel: "Select",
+		filters: {
+			Images: ["png", "jpg", "jpeg", "webp"], // supported by anthropic and openrouter
+		},
+	}
+
+	const fileUris = await vscode.window.showOpenDialog(options)
+
+	if (!fileUris || fileUris.length === 0) {
+		return []
+	}
+
+	return await Promise.all(
+		fileUris.map(async (uri) => {
+			const imagePath = uri.fsPath
+			const buffer = await fs.readFile(imagePath)
+			const base64 = buffer.toString("base64")
+			const mimeType = getMimeType(imagePath)
+			const dataUrl = `data:${mimeType};base64,${base64}`
+			return dataUrl
+		})
+	)
+}
+
+function getMimeType(filePath: string): string {
+	const ext = path.extname(filePath).toLowerCase()
+	switch (ext) {
+		case ".png":
+			return "image/png"
+		case ".jpeg":
+		case ".jpg":
+			return "image/jpeg"
+		case ".webp":
+			return "image/webp"
+		default:
+			throw new Error(`Unsupported file type: ${ext}`)
+	}
+}

tsconfig.json

@@ -12,7 +12,7 @@
 		"noImplicitReturns": true,
 		"noUnusedLocals": false,
 		"resolveJsonModule": true,
-		"rootDir": "src",
+		"rootDir": ".",
 		"skipLibCheck": true,
 		"sourceMap": true,
 		"strict": true,
@@ -20,5 +20,6 @@
 		"useDefineForClassFields": true,
 		"useUnknownInCatchVariables": false
 	},
+	"include": ["src/**/*", "scripts/**/*"],
 	"exclude": ["node_modules", ".vscode-test", "webview-ui"]
-}
+}

webview-ui/scripts/build-react-no-split.js

@@ -13,6 +13,87 @@ const rewire = require("rewire")
 const defaults = rewire("react-scripts/scripts/build.js")
 const config = defaults.__get__("config")
 
+/* Modifying Webpack Configuration for 'shared' dir
+This section uses Rewire to modify Create React App's webpack configuration without ejecting. Rewire allows us to inject and alter the internal build scripts of CRA at runtime. This allows us to maintain a flexible project structure that keeps shared code outside the webview-ui/src directory, while still adhering to CRA's security model that typically restricts imports to within src/. 
+1. Uses the ModuleScopePlugin to whitelist files from the shared directory, allowing them to be imported despite being outside src/. (see: https://stackoverflow.com/questions/44114436/the-create-react-app-imports-restriction-outside-of-src-directory/58321458#58321458)
+2. Modifies the TypeScript rule to include the shared directory in compilation. This essentially transpiles and includes the ts files in shared dir in the output main.js file.
+Before, we would just import types from shared dir and specifying include (and alias to have cleaner paths) in tsconfig.json was enough. But now that we are creating values (i.e. models in api.ts) to import into the react app, we must also include these files in the webpack resolution.
+- Imports from the shared directory must use full paths relative to the src directory, without file extensions.
+- Example: import { someFunction } from '../../src/shared/utils/helpers'
+*/
+const ModuleScopePlugin = require("react-dev-utils/ModuleScopePlugin")
+const path = require("path")
+const fs = require("fs")
+// Get all files in the shared directory
+const sharedDir = path.resolve(__dirname, "../../src/shared")
+function getAllFiles(dir) {
+	let files = []
+	fs.readdirSync(dir).forEach((file) => {
+		const filePath = path.join(dir, file)
+		if (fs.statSync(filePath).isDirectory()) {
+			files = files.concat(getAllFiles(filePath))
+		} else {
+			const withoutExtension = filePath.split(".")[0]
+			files.push(withoutExtension)
+		}
+	})
+	return files
+}
+const sharedFiles = getAllFiles(sharedDir)
+// config.resolve.plugins = config.resolve.plugins.filter((plugin) => !(plugin instanceof ModuleScopePlugin))
+// Instead of excluding the whole ModuleScopePlugin, we just whitelist specific files that can be imported from outside src.
+config.resolve.plugins.forEach((plugin) => {
+	if (plugin instanceof ModuleScopePlugin) {
+		console.log("Whitelisting shared files: ", sharedFiles)
+		sharedFiles.forEach((file) => plugin.allowedFiles.add(file))
+	}
+})
+/* 
+Webpack configuration
+
+Webpack is a module bundler for JavaScript applications. It processes your project files, resolving dependencies and generating a deployable production build.
+The webpack config is an object that tells webpack how to process and bundle your code. It defines entry points, output settings, and how to handle different file types.
+This config.module section of the webpack config deals with how different file types (modules) should be treated.
+config.module.rules:
+Rules define how module files should be processed. Each rule can:
+- Specify which files to process (test)
+	When webpack "processes" a file, it performs several operations:
+	1. Reads the file
+	2. Parses its content and analyzes dependencies
+	3. Applies transformations (e.g., converting TypeScript to JavaScript)
+	4. Potentially modifies the code (e.g., applying polyfills)
+	5. Includes the processed file in the final bundle
+	By specifying which files to process, we're telling webpack which files should go through this pipeline and be included in our application bundle. Files that aren't processed are ignored by webpack.
+	In our case, we're ensuring that TypeScript files in our shared directory are processed, allowing us to use them in our application.
+- Define which folders to include or exclude
+- Set which loaders to use for transformation
+A loader transforms certain types of files into valid modules that webpack can process. For example, the TypeScript loader converts .ts files into JavaScript that webpack can understand.
+By modifying these rules, we can change how webpack processes different files in our project, allowing us to include files from outside the standard src directory.
+
+Why we need to modify the webpack config
+
+Create React App (CRA) is designed to only process files within the src directory for security reasons. (CRA limits processing to the src directory to prevent accidental inclusion of sensitive files, reduce the attack surface, and ensure predictable builds, enhancing overall project security and consistency. Therefore it's essential that if you do include files outside src, you do so explicitly.)
+To use files from the shared directory, we need to:
+1. Modify ModuleScopePlugin to allow imports from the shared directory.
+2. Update the TypeScript loader rule to process TypeScript files from the shared directory.
+These changes tell webpack it's okay to import from the shared directory and ensure that TypeScript files in this directory are properly converted to JavaScript.
+
+Modify webpack configuration to process TypeScript files from shared directory
+
+This code modifies the webpack configuration to allow processing of TypeScript files from our shared directory, which is outside the standard src folder.
+1. config.module.rules[1]: In Create React App's webpack config, the second rule (index 1) typically contains the rules for processing JavaScript and TypeScript files.
+2. .oneOf: This array contains a list of loaders, and webpack will use the first matching loader for each file. We iterate through these to find the TypeScript loader.
+3. We check each rule to see if it applies to TypeScript files by looking for 'ts|tsx' in the test regex.
+4. When we find the TypeScript rule, we add our shared directory to its 'include' array. This tells webpack to also process TypeScript files from the shared directory.
+Note: This code assumes a specific structure in the CRA webpack config. If CRA updates its config structure in future versions, this code might need to be adjusted.
+*/
+config.module.rules[1].oneOf.forEach((rule) => {
+	if (rule.test && rule.test.toString().includes("ts|tsx")) {
+		// rule.include is path to src by default, but we can update rule.include to be an array as it matches an expected schema by react-scripts
+		rule.include = [rule.include, sharedDir].filter(Boolean)
+	}
+})
+
 // Disable code splitting
 config.optimization.splitChunks = {
 	cacheGroups: {

webview-ui/src/App.tsx

@@ -1,26 +1,28 @@
-import React, { useEffect, useState, useCallback } from "react"
+import React, { useCallback, useEffect, useMemo, useState } from "react"
+import { useEvent } from "react-use"
+import { ApiConfiguration } from "../../src/shared/api"
+import { ClaudeMessage, ExtensionMessage } from "../../src/shared/ExtensionMessage"
 import "./App.css"
+import { normalizeApiConfiguration } from "./components/ApiOptions"
 import ChatView from "./components/ChatView"
 import SettingsView from "./components/SettingsView"
-import { ClaudeMessage, ExtensionMessage } from "@shared/ExtensionMessage"
 import WelcomeView from "./components/WelcomeView"
-import { vscode } from "./utilities/vscode"
-import { useEvent } from "react-use"
+import { vscode } from "./utils/vscode"
 
 /*
 The contents of webviews however are created when the webview becomes visible and destroyed when the webview is moved into the background. Any state inside the webview will be lost when the webview is moved to a background tab.
 
 The best way to solve this is to make your webview stateless. Use message passing to save off the webview's state and then restore the state when the webview becomes visible again.
-
-
 */
 
 const App: React.FC = () => {
 	const [didHydrateState, setDidHydrateState] = useState(false)
 	const [showSettings, setShowSettings] = useState(false)
 	const [showWelcome, setShowWelcome] = useState<boolean>(false)
-	const [apiKey, setApiKey] = useState<string>("")
+	const [version, setVersion] = useState<string>("")
+	const [apiConfiguration, setApiConfiguration] = useState<ApiConfiguration | undefined>(undefined)
 	const [maxRequestsPerTask, setMaxRequestsPerTask] = useState<string>("")
+	const [customInstructions, setCustomInstructions] = useState<string>("")
 	const [vscodeThemeName, setVscodeThemeName] = useState<string | undefined>(undefined)
 	const [claudeMessages, setClaudeMessages] = useState<ClaudeMessage[]>([])
 	const [showAnnouncement, setShowAnnouncement] = useState(false)
@@ -33,11 +35,17 @@ const App: React.FC = () => {
 		const message: ExtensionMessage = e.data
 		switch (message.type) {
 			case "state":
-				setShowWelcome(!message.state!.apiKey)
-				setApiKey(message.state!.apiKey || "")
+				setVersion(message.state!.version)
+				const hasKey =
+					message.state!.apiConfiguration?.apiKey !== undefined ||
+					message.state!.apiConfiguration?.openRouterApiKey !== undefined ||
+					message.state!.apiConfiguration?.awsAccessKey !== undefined
+				setShowWelcome(!hasKey)
+				setApiConfiguration(message.state!.apiConfiguration)
 				setMaxRequestsPerTask(
 					message.state!.maxRequestsPerTask !== undefined ? message.state!.maxRequestsPerTask.toString() : ""
 				)
+				setCustomInstructions(message.state!.customInstructions || "")
 				setVscodeThemeName(message.state!.themeName)
 				setClaudeMessages(message.state!.claudeMessages)
 				// don't update showAnnouncement to false if shouldShowAnnouncement is false
@@ -63,6 +71,10 @@ const App: React.FC = () => {
 
 	useEvent("message", handleMessage)
 
+	const { selectedModelInfo } = useMemo(() => {
+		return normalizeApiConfiguration(apiConfiguration)
+	}, [apiConfiguration])
+
 	if (!didHydrateState) {
 		return null
 	}
@@ -70,24 +82,29 @@ const App: React.FC = () => {
 	return (
 		<>
 			{showWelcome ? (
-				<WelcomeView apiKey={apiKey} setApiKey={setApiKey} />
+				<WelcomeView apiConfiguration={apiConfiguration} setApiConfiguration={setApiConfiguration} />
 			) : (
 				<>
 					{showSettings && (
 						<SettingsView
-							apiKey={apiKey}
-							setApiKey={setApiKey}
+							version={version}
+							apiConfiguration={apiConfiguration}
+							setApiConfiguration={setApiConfiguration}
 							maxRequestsPerTask={maxRequestsPerTask}
 							setMaxRequestsPerTask={setMaxRequestsPerTask}
+							customInstructions={customInstructions}
+							setCustomInstructions={setCustomInstructions}
 							onDone={() => setShowSettings(false)}
 						/>
 					)}
 					{/* Do not conditionally load ChatView, it's expensive and there's state we don't want to lose (user input, disableInput, askResponse promise, etc.) */}
 					<ChatView
+						version={version}
 						messages={claudeMessages}
 						isHidden={showSettings}
 						vscodeThemeName={vscodeThemeName}
 						showAnnouncement={showAnnouncement}
+						selectedModelSupportsImages={selectedModelInfo.supportsImages}
 						hideAnnouncement={() => setShowAnnouncement(false)}
 					/>
 				</>

webview-ui/src/components/Announcement.tsx

@@ -1,12 +1,13 @@
 import { VSCodeButton, VSCodeLink } from "@vscode/webview-ui-toolkit/react"
 
 interface AnnouncementProps {
+	version: string
 	hideAnnouncement: () => void
 }
 /*
 You must update the latestAnnouncementId in ClaudeDevProvider for new announcements to show to users. This new id will be compared with whats in state for the 'last announcement shown', and if it's different then the announcement will render. As soon as an announcement is shown, the id will be updated in state. This ensures that announcements are not shown more than once, even if the user doesn't close it themselves.
 */
-const Announcement = ({ hideAnnouncement }: AnnouncementProps) => {
+const Announcement = ({ version, hideAnnouncement }: AnnouncementProps) => {
 	return (
 		<div
 			style={{
@@ -22,34 +23,22 @@ const Announcement = ({ hideAnnouncement }: AnnouncementProps) => {
 				style={{ position: "absolute", top: "8px", right: "8px" }}>
 				<span className="codicon codicon-close"></span>
 			</VSCodeButton>
-			<h3 style={{ margin: "0 0 8px" }}>🎉{"  "}New in v1.0.8</h3>
+			<h3 style={{ margin: "0 0 8px" }}>
+				🎉{"  "}New in v{version}
+			</h3>
 			<ul style={{ margin: "0 0 8px", paddingLeft: "20px" }}>
 				<li>
-					Open in the editor (using{" "}
-					<span
-						className="codicon codicon-link-external"
-						style={{ display: "inline", fontSize: "12.5px", verticalAlign: "text-bottom" }}></span>{" "}
-					or <code>Claude Dev: Open In New Tab</code> in command palette) to see how Claude updates your
-					workspace more clearly
+					Paste images in chat and turn mockups into fully functional applications or fix bugs with
+					screenshots
 				</li>
 				<li>
-					New <code style={{ wordBreak: "break-all" }}>list_files_recursive</code> and{" "}
-					<code style={{ wordBreak: "break-all" }}>view_source_code_definitions_top_level</code> tools to help
-					Claude get a comprehensive overview of your project's file structure and source code definitions
-					<VSCodeLink
-						href="https://github.com/saoudrizwan/claude-dev?tab=readme-ov-file#working-in-existing-projects"
-						style={{ display: "inline" }}>
-						(more on this here)
-					</VSCodeLink>
+					Added option to choose other Claude models (+ GPT-4o, DeepSeek, and Mistral if you use OpenRouter)
 				</li>
 				<li>
-					Interact with CLI commands by sending messages to stdin and terminating long-running processes like
-					servers
+					You can now add custom instructions to the end of the system prompt (e.g. "Always use Python",
+					"Speak in Spanish")
 				</li>
-				<li>Provide feedback to tool use like editing files or running commands</li>
-				<li>Shows diff view of new or edited files right in the editor</li>
-				<li>Added ability to retry failed API requests (helpful for rate limits)</li>
-				<li>Export task to a markdown file (useful as context for future tasks)</li>
+				<li>Improved support for running interactive terminal commands and servers</li>
 			</ul>
 			<p style={{ margin: "0" }}>
 				Follow me for more updates!{" "}

webview-ui/src/components/ApiOptions.tsx

@@ -0,0 +1,272 @@
+import { VSCodeDropdown, VSCodeLink, VSCodeOption, VSCodeTextField } from "@vscode/webview-ui-toolkit/react"
+import React, { useMemo } from "react"
+import {
+	ApiConfiguration,
+	ApiModelId,
+	ModelInfo,
+	anthropicDefaultModelId,
+	anthropicModels,
+	bedrockDefaultModelId,
+	bedrockModels,
+	openRouterDefaultModelId,
+	openRouterModels,
+} from "../../../src/shared/api"
+
+interface ApiOptionsProps {
+	showModelOptions: boolean
+	apiConfiguration?: ApiConfiguration
+	setApiConfiguration: React.Dispatch<React.SetStateAction<ApiConfiguration | undefined>>
+}
+
+const ApiOptions: React.FC<ApiOptionsProps> = ({ showModelOptions, apiConfiguration, setApiConfiguration }) => {
+	const handleInputChange = (field: keyof ApiConfiguration) => (event: any) => {
+		setApiConfiguration((prev) => ({ ...prev, [field]: event.target.value }))
+	}
+
+	const { selectedProvider, selectedModelId, selectedModelInfo } = useMemo(() => {
+		return normalizeApiConfiguration(apiConfiguration)
+	}, [apiConfiguration])
+
+	/*
+	VSCodeDropdown has an open bug where dynamically rendered options don't auto select the provided value prop. You can see this for yourself by comparing  it with normal select/option elements, which work as expected.
+	https://github.com/microsoft/vscode-webview-ui-toolkit/issues/433
+
+	In our case, when the user switches between providers, we recalculate the selectedModelId depending on the provider, the default model for that provider, and a modelId that the user may have selected. Unfortunately, the VSCodeDropdown component wouldn't select this calculated value, and would default to the first "Select a model..." option instead, which makes it seem like the model was cleared out when it wasn't. 
+
+	As a workaround, we create separate instances of the dropdown for each provider, and then conditionally render the one that matches the current provider.
+	*/
+	const createDropdown = (models: Record<string, ModelInfo>) => {
+		return (
+			<VSCodeDropdown
+				id="model-id"
+				value={selectedModelId}
+				onChange={handleInputChange("apiModelId")}
+				style={{ width: "100%" }}>
+				<VSCodeOption value="">Select a model...</VSCodeOption>
+				{Object.keys(models).map((modelId) => (
+					<VSCodeOption
+						key={modelId}
+						value={modelId}
+						style={{
+							whiteSpace: "normal",
+							wordWrap: "break-word",
+							maxWidth: "100%",
+						}}>
+						{modelId}
+					</VSCodeOption>
+				))}
+			</VSCodeDropdown>
+		)
+	}
+
+	return (
+		<div style={{ display: "flex", flexDirection: "column", gap: 5 }}>
+			<div className="dropdown-container">
+				<label htmlFor="api-provider">
+					<span style={{ fontWeight: 500 }}>API Provider</span>
+				</label>
+				<VSCodeDropdown id="api-provider" value={selectedProvider} onChange={handleInputChange("apiProvider")}>
+					<VSCodeOption value="openrouter">OpenRouter</VSCodeOption>
+					<VSCodeOption value="anthropic">Anthropic</VSCodeOption>
+					<VSCodeOption value="bedrock">AWS Bedrock</VSCodeOption>
+				</VSCodeDropdown>
+			</div>
+
+			{selectedProvider === "anthropic" && (
+				<div>
+					<VSCodeTextField
+						value={apiConfiguration?.apiKey || ""}
+						style={{ width: "100%" }}
+						onInput={handleInputChange("apiKey")}
+						placeholder="Enter API Key...">
+						<span style={{ fontWeight: 500 }}>Anthropic API Key</span>
+					</VSCodeTextField>
+					<p
+						style={{
+							fontSize: "12px",
+							marginTop: "5px",
+							color: "var(--vscode-descriptionForeground)",
+						}}>
+						This key is stored locally and only used to make API requests from this extension.
+						<VSCodeLink href="https://console.anthropic.com/" style={{ display: "inline" }}>
+							You can get an Anthropic API key by signing up here.
+						</VSCodeLink>
+					</p>
+				</div>
+			)}
+
+			{selectedProvider === "openrouter" && (
+				<div>
+					<VSCodeTextField
+						value={apiConfiguration?.openRouterApiKey || ""}
+						style={{ width: "100%" }}
+						onInput={handleInputChange("openRouterApiKey")}
+						placeholder="Enter API Key...">
+						<span style={{ fontWeight: 500 }}>OpenRouter API Key</span>
+					</VSCodeTextField>
+					<p
+						style={{
+							fontSize: "12px",
+							marginTop: "5px",
+							color: "var(--vscode-descriptionForeground)",
+						}}>
+						This key is stored locally and only used to make API requests from this extension.
+						<VSCodeLink href="https://openrouter.ai/" style={{ display: "inline" }}>
+							You can get an OpenRouter API key by signing up here.
+						</VSCodeLink>
+					</p>
+				</div>
+			)}
+
+			{selectedProvider === "bedrock" && (
+				<div style={{ display: "flex", flexDirection: "column", gap: 5 }}>
+					<VSCodeTextField
+						value={apiConfiguration?.awsAccessKey || ""}
+						style={{ width: "100%" }}
+						onInput={handleInputChange("awsAccessKey")}
+						placeholder="Enter Access Key...">
+						<span style={{ fontWeight: 500 }}>AWS Access Key</span>
+					</VSCodeTextField>
+					<VSCodeTextField
+						value={apiConfiguration?.awsSecretKey || ""}
+						style={{ width: "100%" }}
+						onInput={handleInputChange("awsSecretKey")}
+						placeholder="Enter Secret Key...">
+						<span style={{ fontWeight: 500 }}>AWS Secret Key</span>
+					</VSCodeTextField>
+					<div className="dropdown-container">
+						<label htmlFor="aws-region-dropdown">
+							<span style={{ fontWeight: 500 }}>AWS Region</span>
+						</label>
+						<VSCodeDropdown
+							id="aws-region-dropdown"
+							value={apiConfiguration?.awsRegion || ""}
+							style={{ width: "100%" }}
+							onChange={handleInputChange("awsRegion")}>
+							<VSCodeOption value="">Select a region...</VSCodeOption>
+							{/* The user will have to choose a region that supports the model they use, but this shouldn't be a problem since they'd have to request access for it in that region in the first place. */}
+							<VSCodeOption value="us-east-1">US East (N. Virginia)</VSCodeOption>
+							{/* <VSCodeOption value="us-east-2">US East (Ohio)</VSCodeOption> */}
+							{/* <VSCodeOption value="us-west-1">US West (N. California)</VSCodeOption> */}
+							<VSCodeOption value="us-west-2">US West (Oregon)</VSCodeOption>
+							{/* <VSCodeOption value="af-south-1">Africa (Cape Town)</VSCodeOption> */}
+							{/* <VSCodeOption value="ap-east-1">Asia Pacific (Hong Kong)</VSCodeOption> */}
+							<VSCodeOption value="ap-south-1">Asia Pacific (Mumbai)</VSCodeOption>
+							<VSCodeOption value="ap-northeast-1">Asia Pacific (Tokyo)</VSCodeOption>
+							{/* <VSCodeOption value="ap-northeast-2">Asia Pacific (Seoul)</VSCodeOption> */}
+							{/* <VSCodeOption value="ap-northeast-3">Asia Pacific (Osaka)</VSCodeOption> */}
+							<VSCodeOption value="ap-southeast-1">Asia Pacific (Singapore)</VSCodeOption>
+							<VSCodeOption value="ap-southeast-2">Asia Pacific (Sydney)</VSCodeOption>
+							<VSCodeOption value="ca-central-1">Canada (Central)</VSCodeOption>
+							<VSCodeOption value="eu-central-1">Europe (Frankfurt)</VSCodeOption>
+							<VSCodeOption value="eu-west-1">Europe (Ireland)</VSCodeOption>
+							<VSCodeOption value="eu-west-2">Europe (London)</VSCodeOption>
+							<VSCodeOption value="eu-west-3">Europe (Paris)</VSCodeOption>
+							{/* <VSCodeOption value="eu-north-1">Europe (Stockholm)</VSCodeOption> */}
+							{/* <VSCodeOption value="me-south-1">Middle East (Bahrain)</VSCodeOption> */}
+							<VSCodeOption value="sa-east-1">South America (São Paulo)</VSCodeOption>
+						</VSCodeDropdown>
+					</div>
+					<p
+						style={{
+							fontSize: "12px",
+							marginTop: "5px",
+							color: "var(--vscode-descriptionForeground)",
+						}}>
+						These credentials are stored locally and only used to make API requests from this extension.
+						<VSCodeLink
+							href="https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html"
+							style={{ display: "inline" }}>
+							You can find your AWS access key and secret key here.
+						</VSCodeLink>
+					</p>
+				</div>
+			)}
+
+			{showModelOptions && (
+				<>
+					<div className="dropdown-container">
+						<label htmlFor="model-id">
+							<span style={{ fontWeight: 500 }}>Model</span>
+						</label>
+						{selectedProvider === "anthropic" && createDropdown(anthropicModels)}
+						{selectedProvider === "openrouter" && createDropdown(openRouterModels)}
+						{selectedProvider === "bedrock" && createDropdown(bedrockModels)}
+					</div>
+
+					<ModelInfoView modelInfo={selectedModelInfo} />
+				</>
+			)}
+		</div>
+	)
+}
+
+const ModelInfoView = ({ modelInfo }: { modelInfo: ModelInfo }) => {
+	const formatPrice = (price: number) => {
+		return new Intl.NumberFormat("en-US", {
+			style: "currency",
+			currency: "USD",
+			minimumFractionDigits: 2,
+			maximumFractionDigits: 2,
+		}).format(price)
+	}
+
+	return (
+		<p style={{ fontSize: "12px", marginTop: "2px", color: "var(--vscode-descriptionForeground)" }}>
+			<span
+				style={{
+					fontWeight: 500,
+					color: modelInfo.supportsImages
+						? "var(--vscode-testing-iconPassed)"
+						: "var(--vscode-errorForeground)",
+				}}>
+				<i
+					className={`codicon codicon-${modelInfo.supportsImages ? "check" : "x"}`}
+					style={{
+						marginRight: 4,
+						marginBottom: modelInfo.supportsImages ? 1 : -1,
+						fontSize: modelInfo.supportsImages ? 11 : 13,
+						fontWeight: 700,
+						display: "inline-block",
+						verticalAlign: "bottom",
+					}}></i>
+				{modelInfo.supportsImages ? "Supports images" : "Does not support images"}
+			</span>
+			<br />
+			<span style={{ fontWeight: 500 }}>Max output:</span> {modelInfo.maxTokens.toLocaleString()} tokens
+			<br />
+			<span style={{ fontWeight: 500 }}>Input price:</span> {formatPrice(modelInfo.inputPrice)} per million tokens
+			<br />
+			<span style={{ fontWeight: 500 }}>Output price:</span> {formatPrice(modelInfo.outputPrice)} per million
+			tokens
+		</p>
+	)
+}
+
+export function normalizeApiConfiguration(apiConfiguration?: ApiConfiguration) {
+	const provider = apiConfiguration?.apiProvider || "anthropic"
+	const modelId = apiConfiguration?.apiModelId
+
+	const getProviderData = (models: Record<string, ModelInfo>, defaultId: ApiModelId) => {
+		let selectedModelId: ApiModelId
+		let selectedModelInfo: ModelInfo
+		if (modelId && modelId in models) {
+			selectedModelId = modelId
+			selectedModelInfo = models[modelId]
+		} else {
+			selectedModelId = defaultId
+			selectedModelInfo = models[defaultId]
+		}
+		return { selectedProvider: provider, selectedModelId, selectedModelInfo }
+	}
+	switch (provider) {
+		case "anthropic":
+			return getProviderData(anthropicModels, anthropicDefaultModelId)
+		case "openrouter":
+			return getProviderData(openRouterModels, openRouterDefaultModelId)
+		case "bedrock":
+			return getProviderData(bedrockModels, bedrockDefaultModelId)
+	}
+}
+
+export default ApiOptions

webview-ui/src/components/ChatRow.tsx

@@ -1,11 +1,12 @@
-import { ClaudeAsk, ClaudeMessage, ClaudeSay, ClaudeSayTool } from "@shared/ExtensionMessage"
 import { VSCodeBadge, VSCodeButton, VSCodeProgressRing } from "@vscode/webview-ui-toolkit/react"
 import React from "react"
-import { COMMAND_OUTPUT_STRING } from "../utilities/combineCommandSequences"
-import { SyntaxHighlighterStyle } from "../utilities/getSyntaxHighlighterStyleFromTheme"
-import CodeBlock from "./CodeBlock/CodeBlock"
 import Markdown from "react-markdown"
 import { Prism as SyntaxHighlighter } from "react-syntax-highlighter"
+import { ClaudeAsk, ClaudeMessage, ClaudeSay, ClaudeSayTool } from "../../../src/shared/ExtensionMessage"
+import { COMMAND_OUTPUT_STRING } from "../utils/combineCommandSequences"
+import { SyntaxHighlighterStyle } from "../utils/getSyntaxHighlighterStyleFromTheme"
+import CodeBlock from "./CodeBlock/CodeBlock"
+import Thumbnails from "./Thumbnails"
 
 interface ChatRowProps {
 	message: ClaudeMessage
@@ -294,7 +295,10 @@ const ChatRow: React.FC<ChatRowProps> = ({
 									whiteSpace: "pre-line",
 									wordWrap: "break-word",
 								}}>
-								<span>{message.text}</span>
+								<span style={{ display: "block" }}>{message.text}</span>
+								{message.images && message.images.length > 0 && (
+									<Thumbnails images={message.images} style={{ marginTop: "8px" }} />
+								)}
 							</div>
 						)
 					case "error":

webview-ui/src/components/ChatView.tsx

@@ -1,28 +1,41 @@
-import { ClaudeAsk, ClaudeMessage, ExtensionMessage } from "@shared/ExtensionMessage"
 import { VSCodeButton, VSCodeLink } from "@vscode/webview-ui-toolkit/react"
 import { KeyboardEvent, useCallback, useEffect, useMemo, useRef, useState } from "react"
 import vsDarkPlus from "react-syntax-highlighter/dist/esm/styles/prism/vsc-dark-plus"
 import DynamicTextArea from "react-textarea-autosize"
 import { useEvent, useMount } from "react-use"
-import { combineApiRequests } from "../utilities/combineApiRequests"
-import { combineCommandSequences } from "../utilities/combineCommandSequences"
-import { getApiMetrics } from "../utilities/getApiMetrics"
-import { getSyntaxHighlighterStyleFromTheme } from "../utilities/getSyntaxHighlighterStyleFromTheme"
-import { vscode } from "../utilities/vscode"
+import { Virtuoso, type VirtuosoHandle } from "react-virtuoso"
+import { ClaudeAsk, ClaudeMessage, ExtensionMessage } from "../../../src/shared/ExtensionMessage"
+import { combineApiRequests } from "../utils/combineApiRequests"
+import { combineCommandSequences } from "../utils/combineCommandSequences"
+import { getApiMetrics } from "../utils/getApiMetrics"
+import { getSyntaxHighlighterStyleFromTheme } from "../utils/getSyntaxHighlighterStyleFromTheme"
+import { vscode } from "../utils/vscode"
+import Announcement from "./Announcement"
 import ChatRow from "./ChatRow"
 import TaskHeader from "./TaskHeader"
-import { Virtuoso, type VirtuosoHandle } from "react-virtuoso"
-import Announcement from "./Announcement"
+import Thumbnails from "./Thumbnails"
 
 interface ChatViewProps {
+	version: string
 	messages: ClaudeMessage[]
 	isHidden: boolean
 	vscodeThemeName?: string
 	showAnnouncement: boolean
+	selectedModelSupportsImages: boolean
 	hideAnnouncement: () => void
 }
-// maybe instead of storing state in App, just make chatview  always show so dont conditionally load/unload? need to make sure messages are persisted (i remember seeing something about how webviews can be frozen in docs)
-const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideAnnouncement }: ChatViewProps) => {
+
+const MAX_IMAGES_PER_MESSAGE = 20 // Anthropic limits to 20 images
+
+const ChatView = ({
+	version,
+	messages,
+	isHidden,
+	vscodeThemeName,
+	showAnnouncement,
+	selectedModelSupportsImages,
+	hideAnnouncement,
+}: ChatViewProps) => {
 	//const task = messages.length > 0 ? (messages[0].say === "task" ? messages[0] : undefined) : undefined
 	const task = messages.length > 0 ? messages[0] : undefined // leaving this less safe version here since if the first message is not a task, then the extension is in a bad state and needs to be debugged (see ClaudeDev.abort)
 	const modifiedMessages = useMemo(() => combineApiRequests(combineCommandSequences(messages.slice(1))), [messages])
@@ -32,6 +45,9 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 	const [inputValue, setInputValue] = useState("")
 	const textAreaRef = useRef<HTMLTextAreaElement>(null)
 	const [textAreaDisabled, setTextAreaDisabled] = useState(false)
+	const [isTextAreaFocused, setIsTextAreaFocused] = useState(false)
+	const [selectedImages, setSelectedImages] = useState<string[]>([])
+	const [thumbnailsHeight, setThumbnailsHeight] = useState(0)
 
 	// we need to hold on to the ask because useEffect > lastMessage will always let us know when an ask comes in and handle it, but by the time handleMessage is called, the last message might not be the ask anymore (it could be a say that followed)
 	const [claudeAsk, setClaudeAsk] = useState<ClaudeAsk | undefined>(undefined)
@@ -39,11 +55,8 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 	const [enableButtons, setEnableButtons] = useState<boolean>(false)
 	const [primaryButtonText, setPrimaryButtonText] = useState<string | undefined>(undefined)
 	const [secondaryButtonText, setSecondaryButtonText] = useState<string | undefined>(undefined)
-
 	const [syntaxHighlighterStyle, setSyntaxHighlighterStyle] = useState(vsDarkPlus)
-
 	const virtuosoRef = useRef<VirtuosoHandle>(null)
-
 	const [expandedRows, setExpandedRows] = useState<Record<number, boolean>>({})
 
 	const toggleRowExpansion = (ts: number) => {
@@ -132,6 +145,14 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 						case "error":
 							break
 						case "api_req_started":
+							if (messages.at(-2)?.ask === "command_output") {
+								// if the last ask is a command_output, and we receive an api_req_started, then that means the command has finished and we don't need input from the user anymore (in every other case, the user has to interact with input field or buttons to continue, which does the following automatically)
+								setInputValue("")
+								setTextAreaDisabled(true)
+								setSelectedImages([])
+								setClaudeAsk(undefined)
+								setEnableButtons(false)
+							}
 							break
 						case "api_req_finished":
 							break
@@ -166,9 +187,9 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 
 	const handleSendMessage = () => {
 		const text = inputValue.trim()
-		if (text) {
+		if (text || selectedImages.length > 0) {
 			if (messages.length === 0) {
-				vscode.postMessage({ type: "newTask", text })
+				vscode.postMessage({ type: "newTask", text, images: selectedImages })
 			} else if (claudeAsk) {
 				switch (claudeAsk) {
 					case "followup":
@@ -176,13 +197,19 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 					case "command": // user can provide feedback to a tool or command use
 					case "command_output": // user can send input to command stdin
 					case "completion_result": // if this happens then the user has feedback for the completion result
-						vscode.postMessage({ type: "askResponse", askResponse: "textResponse", text })
+						vscode.postMessage({
+							type: "askResponse",
+							askResponse: "messageResponse",
+							text,
+							images: selectedImages,
+						})
 						break
 					// there is no other case that a textfield should be enabled
 				}
 			}
 			setInputValue("")
 			setTextAreaDisabled(true)
+			setSelectedImages([])
 			setClaudeAsk(undefined)
 			setEnableButtons(false)
 			// setPrimaryButtonText(undefined)
@@ -248,6 +275,65 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 		vscode.postMessage({ type: "clearTask" })
 	}
 
+	const selectImages = () => {
+		vscode.postMessage({ type: "selectImages" })
+	}
+
+	const handlePaste = async (e: React.ClipboardEvent) => {
+		if (shouldDisableImages) {
+			e.preventDefault()
+			return
+		}
+
+		const items = e.clipboardData.items
+		const acceptedTypes = ["png", "jpeg", "webp"] // supported by anthropic and openrouter (jpg is just a file extension but the image will be recognized as jpeg)
+		const imageItems = Array.from(items).filter((item) => {
+			const [type, subtype] = item.type.split("/")
+			return type === "image" && acceptedTypes.includes(subtype)
+		})
+		if (imageItems.length > 0) {
+			e.preventDefault()
+			const imagePromises = imageItems.map((item) => {
+				return new Promise<string | null>((resolve) => {
+					const blob = item.getAsFile()
+					if (!blob) {
+						resolve(null)
+						return
+					}
+					const reader = new FileReader()
+					reader.onloadend = () => {
+						if (reader.error) {
+							console.error("Error reading file:", reader.error)
+							resolve(null)
+						} else {
+							const result = reader.result
+							resolve(typeof result === "string" ? result : null)
+						}
+					}
+					reader.readAsDataURL(blob)
+				})
+			})
+			const imageDataArray = await Promise.all(imagePromises)
+			const dataUrls = imageDataArray.filter((dataUrl): dataUrl is string => dataUrl !== null)
+			//.map((dataUrl) => dataUrl.split(",")[1]) // strip the mime type prefix, sharp doesn't need it
+			if (dataUrls.length > 0) {
+				setSelectedImages((prevImages) => [...prevImages, ...dataUrls].slice(0, MAX_IMAGES_PER_MESSAGE))
+			} else {
+				console.warn("No valid images were processed")
+			}
+		}
+	}
+
+	useEffect(() => {
+		if (selectedImages.length === 0) {
+			setThumbnailsHeight(0)
+		}
+	}, [selectedImages])
+
+	const handleThumbnailsHeightChange = useCallback((height: number) => {
+		setThumbnailsHeight(height)
+	}, [])
+
 	const handleMessage = useCallback(
 		(e: MessageEvent) => {
 			const message: ExtensionMessage = e.data
@@ -261,6 +347,14 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 							break
 					}
 					break
+				case "selectedImages":
+					const newImages = message.images ?? []
+					if (newImages.length > 0) {
+						setSelectedImages((prevImages) =>
+							[...prevImages, ...newImages].slice(0, MAX_IMAGES_PER_MESSAGE)
+						)
+					}
+					break
 			}
 			// textAreaRef.current is not explicitly required here since react gaurantees that ref will be stable across re-renders, and we're not using its value but its reference.
 		},
@@ -301,6 +395,12 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 				case "api_req_finished": // combineApiRequests removes this from modifiedMessages anyways
 				case "api_req_retried": // this message is used to update the latest api_req_started that the request was retried
 					return false
+				case "text":
+					// Sometimes Claude returns an empty text message, we don't want to render these. (We also use a say text for user messages, so in case they just sent images we still render that)
+					if ((message.text ?? "") === "" && (message.images?.length ?? 0) === 0) {
+						return false
+					}
+					break
 			}
 			return true
 		})
@@ -317,13 +417,20 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 		return () => clearTimeout(timer)
 	}, [visibleMessages])
 
-	const placeholderText = useMemo(() => {
+	const [placeholderText, isInputPipingToStdin] = useMemo(() => {
 		if (messages.at(-1)?.ask === "command_output") {
-			return "Type input to command stdin..."
+			return ["Type input to command stdin...", true]
 		}
-		return task ? "Type a message..." : "Type your task here..."
+		const text = task ? "Type a message..." : "Type your task here..."
+		return [text, false]
 	}, [task, messages])
 
+	const shouldDisableImages =
+		!selectedModelSupportsImages ||
+		textAreaDisabled ||
+		selectedImages.length >= MAX_IMAGES_PER_MESSAGE ||
+		isInputPipingToStdin
+
 	return (
 		<div
 			style={{
@@ -338,7 +445,7 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 			}}>
 			{task ? (
 				<TaskHeader
-					taskText={task.text || ""}
+					task={task}
 					tokensIn={apiMetrics.totalTokensIn}
 					tokensOut={apiMetrics.totalTokensOut}
 					totalCost={apiMetrics.totalCost}
@@ -347,7 +454,7 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 				/>
 			) : (
 				<>
-					{showAnnouncement && <Announcement hideAnnouncement={hideAnnouncement} />}
+					{showAnnouncement && <Announcement version={version} hideAnnouncement={hideAnnouncement} />}
 					<div style={{ padding: "0 20px" }}>
 						<h2>What can I do for you?</h2>
 						<p>
@@ -357,8 +464,8 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 								style={{ display: "inline" }}>
 								Claude 3.5 Sonnet's agentic coding capabilities,
 							</VSCodeLink>{" "}
-							I can handle complex software development tasks step-by-step. With tools that let me read &
-							write files, analyze project source code, and execute terminal commands (after you grant
+							I can handle complex software development tasks step-by-step. With tools that let me create
+							& edit files, explore complex projects, and execute terminal commands (after you grant
 							permission), I can assist you in ways that go beyond simple code completion or tech support.
 						</p>
 					</div>
@@ -420,13 +527,33 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 					</VSCodeButton>
 				)}
 			</div>
-			<div style={{ padding: "10px 15px", opacity: textAreaDisabled ? 0.5 : 1, position: "relative" }}>
+			<div
+				style={{
+					padding: "10px 15px",
+					opacity: textAreaDisabled ? 0.5 : 1,
+					position: "relative",
+					display: "flex",
+				}}>
+				{!isTextAreaFocused && (
+					<div
+						style={{
+							position: "absolute",
+							inset: "10px 15px",
+							border: "1px solid var(--vscode-input-border)",
+							borderRadius: 2,
+							pointerEvents: "none",
+						}}
+					/>
+				)}
 				<DynamicTextArea
 					ref={textAreaRef}
 					value={inputValue}
 					disabled={textAreaDisabled}
 					onChange={(e) => setInputValue(e.target.value)}
 					onKeyDown={handleKeyDown}
+					onFocus={() => setIsTextAreaFocused(true)}
+					onBlur={() => setIsTextAreaFocused(false)}
+					onPaste={handlePaste}
 					onHeightChange={() =>
 						//virtuosoRef.current?.scrollToIndex({ index: "LAST", align: "end", behavior: "auto" })
 						virtuosoRef.current?.scrollTo({ top: Number.MAX_SAFE_INTEGER, behavior: "auto" })
@@ -439,26 +566,58 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 						boxSizing: "border-box",
 						backgroundColor: "var(--vscode-input-background)",
 						color: "var(--vscode-input-foreground)",
-						border: "1px solid var(--vscode-input-border)",
+						//border: "1px solid var(--vscode-input-border)",
 						borderRadius: 2,
 						fontFamily: "var(--vscode-font-family)",
 						fontSize: "var(--vscode-editor-font-size)",
 						lineHeight: "var(--vscode-editor-line-height)",
 						resize: "none",
 						overflow: "hidden",
-						padding: "8px 36px 8px 8px",
+						// Since we have maxRows, when text is long enough it starts to overflow the bottom padding, appearing behind the thumbnails. To fix this, we use a transparent border to push the text up instead. (https://stackoverflow.com/questions/42631947/maintaining-a-padding-inside-of-text-area/52538410#52538410)
+						borderTop: "9px solid transparent",
+						borderBottom: `${thumbnailsHeight + 9}px solid transparent`,
+						borderRight: "54px solid transparent",
+						borderLeft: "9px solid transparent",
+						// Instead of using boxShadow, we use a div with a border to better replicate the behavior when the textarea is focused
+						// boxShadow: "0px 0px 0px 1px var(--vscode-input-border)",
+						padding: 0,
 						cursor: textAreaDisabled ? "not-allowed" : undefined,
+						flex: 1,
 					}}
 				/>
+				{selectedImages.length > 0 && (
+					<Thumbnails
+						images={selectedImages}
+						setImages={setSelectedImages}
+						onHeightChange={handleThumbnailsHeightChange}
+						style={{
+							position: "absolute",
+							paddingTop: 4,
+							bottom: 14,
+							left: 22,
+							right: 67, // (54 + 9) + 4 extra padding
+						}}
+					/>
+				)}
 				<div
 					style={{
 						position: "absolute",
 						right: 20,
+						bottom: 14, // Align with the bottom padding of the container
 						display: "flex",
-						alignItems: "center",
-						top: 0,
-						bottom: 1.5,
+						alignItems: "flex-end",
+						height: "calc(100% - 20px)", // Full height minus top and bottom padding
 					}}>
+					<VSCodeButton
+						disabled={shouldDisableImages}
+						appearance="icon"
+						aria-label="Attach Images"
+						onClick={selectImages}
+						style={{ marginRight: "4px" }}>
+						<span
+							className="codicon codicon-device-camera"
+							style={{ fontSize: 18.5, marginLeft: -2, marginBottom: 1.5 }}></span>
+					</VSCodeButton>
 					<VSCodeButton
 						disabled={textAreaDisabled}
 						appearance="icon"

webview-ui/src/components/CodeBlock/CodeBlock.tsx

@@ -1,7 +1,7 @@
 import { useMemo } from "react"
 import { Prism as SyntaxHighlighter } from "react-syntax-highlighter"
-import { getLanguageFromPath } from "../../utilities/getLanguageFromPath"
-import { SyntaxHighlighterStyle } from "../../utilities/getSyntaxHighlighterStyleFromTheme"
+import { getLanguageFromPath } from "../../utils/getLanguageFromPath"
+import { SyntaxHighlighterStyle } from "../../utils/getSyntaxHighlighterStyleFromTheme"
 
 /*
 const vscodeSyntaxStyle: React.CSSProperties = {

webview-ui/src/components/SettingsView.tsx

@@ -1,64 +1,57 @@
-import { VSCodeButton, VSCodeDivider, VSCodeLink, VSCodeTextField } from "@vscode/webview-ui-toolkit/react"
-import React, { useState } from "react"
-import { useEffectOnce } from "react-use"
-import { vscode } from "../utilities/vscode"
+import { VSCodeButton, VSCodeLink, VSCodeTextArea, VSCodeTextField } from "@vscode/webview-ui-toolkit/react"
+import React, { useEffect, useState } from "react"
+import { ApiConfiguration } from "../../../src/shared/api"
+import { validateApiConfiguration, validateMaxRequestsPerTask } from "../utils/validate"
+import { vscode } from "../utils/vscode"
+import ApiOptions from "./ApiOptions"
 
 type SettingsViewProps = {
-	apiKey: string
-	setApiKey: React.Dispatch<React.SetStateAction<string>>
+	version: string
+	apiConfiguration?: ApiConfiguration
+	setApiConfiguration: React.Dispatch<React.SetStateAction<ApiConfiguration | undefined>>
 	maxRequestsPerTask: string
 	setMaxRequestsPerTask: React.Dispatch<React.SetStateAction<string>>
-	onDone: () => void // Define the type of the onDone prop
+	customInstructions: string
+	setCustomInstructions: React.Dispatch<React.SetStateAction<string>>
+	onDone: () => void
 }
 
-const SettingsView = ({ apiKey, setApiKey, maxRequestsPerTask, setMaxRequestsPerTask, onDone }: SettingsViewProps) => {
-	const [apiKeyErrorMessage, setApiKeyErrorMessage] = useState<string | undefined>(undefined)
+const SettingsView = ({
+	version,
+	apiConfiguration,
+	setApiConfiguration,
+	maxRequestsPerTask,
+	setMaxRequestsPerTask,
+	customInstructions,
+	setCustomInstructions,
+	onDone,
+}: SettingsViewProps) => {
+	const [apiErrorMessage, setApiErrorMessage] = useState<string | undefined>(undefined)
 	const [maxRequestsErrorMessage, setMaxRequestsErrorMessage] = useState<string | undefined>(undefined)
 
-	const disableDoneButton = apiKeyErrorMessage != null || maxRequestsErrorMessage != null
-
-	const handleApiKeyChange = (event: any) => {
-		const input = event.target.value
-		setApiKey(input)
-		validateApiKey(input)
-	}
-
-	const validateApiKey = (value: string) => {
-		if (value.trim() === "") {
-			setApiKeyErrorMessage("API Key cannot be empty")
-		} else {
-			setApiKeyErrorMessage(undefined)
-		}
-	}
-
-	const handleMaxRequestsChange = (event: any) => {
-		const input = event.target.value
-		setMaxRequestsPerTask(input)
-		validateMaxRequests(input)
-	}
-
-	const validateMaxRequests = (value: string | undefined) => {
-		if (value?.trim()) {
-			const num = Number(value)
-			if (isNaN(num)) {
-				setMaxRequestsErrorMessage("Maximum requests must be a number")
-			} else if (num < 3 || num > 100) {
-				setMaxRequestsErrorMessage("Maximum requests must be between 3 and 100")
-			} else {
-				setMaxRequestsErrorMessage(undefined)
-			}
-		} else {
-			setMaxRequestsErrorMessage(undefined)
-		}
-	}
-
 	const handleSubmit = () => {
-		vscode.postMessage({ type: "apiKey", text: apiKey })
-		vscode.postMessage({ type: "maxRequestsPerTask", text: maxRequestsPerTask })
+		const apiValidationResult = validateApiConfiguration(apiConfiguration)
+		const maxRequestsValidationResult = validateMaxRequestsPerTask(maxRequestsPerTask)
 
-		onDone()
+		setApiErrorMessage(apiValidationResult)
+		setMaxRequestsErrorMessage(maxRequestsValidationResult)
+
+		if (!apiValidationResult && !maxRequestsValidationResult) {
+			vscode.postMessage({ type: "apiConfiguration", apiConfiguration })
+			vscode.postMessage({ type: "maxRequestsPerTask", text: maxRequestsPerTask })
+			vscode.postMessage({ type: "customInstructions", text: customInstructions })
+			onDone()
+		}
 	}
 
+	useEffect(() => {
+		setApiErrorMessage(undefined)
+	}, [apiConfiguration])
+
+	useEffect(() => {
+		setMaxRequestsErrorMessage(undefined)
+	}, [maxRequestsPerTask])
+
 	// validate as soon as the component is mounted
 	/*
 	useEffect will use stale values of variables if they are not included in the dependency array. so trying to use useEffect with a dependency array of only one value for example will use any other variables' old values. In most cases you don't want this, and should opt to use react-use hooks.
@@ -70,106 +63,115 @@ const SettingsView = ({ apiKey, setApiKey, maxRequestsPerTask, setMaxRequestsPer
 
 	If we only want to run code once on mount we can use react-use's useEffectOnce or useMount
 	*/
-	useEffectOnce(() => {
-		const timeoutId = setTimeout(() => {
-			validateApiKey(apiKey)
-			validateMaxRequests(maxRequestsPerTask)
-		}, 1000)
-
-		return () => clearTimeout(timeoutId)
-	})
 
 	return (
-		<div style={{ margin: "0 auto", paddingTop: "10px" }}>
-			<div
-				style={{
-					display: "flex",
-					justifyContent: "space-between",
-					alignItems: "center",
-					marginBottom: "17px",
-				}}>
-				<h3 style={{ color: "var(--vscode-foreground)", margin: 0 }}>Settings</h3>
-				<VSCodeButton onClick={handleSubmit} disabled={disableDoneButton}>
-					Done
-				</VSCodeButton>
-			</div>
+		<div
+			style={{
+				position: "fixed",
+				top: 0,
+				left: 0,
+				right: 0,
+				bottom: 0,
+				padding: "10px 18px 18px 20px",
+				display: "flex",
+				flexDirection: "column",
+				overflow: "hidden",
+			}}>
+			<div style={{ flexGrow: 1, overflow: "auto" }}>
+				<div
+					style={{
+						display: "flex",
+						justifyContent: "space-between",
+						alignItems: "center",
+						marginBottom: "17px",
+					}}>
+					<h3 style={{ color: "var(--vscode-foreground)", margin: 0 }}>Settings</h3>
+					<VSCodeButton onClick={handleSubmit}>Done</VSCodeButton>
+				</div>
 
-			<div style={{ marginBottom: "20px" }}>
-				<VSCodeTextField
-					value={apiKey}
-					style={{ width: "100%" }}
-					placeholder="Enter your Anthropic API Key"
-					onInput={handleApiKeyChange}>
-					<span style={{ fontWeight: "500" }}>Anthropic API Key</span>
-				</VSCodeTextField>
-				{apiKeyErrorMessage && (
+				<div style={{ marginBottom: 5 }}>
+					<ApiOptions
+						apiConfiguration={apiConfiguration}
+						setApiConfiguration={setApiConfiguration}
+						showModelOptions={true}
+					/>
+					{apiErrorMessage && (
+						<p
+							style={{
+								margin: "-5px 0 12px 0",
+								fontSize: "12px",
+								color: "var(--vscode-errorForeground)",
+							}}>
+							{apiErrorMessage}
+						</p>
+					)}
+				</div>
+
+				<div style={{ marginBottom: 15 }}>
+					<VSCodeTextArea
+						value={customInstructions}
+						style={{ width: "100%" }}
+						rows={4}
+						placeholder={
+							'e.g. "Run unit tests at the end", "Use TypeScript with async/await", "Speak in Spanish"'
+						}
+						onInput={(e: any) => setCustomInstructions(e.target?.value || "")}>
+						<span style={{ fontWeight: "500" }}>Custom Instructions</span>
+					</VSCodeTextArea>
 					<p
 						style={{
 							fontSize: "12px",
 							marginTop: "5px",
-							color: "var(--vscode-errorForeground)",
+							color: "var(--vscode-descriptionForeground)",
 						}}>
-						{apiKeyErrorMessage}
+						These instructions are added to the end of the system prompt sent with every request.
 					</p>
-				)}
-				<p
-					style={{
-						fontSize: "12px",
-						marginTop: "5px",
-						color: "var(--vscode-descriptionForeground)",
-					}}>
-					This key is not shared with anyone and only used to make API requests from the extension.
-					<VSCodeLink href="https://console.anthropic.com/" style={{ display: "inline" }}>
-						You can get an API key by signing up here.
-					</VSCodeLink>
-				</p>
-			</div>
+				</div>
 
-			<div style={{ marginBottom: "20px" }}>
-				<VSCodeTextField
-					value={maxRequestsPerTask}
-					style={{ width: "100%" }}
-					placeholder="20"
-					onInput={handleMaxRequestsChange}>
-					<span style={{ fontWeight: "500" }}>Maximum # Requests Per Task</span>
-				</VSCodeTextField>
-				{maxRequestsErrorMessage && (
+				<div style={{ marginBottom: 20 }}>
+					<VSCodeTextField
+						value={maxRequestsPerTask}
+						style={{ width: "100%" }}
+						placeholder="20"
+						onInput={(e: any) => setMaxRequestsPerTask(e.target?.value)}>
+						<span style={{ fontWeight: "500" }}>Maximum # Requests Per Task</span>
+					</VSCodeTextField>
 					<p
 						style={{
 							fontSize: "12px",
 							marginTop: "5px",
-							color: "var(--vscode-errorForeground)",
+							color: "var(--vscode-descriptionForeground)",
 						}}>
-						{maxRequestsErrorMessage}
+						If Claude Dev reaches this limit, it will pause and ask for your permission before making
+						additional requests.
 					</p>
-				)}
-				<p
-					style={{
-						fontSize: "12px",
-						marginTop: "5px",
-						color: "var(--vscode-descriptionForeground)",
-					}}>
-					If Claude Dev reaches this limit, it will pause and ask for your permission before making additional
-					requests.
-				</p>
+					{maxRequestsErrorMessage && (
+						<p
+							style={{
+								fontSize: "12px",
+								marginTop: "5px",
+								color: "var(--vscode-errorForeground)",
+							}}>
+							{maxRequestsErrorMessage}
+						</p>
+					)}
+				</div>
 			</div>
 
-			<VSCodeDivider />
-
 			<div
 				style={{
-					marginTop: "20px",
 					textAlign: "center",
 					color: "var(--vscode-descriptionForeground)",
 					fontSize: "12px",
 					lineHeight: "1.2",
-					fontStyle: "italic",
 				}}>
-				<p>
-					<VSCodeLink href="https://github.com/saoudrizwan/claude-dev">
+				<p style={{ wordWrap: "break-word", margin: 0, padding: 0 }}>
+					If you have any questions or feedback, feel free to open an issue at{" "}
+					<VSCodeLink href="https://github.com/saoudrizwan/claude-dev" style={{ display: "inline" }}>
 						https://github.com/saoudrizwan/claude-dev
 					</VSCodeLink>
 				</p>
+				<p style={{ fontStyle: "italic", margin: "10px 0 0 0", padding: 0 }}>v{version}</p>
 			</div>
 		</div>
 	)

webview-ui/src/components/TaskHeader.tsx

@@ -2,10 +2,12 @@ import { VSCodeButton } from "@vscode/webview-ui-toolkit/react"
 import React, { useEffect, useRef, useState } from "react"
 import TextTruncate from "react-text-truncate"
 import { useWindowSize } from "react-use"
-import { vscode } from "../utilities/vscode"
+import { ClaudeMessage } from "../../../src/shared/ExtensionMessage"
+import { vscode } from "../utils/vscode"
+import Thumbnails from "./Thumbnails"
 
 interface TaskHeaderProps {
-	taskText: string
+	task: ClaudeMessage
 	tokensIn: number
 	tokensOut: number
 	totalCost: number
@@ -13,7 +15,7 @@ interface TaskHeaderProps {
 	isHidden: boolean
 }
 
-const TaskHeader: React.FC<TaskHeaderProps> = ({ taskText, tokensIn, tokensOut, totalCost, onClose, isHidden }) => {
+const TaskHeader: React.FC<TaskHeaderProps> = ({ task, tokensIn, tokensOut, totalCost, onClose, isHidden }) => {
 	const [isExpanded, setIsExpanded] = useState(false)
 	const [textTruncateKey, setTextTruncateKey] = useState(0)
 	const textContainerRef = useRef<HTMLDivElement>(null)
@@ -116,7 +118,7 @@ const TaskHeader: React.FC<TaskHeaderProps> = ({ taskText, tokensIn, tokensOut,
 						line={isExpanded ? 0 : 3}
 						element="span"
 						truncateText="…"
-						text={taskText}
+						text={task.text}
 						textTruncateChild={
 							<span
 								style={{
@@ -141,16 +143,10 @@ const TaskHeader: React.FC<TaskHeaderProps> = ({ taskText, tokensIn, tokensOut,
 						</span>
 					)}
 				</div>
+				{task.images && task.images.length > 0 && <Thumbnails images={task.images} />}
 				<div style={{ display: "flex", flexDirection: "column", gap: "4px" }}>
 					<div style={{ display: "flex", alignItems: "center", gap: "4px" }}>
 						<span style={{ fontWeight: "bold" }}>Tokens:</span>
-						<span style={{ display: "flex", alignItems: "center", gap: "3px" }}>
-							<i
-								className="codicon codicon-arrow-down"
-								style={{ fontSize: "12px", fontWeight: "bold", marginBottom: "-2px" }}
-							/>
-							{tokensOut.toLocaleString()}
-						</span>
 						<span style={{ display: "flex", alignItems: "center", gap: "3px" }}>
 							<i
 								className="codicon codicon-arrow-up"
@@ -158,6 +154,13 @@ const TaskHeader: React.FC<TaskHeaderProps> = ({ taskText, tokensIn, tokensOut,
 							/>
 							{tokensIn.toLocaleString()}
 						</span>
+						<span style={{ display: "flex", alignItems: "center", gap: "3px" }}>
+							<i
+								className="codicon codicon-arrow-down"
+								style={{ fontSize: "12px", fontWeight: "bold", marginBottom: "-2px" }}
+							/>
+							{tokensOut.toLocaleString()}
+						</span>
 					</div>
 					<div
 						style={{

webview-ui/src/components/Thumbnails.tsx

@@ -0,0 +1,91 @@
+import React, { useState, useRef, useLayoutEffect } from "react"
+import { useWindowSize } from "react-use"
+
+interface ThumbnailsProps {
+	images: string[]
+	style?: React.CSSProperties
+	setImages?: React.Dispatch<React.SetStateAction<string[]>>
+	onHeightChange?: (height: number) => void
+}
+
+const Thumbnails: React.FC<ThumbnailsProps> = ({ images, style, setImages, onHeightChange }) => {
+	const [hoveredIndex, setHoveredIndex] = useState<number | null>(null)
+	const containerRef = useRef<HTMLDivElement>(null)
+	const { width } = useWindowSize()
+
+	useLayoutEffect(() => {
+		if (containerRef.current) {
+			let height = containerRef.current.clientHeight
+			// some browsers return 0 for clientHeight
+			if (!height) {
+				height = containerRef.current.getBoundingClientRect().height
+			}
+			onHeightChange?.(height)
+		}
+		setHoveredIndex(null)
+	}, [images, width, onHeightChange])
+
+	const handleDelete = (index: number) => {
+		setImages?.((prevImages) => prevImages.filter((_, i) => i !== index))
+	}
+
+	const isDeletable = setImages !== undefined
+
+	return (
+		<div
+			ref={containerRef}
+			style={{
+				display: "flex",
+				flexWrap: "wrap",
+				gap: 5,
+				rowGap: 3,
+				...style,
+			}}>
+			{images.map((image, index) => (
+				<div
+					key={index}
+					style={{ position: "relative" }}
+					onMouseEnter={() => setHoveredIndex(index)}
+					onMouseLeave={() => setHoveredIndex(null)}>
+					<img
+						src={image}
+						alt={`Thumbnail ${index + 1}`}
+						style={{
+							width: 34,
+							height: 34,
+							objectFit: "cover",
+							borderRadius: 4,
+						}}
+					/>
+					{isDeletable && hoveredIndex === index && (
+						<div
+							onClick={() => handleDelete(index)}
+							style={{
+								position: "absolute",
+								top: -4,
+								right: -4,
+								width: 13,
+								height: 13,
+								borderRadius: "50%",
+								backgroundColor: "var(--vscode-badge-background)",
+								display: "flex",
+								justifyContent: "center",
+								alignItems: "center",
+								cursor: "pointer",
+							}}>
+							<span
+								className="codicon codicon-close"
+								style={{
+									color: "var(--vscode-foreground)",
+									fontSize: 10,
+									fontWeight: "bold",
+								}}></span>
+						</div>
+					)}
+				</div>
+			))}
+		</div>
+	)
+}
+
+export default Thumbnails

webview-ui/src/components/WelcomeView.tsx

@@ -1,32 +1,27 @@
-import React, { useState, useEffect } from "react"
-import { VSCodeButton, VSCodeTextField, VSCodeLink, VSCodeDivider } from "@vscode/webview-ui-toolkit/react"
-import { vscode } from "../utilities/vscode"
+import { VSCodeButton, VSCodeLink } from "@vscode/webview-ui-toolkit/react"
+import React, { useEffect, useState } from "react"
+import { ApiConfiguration } from "../../../src/shared/api"
+import { validateApiConfiguration } from "../utils/validate"
+import { vscode } from "../utils/vscode"
+import ApiOptions from "./ApiOptions"
 
 interface WelcomeViewProps {
-	apiKey: string
-	setApiKey: React.Dispatch<React.SetStateAction<string>>
+	apiConfiguration?: ApiConfiguration
+	setApiConfiguration: React.Dispatch<React.SetStateAction<ApiConfiguration | undefined>>
 }
 
-const WelcomeView: React.FC<WelcomeViewProps> = ({ apiKey, setApiKey }) => {
-	const [apiKeyErrorMessage, setApiKeyErrorMessage] = useState<string | undefined>(undefined)
+const WelcomeView: React.FC<WelcomeViewProps> = ({ apiConfiguration, setApiConfiguration }) => {
+	const [apiErrorMessage, setApiErrorMessage] = useState<string | undefined>(undefined)
 
-	const disableLetsGoButton = apiKeyErrorMessage != null
-
-	const validateApiKey = (value: string) => {
-		if (value.trim() === "") {
-			setApiKeyErrorMessage("API Key cannot be empty")
-		} else {
-			setApiKeyErrorMessage(undefined)
-		}
-	}
+	const disableLetsGoButton = apiErrorMessage != null
 
 	const handleSubmit = () => {
-		vscode.postMessage({ type: "apiKey", text: apiKey })
+		vscode.postMessage({ type: "apiConfiguration", apiConfiguration })
 	}
 
 	useEffect(() => {
-		validateApiKey(apiKey)
-	}, [apiKey])
+		setApiErrorMessage(validateApiConfiguration(apiConfiguration))
+	}, [apiConfiguration])
 
 	return (
 		<div style={{ position: "fixed", top: 0, left: 0, right: 0, bottom: 0, padding: "0 20px" }}>
@@ -38,39 +33,22 @@ const WelcomeView: React.FC<WelcomeViewProps> = ({ apiKey, setApiKey }) => {
 					style={{ display: "inline" }}>
 					Claude 3.5 Sonnet's agentic coding capabilities.
 				</VSCodeLink>{" "}
-				I am prompted to think through tasks step-by-step and have access to tools that let me read & write
-				files, analyze project source code, and execute terminal commands (with your permission, of course).
+				I am prompted to think through tasks step-by-step and have access to tools that let me create & edit
+				files, explore complex projects, and execute terminal commands (with your permission, of course).
 			</p>
 
-			<b>To get started, this extension needs an Anthropic API key:</b>
-			<ol style={{ paddingLeft: "15px" }}>
-				<li>
-					Go to{" "}
-					<VSCodeLink href="https://console.anthropic.com" style={{ display: "inline" }}>
-						https://console.anthropic.com
-					</VSCodeLink>
-				</li>
-				<li>You may need to buy some credits (although Anthropic is offering $5 free credit for new users)</li>
-				<li>Click 'Get API Keys' and create a new key (you can delete it any time)</li>
-			</ol>
+			<b>To get started, this extension needs an API key for Claude 3.5 Sonnet:</b>
 
-			<VSCodeDivider />
-
-			<div style={{ marginTop: "20px", display: "flex", alignItems: "center" }}>
-				<VSCodeTextField
-					style={{ flexGrow: 1, marginRight: "10px" }}
-					placeholder="Enter API Key..."
-					value={apiKey}
-					onInput={(e: any) => setApiKey(e.target.value)}
+			<div style={{ marginTop: "15px" }}>
+				<ApiOptions
+					apiConfiguration={apiConfiguration}
+					setApiConfiguration={setApiConfiguration}
+					showModelOptions={false}
 				/>
-				<VSCodeButton onClick={handleSubmit} disabled={disableLetsGoButton}>
-					Submit
+				<VSCodeButton onClick={handleSubmit} disabled={disableLetsGoButton} style={{ marginTop: "3px" }}>
+					Let's go!
 				</VSCodeButton>
 			</div>
-
-			<p style={{ fontSize: "12px", marginTop: "10px", color: "var(--vscode-descriptionForeground)" }}>
-				Your API key is stored securely on your computer and used only for interacting with the Anthropic API.
-			</p>
 		</div>
 	)
 }

webview-ui/src/index.css

@@ -109,3 +109,31 @@ The above scrollbar styling uses some transparent background color magic to acco
 .code-block-scrollable::-webkit-scrollbar-corner {
 	background-color: transparent;
 }
+
+/*
+Dropdown label
+https://github.com/microsoft/vscode-webview-ui-toolkit/tree/main/src/dropdown#with-label
+*/
+.dropdown-container {
+	box-sizing: border-box;
+	display: flex;
+	flex-flow: column nowrap;
+	align-items: flex-start;
+	justify-content: flex-start;
+}
+.dropdown-container label {
+	display: block;
+	color: var(--vscode-foreground);
+	cursor: pointer;
+	font-size: var(--vscode-font-size);
+	line-height: normal;
+	margin-bottom: 2px;
+}
+
+/* Fix scrollbar in dropdown */
+
+vscode-dropdown::part(listbox) {
+	border-color: var(--vscode-scrollbarSlider-background);
+	transition: none;
+	scrollbar-color: var(--vscode-scrollbarSlider-background) transparent;
+}

webview-ui/src/utils/combineApiRequests.ts

@@ -1,15 +1,15 @@
-import { ClaudeMessage } from "@shared/ExtensionMessage"
+import { ClaudeMessage } from "../../../src/shared/ExtensionMessage"
 
 /**
  * Combines API request start and finish messages in an array of ClaudeMessages.
- * 
+ *
  * This function looks for pairs of 'api_req_started' and 'api_req_finished' messages.
  * When it finds a pair, it combines them into a single 'api_req_combined' message.
  * The JSON data in the text fields of both messages are merged.
- * 
+ *
  * @param messages - An array of ClaudeMessage objects to process.
  * @returns A new array of ClaudeMessage objects with API requests combined.
- * 
+ *
  * @example
  * const messages = [
  *   { type: "say", say: "api_req_started", text: '{"request":"GET /api/data"}', ts: 1000 },

webview-ui/src/utils/combineCommandSequences.ts

@@ -1,4 +1,4 @@
-import { ClaudeMessage } from "@shared/ExtensionMessage"
+import { ClaudeMessage } from "../../../src/shared/ExtensionMessage"
 
 /**
  * Combines sequences of command and command_output messages in an array of ClaudeMessages.
@@ -41,7 +41,11 @@ export function combineCommandSequences(messages: ClaudeMessage[]): ClaudeMessag
 						combinedText += `\n${COMMAND_OUTPUT_STRING}`
 						didAddOutput = true
 					}
-					combinedText += "\n" + (messages[j].text || "")
+					// handle cases where we receive empty command_output (ie when extension is relinquishing control over exit command button)
+					const output = messages[j].text || ""
+					if (output.length > 0) {
+						combinedText += "\n" + output
+					}
 				}
 				j++
 			}

webview-ui/src/utils/getApiMetrics.ts

@@ -1,4 +1,4 @@
-import { ClaudeMessage } from "@shared/ExtensionMessage"
+import { ClaudeMessage } from "../../../src/shared/ExtensionMessage"
 
 interface ApiMetrics {
 	totalTokensIn: number

webview-ui/src/utils/getLanguageFromPath.ts

@@ -85,6 +85,5 @@ const extensionToLanguage: { [key: string]: string } = {
 
 export function getLanguageFromPath(path: string): string | undefined {
 	const extension = path.split(".").pop()?.toLowerCase() || ""
-    console.log(`converted path to language: ${path} -> ${extensionToLanguage[extension]}`)
 	return extensionToLanguage[extension]
 }

webview-ui/src/utils/mockMessages.ts

@@ -1,4 +1,4 @@
-import { ClaudeMessage } from "@shared/ExtensionMessage";
+import { ClaudeMessage } from "../../../src/shared/ExtensionMessage"
 
 export const mockMessages: ClaudeMessage[] = [
 	{

webview-ui/src/utils/validate.ts

@@ -0,0 +1,34 @@
+import { ApiConfiguration } from "../../../src/shared/api"
+
+export function validateApiConfiguration(apiConfiguration?: ApiConfiguration): string | undefined {
+	if (apiConfiguration) {
+		switch (apiConfiguration.apiProvider) {
+			case "anthropic":
+				if (!apiConfiguration.apiKey) {
+					return "You must provide a valid API key or choose a different provider."
+				}
+				break
+			case "bedrock":
+				if (!apiConfiguration.awsAccessKey || !apiConfiguration.awsSecretKey || !apiConfiguration.awsRegion) {
+					return "You must provide a valid AWS access key, secret key, and region."
+				}
+				break
+			case "openrouter":
+				if (!apiConfiguration.openRouterApiKey) {
+					return "You must provide a valid API key or choose a different provider."
+				}
+				break
+		}
+	}
+	return undefined
+}
+
+export function validateMaxRequestsPerTask(maxRequestsPerTask?: string): string | undefined {
+	if (maxRequestsPerTask && maxRequestsPerTask.trim()) {
+		const num = Number(maxRequestsPerTask)
+		if (isNaN(num) || num < 3 || num > 100) {
+			return "Maximum requests must be between 3 and 100"
+		}
+	}
+	return undefined
+}

webview-ui/src/utils/vscode.ts

@@ -1,4 +1,4 @@
-import { WebviewMessage } from "@shared/WebviewMessage"
+import { WebviewMessage } from "../../../src/shared/WebviewMessage"
 import type { WebviewApi } from "vscode-webview"
 
 /**

webview-ui/tsconfig.json

@@ -14,10 +14,7 @@
 		"resolveJsonModule": true,
 		"isolatedModules": true,
 		"noEmit": true,
-		"jsx": "react-jsx",
-		"paths": {
-			"@shared/*": ["../src/shared/*"]
-		}
+		"jsx": "react-jsx"
 	},
 	"include": ["src", "../src/shared"]
 }