Update version

CHANGELOG.md

@@ -4,6 +4,19 @@ All notable changes to the "claude-dev" extension will be documented in this fil
 
 <!-- Check [Keep a Changelog](http://keepachangelog.com/) for recommendations on how to structure this file. -->
 
+## [1.1.1]
+
+- Adds option to choose other Claude models (+ GPT-4o, DeepSeek, and Mistral if you use OpenRouter)
+- Adds option to add custom instructions to the end of the system prompt
+
+## [1.1.0]
+
+- Paste images in chat to use Claude's vision capabilities and turn mockups into fully functional applications or fix bugs with screenshots
+
+## [1.0.9]
+
+- Add support for OpenRouter and AWS Bedrock
+
 ## [1.0.8]
 
 - Shows diff view of new or edited files right in the editor

README.md

@@ -8,35 +8,34 @@
   <a href="https://marketplace.visualstudio.com/items?itemName=saoudrizwan.claude-dev"><strong>Download VSCode Extension</strong></a>
 </p>
 
-Thanks to [Claude 3.5 Sonnet's agentic coding capabilities](https://www-cdn.anthropic.com/fed9cc193a14b84131812372d8d5857f8f304c52/Model_Card_Claude_3_Addendum.pdf) Claude Dev can handle complex software development tasks step-by-step. With tools that let him read & write files, analyze project source code, and execute terminal commands (after you grant permission), he can assist you in ways that go beyond simple code completion or tech support. From building software projects to running system operations, Claude Dev is only limited by your imagination.
-
-While autonomous AI scripts traditionally run in sandboxed environments, Claude Dev provides a human-in-the-loop GUI to supervise every file changed and command executed, providing a safe and accessible way to explore the potential of agentic AI.
+Thanks to [Claude 3.5 Sonnet's agentic coding capabilities](https://www-cdn.anthropic.com/fed9cc193a14b84131812372d8d5857f8f304c52/Model_Card_Claude_3_Addendum.pdf) Claude Dev can handle complex software development tasks step-by-step. With tools that let him create & edit files, explore complex projects, and execute terminal commands (after you grant permission), he can assist you in ways that go beyond simple code completion or tech support. While autonomous AI scripts traditionally run in sandboxed environments, Claude Dev provides a human-in-the-loop GUI to supervise every file changed and command executed, providing a safe and accessible way to explore the potential of agentic AI.
 
+-   Paste images in chat to use Claude's vision capabilities and turn mockups into fully functional applications or fix bugs with screenshots
 -   Inspect diffs of every change Claude makes right in the editor, and keep track with syntax highlighted previews in chat
--   Runs CLI commands directly in chat, so you never have to open a terminal yourself (+ respond to interactive commands by sending a message)
--   Presents permission buttons (i.e. 'Approve CLI command') before tool use or sending information to the API
+-   Runs terminal commands directly in chat, so you never have to open a terminal yourself (+ respond to interactive commands by sending a message)
+-   Presents permission buttons (i.e. 'Approve terminal command') before tool use or sending information to the API
 -   Keep track of total tokens and API usage cost for the entire task loop and individual requests
 -   Set a maximum # of API requests allowed for a task before being prompted for permission to proceed
--   When a task is completed, Claude determines if he can present the result to you with a CLI command like `open -a "Google Chrome" index.html`, which you run with a click of a button
+-   When a task is completed, Claude determines if he can present the result to you with a terminal command like `open -a "Google Chrome" index.html`, which you run with a click of a button
 
 _**Pro tip**: Use the `Cmd + Shift + P` shortcut to open the command palette and type `Claude Dev: Open In New Tab` to start a new task right in your editor._
 
 ## How it works
 
-Claude Dev uses an agentic loop style implementation with chain-of-thought prompting and access to powerful tools that give him the ability to accomplish nearly any task. Start by providing a task and the agentic loop fires off, where it might use certain tools (with your permission) to accomplish each step in its thought process.
+Claude Dev uses an autonomous task execution loop with chain-of-thought prompting and access to powerful tools that give him the ability to accomplish nearly any task. Start by providing a task and the loop fires off, where it might use certain tools (with your permission) to accomplish each step in its thought process.
 
 ### Tools
 
 Claude Dev has access to the following capabilities:
 
-1. **`execute_command`**: Execute CLI commands on the system (only with your permission, output is streamed into the chat and you can respond to stdin or exit long-running processes when you're ready)
+1. **`execute_command`**: Execute terminal commands on the system (only with your permission, output is streamed into the chat and you can respond to stdin or exit long-running processes when you're ready)
 2. **`list_files_top_level`**: List all paths for files at the top level of the specified directory (useful for generic file operations like retrieving a file from your Desktop)
 3. **`list_files_recursive`**: List all paths for files in the specified directory and nested subdirectories (excludes files in .gitignore)
 4. **`view_source_code_definitions_top_level`**: Parses all source code files at the top level of the specified directory to extract names of key elements like classes and functions (see more below)
 5. **`read_file`**: Read the contents of a file at the specified path
 6. **`write_to_file`**: Write content to a file at the specified path, automatically creating any necessary directories
 7. **`ask_followup_question`**: Ask the user a question to gather additional information needed to complete a task (due to the autonomous nature of the program, this isn't a typical chatbot–Claude Dev must explicitly interrupt his task loop to ask for more information)
-8. **`attempt_completion`**: Present the result to the user after completing a task, potentially with a CLI command to kickoff a demonstration
+8. **`attempt_completion`**: Present the result to the user after completing a task, potentially with a terminal command to kickoff a demonstration
 
 ### Working in Existing Projects
 
@@ -60,7 +59,8 @@ Claude always asks for your permission first before any tools are executed or in
 
 ## Contribution
 
-Feel free to contribute to this project by submitting issues and pull requests. Contributions are welcome and appreciated!
+Paul Graham said it best, "if you build something now that barely works with AI, the next models will make it _really_ work." I've built this project with the assumption that scaling laws will continue to improve the quality (and cost) of AI models, and what might be difficult for Claude 3.5 Sonnet today will be effortless for future generations. That is the design philosophy I'd like to develop this project with, so it will always be updated with the best models, tools, and capabilities available–without wasting effort on implementing stopgaps like cheaper agents. With that said, I'm always open to suggestions and feedback, so please feel free to contribute to this project by submitting issues and pull requests. Contributions are welcome and appreciated!
+
 To build Claude Dev locally, follow these steps:
 
 1. Clone the repository:
@@ -80,10 +80,17 @@ To build Claude Dev locally, follow these steps:
 ## Reviews
 
 -   ["Claude Sonnet 3.5 Artifacts in VSCode With This Extension"](https://www.youtube.com/watch?v=5FbZ8ALfSTs) by [CoderOne](https://www.youtube.com/@CoderOne)
--   ["Meet Claude Dev — An Open-Source AI Programmer In VS Code"](https://generativeai.pub/meet-claude-dev-an-open-source-autonomous-ai-programmer-in-vs-code-f457f9821b7b) by [Jim Clyde Monge](https://jimclydemonge.medium.com/)
+-   ["ClaudeDev: NEW Coding Agent Can Generate Applications within VS Code!"](https://www.youtube.com/watch?v=UNsQHosbIoE) by [WorldofAI](https://www.youtube.com/@intheworldofai)
+-   ["ClaudeDev: This Mind-Blowing Coding Agent Can Build SaaS Apps in Minutes!"](https://www.youtube.com/watch?v=Ki0nuOeUpT0) by [AI for Devs](https://www.youtube.com/@ai-for-devs)
+-   ["Claude Dev: This Coding Agent can Generate Applications Within VS Code!"](https://www.youtube.com/watch?v=ufq6sHGe0zs) by [AICodeKing](https://www.youtube.com/@AICodeKing)
+-   ["ClaudeDev + NextJS + Supabase: Generate FULL-STACK Apps with Claude 3.5 Sonnet"](https://www.youtube.com/watch?v=GeZBfO1kxA4) by [AICodeKing](https://www.youtube.com/@AICodeKing)
+-   ["Meet Claude Dev — An Open-Source AI Programmer In VS Code"](https://generativeai.pub/meet-claude-dev-an-open-source-autonomous-ai-programmer-in-vs-code-f457f9821b7b) and ["Build games with zero code using Claude Dev in VS Code"](https://www.youtube.com/watch?v=VT-JYVi81ew) by [Jim Clyde Monge](https://jimclydemonge.medium.com/)
+-   ["Claude Dev Builds NextJS App! Aider, Continue Dev & GitHub Copilot Open-Source Alternative"](https://www.youtube.com/watch?v=Rv0wJZRpnCQ) by [Josh Pocock](https://www.youtube.com/@joshfpocock)
+-   ["Claude Dev Is Mindblowing. The Best Coding Assistant tool?"](https://www.youtube.com/watch?v=Vp1Z3VGZroA) by [Yaron Been](https://www.youtube.com/@ecomxfactor-YaronBeen)
+-   ["ClaudeDev: The Ultimate Coding Agent for VS Code"](https://www.youtube.com/watch?v=aq0yw_DtphQ) by [Blas](https://www.youtube.com/@blascerecer)
 -   ["AI Development with Claude Dev"](https://www.linkedin.com/pulse/ai-development-claude-dev-shannon-lal-3ql3e/) by Shannon Lal
 -   ["Code Smarter with Claude Dev: An AI Programmer for Your Projects"](https://www.linkedin.com/pulse/code-smarter-claude-dev-ai-programmer-your-projects-iana-detochka-jiqpe) by Iana D.
--   [Claude Dev also hit top 10 posts of all time on r/ClaudeAI (thank you for all the lovely comments)](https://www.reddit.com/r/ClaudeAI/comments/1e3h0f1/my_submission_to_anthropics_build_with_claude/)
+-   [Claude Dev also hit top 10 posts of all time on r/ClaudeAI](https://www.reddit.com/r/ClaudeAI/comments/1e3h0f1/my_submission_to_anthropics_build_with_claude/)
 
 ## License
 

package.json

@@ -2,7 +2,7 @@
   "name": "claude-dev",
   "displayName": "Claude Dev",
   "description": "Autonomous software engineer right in your IDE, capable of creating/editing files, executing commands, and more with your permission every step of the way.",
-  "version": "1.0.82",
+  "version": "1.1.14",
   "icon": "icon.png",
   "engines": {
     "vscode": "^1.84.0"
@@ -122,16 +122,20 @@
     "typescript": "^5.4.5"
   },
   "dependencies": {
+    "@anthropic-ai/bedrock-sdk": "^0.10.2",
     "@anthropic-ai/sdk": "^0.24.3",
     "@vscode/codicons": "^0.0.36",
     "default-shell": "^2.2.0",
+    "delay": "^6.0.0",
     "diff": "^5.2.0",
     "execa": "^9.3.0",
     "globby": "^14.0.2",
+    "openai": "^4.54.0",
     "os-name": "^6.0.0",
     "p-wait-for": "^5.0.2",
     "serialize-error": "^11.0.3",
+    "tree-kill": "^1.2.2",
     "tree-sitter-wasms": "^0.1.11",
     "web-tree-sitter": "^0.22.6"
   }
-}
+}

src/api/anthropic.ts

@@ -0,0 +1,64 @@
+import { Anthropic } from "@anthropic-ai/sdk"
+import { ApiHandler, withoutImageData } from "."
+import { anthropicDefaultModelId, AnthropicModelId, anthropicModels, ApiHandlerOptions, ModelInfo } from "../shared/api"
+
+export class AnthropicHandler implements ApiHandler {
+	private options: ApiHandlerOptions
+	private client: Anthropic
+
+	constructor(options: ApiHandlerOptions) {
+		this.options = options
+		this.client = new Anthropic({ apiKey: this.options.apiKey })
+	}
+
+	async createMessage(
+		systemPrompt: string,
+		messages: Anthropic.Messages.MessageParam[],
+		tools: Anthropic.Messages.Tool[]
+	): Promise<Anthropic.Messages.Message> {
+		return await this.client.messages.create(
+			{
+				model: this.getModel().id,
+				max_tokens: this.getModel().info.maxTokens,
+				system: systemPrompt,
+				messages,
+				tools,
+				tool_choice: { type: "auto" },
+			},
+			// https://x.com/alexalbert__/status/1812921642143900036
+			// https://github.com/anthropics/anthropic-sdk-typescript?tab=readme-ov-file#default-headers
+			this.getModel().id === "claude-3-5-sonnet-20240620"
+				? {
+						headers: { "anthropic-beta": "max-tokens-3-5-sonnet-2024-07-15" },
+				  }
+				: undefined
+		)
+	}
+
+	createUserReadableRequest(
+		userContent: Array<
+			| Anthropic.TextBlockParam
+			| Anthropic.ImageBlockParam
+			| Anthropic.ToolUseBlockParam
+			| Anthropic.ToolResultBlockParam
+		>
+	): any {
+		return {
+			model: this.getModel().id,
+			max_tokens: this.getModel().info.maxTokens,
+			system: "(see SYSTEM_PROMPT in src/ClaudeDev.ts)",
+			messages: [{ conversation_history: "..." }, { role: "user", content: withoutImageData(userContent) }],
+			tools: "(see tools in src/ClaudeDev.ts)",
+			tool_choice: { type: "auto" },
+		}
+	}
+
+	getModel(): { id: AnthropicModelId; info: ModelInfo } {
+		const modelId = this.options.apiModelId
+		if (modelId && modelId in anthropicModels) {
+			const id = modelId as AnthropicModelId
+			return { id, info: anthropicModels[id] }
+		}
+		return { id: anthropicDefaultModelId, info: anthropicModels[anthropicDefaultModelId] }
+	}
+}

src/api/bedrock.ts

@@ -0,0 +1,66 @@
+import AnthropicBedrock from "@anthropic-ai/bedrock-sdk"
+import { Anthropic } from "@anthropic-ai/sdk"
+import { ApiHandler, withoutImageData } from "."
+import { ApiHandlerOptions, bedrockDefaultModelId, BedrockModelId, bedrockModels, ModelInfo } from "../shared/api"
+
+// https://docs.anthropic.com/en/api/claude-on-amazon-bedrock
+export class AwsBedrockHandler implements ApiHandler {
+	private options: ApiHandlerOptions
+	private client: AnthropicBedrock
+
+	constructor(options: ApiHandlerOptions) {
+		this.options = options
+		this.client = new AnthropicBedrock({
+			// Authenticate by either providing the keys below or use the default AWS credential providers, such as
+			// using ~/.aws/credentials or the "AWS_SECRET_ACCESS_KEY" and "AWS_ACCESS_KEY_ID" environment variables.
+			awsAccessKey: this.options.awsAccessKey,
+			awsSecretKey: this.options.awsSecretKey,
+
+			// awsRegion changes the aws region to which the request is made. By default, we read AWS_REGION,
+			// and if that's not present, we default to us-east-1. Note that we do not read ~/.aws/config for the region.
+			awsRegion: this.options.awsRegion,
+		})
+	}
+
+	async createMessage(
+		systemPrompt: string,
+		messages: Anthropic.Messages.MessageParam[],
+		tools: Anthropic.Messages.Tool[]
+	): Promise<Anthropic.Messages.Message> {
+		return await this.client.messages.create({
+			model: this.getModel().id,
+			max_tokens: this.getModel().info.maxTokens,
+			system: systemPrompt,
+			messages,
+			tools,
+			tool_choice: { type: "auto" },
+		})
+	}
+
+	createUserReadableRequest(
+		userContent: Array<
+			| Anthropic.TextBlockParam
+			| Anthropic.ImageBlockParam
+			| Anthropic.ToolUseBlockParam
+			| Anthropic.ToolResultBlockParam
+		>
+	): any {
+		return {
+			model: this.getModel().id,
+			max_tokens: this.getModel().info.maxTokens,
+			system: "(see SYSTEM_PROMPT in src/ClaudeDev.ts)",
+			messages: [{ conversation_history: "..." }, { role: "user", content: withoutImageData(userContent) }],
+			tools: "(see tools in src/ClaudeDev.ts)",
+			tool_choice: { type: "auto" },
+		}
+	}
+
+	getModel(): { id: BedrockModelId; info: ModelInfo } {
+		const modelId = this.options.apiModelId
+		if (modelId && modelId in bedrockModels) {
+			const id = modelId as BedrockModelId
+			return { id, info: bedrockModels[id] }
+		}
+		return { id: bedrockDefaultModelId, info: bedrockModels[bedrockDefaultModelId] }
+	}
+}

src/api/index.ts

@@ -0,0 +1,66 @@
+import { Anthropic } from "@anthropic-ai/sdk"
+import { ApiConfiguration, ApiModelId, ModelInfo } from "../shared/api"
+import { AnthropicHandler } from "./anthropic"
+import { AwsBedrockHandler } from "./bedrock"
+import { OpenRouterHandler } from "./openrouter"
+
+export interface ApiHandler {
+	createMessage(
+		systemPrompt: string,
+		messages: Anthropic.Messages.MessageParam[],
+		tools: Anthropic.Messages.Tool[]
+	): Promise<Anthropic.Messages.Message>
+
+	createUserReadableRequest(
+		userContent: Array<
+			| Anthropic.TextBlockParam
+			| Anthropic.ImageBlockParam
+			| Anthropic.ToolUseBlockParam
+			| Anthropic.ToolResultBlockParam
+		>
+	): any
+
+	getModel(): { id: ApiModelId; info: ModelInfo }
+}
+
+export function buildApiHandler(configuration: ApiConfiguration): ApiHandler {
+	const { apiProvider, ...options } = configuration
+	switch (apiProvider) {
+		case "anthropic":
+			return new AnthropicHandler(options)
+		case "openrouter":
+			return new OpenRouterHandler(options)
+		case "bedrock":
+			return new AwsBedrockHandler(options)
+		default:
+			return new AnthropicHandler(options)
+	}
+}
+
+export function withoutImageData(
+	userContent: Array<
+		| Anthropic.TextBlockParam
+		| Anthropic.ImageBlockParam
+		| Anthropic.ToolUseBlockParam
+		| Anthropic.ToolResultBlockParam
+	>
+): Array<
+	Anthropic.TextBlockParam | Anthropic.ImageBlockParam | Anthropic.ToolUseBlockParam | Anthropic.ToolResultBlockParam
+> {
+	return userContent.map((part) => {
+		if (part.type === "image") {
+			return { ...part, source: { ...part.source, data: "..." } }
+		} else if (part.type === "tool_result" && typeof part.content !== "string") {
+			return {
+				...part,
+				content: part.content?.map((contentPart) => {
+					if (contentPart.type === "image") {
+						return { ...contentPart, source: { ...contentPart.source, data: "..." } }
+					}
+					return contentPart
+				}),
+			}
+		}
+		return part
+	})
+}

src/api/openrouter.ts

@@ -0,0 +1,396 @@
+import { Anthropic } from "@anthropic-ai/sdk"
+import OpenAI from "openai"
+import { ApiHandler, withoutImageData } from "."
+import {
+	ApiHandlerOptions,
+	ModelInfo,
+	openRouterDefaultModelId,
+	OpenRouterModelId,
+	openRouterModels,
+} from "../shared/api"
+
+export class OpenRouterHandler implements ApiHandler {
+	private options: ApiHandlerOptions
+	private client: OpenAI
+
+	constructor(options: ApiHandlerOptions) {
+		this.options = options
+		this.client = new OpenAI({
+			baseURL: "https://openrouter.ai/api/v1",
+			apiKey: this.options.openRouterApiKey,
+			defaultHeaders: {
+				"HTTP-Referer": "https://github.com/saoudrizwan/claude-dev", // Optional, for including your app on openrouter.ai rankings.
+				"X-Title": "claude-dev", // Optional. Shows in rankings on openrouter.ai.
+			},
+		})
+	}
+
+	async createMessage(
+		systemPrompt: string,
+		messages: Anthropic.Messages.MessageParam[],
+		tools: Anthropic.Messages.Tool[]
+	): Promise<Anthropic.Messages.Message> {
+		// Convert Anthropic messages to OpenAI format
+		const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = [
+			{ role: "system", content: systemPrompt },
+			...this.convertToOpenAiMessages(messages),
+		]
+
+		// Convert Anthropic tools to OpenAI tools
+		const openAiTools: OpenAI.Chat.ChatCompletionTool[] = tools.map((tool) => ({
+			type: "function",
+			function: {
+				name: tool.name,
+				description: tool.description,
+				parameters: tool.input_schema, // matches anthropic tool input schema (see https://platform.openai.com/docs/guides/function-calling)
+			},
+		}))
+
+		const createParams: OpenAI.Chat.Completions.ChatCompletionCreateParamsNonStreaming = {
+			model: this.getModel().id,
+			max_tokens: this.getModel().info.maxTokens,
+			messages: openAiMessages,
+			tools: openAiTools,
+			tool_choice: "auto",
+		}
+
+		let completion: OpenAI.Chat.Completions.ChatCompletion
+		try {
+			completion = await this.client.chat.completions.create(createParams)
+		} catch (error) {
+			console.error("Error creating message from normal request. Using streaming fallback...", error)
+			completion = await this.streamCompletion(createParams)
+		}
+
+		const errorMessage = (completion as any).error?.message // openrouter returns an error object instead of the openai sdk throwing an error
+		if (errorMessage) {
+			throw new Error(errorMessage)
+		}
+
+		// Convert OpenAI response to Anthropic format
+		const openAiMessage = completion.choices[0].message
+		const anthropicMessage: Anthropic.Messages.Message = {
+			id: completion.id,
+			type: "message",
+			role: openAiMessage.role, // always "assistant"
+			content: [
+				{
+					type: "text",
+					text: openAiMessage.content || "",
+				},
+			],
+			model: completion.model,
+			stop_reason: (() => {
+				switch (completion.choices[0].finish_reason) {
+					case "stop":
+						return "end_turn"
+					case "length":
+						return "max_tokens"
+					case "tool_calls":
+						return "tool_use"
+					case "content_filter": // Anthropic doesn't have an exact equivalent
+					default:
+						return null
+				}
+			})(),
+			stop_sequence: null, // which custom stop_sequence was generated, if any (not applicable if you don't use stop_sequence)
+			usage: {
+				input_tokens: completion.usage?.prompt_tokens || 0,
+				output_tokens: completion.usage?.completion_tokens || 0,
+			},
+		}
+
+		if (openAiMessage.tool_calls && openAiMessage.tool_calls.length > 0) {
+			anthropicMessage.content.push(
+				...openAiMessage.tool_calls.map((toolCall): Anthropic.ToolUseBlock => {
+					let parsedInput = {}
+					try {
+						parsedInput = JSON.parse(toolCall.function.arguments || "{}")
+					} catch (error) {
+						console.error("Failed to parse tool arguments:", error)
+					}
+					return {
+						type: "tool_use",
+						id: toolCall.id,
+						name: toolCall.function.name,
+						input: parsedInput,
+					}
+				})
+			)
+		}
+
+		return anthropicMessage
+	}
+
+	convertToOpenAiMessages(
+		anthropicMessages: Anthropic.Messages.MessageParam[]
+	): OpenAI.Chat.ChatCompletionMessageParam[] {
+		const openAiMessages: OpenAI.Chat.ChatCompletionMessageParam[] = []
+
+		for (const anthropicMessage of anthropicMessages) {
+			if (typeof anthropicMessage.content === "string") {
+				openAiMessages.push({ role: anthropicMessage.role, content: anthropicMessage.content })
+			} else {
+				// image_url.url is base64 encoded image data
+				// ensure it contains the content-type of the image: data:image/png;base64,
+				/*
+			{ role: "user", content: "" | { type: "text", text: string } | { type: "image_url", image_url: { url: string } } },
+			 // content required unless tool_calls is present
+			{ role: "assistant", content?: "" | null, tool_calls?: [{ id: "", function: { name: "", arguments: "" }, type: "function" }] },
+			{ role: "tool", tool_call_id: "", content: ""}
+			 */
+				if (anthropicMessage.role === "user") {
+					const { nonToolMessages, toolMessages } = anthropicMessage.content.reduce<{
+						nonToolMessages: (Anthropic.TextBlockParam | Anthropic.ImageBlockParam)[]
+						toolMessages: Anthropic.ToolResultBlockParam[]
+					}>(
+						(acc, part) => {
+							if (part.type === "tool_result") {
+								acc.toolMessages.push(part)
+							} else if (part.type === "text" || part.type === "image") {
+								acc.nonToolMessages.push(part)
+							} // user cannot send tool_use messages
+							return acc
+						},
+						{ nonToolMessages: [], toolMessages: [] }
+					)
+
+					// Process non-tool messages
+					if (nonToolMessages.length > 0) {
+						openAiMessages.push({
+							role: "user",
+							content: nonToolMessages.map((part) => {
+								if (part.type === "image") {
+									return {
+										type: "image_url",
+										image_url: { url: `data:${part.source.media_type};base64,${part.source.data}` },
+									}
+								}
+								return { type: "text", text: part.text }
+							}),
+						})
+					}
+
+					// Process tool result messages
+					let toolResultImages: Anthropic.Messages.ImageBlockParam[] = []
+					toolMessages.forEach((toolMessage) => {
+						// The Anthropic SDK allows tool results to be a string or an array of text and image blocks, enabling rich and structured content. In contrast, the OpenAI SDK only supports tool results as a single string, so we map the Anthropic tool result parts into one concatenated string to maintain compatibility.
+						let content: string
+
+						if (typeof toolMessage.content === "string") {
+							content = toolMessage.content
+						} else {
+							content =
+								toolMessage.content
+									?.map((part) => {
+										if (part.type === "image") {
+											toolResultImages.push(part)
+											return "(see following user message for image)"
+										}
+										return part.text
+									})
+									.join("\n") ?? ""
+						}
+						openAiMessages.push({
+							role: "tool",
+							tool_call_id: toolMessage.tool_use_id,
+							content: content,
+						})
+					})
+
+					// If tool results contain images, send as a separate user message
+					// I ran into an issue where if I gave feedback for one of many tool uses, the request would fail.
+					// "Messages following `tool_use` blocks must begin with a matching number of `tool_result` blocks."
+					// Therefore we need to send these images after the tool result messages
+					if (toolResultImages.length > 0) {
+						openAiMessages.push({
+							role: "user",
+							content: toolResultImages.map((part) => ({
+								type: "image_url",
+								image_url: { url: `data:${part.source.media_type};base64,${part.source.data}` },
+							})),
+						})
+					}
+				} else if (anthropicMessage.role === "assistant") {
+					const { nonToolMessages, toolMessages } = anthropicMessage.content.reduce<{
+						nonToolMessages: (Anthropic.TextBlockParam | Anthropic.ImageBlockParam)[]
+						toolMessages: Anthropic.ToolUseBlockParam[]
+					}>(
+						(acc, part) => {
+							if (part.type === "tool_use") {
+								acc.toolMessages.push(part)
+							} else if (part.type === "text" || part.type === "image") {
+								acc.nonToolMessages.push(part)
+							} // assistant cannot send tool_result messages
+							return acc
+						},
+						{ nonToolMessages: [], toolMessages: [] }
+					)
+
+					// Process non-tool messages
+					let content: string | undefined
+					if (nonToolMessages.length > 0) {
+						content = nonToolMessages
+							.map((part) => {
+								if (part.type === "image") {
+									return "" // impossible as the assistant cannot send images
+								}
+								return part.text
+							})
+							.join("\n")
+					}
+
+					// Process tool use messages
+					let tool_calls: OpenAI.Chat.ChatCompletionMessageToolCall[] = toolMessages.map((toolMessage) => ({
+						id: toolMessage.id,
+						type: "function",
+						function: {
+							name: toolMessage.name,
+							// json string
+							arguments: JSON.stringify(toolMessage.input),
+						},
+					}))
+
+					openAiMessages.push({
+						role: "assistant",
+						content,
+						// Cannot be an empty array. API expects an array with minimum length 1, and will respond with an error if it's empty
+						tool_calls: tool_calls.length > 0 ? tool_calls : undefined,
+					})
+				}
+			}
+		}
+
+		return openAiMessages
+	}
+
+	/*
+	Streaming the completion is a fallback behavior for when a normal request responds with an invalid JSON object ("Unexpected end of JSON input"). This would usually happen in cases where the model makes tool calls with large arguments. After talking with OpenRouter folks, streaming mitigates this issue for now until they fix the underlying problem ("some weird data from anthropic got decoded wrongly and crashed the buffer")
+	*/
+	async streamCompletion(
+		createParams: OpenAI.Chat.Completions.ChatCompletionCreateParamsNonStreaming
+	): Promise<OpenAI.Chat.Completions.ChatCompletion> {
+		const stream = await this.client.chat.completions.create({
+			...createParams,
+			stream: true,
+		})
+
+		let textContent: string = ""
+		let toolCalls: OpenAI.Chat.ChatCompletionMessageToolCall[] = []
+
+		try {
+			let currentToolCall: (OpenAI.Chat.ChatCompletionMessageToolCall & { index?: number }) | null = null
+			for await (const chunk of stream) {
+				const delta = chunk.choices[0]?.delta
+				if (delta?.content) {
+					textContent += delta.content
+				}
+				if (delta?.tool_calls) {
+					for (const toolCallDelta of delta.tool_calls) {
+						if (toolCallDelta.index === undefined) {
+							continue
+						}
+						if (!currentToolCall || currentToolCall.index !== toolCallDelta.index) {
+							// new index means new tool call, so add the previous one to the list
+							if (currentToolCall) {
+								toolCalls.push(currentToolCall)
+							}
+							currentToolCall = {
+								index: toolCallDelta.index,
+								id: toolCallDelta.id || "",
+								type: "function",
+								function: { name: "", arguments: "" },
+							}
+						}
+						if (toolCallDelta.id) {
+							currentToolCall.id = toolCallDelta.id
+						}
+						if (toolCallDelta.type) {
+							currentToolCall.type = toolCallDelta.type
+						}
+						if (toolCallDelta.function) {
+							if (toolCallDelta.function.name) {
+								currentToolCall.function.name = toolCallDelta.function.name
+							}
+							if (toolCallDelta.function.arguments) {
+								currentToolCall.function.arguments =
+									(currentToolCall.function.arguments || "") + toolCallDelta.function.arguments
+							}
+						}
+					}
+				}
+			}
+			if (currentToolCall) {
+				toolCalls.push(currentToolCall)
+			}
+		} catch (error) {
+			console.error("Error streaming completion:", error)
+			throw error
+		}
+
+		// Usage information is not available in streaming responses, so we need to estimate token counts
+		function approximateTokenCount(text: string): number {
+			return Math.ceil(new TextEncoder().encode(text).length / 4)
+		}
+		const promptTokens = approximateTokenCount(
+			createParams.messages
+				.map((m) => (typeof m.content === "string" ? m.content : JSON.stringify(m.content)))
+				.join(" ")
+		)
+		const completionTokens = approximateTokenCount(
+			textContent + toolCalls.map((toolCall) => toolCall.function.arguments || "").join(" ")
+		)
+
+		const completion: OpenAI.Chat.Completions.ChatCompletion = {
+			created: Date.now(),
+			object: "chat.completion",
+			id: `openrouter-${Date.now()}-${Math.random().toString(36).slice(2, 11)}`, // this ID won't be traceable back to OpenRouter's systems if you need to debug issues
+			choices: [
+				{
+					message: {
+						role: "assistant",
+						content: textContent,
+						tool_calls: toolCalls.length > 0 ? toolCalls : undefined,
+					},
+					finish_reason: toolCalls.length > 0 ? "tool_calls" : "stop",
+					index: 0,
+					logprobs: null,
+				},
+			],
+			model: this.getModel().id,
+			usage: {
+				prompt_tokens: promptTokens,
+				completion_tokens: completionTokens,
+				total_tokens: promptTokens + completionTokens,
+			},
+		}
+
+		return completion
+	}
+
+	createUserReadableRequest(
+		userContent: Array<
+			| Anthropic.TextBlockParam
+			| Anthropic.ImageBlockParam
+			| Anthropic.ToolUseBlockParam
+			| Anthropic.ToolResultBlockParam
+		>
+	): any {
+		return {
+			model: this.getModel().id,
+			max_tokens: this.getModel().info.maxTokens,
+			messages: [{ conversation_history: "..." }, { role: "user", content: withoutImageData(userContent) }],
+			tools: "(see tools in src/ClaudeDev.ts)",
+			tool_choice: "auto",
+		}
+	}
+
+	getModel(): { id: OpenRouterModelId; info: ModelInfo } {
+		const modelId = this.options.apiModelId
+		if (modelId && modelId in openRouterModels) {
+			const id = modelId as OpenRouterModelId
+			return { id, info: openRouterModels[id] }
+		}
+		return { id: openRouterDefaultModelId, info: openRouterModels[openRouterDefaultModelId] }
+	}
+}

src/parse-source-code/index.ts

@@ -1,10 +1,20 @@
 import * as fs from "fs/promises"
 import { globby } from "globby"
+import os from "os"
 import * as path from "path"
 import { LanguageParser, loadRequiredLanguageParsers } from "./languageParser"
 
 // TODO: implement caching behavior to avoid having to keep analyzing project for new tasks.
 export async function parseSourceCodeForDefinitionsTopLevel(dirPath: string): Promise<string> {
+	// check if the path exists
+	const dirExists = await fs
+		.access(path.resolve(dirPath))
+		.then(() => true)
+		.catch(() => false)
+	if (!dirExists) {
+		return "This directory does not exist or you do not have permission to access it."
+	}
+
 	// Get all files at top level (not gitignored)
 	const allFiles = await listFiles(dirPath, false)
 
@@ -45,11 +55,17 @@ export async function parseSourceCodeForDefinitionsTopLevel(dirPath: string): Pr
 
 export async function listFiles(dirPath: string, recursive: boolean): Promise<string[]> {
 	const absolutePath = path.resolve(dirPath)
+	// Do not allow listing files in root or home directory, which Claude tends to want to do when the user's prompt is vague.
 	const root = process.platform === "win32" ? path.parse(absolutePath).root : "/"
 	const isRoot = absolutePath === root
 	if (isRoot) {
 		return [root]
 	}
+	const homeDir = os.homedir()
+	const isHomeDir = absolutePath === homeDir
+	if (isHomeDir) {
+		return [homeDir]
+	}
 
 	const dirsToIgnore = [
 		"node_modules",
@@ -77,7 +93,7 @@ export async function listFiles(dirPath: string, recursive: boolean): Promise<st
 		markDirectories: true, // Append a / on any directories matched
 		gitignore: recursive, // globby ignores any files that are gitignored
 		ignore: recursive ? dirsToIgnore : undefined, // just in case there is no gitignore, we ignore sensible defaults
-		onlyFiles: recursive, // true by default, false means it will list directories on their own too
+		onlyFiles: false, // true by default, false means it will list directories on their own too
 	}
 	// * globs all files in one dir, ** globs files in nested directories
 	const files = await globby(recursive ? "**" : "*", options)

src/providers/ClaudeDevProvider.ts

@@ -1,27 +1,31 @@
-import { Uri, Webview } from "vscode"
-//import * as weather from "weather-js"
 import * as vscode from "vscode"
 import { ClaudeDev } from "../ClaudeDev"
-import { ClaudeMessage, ExtensionMessage } from "../shared/ExtensionMessage"
+import { ApiModelId, ApiProvider } from "../shared/api"
+import { ExtensionMessage } from "../shared/ExtensionMessage"
 import { WebviewMessage } from "../shared/WebviewMessage"
-import { Anthropic } from "@anthropic-ai/sdk"
-import * as path from "path"
-import os from "os"
-
+import { downloadTask, getNonce, getUri, selectImages } from "../utils"
 /*
 https://github.com/microsoft/vscode-webview-ui-toolkit-samples/blob/main/default/weather-webview/src/providers/WeatherViewProvider.ts
 
 https://github.com/KumarVariable/vscode-extension-sidebar-html/blob/master/src/customSidebarViewProvider.ts
 */
 
+type SecretKey = "apiKey" | "openRouterApiKey" | "awsAccessKey" | "awsSecretKey"
+type GlobalStateKey =
+	| "apiProvider"
+	| "apiModelId"
+	| "awsRegion"
+	| "maxRequestsPerTask"
+	| "lastShownAnnouncementId"
+	| "customInstructions"
+
 export class ClaudeDevProvider implements vscode.WebviewViewProvider {
 	public static readonly sideBarId = "claude-dev.SidebarProvider" // used in package.json as the view's id. This value cannot be changed due to how vscode caches views based on their id, and updating the id would break existing instances of the extension.
 	public static readonly tabPanelId = "claude-dev.TabPanelProvider"
 	private disposables: vscode.Disposable[] = []
 	private view?: vscode.WebviewView | vscode.WebviewPanel
-	private providerInstanceIdentifier = Date.now()
 	private claudeDev?: ClaudeDev
-	private latestAnnouncementId = "jul-29-2024" // update to some unique identifier when we add a new announcement
+	private latestAnnouncementId = "aug-11-2024" // update to some unique identifier when we add a new announcement
 
 	constructor(
 		private readonly context: vscode.ExtensionContext,
@@ -132,15 +136,10 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
 		this.outputChannel.appendLine("Webview view resolved")
 	}
 
-	async tryToInitClaudeDevWithTask(task: string) {
+	async initClaudeDevWithTask(task?: string, images?: string[]) {
 		await this.clearTask() // ensures that an exising task doesn't exist before starting a new one, although this shouldn't be possible since user must clear task before starting a new one
-		const [apiKey, maxRequestsPerTask] = await Promise.all([
-			this.getSecret("apiKey") as Promise<string | undefined>,
-			this.getGlobalState("maxRequestsPerTask") as Promise<number | undefined>,
-		])
-		if (this.view && apiKey) {
-			this.claudeDev = new ClaudeDev(this, task, apiKey, maxRequestsPerTask)
-		}
+		const { maxRequestsPerTask, apiConfiguration, customInstructions } = await this.getState()
+		this.claudeDev = new ClaudeDev(this, apiConfiguration, maxRequestsPerTask, customInstructions, task, images)
 	}
 
 	// Send any JSON serializable data to the react app
@@ -200,7 +199,8 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
         create a content security policy meta tag so that only loading scripts with a nonce is allowed
         As your extension grows you will likely want to add custom styles, fonts, and/or images to your webview. If you do, you will need to update the content security policy meta tag to explicity allow for these resources. E.g.
                 <meta http-equiv="Content-Security-Policy" content="default-src 'none'; style-src ${webview.cspSource}; font-src ${webview.cspSource}; img-src ${webview.cspSource} https:; script-src 'nonce-${nonce}';">
-
+		- 'unsafe-inline' is required for styles due to vscode-webview-toolkit's dynamic style injection
+		- since we pass base64 images to the webview, we need to specify img-src ${webview.cspSource} data:;
 
         in meta tag we add nonce attribute: A cryptographic nonce (only used once) to allow scripts. The server must generate a unique nonce value each time it transmits a policy. It is critical to provide a nonce that cannot be guessed as bypassing a resource's policy is otherwise trivial.
         */
@@ -214,7 +214,7 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
             <meta charset="utf-8">
             <meta name="viewport" content="width=device-width,initial-scale=1,shrink-to-fit=no">
             <meta name="theme-color" content="#000000">
-            <meta http-equiv="Content-Security-Policy" content="default-src 'none'; font-src ${webview.cspSource}; style-src ${webview.cspSource}; script-src 'nonce-${nonce}';">
+            <meta http-equiv="Content-Security-Policy" content="default-src 'none'; font-src ${webview.cspSource}; style-src ${webview.cspSource} 'unsafe-inline'; img-src ${webview.cspSource} data:; script-src 'nonce-${nonce}';">
             <link rel="stylesheet" type="text/css" href="${stylesUri}">
 			<link href="${codiconsUri}" rel="stylesheet" />
             <title>Claude Dev</title>
@@ -250,11 +250,28 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
 						// Could also do this in extension .ts
 						//this.postMessageToWebview({ type: "text", text: `Extension: ${Date.now()}` })
 						// initializing new instance of ClaudeDev will make sure that any agentically running promises in old instance don't affect our new task. this essentially creates a fresh slate for the new task
-						await this.tryToInitClaudeDevWithTask(message.text!)
+						await this.initClaudeDevWithTask(message.text, message.images)
 						break
-					case "apiKey":
-						await this.storeSecret("apiKey", message.text!)
-						this.claudeDev?.updateApiKey(message.text!)
+					case "apiConfiguration":
+						if (message.apiConfiguration) {
+							const {
+								apiProvider,
+								apiModelId,
+								apiKey,
+								openRouterApiKey,
+								awsAccessKey,
+								awsSecretKey,
+								awsRegion,
+							} = message.apiConfiguration
+							await this.updateGlobalState("apiProvider", apiProvider)
+							await this.updateGlobalState("apiModelId", apiModelId)
+							await this.storeSecret("apiKey", apiKey)
+							await this.storeSecret("openRouterApiKey", openRouterApiKey)
+							await this.storeSecret("awsAccessKey", awsAccessKey)
+							await this.storeSecret("awsSecretKey", awsSecretKey)
+							await this.updateGlobalState("awsRegion", awsRegion)
+							this.claudeDev?.updateApi(message.apiConfiguration)
+						}
 						await this.postStateToWebview()
 						break
 					case "maxRequestsPerTask":
@@ -269,8 +286,14 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
 						this.claudeDev?.updateMaxRequestsPerTask(result)
 						await this.postStateToWebview()
 						break
+					case "customInstructions":
+						// User may be clearing the field
+						await this.updateGlobalState("customInstructions", message.text || undefined)
+						this.claudeDev?.updateCustomInstructions(message.text || undefined)
+						await this.postStateToWebview()
+						break
 					case "askResponse":
-						this.claudeDev?.handleWebviewAskResponse(message.askResponse!, message.text)
+						this.claudeDev?.handleWebviewAskResponse(message.askResponse!, message.text, message.images)
 						break
 					case "clearTask":
 						// newTask will start a new task with a given task text, while clear task resets the current session and allows for a new task to be started
@@ -282,7 +305,11 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
 						await this.postStateToWebview()
 						break
 					case "downloadTask":
-						this.downloadTask()
+						downloadTask(this.claudeDev?.apiConversationHistory ?? [])
+						break
+					case "selectImages":
+						const images = await selectImages()
+						await this.postMessageToWebview({ type: "selectedImages", images })
 						break
 					// Add more switch case statements here as more webview message commands
 					// are created within the webview context (i.e. inside media/main.js)
@@ -293,114 +320,32 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
 		)
 	}
 
-	async downloadTask() {
-		// File name
-		const date = new Date()
-		const month = date.toLocaleString("en-US", { month: "short" }).toLowerCase()
-		const day = date.getDate()
-		const year = date.getFullYear()
-		let hours = date.getHours()
-		const minutes = date.getMinutes().toString().padStart(2, "0")
-		const ampm = hours >= 12 ? "pm" : "am"
-		hours = hours % 12
-		hours = hours ? hours : 12 // the hour '0' should be '12'
-		const fileName = `claude_dev_task_${month}-${day}-${year}_${hours}-${minutes}-${ampm}.md`
-
-		// Generate markdown
-		const conversationHistory = await this.getApiConversationHistory()
-		const markdownContent = conversationHistory
-			.map((message) => {
-				const role = message.role === "user" ? "**User:**" : "**Assistant:**"
-				const content = Array.isArray(message.content)
-					? message.content.map(this.formatContentBlockToMarkdown).join("\n")
-					: message.content
-
-				return `${role}\n\n${content}\n\n`
-			})
-			.join("---\n\n")
-
-		// Prompt user for save location
-		const saveUri = await vscode.window.showSaveDialog({
-			filters: { Markdown: ["md"] },
-			defaultUri: vscode.Uri.file(path.join(os.homedir(), "Downloads", fileName)),
-		})
-
-		if (saveUri) {
-			// Write content to the selected location
-			await vscode.workspace.fs.writeFile(saveUri, Buffer.from(markdownContent))
-			vscode.window.showTextDocument(saveUri, { preview: true })
-		}
-	}
-
-	private formatContentBlockToMarkdown(
-		block:
-			| Anthropic.TextBlockParam
-			| Anthropic.ImageBlockParam
-			| Anthropic.ToolUseBlockParam
-			| Anthropic.ToolResultBlockParam
-	): string {
-		switch (block.type) {
-			case "text":
-				return block.text
-			case "image":
-				return `[Image: ${block.source.media_type}]`
-			case "tool_use":
-				let input: string
-				if (typeof block.input === "object" && block.input !== null) {
-					input = Object.entries(block.input)
-						.map(([key, value]) => `${key.charAt(0).toUpperCase() + key.slice(1)}: ${value}`)
-						.join("\n")
-				} else {
-					input = String(block.input)
-				}
-				return `[Tool Use: ${block.name}]\n${input}`
-			case "tool_result":
-				if (typeof block.content === "string") {
-					return `[Tool Result${block.is_error ? " (Error)" : ""}]\n${block.content}`
-				} else if (Array.isArray(block.content)) {
-					return `[Tool Result${block.is_error ? " (Error)" : ""}]\n${block.content
-						.map(this.formatContentBlockToMarkdown)
-						.join("\n")}`
-				} else {
-					return `[Tool Result${block.is_error ? " (Error)" : ""}]`
-				}
-			default:
-				return "[Unexpected content type]"
-		}
-	}
-
 	async postStateToWebview() {
-		const [apiKey, maxRequestsPerTask, claudeMessages, lastShownAnnouncementId] = await Promise.all([
-			this.getSecret("apiKey") as Promise<string | undefined>,
-			this.getGlobalState("maxRequestsPerTask") as Promise<number | undefined>,
-			this.getClaudeMessages(),
-			this.getGlobalState("lastShownAnnouncementId") as Promise<string | undefined>,
-		])
+		const { apiConfiguration, maxRequestsPerTask, lastShownAnnouncementId, customInstructions } =
+			await this.getState()
 		this.postMessageToWebview({
 			type: "state",
 			state: {
-				apiKey,
+				version: this.context.extension?.packageJSON?.version ?? "",
+				apiConfiguration,
 				maxRequestsPerTask,
+				customInstructions,
 				themeName: vscode.workspace.getConfiguration("workbench").get<string>("colorTheme"),
-				claudeMessages,
+				claudeMessages: this.claudeDev?.claudeMessages || [],
 				shouldShowAnnouncement: lastShownAnnouncementId !== this.latestAnnouncementId,
 			},
 		})
 	}
 
 	async clearTask() {
-		if (this.claudeDev) {
-			this.claudeDev.abort = true // will stop any agentically running promises
-			this.claudeDev = undefined // removes reference to it, so once promises end it will be garbage collected
-		}
-		await this.setApiConversationHistory(undefined)
-		await this.setClaudeMessages(undefined)
+		this.claudeDev?.abortTask()
+		this.claudeDev = undefined // removes reference to it, so once promises end it will be garbage collected
 	}
 
 	// Caching mechanism to keep track of webview messages + API conversation history per provider instance
 
 	/*
-	Now that we use retainContextWhenHidden, we don't have to store a cache of claude messages in the user's state, but we do to reduce memory footprint in long conversations.
+	Now that we use retainContextWhenHidden, we don't have to store a cache of claude messages in the user's state, but we could to reduce memory footprint in long conversations.
 
 	- We have to be careful of what state is shared between ClaudeDevProvider instances since there could be multiple instances of the extension running at once. For example when we cached claude messages using the same key, two instances of the extension could end up using the same key and overwriting each other's messages.
 	- Some state does need to be shared between the instances, i.e. the API key--however there doesn't seem to be a good way to notfy the other instances that the API key has changed.
@@ -411,52 +356,66 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
 	However in the future when we implement task history, we'll need to use a unique identifier for each task. As well as manage a data structure that keeps track of task history with their associated identifiers and the task message itself, to present in a 'Task History' view.
 	Task history is a significant undertaking as it would require refactoring how we wait for ask responses--it would need to be a hidden claudeMessage, so that user's can resume tasks that ended with an ask.
 	*/
+	// private providerInstanceIdentifier = Date.now()
+	// getClaudeMessagesStateKey() {
+	// 	return `claudeMessages-${this.providerInstanceIdentifier}`
+	// }
 
-	getClaudeMessagesStateKey() {
-		return `claudeMessages-${this.providerInstanceIdentifier}`
-	}
-
-	getApiConversationHistoryStateKey() {
-		return `apiConversationHistory-${this.providerInstanceIdentifier}`
-	}
+	// getApiConversationHistoryStateKey() {
+	// 	return `apiConversationHistory-${this.providerInstanceIdentifier}`
+	// }
 
 	// claude messages to present in the webview
 
-	async getClaudeMessages(): Promise<ClaudeMessage[]> {
-		const messages = (await this.getGlobalState(this.getClaudeMessagesStateKey())) as ClaudeMessage[]
-		return messages || []
-	}
+	// getClaudeMessages(): ClaudeMessage[] {
+	// 	// const messages = (await this.getGlobalState(this.getClaudeMessagesStateKey())) as ClaudeMessage[]
+	// 	// return messages || []
+	// 	return this.claudeMessages
+	// }
 
-	async setClaudeMessages(messages: ClaudeMessage[] | undefined) {
-		await this.updateGlobalState(this.getClaudeMessagesStateKey(), messages)
-	}
+	// setClaudeMessages(messages: ClaudeMessage[] | undefined) {
+	// 	// await this.updateGlobalState(this.getClaudeMessagesStateKey(), messages)
+	// 	this.claudeMessages = messages || []
+	// }
 
-	async addClaudeMessage(message: ClaudeMessage): Promise<ClaudeMessage[]> {
-		const messages = await this.getClaudeMessages()
-		messages.push(message)
-		await this.setClaudeMessages(messages)
-		return messages
-	}
+	// addClaudeMessage(message: ClaudeMessage): ClaudeMessage[] {
+	// 	// const messages = await this.getClaudeMessages()
+	// 	// messages.push(message)
+	// 	// await this.setClaudeMessages(messages)
+	// 	// return messages
+	// 	this.claudeMessages.push(message)
+	// 	return this.claudeMessages
+	// }
 
 	// conversation history to send in API requests
 
-	async getApiConversationHistory(): Promise<Anthropic.MessageParam[]> {
-		const history = (await this.getGlobalState(
-			this.getApiConversationHistoryStateKey()
-		)) as Anthropic.MessageParam[]
-		return history || []
-	}
+	/*
+	It seems that some API messages do not comply with vscode state requirements. Either the Anthropic library is manipulating these values somehow in the backend in a way thats creating cyclic references, or the API returns a function or a Symbol as part of the message content.
+	VSCode docs about state: "The value must be JSON-stringifyable ... value — A value. MUST not contain cyclic references."
+	For now we'll store the conversation history in memory, and if we need to store in state directly we'd need to do a manual conversion to ensure proper json stringification.
+	*/
 
-	async setApiConversationHistory(history: Anthropic.MessageParam[] | undefined) {
-		await this.updateGlobalState(this.getApiConversationHistoryStateKey(), history)
-	}
+	// getApiConversationHistory(): Anthropic.MessageParam[] {
+	// 	// const history = (await this.getGlobalState(
+	// 	// 	this.getApiConversationHistoryStateKey()
+	// 	// )) as Anthropic.MessageParam[]
+	// 	// return history || []
+	// 	return this.apiConversationHistory
+	// }
 
-	async addMessageToApiConversationHistory(message: Anthropic.MessageParam): Promise<Anthropic.MessageParam[]> {
-		const history = await this.getApiConversationHistory()
-		history.push(message)
-		await this.setApiConversationHistory(history)
-		return history
-	}
+	// setApiConversationHistory(history: Anthropic.MessageParam[] | undefined) {
+	// 	// await this.updateGlobalState(this.getApiConversationHistoryStateKey(), history)
+	// 	this.apiConversationHistory = history || []
+	// }
+
+	// addMessageToApiConversationHistory(message: Anthropic.MessageParam): Anthropic.MessageParam[] {
+	// 	// const history = await this.getApiConversationHistory()
+	// 	// history.push(message)
+	// 	// await this.setApiConversationHistory(history)
+	// 	// return history
+	// 	this.apiConversationHistory.push(message)
+	// 	return this.apiConversationHistory
+	// }
 
 	/*
 	Storage
@@ -464,13 +423,68 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
 	https://www.eliostruyf.com/devhack-code-extension-storage-options/
 	*/
 
+	async getState() {
+		const [
+			storedApiProvider,
+			apiModelId,
+			apiKey,
+			openRouterApiKey,
+			awsAccessKey,
+			awsSecretKey,
+			awsRegion,
+			maxRequestsPerTask,
+			lastShownAnnouncementId,
+			customInstructions,
+		] = await Promise.all([
+			this.getGlobalState("apiProvider") as Promise<ApiProvider | undefined>,
+			this.getGlobalState("apiModelId") as Promise<ApiModelId | undefined>,
+			this.getSecret("apiKey") as Promise<string | undefined>,
+			this.getSecret("openRouterApiKey") as Promise<string | undefined>,
+			this.getSecret("awsAccessKey") as Promise<string | undefined>,
+			this.getSecret("awsSecretKey") as Promise<string | undefined>,
+			this.getGlobalState("awsRegion") as Promise<string | undefined>,
+			this.getGlobalState("maxRequestsPerTask") as Promise<number | undefined>,
+			this.getGlobalState("lastShownAnnouncementId") as Promise<string | undefined>,
+			this.getGlobalState("customInstructions") as Promise<string | undefined>,
+		])
+
+		let apiProvider: ApiProvider
+		if (storedApiProvider) {
+			apiProvider = storedApiProvider
+		} else {
+			// Either new user or legacy user that doesn't have the apiProvider stored in state
+			// (If they're using OpenRouter or Bedrock, then apiProvider state will exist)
+			if (apiKey) {
+				apiProvider = "anthropic"
+			} else {
+				// New users should default to openrouter (better rate limits and wider model selection)
+				apiProvider = "openrouter"
+			}
+		}
+
+		return {
+			apiConfiguration: {
+				apiProvider,
+				apiModelId,
+				apiKey,
+				openRouterApiKey,
+				awsAccessKey,
+				awsSecretKey,
+				awsRegion,
+			},
+			maxRequestsPerTask,
+			lastShownAnnouncementId,
+			customInstructions,
+		}
+	}
+
 	// global
 
-	private async updateGlobalState(key: string, value: any) {
+	private async updateGlobalState(key: GlobalStateKey, value: any) {
 		await this.context.globalState.update(key, value)
 	}
 
-	private async getGlobalState(key: string) {
+	private async getGlobalState(key: GlobalStateKey) {
 		return await this.context.globalState.get(key)
 	}
 
@@ -496,43 +510,15 @@ export class ClaudeDevProvider implements vscode.WebviewViewProvider {
 
 	// secrets
 
-	private async storeSecret(key: string, value: any) {
-		await this.context.secrets.store(key, value)
+	private async storeSecret(key: SecretKey, value?: string) {
+		if (value) {
+			await this.context.secrets.store(key, value)
+		} else {
+			await this.context.secrets.delete(key)
+		}
 	}
 
-	private async getSecret(key: string) {
+	private async getSecret(key: SecretKey) {
 		return await this.context.secrets.get(key)
 	}
 }
-
-/**
- * A helper function that returns a unique alphanumeric identifier called a nonce.
- *
- * @remarks This function is primarily used to help enforce content security
- * policies for resources/scripts being executed in a webview context.
- *
- * @returns A nonce
- */
-export function getNonce() {
-	let text = ""
-	const possible = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
-	for (let i = 0; i < 32; i++) {
-		text += possible.charAt(Math.floor(Math.random() * possible.length))
-	}
-	return text
-}
-
-/**
- * A helper function which will get the webview URI of a given file or resource.
- *
- * @remarks This URI can be used within a webview's HTML as a link to the
- * given file/resource.
- *
- * @param webview A reference to the extension webview
- * @param extensionUri The URI of the directory containing the extension
- * @param pathList An array of strings representing the path to a file/resource
- * @returns A URI pointing to the file/resource
- */
-export function getUri(webview: Webview, extensionUri: Uri, pathList: string[]) {
-	return webview.asWebviewUri(Uri.joinPath(extensionUri, ...pathList))
-}

src/shared/ExtensionMessage.ts

@@ -1,16 +1,21 @@
 // type that represents json data that is sent from extension to webview, called ExtensionMessage and has 'type' enum which can be 'plusButtonTapped' or 'settingsButtonTapped' or 'hello'
 
+import { ApiConfiguration } from "./api"
+
 // webview will hold state
 export interface ExtensionMessage {
-	type: "action" | "state"
+	type: "action" | "state" | "selectedImages"
 	text?: string
 	action?: "plusButtonTapped" | "settingsButtonTapped" | "didBecomeVisible"
 	state?: ExtensionState
+	images?: string[]
 }
 
 export interface ExtensionState {
-	apiKey?: string
+	version: string
+	apiConfiguration?: ApiConfiguration
 	maxRequestsPerTask?: number
+	customInstructions?: string
 	themeName?: string
 	claudeMessages: ClaudeMessage[]
 	shouldShowAnnouncement: boolean
@@ -22,6 +27,7 @@ export interface ClaudeMessage {
 	ask?: ClaudeAsk
 	say?: ClaudeSay
 	text?: string
+	images?: string[]
 }
 
 export type ClaudeAsk =

src/shared/WebviewMessage.ts

@@ -1,15 +1,21 @@
+import { ApiConfiguration, ApiProvider } from "./api"
+
 export interface WebviewMessage {
 	type:
-		| "apiKey"
+		| "apiConfiguration"
 		| "maxRequestsPerTask"
+		| "customInstructions"
 		| "webviewDidLaunch"
 		| "newTask"
 		| "askResponse"
 		| "clearTask"
 		| "didShowAnnouncement"
 		| "downloadTask"
+		| "selectImages"
 	text?: string
 	askResponse?: ClaudeAskResponse
+	apiConfiguration?: ApiConfiguration
+	images?: string[]
 }
 
-export type ClaudeAskResponse = "yesButtonTapped" | "noButtonTapped" | "textResponse"
+export type ClaudeAskResponse = "yesButtonTapped" | "noButtonTapped" | "messageResponse"

src/shared/api.ts

@@ -0,0 +1,208 @@
+export type ApiProvider = "anthropic" | "openrouter" | "bedrock"
+
+export interface ApiHandlerOptions {
+	apiModelId?: ApiModelId
+	apiKey?: string // anthropic
+	openRouterApiKey?: string
+	awsAccessKey?: string
+	awsSecretKey?: string
+	awsRegion?: string
+}
+
+export type ApiConfiguration = ApiHandlerOptions & {
+	apiProvider?: ApiProvider
+}
+
+// Models
+
+export interface ModelInfo {
+	maxTokens: number
+	supportsImages: boolean
+	inputPrice: number
+	outputPrice: number
+}
+
+export type ApiModelId = AnthropicModelId | OpenRouterModelId | BedrockModelId
+
+// Anthropic
+// https://docs.anthropic.com/en/docs/about-claude/models
+export type AnthropicModelId = keyof typeof anthropicModels
+export const anthropicDefaultModelId: AnthropicModelId = "claude-3-5-sonnet-20240620"
+export const anthropicModels = {
+	"claude-3-5-sonnet-20240620": {
+		maxTokens: 8192,
+		supportsImages: true,
+		inputPrice: 3.0, // $3 per million input tokens
+		outputPrice: 15.0, // $15 per million output tokens
+	},
+	"claude-3-opus-20240229": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 15.0,
+		outputPrice: 75.0,
+	},
+	"claude-3-sonnet-20240229": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 3.0,
+		outputPrice: 15.0,
+	},
+	"claude-3-haiku-20240307": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 0.25,
+		outputPrice: 1.25,
+	},
+} as const satisfies Record<string, ModelInfo> // as const assertion makes the object deeply readonly
+
+// AWS Bedrock
+// https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html
+export type BedrockModelId = keyof typeof bedrockModels
+export const bedrockDefaultModelId: BedrockModelId = "anthropic.claude-3-5-sonnet-20240620-v1:0"
+export const bedrockModels = {
+	"anthropic.claude-3-5-sonnet-20240620-v1:0": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 3.0,
+		outputPrice: 15.0,
+	},
+	"anthropic.claude-3-opus-20240229-v1:0": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 15.0,
+		outputPrice: 75.0,
+	},
+	"anthropic.claude-3-sonnet-20240229-v1:0": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 3.0,
+		outputPrice: 15.0,
+	},
+	"anthropic.claude-3-haiku-20240307-v1:0": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 0.25,
+		outputPrice: 1.25,
+	},
+} as const satisfies Record<string, ModelInfo>
+
+// OpenRouter
+// https://openrouter.ai/models?order=newest&supported_parameters=tools
+export type OpenRouterModelId = keyof typeof openRouterModels
+export const openRouterDefaultModelId: OpenRouterModelId = "anthropic/claude-3.5-sonnet:beta"
+export const openRouterModels = {
+	"anthropic/claude-3.5-sonnet:beta": {
+		maxTokens: 8192,
+		supportsImages: true,
+		inputPrice: 3.0,
+		outputPrice: 15.0,
+	},
+	"anthropic/claude-3-opus:beta": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 15,
+		outputPrice: 75,
+	},
+	"anthropic/claude-3-sonnet:beta": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 3,
+		outputPrice: 15,
+	},
+	"anthropic/claude-3-haiku:beta": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 0.25,
+		outputPrice: 1.25,
+	},
+	"openai/gpt-4o-2024-08-06": {
+		maxTokens: 16384,
+		supportsImages: true,
+		inputPrice: 2.5,
+		outputPrice: 10,
+	},
+	"openai/gpt-4o-mini-2024-07-18": {
+		maxTokens: 16384,
+		supportsImages: true,
+		inputPrice: 0.15,
+		outputPrice: 0.6,
+	},
+	"openai/gpt-4-turbo": {
+		maxTokens: 4096,
+		supportsImages: true,
+		inputPrice: 10,
+		outputPrice: 30,
+	},
+	// llama 3.1 models cannot use tools yet
+	// "meta-llama/llama-3.1-405b-instruct": {
+	// 	maxTokens: 2048,
+	// 	supportsImages: false,
+	// 	inputPrice: 2.7,
+	// 	outputPrice: 2.7,
+	// },
+	// "meta-llama/llama-3.1-70b-instruct": {
+	// 	maxTokens: 2048,
+	// 	supportsImages: false,
+	// 	inputPrice: 0.52,
+	// 	outputPrice: 0.75,
+	// },
+	// "meta-llama/llama-3.1-8b-instruct": {
+	// 	maxTokens: 2048,
+	// 	supportsImages: false,
+	// 	inputPrice: 0.06,
+	// 	outputPrice: 0.06,
+	// },
+	// OpenRouter needs to fix mapping gemini 1.5 responses for tool calls properly, they return content with line breaks formatted wrong (too many escapes), and throw errors for being in the wrong order when they're not. They also cannot handle feedback given to a request with multiple tools. Giving feedback to one tool use requests works fine. ("Please ensure that function response turn comes immediately after a function call turn. And the number of function response parts should be equal to number of function call parts of the function call turn.")
+	// "google/gemini-pro-1.5": {
+	// 	maxTokens: 8192,
+	// 	supportsImages: false, // "Function Calling is not supported with non-text input"
+	// 	inputPrice: 2.5,
+	// 	outputPrice: 7.5,
+	// },
+	// "google/gemini-flash-1.5": {
+	// 	maxTokens: 8192,
+	// 	supportsImages: false, // "Function Calling is not supported with non-text input"
+	// 	inputPrice: 0.25,
+	// 	outputPrice: 0.75,
+	// },
+	// "google/gemini-pro": {
+	// 	maxTokens: 8192,
+	// 	supportsImages: false, // "Function Calling is not supported with non-text input"
+	// 	inputPrice: 0.125,
+	// 	outputPrice: 0.375,
+	// },
+	// while deepseek coder can use tools, it may sometimes send tool invocation as a text block
+	"deepseek/deepseek-coder": {
+		maxTokens: 4096,
+		supportsImages: false,
+		inputPrice: 0.14,
+		outputPrice: 0.28,
+	},
+	// mistral models can use tools but aren't great at going step-by-step and proceeding to the next step
+	"mistralai/mistral-large": {
+		maxTokens: 8192,
+		supportsImages: false,
+		inputPrice: 3,
+		outputPrice: 9,
+	},
+	// This model is not capable of complex system/tool prompts
+	// "mistralai/mistral-7b-instruct-v0.1": {
+	// 	maxTokens: 4096,
+	// 	supportsImages: false,
+	// 	inputPrice: 0.06,
+	// 	outputPrice: 0.06,
+	// },
+	// cohere models are not capable of complex system/tool prompts
+	// "cohere/command-r-plus": {
+	// 	maxTokens: 4000,
+	// 	supportsImages: false,
+	// 	inputPrice: 3,
+	// 	outputPrice: 15,
+	// },
+	// "cohere/command-r": {
+	// 	maxTokens: 4000,
+	// 	supportsImages: false,
+	// 	inputPrice: 0.5,
+	// 	outputPrice: 1.5,
+	// },
+} as const satisfies Record<string, ModelInfo>

src/utils/export-markdown.ts

@@ -0,0 +1,79 @@
+import { Anthropic } from "@anthropic-ai/sdk"
+import os from "os"
+import * as path from "path"
+import * as vscode from "vscode"
+
+export async function downloadTask(conversationHistory: Anthropic.MessageParam[]) {
+	// File name
+	const date = new Date()
+	const month = date.toLocaleString("en-US", { month: "short" }).toLowerCase()
+	const day = date.getDate()
+	const year = date.getFullYear()
+	let hours = date.getHours()
+	const minutes = date.getMinutes().toString().padStart(2, "0")
+	const ampm = hours >= 12 ? "pm" : "am"
+	hours = hours % 12
+	hours = hours ? hours : 12 // the hour '0' should be '12'
+	const fileName = `claude_dev_task_${month}-${day}-${year}_${hours}-${minutes}-${ampm}.md`
+
+	// Generate markdown
+	const markdownContent = conversationHistory
+		.map((message) => {
+			const role = message.role === "user" ? "**User:**" : "**Assistant:**"
+			const content = Array.isArray(message.content)
+				? message.content.map(formatContentBlockToMarkdown).join("\n")
+				: message.content
+
+			return `${role}\n\n${content}\n\n`
+		})
+		.join("---\n\n")
+
+	// Prompt user for save location
+	const saveUri = await vscode.window.showSaveDialog({
+		filters: { Markdown: ["md"] },
+		defaultUri: vscode.Uri.file(path.join(os.homedir(), "Downloads", fileName)),
+	})
+
+	if (saveUri) {
+		// Write content to the selected location
+		await vscode.workspace.fs.writeFile(saveUri, Buffer.from(markdownContent))
+		vscode.window.showTextDocument(saveUri, { preview: true })
+	}
+}
+
+function formatContentBlockToMarkdown(
+	block:
+		| Anthropic.TextBlockParam
+		| Anthropic.ImageBlockParam
+		| Anthropic.ToolUseBlockParam
+		| Anthropic.ToolResultBlockParam
+): string {
+	switch (block.type) {
+		case "text":
+			return block.text
+		case "image":
+			return `[Image]`
+		case "tool_use":
+			let input: string
+			if (typeof block.input === "object" && block.input !== null) {
+				input = Object.entries(block.input)
+					.map(([key, value]) => `${key.charAt(0).toUpperCase() + key.slice(1)}: ${value}`)
+					.join("\n")
+			} else {
+				input = String(block.input)
+			}
+			return `[Tool Use: ${block.name}]\n${input}`
+		case "tool_result":
+			if (typeof block.content === "string") {
+				return `[Tool Result${block.is_error ? " (Error)" : ""}]\n${block.content}`
+			} else if (Array.isArray(block.content)) {
+				return `[Tool Result${block.is_error ? " (Error)" : ""}]\n${block.content
+					.map(formatContentBlockToMarkdown)
+					.join("\n")}`
+			} else {
+				return `[Tool Result${block.is_error ? " (Error)" : ""}]`
+			}
+		default:
+			return "[Unexpected content type]"
+	}
+}

src/utils/getNonce.ts

@@ -0,0 +1,16 @@
+/**
+ * A helper function that returns a unique alphanumeric identifier called a nonce.
+ *
+ * @remarks This function is primarily used to help enforce content security
+ * policies for resources/scripts being executed in a webview context.
+ *
+ * @returns A nonce
+ */
+export function getNonce() {
+	let text = ""
+	const possible = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
+	for (let i = 0; i < 32; i++) {
+		text += possible.charAt(Math.floor(Math.random() * possible.length))
+	}
+	return text
+}

src/utils/getUri.ts

@@ -0,0 +1,15 @@
+import { Uri, Webview } from "vscode"
+/**
+ * A helper function which will get the webview URI of a given file or resource.
+ *
+ * @remarks This URI can be used within a webview's HTML as a link to the
+ * given file/resource.
+ *
+ * @param webview A reference to the extension webview
+ * @param extensionUri The URI of the directory containing the extension
+ * @param pathList An array of strings representing the path to a file/resource
+ * @returns A URI pointing to the file/resource
+ */
+export function getUri(webview: Webview, extensionUri: Uri, pathList: string[]) {
+	return webview.asWebviewUri(Uri.joinPath(extensionUri, ...pathList))
+}

src/utils/index.ts

@@ -0,0 +1,4 @@
+export * from "./getNonce"
+export * from "./getUri"
+export * from "./process-images"
+export * from "./export-markdown"

src/utils/process-images.ts

@@ -0,0 +1,45 @@
+import * as vscode from "vscode"
+import fs from "fs/promises"
+import * as path from "path"
+
+export async function selectImages(): Promise<string[]> {
+	const options: vscode.OpenDialogOptions = {
+		canSelectMany: true,
+		openLabel: "Select",
+		filters: {
+			Images: ["png", "jpg", "jpeg", "webp"], // supported by anthropic and openrouter
+		},
+	}
+
+	const fileUris = await vscode.window.showOpenDialog(options)
+
+	if (!fileUris || fileUris.length === 0) {
+		return []
+	}
+
+	return await Promise.all(
+		fileUris.map(async (uri) => {
+			const imagePath = uri.fsPath
+			const buffer = await fs.readFile(imagePath)
+			const base64 = buffer.toString("base64")
+			const mimeType = getMimeType(imagePath)
+			const dataUrl = `data:${mimeType};base64,${base64}`
+			return dataUrl
+		})
+	)
+}
+
+function getMimeType(filePath: string): string {
+	const ext = path.extname(filePath).toLowerCase()
+	switch (ext) {
+		case ".png":
+			return "image/png"
+		case ".jpeg":
+		case ".jpg":
+			return "image/jpeg"
+		case ".webp":
+			return "image/webp"
+		default:
+			throw new Error(`Unsupported file type: ${ext}`)
+	}
+}

tsconfig.json

@@ -12,7 +12,7 @@
 		"noImplicitReturns": true,
 		"noUnusedLocals": false,
 		"resolveJsonModule": true,
-		"rootDir": "src",
+		"rootDir": ".",
 		"skipLibCheck": true,
 		"sourceMap": true,
 		"strict": true,
@@ -20,5 +20,6 @@
 		"useDefineForClassFields": true,
 		"useUnknownInCatchVariables": false
 	},
+	"include": ["src/**/*", "scripts/**/*"],
 	"exclude": ["node_modules", ".vscode-test", "webview-ui"]
-}
+}

webview-ui/scripts/build-react-no-split.js

@@ -13,6 +13,87 @@ const rewire = require("rewire")
 const defaults = rewire("react-scripts/scripts/build.js")
 const config = defaults.__get__("config")
 
+/* Modifying Webpack Configuration for 'shared' dir
+This section uses Rewire to modify Create React App's webpack configuration without ejecting. Rewire allows us to inject and alter the internal build scripts of CRA at runtime. This allows us to maintain a flexible project structure that keeps shared code outside the webview-ui/src directory, while still adhering to CRA's security model that typically restricts imports to within src/. 
+1. Uses the ModuleScopePlugin to whitelist files from the shared directory, allowing them to be imported despite being outside src/. (see: https://stackoverflow.com/questions/44114436/the-create-react-app-imports-restriction-outside-of-src-directory/58321458#58321458)
+2. Modifies the TypeScript rule to include the shared directory in compilation. This essentially transpiles and includes the ts files in shared dir in the output main.js file.
+Before, we would just import types from shared dir and specifying include (and alias to have cleaner paths) in tsconfig.json was enough. But now that we are creating values (i.e. models in api.ts) to import into the react app, we must also include these files in the webpack resolution.
+- Imports from the shared directory must use full paths relative to the src directory, without file extensions.
+- Example: import { someFunction } from '../../src/shared/utils/helpers'
+*/
+const ModuleScopePlugin = require("react-dev-utils/ModuleScopePlugin")
+const path = require("path")
+const fs = require("fs")
+// Get all files in the shared directory
+const sharedDir = path.resolve(__dirname, "../../src/shared")
+function getAllFiles(dir) {
+	let files = []
+	fs.readdirSync(dir).forEach((file) => {
+		const filePath = path.join(dir, file)
+		if (fs.statSync(filePath).isDirectory()) {
+			files = files.concat(getAllFiles(filePath))
+		} else {
+			const withoutExtension = filePath.split(".")[0]
+			files.push(withoutExtension)
+		}
+	})
+	return files
+}
+const sharedFiles = getAllFiles(sharedDir)
+// config.resolve.plugins = config.resolve.plugins.filter((plugin) => !(plugin instanceof ModuleScopePlugin))
+// Instead of excluding the whole ModuleScopePlugin, we just whitelist specific files that can be imported from outside src.
+config.resolve.plugins.forEach((plugin) => {
+	if (plugin instanceof ModuleScopePlugin) {
+		console.log("Whitelisting shared files: ", sharedFiles)
+		sharedFiles.forEach((file) => plugin.allowedFiles.add(file))
+	}
+})
+/* 
+Webpack configuration
+
+Webpack is a module bundler for JavaScript applications. It processes your project files, resolving dependencies and generating a deployable production build.
+The webpack config is an object that tells webpack how to process and bundle your code. It defines entry points, output settings, and how to handle different file types.
+This config.module section of the webpack config deals with how different file types (modules) should be treated.
+config.module.rules:
+Rules define how module files should be processed. Each rule can:
+- Specify which files to process (test)
+	When webpack "processes" a file, it performs several operations:
+	1. Reads the file
+	2. Parses its content and analyzes dependencies
+	3. Applies transformations (e.g., converting TypeScript to JavaScript)
+	4. Potentially modifies the code (e.g., applying polyfills)
+	5. Includes the processed file in the final bundle
+	By specifying which files to process, we're telling webpack which files should go through this pipeline and be included in our application bundle. Files that aren't processed are ignored by webpack.
+	In our case, we're ensuring that TypeScript files in our shared directory are processed, allowing us to use them in our application.
+- Define which folders to include or exclude
+- Set which loaders to use for transformation
+A loader transforms certain types of files into valid modules that webpack can process. For example, the TypeScript loader converts .ts files into JavaScript that webpack can understand.
+By modifying these rules, we can change how webpack processes different files in our project, allowing us to include files from outside the standard src directory.
+
+Why we need to modify the webpack config
+
+Create React App (CRA) is designed to only process files within the src directory for security reasons. (CRA limits processing to the src directory to prevent accidental inclusion of sensitive files, reduce the attack surface, and ensure predictable builds, enhancing overall project security and consistency. Therefore it's essential that if you do include files outside src, you do so explicitly.)
+To use files from the shared directory, we need to:
+1. Modify ModuleScopePlugin to allow imports from the shared directory.
+2. Update the TypeScript loader rule to process TypeScript files from the shared directory.
+These changes tell webpack it's okay to import from the shared directory and ensure that TypeScript files in this directory are properly converted to JavaScript.
+
+Modify webpack configuration to process TypeScript files from shared directory
+
+This code modifies the webpack configuration to allow processing of TypeScript files from our shared directory, which is outside the standard src folder.
+1. config.module.rules[1]: In Create React App's webpack config, the second rule (index 1) typically contains the rules for processing JavaScript and TypeScript files.
+2. .oneOf: This array contains a list of loaders, and webpack will use the first matching loader for each file. We iterate through these to find the TypeScript loader.
+3. We check each rule to see if it applies to TypeScript files by looking for 'ts|tsx' in the test regex.
+4. When we find the TypeScript rule, we add our shared directory to its 'include' array. This tells webpack to also process TypeScript files from the shared directory.
+Note: This code assumes a specific structure in the CRA webpack config. If CRA updates its config structure in future versions, this code might need to be adjusted.
+*/
+config.module.rules[1].oneOf.forEach((rule) => {
+	if (rule.test && rule.test.toString().includes("ts|tsx")) {
+		// rule.include is path to src by default, but we can update rule.include to be an array as it matches an expected schema by react-scripts
+		rule.include = [rule.include, sharedDir].filter(Boolean)
+	}
+})
+
 // Disable code splitting
 config.optimization.splitChunks = {
 	cacheGroups: {

webview-ui/src/App.tsx

@@ -1,26 +1,28 @@
-import React, { useEffect, useState, useCallback } from "react"
+import React, { useCallback, useEffect, useMemo, useState } from "react"
+import { useEvent } from "react-use"
+import { ApiConfiguration } from "../../src/shared/api"
+import { ClaudeMessage, ExtensionMessage } from "../../src/shared/ExtensionMessage"
 import "./App.css"
+import { normalizeApiConfiguration } from "./components/ApiOptions"
 import ChatView from "./components/ChatView"
 import SettingsView from "./components/SettingsView"
-import { ClaudeMessage, ExtensionMessage } from "@shared/ExtensionMessage"
 import WelcomeView from "./components/WelcomeView"
-import { vscode } from "./utilities/vscode"
-import { useEvent } from "react-use"
+import { vscode } from "./utils/vscode"
 
 /*
 The contents of webviews however are created when the webview becomes visible and destroyed when the webview is moved into the background. Any state inside the webview will be lost when the webview is moved to a background tab.
 
 The best way to solve this is to make your webview stateless. Use message passing to save off the webview's state and then restore the state when the webview becomes visible again.
-
-
 */
 
 const App: React.FC = () => {
 	const [didHydrateState, setDidHydrateState] = useState(false)
 	const [showSettings, setShowSettings] = useState(false)
 	const [showWelcome, setShowWelcome] = useState<boolean>(false)
-	const [apiKey, setApiKey] = useState<string>("")
+	const [version, setVersion] = useState<string>("")
+	const [apiConfiguration, setApiConfiguration] = useState<ApiConfiguration | undefined>(undefined)
 	const [maxRequestsPerTask, setMaxRequestsPerTask] = useState<string>("")
+	const [customInstructions, setCustomInstructions] = useState<string>("")
 	const [vscodeThemeName, setVscodeThemeName] = useState<string | undefined>(undefined)
 	const [claudeMessages, setClaudeMessages] = useState<ClaudeMessage[]>([])
 	const [showAnnouncement, setShowAnnouncement] = useState(false)
@@ -33,11 +35,17 @@ const App: React.FC = () => {
 		const message: ExtensionMessage = e.data
 		switch (message.type) {
 			case "state":
-				setShowWelcome(!message.state!.apiKey)
-				setApiKey(message.state!.apiKey || "")
+				setVersion(message.state!.version)
+				const hasKey =
+					message.state!.apiConfiguration?.apiKey !== undefined ||
+					message.state!.apiConfiguration?.openRouterApiKey !== undefined ||
+					message.state!.apiConfiguration?.awsAccessKey !== undefined
+				setShowWelcome(!hasKey)
+				setApiConfiguration(message.state!.apiConfiguration)
 				setMaxRequestsPerTask(
 					message.state!.maxRequestsPerTask !== undefined ? message.state!.maxRequestsPerTask.toString() : ""
 				)
+				setCustomInstructions(message.state!.customInstructions || "")
 				setVscodeThemeName(message.state!.themeName)
 				setClaudeMessages(message.state!.claudeMessages)
 				// don't update showAnnouncement to false if shouldShowAnnouncement is false
@@ -63,6 +71,10 @@ const App: React.FC = () => {
 
 	useEvent("message", handleMessage)
 
+	const { selectedModelInfo } = useMemo(() => {
+		return normalizeApiConfiguration(apiConfiguration)
+	}, [apiConfiguration])
+
 	if (!didHydrateState) {
 		return null
 	}
@@ -70,24 +82,29 @@ const App: React.FC = () => {
 	return (
 		<>
 			{showWelcome ? (
-				<WelcomeView apiKey={apiKey} setApiKey={setApiKey} />
+				<WelcomeView apiConfiguration={apiConfiguration} setApiConfiguration={setApiConfiguration} />
 			) : (
 				<>
 					{showSettings && (
 						<SettingsView
-							apiKey={apiKey}
-							setApiKey={setApiKey}
+							version={version}
+							apiConfiguration={apiConfiguration}
+							setApiConfiguration={setApiConfiguration}
 							maxRequestsPerTask={maxRequestsPerTask}
 							setMaxRequestsPerTask={setMaxRequestsPerTask}
+							customInstructions={customInstructions}
+							setCustomInstructions={setCustomInstructions}
 							onDone={() => setShowSettings(false)}
 						/>
 					)}
 					{/* Do not conditionally load ChatView, it's expensive and there's state we don't want to lose (user input, disableInput, askResponse promise, etc.) */}
 					<ChatView
+						version={version}
 						messages={claudeMessages}
 						isHidden={showSettings}
 						vscodeThemeName={vscodeThemeName}
 						showAnnouncement={showAnnouncement}
+						selectedModelSupportsImages={selectedModelInfo.supportsImages}
 						hideAnnouncement={() => setShowAnnouncement(false)}
 					/>
 				</>

webview-ui/src/components/Announcement.tsx

@@ -1,12 +1,13 @@
 import { VSCodeButton, VSCodeLink } from "@vscode/webview-ui-toolkit/react"
 
 interface AnnouncementProps {
+	version: string
 	hideAnnouncement: () => void
 }
 /*
 You must update the latestAnnouncementId in ClaudeDevProvider for new announcements to show to users. This new id will be compared with whats in state for the 'last announcement shown', and if it's different then the announcement will render. As soon as an announcement is shown, the id will be updated in state. This ensures that announcements are not shown more than once, even if the user doesn't close it themselves.
 */
-const Announcement = ({ hideAnnouncement }: AnnouncementProps) => {
+const Announcement = ({ version, hideAnnouncement }: AnnouncementProps) => {
 	return (
 		<div
 			style={{
@@ -22,34 +23,22 @@ const Announcement = ({ hideAnnouncement }: AnnouncementProps) => {
 				style={{ position: "absolute", top: "8px", right: "8px" }}>
 				<span className="codicon codicon-close"></span>
 			</VSCodeButton>
-			<h3 style={{ margin: "0 0 8px" }}>🎉{"  "}New in v1.0.8</h3>
+			<h3 style={{ margin: "0 0 8px" }}>
+				🎉{"  "}New in v{version}
+			</h3>
 			<ul style={{ margin: "0 0 8px", paddingLeft: "20px" }}>
 				<li>
-					Open in the editor (using{" "}
-					<span
-						className="codicon codicon-link-external"
-						style={{ display: "inline", fontSize: "12.5px", verticalAlign: "text-bottom" }}></span>{" "}
-					or <code>Claude Dev: Open In New Tab</code> in command palette) to see how Claude updates your
-					workspace more clearly
+					Paste images in chat and turn mockups into fully functional applications or fix bugs with
+					screenshots
 				</li>
 				<li>
-					New <code style={{ wordBreak: "break-all" }}>list_files_recursive</code> and{" "}
-					<code style={{ wordBreak: "break-all" }}>view_source_code_definitions_top_level</code> tools to help
-					Claude get a comprehensive overview of your project's file structure and source code definitions
-					<VSCodeLink
-						href="https://github.com/saoudrizwan/claude-dev?tab=readme-ov-file#working-in-existing-projects"
-						style={{ display: "inline" }}>
-						(more on this here)
-					</VSCodeLink>
+					Added option to choose other Claude models (+ GPT-4o, DeepSeek, and Mistral if you use OpenRouter)
 				</li>
 				<li>
-					Interact with CLI commands by sending messages to stdin and terminating long-running processes like
-					servers
+					You can now add custom instructions to the end of the system prompt (e.g. "Always use Python",
+					"Speak in Spanish")
 				</li>
-				<li>Provide feedback to tool use like editing files or running commands</li>
-				<li>Shows diff view of new or edited files right in the editor</li>
-				<li>Added ability to retry failed API requests (helpful for rate limits)</li>
-				<li>Export task to a markdown file (useful as context for future tasks)</li>
+				<li>Improved support for running interactive terminal commands and servers</li>
 			</ul>
 			<p style={{ margin: "0" }}>
 				Follow me for more updates!{" "}

webview-ui/src/components/ApiOptions.tsx

@@ -0,0 +1,272 @@
+import { VSCodeDropdown, VSCodeLink, VSCodeOption, VSCodeTextField } from "@vscode/webview-ui-toolkit/react"
+import React, { useMemo } from "react"
+import {
+	ApiConfiguration,
+	ApiModelId,
+	ModelInfo,
+	anthropicDefaultModelId,
+	anthropicModels,
+	bedrockDefaultModelId,
+	bedrockModels,
+	openRouterDefaultModelId,
+	openRouterModels,
+} from "../../../src/shared/api"
+
+interface ApiOptionsProps {
+	showModelOptions: boolean
+	apiConfiguration?: ApiConfiguration
+	setApiConfiguration: React.Dispatch<React.SetStateAction<ApiConfiguration | undefined>>
+}
+
+const ApiOptions: React.FC<ApiOptionsProps> = ({ showModelOptions, apiConfiguration, setApiConfiguration }) => {
+	const handleInputChange = (field: keyof ApiConfiguration) => (event: any) => {
+		setApiConfiguration((prev) => ({ ...prev, [field]: event.target.value }))
+	}
+
+	const { selectedProvider, selectedModelId, selectedModelInfo } = useMemo(() => {
+		return normalizeApiConfiguration(apiConfiguration)
+	}, [apiConfiguration])
+
+	/*
+	VSCodeDropdown has an open bug where dynamically rendered options don't auto select the provided value prop. You can see this for yourself by comparing  it with normal select/option elements, which work as expected.
+	https://github.com/microsoft/vscode-webview-ui-toolkit/issues/433
+
+	In our case, when the user switches between providers, we recalculate the selectedModelId depending on the provider, the default model for that provider, and a modelId that the user may have selected. Unfortunately, the VSCodeDropdown component wouldn't select this calculated value, and would default to the first "Select a model..." option instead, which makes it seem like the model was cleared out when it wasn't. 
+
+	As a workaround, we create separate instances of the dropdown for each provider, and then conditionally render the one that matches the current provider.
+	*/
+	const createDropdown = (models: Record<string, ModelInfo>) => {
+		return (
+			<VSCodeDropdown
+				id="model-id"
+				value={selectedModelId}
+				onChange={handleInputChange("apiModelId")}
+				style={{ width: "100%" }}>
+				<VSCodeOption value="">Select a model...</VSCodeOption>
+				{Object.keys(models).map((modelId) => (
+					<VSCodeOption
+						key={modelId}
+						value={modelId}
+						style={{
+							whiteSpace: "normal",
+							wordWrap: "break-word",
+							maxWidth: "100%",
+						}}>
+						{modelId}
+					</VSCodeOption>
+				))}
+			</VSCodeDropdown>
+		)
+	}
+
+	return (
+		<div style={{ display: "flex", flexDirection: "column", gap: 5 }}>
+			<div className="dropdown-container">
+				<label htmlFor="api-provider">
+					<span style={{ fontWeight: 500 }}>API Provider</span>
+				</label>
+				<VSCodeDropdown id="api-provider" value={selectedProvider} onChange={handleInputChange("apiProvider")}>
+					<VSCodeOption value="openrouter">OpenRouter</VSCodeOption>
+					<VSCodeOption value="anthropic">Anthropic</VSCodeOption>
+					<VSCodeOption value="bedrock">AWS Bedrock</VSCodeOption>
+				</VSCodeDropdown>
+			</div>
+
+			{selectedProvider === "anthropic" && (
+				<div>
+					<VSCodeTextField
+						value={apiConfiguration?.apiKey || ""}
+						style={{ width: "100%" }}
+						onInput={handleInputChange("apiKey")}
+						placeholder="Enter API Key...">
+						<span style={{ fontWeight: 500 }}>Anthropic API Key</span>
+					</VSCodeTextField>
+					<p
+						style={{
+							fontSize: "12px",
+							marginTop: "5px",
+							color: "var(--vscode-descriptionForeground)",
+						}}>
+						This key is stored locally and only used to make API requests from this extension.
+						<VSCodeLink href="https://console.anthropic.com/" style={{ display: "inline" }}>
+							You can get an Anthropic API key by signing up here.
+						</VSCodeLink>
+					</p>
+				</div>
+			)}
+
+			{selectedProvider === "openrouter" && (
+				<div>
+					<VSCodeTextField
+						value={apiConfiguration?.openRouterApiKey || ""}
+						style={{ width: "100%" }}
+						onInput={handleInputChange("openRouterApiKey")}
+						placeholder="Enter API Key...">
+						<span style={{ fontWeight: 500 }}>OpenRouter API Key</span>
+					</VSCodeTextField>
+					<p
+						style={{
+							fontSize: "12px",
+							marginTop: "5px",
+							color: "var(--vscode-descriptionForeground)",
+						}}>
+						This key is stored locally and only used to make API requests from this extension.
+						<VSCodeLink href="https://openrouter.ai/" style={{ display: "inline" }}>
+							You can get an OpenRouter API key by signing up here.
+						</VSCodeLink>
+					</p>
+				</div>
+			)}
+
+			{selectedProvider === "bedrock" && (
+				<div style={{ display: "flex", flexDirection: "column", gap: 5 }}>
+					<VSCodeTextField
+						value={apiConfiguration?.awsAccessKey || ""}
+						style={{ width: "100%" }}
+						onInput={handleInputChange("awsAccessKey")}
+						placeholder="Enter Access Key...">
+						<span style={{ fontWeight: 500 }}>AWS Access Key</span>
+					</VSCodeTextField>
+					<VSCodeTextField
+						value={apiConfiguration?.awsSecretKey || ""}
+						style={{ width: "100%" }}
+						onInput={handleInputChange("awsSecretKey")}
+						placeholder="Enter Secret Key...">
+						<span style={{ fontWeight: 500 }}>AWS Secret Key</span>
+					</VSCodeTextField>
+					<div className="dropdown-container">
+						<label htmlFor="aws-region-dropdown">
+							<span style={{ fontWeight: 500 }}>AWS Region</span>
+						</label>
+						<VSCodeDropdown
+							id="aws-region-dropdown"
+							value={apiConfiguration?.awsRegion || ""}
+							style={{ width: "100%" }}
+							onChange={handleInputChange("awsRegion")}>
+							<VSCodeOption value="">Select a region...</VSCodeOption>
+							{/* The user will have to choose a region that supports the model they use, but this shouldn't be a problem since they'd have to request access for it in that region in the first place. */}
+							<VSCodeOption value="us-east-1">US East (N. Virginia)</VSCodeOption>
+							{/* <VSCodeOption value="us-east-2">US East (Ohio)</VSCodeOption> */}
+							{/* <VSCodeOption value="us-west-1">US West (N. California)</VSCodeOption> */}
+							<VSCodeOption value="us-west-2">US West (Oregon)</VSCodeOption>
+							{/* <VSCodeOption value="af-south-1">Africa (Cape Town)</VSCodeOption> */}
+							{/* <VSCodeOption value="ap-east-1">Asia Pacific (Hong Kong)</VSCodeOption> */}
+							<VSCodeOption value="ap-south-1">Asia Pacific (Mumbai)</VSCodeOption>
+							<VSCodeOption value="ap-northeast-1">Asia Pacific (Tokyo)</VSCodeOption>
+							{/* <VSCodeOption value="ap-northeast-2">Asia Pacific (Seoul)</VSCodeOption> */}
+							{/* <VSCodeOption value="ap-northeast-3">Asia Pacific (Osaka)</VSCodeOption> */}
+							<VSCodeOption value="ap-southeast-1">Asia Pacific (Singapore)</VSCodeOption>
+							<VSCodeOption value="ap-southeast-2">Asia Pacific (Sydney)</VSCodeOption>
+							<VSCodeOption value="ca-central-1">Canada (Central)</VSCodeOption>
+							<VSCodeOption value="eu-central-1">Europe (Frankfurt)</VSCodeOption>
+							<VSCodeOption value="eu-west-1">Europe (Ireland)</VSCodeOption>
+							<VSCodeOption value="eu-west-2">Europe (London)</VSCodeOption>
+							<VSCodeOption value="eu-west-3">Europe (Paris)</VSCodeOption>
+							{/* <VSCodeOption value="eu-north-1">Europe (Stockholm)</VSCodeOption> */}
+							{/* <VSCodeOption value="me-south-1">Middle East (Bahrain)</VSCodeOption> */}
+							<VSCodeOption value="sa-east-1">South America (São Paulo)</VSCodeOption>
+						</VSCodeDropdown>
+					</div>
+					<p
+						style={{
+							fontSize: "12px",
+							marginTop: "5px",
+							color: "var(--vscode-descriptionForeground)",
+						}}>
+						These credentials are stored locally and only used to make API requests from this extension.
+						<VSCodeLink
+							href="https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html"
+							style={{ display: "inline" }}>
+							You can find your AWS access key and secret key here.
+						</VSCodeLink>
+					</p>
+				</div>
+			)}
+
+			{showModelOptions && (
+				<>
+					<div className="dropdown-container">
+						<label htmlFor="model-id">
+							<span style={{ fontWeight: 500 }}>Model</span>
+						</label>
+						{selectedProvider === "anthropic" && createDropdown(anthropicModels)}
+						{selectedProvider === "openrouter" && createDropdown(openRouterModels)}
+						{selectedProvider === "bedrock" && createDropdown(bedrockModels)}
+					</div>
+
+					<ModelInfoView modelInfo={selectedModelInfo} />
+				</>
+			)}
+		</div>
+	)
+}
+
+const ModelInfoView = ({ modelInfo }: { modelInfo: ModelInfo }) => {
+	const formatPrice = (price: number) => {
+		return new Intl.NumberFormat("en-US", {
+			style: "currency",
+			currency: "USD",
+			minimumFractionDigits: 2,
+			maximumFractionDigits: 2,
+		}).format(price)
+	}
+
+	return (
+		<p style={{ fontSize: "12px", marginTop: "2px", color: "var(--vscode-descriptionForeground)" }}>
+			<span
+				style={{
+					fontWeight: 500,
+					color: modelInfo.supportsImages
+						? "var(--vscode-testing-iconPassed)"
+						: "var(--vscode-errorForeground)",
+				}}>
+				<i
+					className={`codicon codicon-${modelInfo.supportsImages ? "check" : "x"}`}
+					style={{
+						marginRight: 4,
+						marginBottom: modelInfo.supportsImages ? 1 : -1,
+						fontSize: modelInfo.supportsImages ? 11 : 13,
+						fontWeight: 700,
+						display: "inline-block",
+						verticalAlign: "bottom",
+					}}></i>
+				{modelInfo.supportsImages ? "Supports images" : "Does not support images"}
+			</span>
+			<br />
+			<span style={{ fontWeight: 500 }}>Max output:</span> {modelInfo.maxTokens.toLocaleString()} tokens
+			<br />
+			<span style={{ fontWeight: 500 }}>Input price:</span> {formatPrice(modelInfo.inputPrice)} per million tokens
+			<br />
+			<span style={{ fontWeight: 500 }}>Output price:</span> {formatPrice(modelInfo.outputPrice)} per million
+			tokens
+		</p>
+	)
+}
+
+export function normalizeApiConfiguration(apiConfiguration?: ApiConfiguration) {
+	const provider = apiConfiguration?.apiProvider || "anthropic"
+	const modelId = apiConfiguration?.apiModelId
+
+	const getProviderData = (models: Record<string, ModelInfo>, defaultId: ApiModelId) => {
+		let selectedModelId: ApiModelId
+		let selectedModelInfo: ModelInfo
+		if (modelId && modelId in models) {
+			selectedModelId = modelId
+			selectedModelInfo = models[modelId]
+		} else {
+			selectedModelId = defaultId
+			selectedModelInfo = models[defaultId]
+		}
+		return { selectedProvider: provider, selectedModelId, selectedModelInfo }
+	}
+	switch (provider) {
+		case "anthropic":
+			return getProviderData(anthropicModels, anthropicDefaultModelId)
+		case "openrouter":
+			return getProviderData(openRouterModels, openRouterDefaultModelId)
+		case "bedrock":
+			return getProviderData(bedrockModels, bedrockDefaultModelId)
+	}
+}
+
+export default ApiOptions

webview-ui/src/components/ChatRow.tsx

@@ -1,11 +1,12 @@
-import { ClaudeAsk, ClaudeMessage, ClaudeSay, ClaudeSayTool } from "@shared/ExtensionMessage"
 import { VSCodeBadge, VSCodeButton, VSCodeProgressRing } from "@vscode/webview-ui-toolkit/react"
 import React from "react"
-import { COMMAND_OUTPUT_STRING } from "../utilities/combineCommandSequences"
-import { SyntaxHighlighterStyle } from "../utilities/getSyntaxHighlighterStyleFromTheme"
-import CodeBlock from "./CodeBlock/CodeBlock"
 import Markdown from "react-markdown"
 import { Prism as SyntaxHighlighter } from "react-syntax-highlighter"
+import { ClaudeAsk, ClaudeMessage, ClaudeSay, ClaudeSayTool } from "../../../src/shared/ExtensionMessage"
+import { COMMAND_OUTPUT_STRING } from "../utils/combineCommandSequences"
+import { SyntaxHighlighterStyle } from "../utils/getSyntaxHighlighterStyleFromTheme"
+import CodeBlock from "./CodeBlock/CodeBlock"
+import Thumbnails from "./Thumbnails"
 
 interface ChatRowProps {
 	message: ClaudeMessage
@@ -294,7 +295,10 @@ const ChatRow: React.FC<ChatRowProps> = ({
 									whiteSpace: "pre-line",
 									wordWrap: "break-word",
 								}}>
-								<span>{message.text}</span>
+								<span style={{ display: "block" }}>{message.text}</span>
+								{message.images && message.images.length > 0 && (
+									<Thumbnails images={message.images} style={{ marginTop: "8px" }} />
+								)}
 							</div>
 						)
 					case "error":

webview-ui/src/components/ChatView.tsx

@@ -1,28 +1,41 @@
-import { ClaudeAsk, ClaudeMessage, ExtensionMessage } from "@shared/ExtensionMessage"
 import { VSCodeButton, VSCodeLink } from "@vscode/webview-ui-toolkit/react"
 import { KeyboardEvent, useCallback, useEffect, useMemo, useRef, useState } from "react"
 import vsDarkPlus from "react-syntax-highlighter/dist/esm/styles/prism/vsc-dark-plus"
 import DynamicTextArea from "react-textarea-autosize"
 import { useEvent, useMount } from "react-use"
-import { combineApiRequests } from "../utilities/combineApiRequests"
-import { combineCommandSequences } from "../utilities/combineCommandSequences"
-import { getApiMetrics } from "../utilities/getApiMetrics"
-import { getSyntaxHighlighterStyleFromTheme } from "../utilities/getSyntaxHighlighterStyleFromTheme"
-import { vscode } from "../utilities/vscode"
+import { Virtuoso, type VirtuosoHandle } from "react-virtuoso"
+import { ClaudeAsk, ClaudeMessage, ExtensionMessage } from "../../../src/shared/ExtensionMessage"
+import { combineApiRequests } from "../utils/combineApiRequests"
+import { combineCommandSequences } from "../utils/combineCommandSequences"
+import { getApiMetrics } from "../utils/getApiMetrics"
+import { getSyntaxHighlighterStyleFromTheme } from "../utils/getSyntaxHighlighterStyleFromTheme"
+import { vscode } from "../utils/vscode"
+import Announcement from "./Announcement"
 import ChatRow from "./ChatRow"
 import TaskHeader from "./TaskHeader"
-import { Virtuoso, type VirtuosoHandle } from "react-virtuoso"
-import Announcement from "./Announcement"
+import Thumbnails from "./Thumbnails"
 
 interface ChatViewProps {
+	version: string
 	messages: ClaudeMessage[]
 	isHidden: boolean
 	vscodeThemeName?: string
 	showAnnouncement: boolean
+	selectedModelSupportsImages: boolean
 	hideAnnouncement: () => void
 }
-// maybe instead of storing state in App, just make chatview  always show so dont conditionally load/unload? need to make sure messages are persisted (i remember seeing something about how webviews can be frozen in docs)
-const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideAnnouncement }: ChatViewProps) => {
+
+const MAX_IMAGES_PER_MESSAGE = 20 // Anthropic limits to 20 images
+
+const ChatView = ({
+	version,
+	messages,
+	isHidden,
+	vscodeThemeName,
+	showAnnouncement,
+	selectedModelSupportsImages,
+	hideAnnouncement,
+}: ChatViewProps) => {
 	//const task = messages.length > 0 ? (messages[0].say === "task" ? messages[0] : undefined) : undefined
 	const task = messages.length > 0 ? messages[0] : undefined // leaving this less safe version here since if the first message is not a task, then the extension is in a bad state and needs to be debugged (see ClaudeDev.abort)
 	const modifiedMessages = useMemo(() => combineApiRequests(combineCommandSequences(messages.slice(1))), [messages])
@@ -31,8 +44,10 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 
 	const [inputValue, setInputValue] = useState("")
 	const textAreaRef = useRef<HTMLTextAreaElement>(null)
-	const [textAreaHeight, setTextAreaHeight] = useState<number | undefined>(undefined)
 	const [textAreaDisabled, setTextAreaDisabled] = useState(false)
+	const [isTextAreaFocused, setIsTextAreaFocused] = useState(false)
+	const [selectedImages, setSelectedImages] = useState<string[]>([])
+	const [thumbnailsHeight, setThumbnailsHeight] = useState(0)
 
 	// we need to hold on to the ask because useEffect > lastMessage will always let us know when an ask comes in and handle it, but by the time handleMessage is called, the last message might not be the ask anymore (it could be a say that followed)
 	const [claudeAsk, setClaudeAsk] = useState<ClaudeAsk | undefined>(undefined)
@@ -40,11 +55,8 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 	const [enableButtons, setEnableButtons] = useState<boolean>(false)
 	const [primaryButtonText, setPrimaryButtonText] = useState<string | undefined>(undefined)
 	const [secondaryButtonText, setSecondaryButtonText] = useState<string | undefined>(undefined)
-
 	const [syntaxHighlighterStyle, setSyntaxHighlighterStyle] = useState(vsDarkPlus)
-
 	const virtuosoRef = useRef<VirtuosoHandle>(null)
-
 	const [expandedRows, setExpandedRows] = useState<Record<number, boolean>>({})
 
 	const toggleRowExpansion = (ts: number) => {
@@ -133,6 +145,14 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 						case "error":
 							break
 						case "api_req_started":
+							if (messages.at(-2)?.ask === "command_output") {
+								// if the last ask is a command_output, and we receive an api_req_started, then that means the command has finished and we don't need input from the user anymore (in every other case, the user has to interact with input field or buttons to continue, which does the following automatically)
+								setInputValue("")
+								setTextAreaDisabled(true)
+								setSelectedImages([])
+								setClaudeAsk(undefined)
+								setEnableButtons(false)
+							}
 							break
 						case "api_req_finished":
 							break
@@ -167,9 +187,9 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 
 	const handleSendMessage = () => {
 		const text = inputValue.trim()
-		if (text) {
+		if (text || selectedImages.length > 0) {
 			if (messages.length === 0) {
-				vscode.postMessage({ type: "newTask", text })
+				vscode.postMessage({ type: "newTask", text, images: selectedImages })
 			} else if (claudeAsk) {
 				switch (claudeAsk) {
 					case "followup":
@@ -177,13 +197,19 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 					case "command": // user can provide feedback to a tool or command use
 					case "command_output": // user can send input to command stdin
 					case "completion_result": // if this happens then the user has feedback for the completion result
-						vscode.postMessage({ type: "askResponse", askResponse: "textResponse", text })
+						vscode.postMessage({
+							type: "askResponse",
+							askResponse: "messageResponse",
+							text,
+							images: selectedImages,
+						})
 						break
 					// there is no other case that a textfield should be enabled
 				}
 			}
 			setInputValue("")
 			setTextAreaDisabled(true)
+			setSelectedImages([])
 			setClaudeAsk(undefined)
 			setEnableButtons(false)
 			// setPrimaryButtonText(undefined)
@@ -249,6 +275,65 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 		vscode.postMessage({ type: "clearTask" })
 	}
 
+	const selectImages = () => {
+		vscode.postMessage({ type: "selectImages" })
+	}
+
+	const handlePaste = async (e: React.ClipboardEvent) => {
+		if (shouldDisableImages) {
+			e.preventDefault()
+			return
+		}
+
+		const items = e.clipboardData.items
+		const acceptedTypes = ["png", "jpeg", "webp"] // supported by anthropic and openrouter (jpg is just a file extension but the image will be recognized as jpeg)
+		const imageItems = Array.from(items).filter((item) => {
+			const [type, subtype] = item.type.split("/")
+			return type === "image" && acceptedTypes.includes(subtype)
+		})
+		if (imageItems.length > 0) {
+			e.preventDefault()
+			const imagePromises = imageItems.map((item) => {
+				return new Promise<string | null>((resolve) => {
+					const blob = item.getAsFile()
+					if (!blob) {
+						resolve(null)
+						return
+					}
+					const reader = new FileReader()
+					reader.onloadend = () => {
+						if (reader.error) {
+							console.error("Error reading file:", reader.error)
+							resolve(null)
+						} else {
+							const result = reader.result
+							resolve(typeof result === "string" ? result : null)
+						}
+					}
+					reader.readAsDataURL(blob)
+				})
+			})
+			const imageDataArray = await Promise.all(imagePromises)
+			const dataUrls = imageDataArray.filter((dataUrl): dataUrl is string => dataUrl !== null)
+			//.map((dataUrl) => dataUrl.split(",")[1]) // strip the mime type prefix, sharp doesn't need it
+			if (dataUrls.length > 0) {
+				setSelectedImages((prevImages) => [...prevImages, ...dataUrls].slice(0, MAX_IMAGES_PER_MESSAGE))
+			} else {
+				console.warn("No valid images were processed")
+			}
+		}
+	}
+
+	useEffect(() => {
+		if (selectedImages.length === 0) {
+			setThumbnailsHeight(0)
+		}
+	}, [selectedImages])
+
+	const handleThumbnailsHeightChange = useCallback((height: number) => {
+		setThumbnailsHeight(height)
+	}, [])
+
 	const handleMessage = useCallback(
 		(e: MessageEvent) => {
 			const message: ExtensionMessage = e.data
@@ -262,6 +347,14 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 							break
 					}
 					break
+				case "selectedImages":
+					const newImages = message.images ?? []
+					if (newImages.length > 0) {
+						setSelectedImages((prevImages) =>
+							[...prevImages, ...newImages].slice(0, MAX_IMAGES_PER_MESSAGE)
+						)
+					}
+					break
 			}
 			// textAreaRef.current is not explicitly required here since react gaurantees that ref will be stable across re-renders, and we're not using its value but its reference.
 		},
@@ -275,12 +368,6 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 		textAreaRef.current?.focus()
 	})
 
-	useEffect(() => {
-		if (textAreaRef.current && !textAreaHeight) {
-			setTextAreaHeight(textAreaRef.current.offsetHeight)
-		}
-	}, [textAreaHeight])
-
 	useEffect(() => {
 		const timer = setTimeout(() => {
 			if (!isHidden && !textAreaDisabled && !enableButtons) {
@@ -308,6 +395,12 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 				case "api_req_finished": // combineApiRequests removes this from modifiedMessages anyways
 				case "api_req_retried": // this message is used to update the latest api_req_started that the request was retried
 					return false
+				case "text":
+					// Sometimes Claude returns an empty text message, we don't want to render these. (We also use a say text for user messages, so in case they just sent images we still render that)
+					if ((message.text ?? "") === "" && (message.images?.length ?? 0) === 0) {
+						return false
+					}
+					break
 			}
 			return true
 		})
@@ -324,13 +417,20 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 		return () => clearTimeout(timer)
 	}, [visibleMessages])
 
-	const placeholderText = useMemo(() => {
+	const [placeholderText, isInputPipingToStdin] = useMemo(() => {
 		if (messages.at(-1)?.ask === "command_output") {
-			return "Type input to command stdin..."
+			return ["Type input to command stdin...", true]
 		}
-		return task ? "Type a message..." : "Type your task here..."
+		const text = task ? "Type a message..." : "Type your task here..."
+		return [text, false]
 	}, [task, messages])
 
+	const shouldDisableImages =
+		!selectedModelSupportsImages ||
+		textAreaDisabled ||
+		selectedImages.length >= MAX_IMAGES_PER_MESSAGE ||
+		isInputPipingToStdin
+
 	return (
 		<div
 			style={{
@@ -345,7 +445,7 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 			}}>
 			{task ? (
 				<TaskHeader
-					taskText={task.text || ""}
+					task={task}
 					tokensIn={apiMetrics.totalTokensIn}
 					tokensOut={apiMetrics.totalTokensOut}
 					totalCost={apiMetrics.totalCost}
@@ -354,7 +454,7 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 				/>
 			) : (
 				<>
-					{showAnnouncement && <Announcement hideAnnouncement={hideAnnouncement} />}
+					{showAnnouncement && <Announcement version={version} hideAnnouncement={hideAnnouncement} />}
 					<div style={{ padding: "0 20px" }}>
 						<h2>What can I do for you?</h2>
 						<p>
@@ -364,8 +464,8 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 								style={{ display: "inline" }}>
 								Claude 3.5 Sonnet's agentic coding capabilities,
 							</VSCodeLink>{" "}
-							I can handle complex software development tasks step-by-step. With tools that let me read &
-							write files, analyze project source code, and execute terminal commands (after you grant
+							I can handle complex software development tasks step-by-step. With tools that let me create
+							& edit files, explore complex projects, and execute terminal commands (after you grant
 							permission), I can assist you in ways that go beyond simple code completion or tech support.
 						</p>
 					</div>
@@ -427,13 +527,33 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 					</VSCodeButton>
 				)}
 			</div>
-			<div style={{ padding: "10px 15px", opacity: textAreaDisabled ? 0.5 : 1, position: "relative" }}>
+			<div
+				style={{
+					padding: "10px 15px",
+					opacity: textAreaDisabled ? 0.5 : 1,
+					position: "relative",
+					display: "flex",
+				}}>
+				{!isTextAreaFocused && (
+					<div
+						style={{
+							position: "absolute",
+							inset: "10px 15px",
+							border: "1px solid var(--vscode-input-border)",
+							borderRadius: 2,
+							pointerEvents: "none",
+						}}
+					/>
+				)}
 				<DynamicTextArea
 					ref={textAreaRef}
 					value={inputValue}
 					disabled={textAreaDisabled}
 					onChange={(e) => setInputValue(e.target.value)}
 					onKeyDown={handleKeyDown}
+					onFocus={() => setIsTextAreaFocused(true)}
+					onBlur={() => setIsTextAreaFocused(false)}
+					onPaste={handlePaste}
 					onHeightChange={() =>
 						//virtuosoRef.current?.scrollToIndex({ index: "LAST", align: "end", behavior: "auto" })
 						virtuosoRef.current?.scrollTo({ top: Number.MAX_SAFE_INTEGER, behavior: "auto" })
@@ -446,25 +566,58 @@ const ChatView = ({ messages, isHidden, vscodeThemeName, showAnnouncement, hideA
 						boxSizing: "border-box",
 						backgroundColor: "var(--vscode-input-background)",
 						color: "var(--vscode-input-foreground)",
-						border: "1px solid var(--vscode-input-border)",
+						//border: "1px solid var(--vscode-input-border)",
 						borderRadius: 2,
 						fontFamily: "var(--vscode-font-family)",
 						fontSize: "var(--vscode-editor-font-size)",
 						lineHeight: "var(--vscode-editor-line-height)",
 						resize: "none",
 						overflow: "hidden",
-						padding: "8px 36px 8px 8px",
+						// Since we have maxRows, when text is long enough it starts to overflow the bottom padding, appearing behind the thumbnails. To fix this, we use a transparent border to push the text up instead. (https://stackoverflow.com/questions/42631947/maintaining-a-padding-inside-of-text-area/52538410#52538410)
+						borderTop: "9px solid transparent",
+						borderBottom: `${thumbnailsHeight + 9}px solid transparent`,
+						borderRight: "54px solid transparent",
+						borderLeft: "9px solid transparent",
+						// Instead of using boxShadow, we use a div with a border to better replicate the behavior when the textarea is focused
+						// boxShadow: "0px 0px 0px 1px var(--vscode-input-border)",
+						padding: 0,
 						cursor: textAreaDisabled ? "not-allowed" : undefined,
+						flex: 1,
 					}}
 				/>
+				{selectedImages.length > 0 && (
+					<Thumbnails
+						images={selectedImages}
+						setImages={setSelectedImages}
+						onHeightChange={handleThumbnailsHeightChange}
+						style={{
+							position: "absolute",
+							paddingTop: 4,
+							bottom: 14,
+							left: 22,
+							right: 67, // (54 + 9) + 4 extra padding
+						}}
+					/>
+				)}
 				<div
 					style={{
 						position: "absolute",
 						right: 20,
+						bottom: 14, // Align with the bottom padding of the container
 						display: "flex",
-						alignItems: "center",
-						...(!!textAreaHeight ? { height: textAreaHeight, bottom: 12 } : { top: 0, bottom: 1.5 }),
+						alignItems: "flex-end",
+						height: "calc(100% - 20px)", // Full height minus top and bottom padding
 					}}>
+					<VSCodeButton
+						disabled={shouldDisableImages}
+						appearance="icon"
+						aria-label="Attach Images"
+						onClick={selectImages}
+						style={{ marginRight: "4px" }}>
+						<span
+							className="codicon codicon-device-camera"
+							style={{ fontSize: 18.5, marginLeft: -2, marginBottom: 1.5 }}></span>
+					</VSCodeButton>
 					<VSCodeButton
 						disabled={textAreaDisabled}
 						appearance="icon"

webview-ui/src/components/CodeBlock/CodeBlock.tsx

@@ -1,7 +1,7 @@
 import { useMemo } from "react"
 import { Prism as SyntaxHighlighter } from "react-syntax-highlighter"
-import { getLanguageFromPath } from "../../utilities/getLanguageFromPath"
-import { SyntaxHighlighterStyle } from "../../utilities/getSyntaxHighlighterStyleFromTheme"
+import { getLanguageFromPath } from "../../utils/getLanguageFromPath"
+import { SyntaxHighlighterStyle } from "../../utils/getSyntaxHighlighterStyleFromTheme"
 
 /*
 const vscodeSyntaxStyle: React.CSSProperties = {

webview-ui/src/components/SettingsView.tsx

@@ -1,64 +1,57 @@
-import { VSCodeButton, VSCodeDivider, VSCodeLink, VSCodeTextField } from "@vscode/webview-ui-toolkit/react"
-import React, { useState } from "react"
-import { useEffectOnce } from "react-use"
-import { vscode } from "../utilities/vscode"
+import { VSCodeButton, VSCodeLink, VSCodeTextArea, VSCodeTextField } from "@vscode/webview-ui-toolkit/react"
+import React, { useEffect, useState } from "react"
+import { ApiConfiguration } from "../../../src/shared/api"
+import { validateApiConfiguration, validateMaxRequestsPerTask } from "../utils/validate"
+import { vscode } from "../utils/vscode"
+import ApiOptions from "./ApiOptions"
 
 type SettingsViewProps = {
-	apiKey: string
-	setApiKey: React.Dispatch<React.SetStateAction<string>>
+	version: string
+	apiConfiguration?: ApiConfiguration
+	setApiConfiguration: React.Dispatch<React.SetStateAction<ApiConfiguration | undefined>>
 	maxRequestsPerTask: string
 	setMaxRequestsPerTask: React.Dispatch<React.SetStateAction<string>>
-	onDone: () => void // Define the type of the onDone prop
+	customInstructions: string
+	setCustomInstructions: React.Dispatch<React.SetStateAction<string>>
+	onDone: () => void
 }
 
-const SettingsView = ({ apiKey, setApiKey, maxRequestsPerTask, setMaxRequestsPerTask, onDone }: SettingsViewProps) => {
-	const [apiKeyErrorMessage, setApiKeyErrorMessage] = useState<string | undefined>(undefined)
+const SettingsView = ({
+	version,
+	apiConfiguration,
+	setApiConfiguration,
+	maxRequestsPerTask,
+	setMaxRequestsPerTask,
+	customInstructions,
+	setCustomInstructions,
+	onDone,
+}: SettingsViewProps) => {
+	const [apiErrorMessage, setApiErrorMessage] = useState<string | undefined>(undefined)
 	const [maxRequestsErrorMessage, setMaxRequestsErrorMessage] = useState<string | undefined>(undefined)
 
-	const disableDoneButton = apiKeyErrorMessage != null || maxRequestsErrorMessage != null
-
-	const handleApiKeyChange = (event: any) => {
-		const input = event.target.value
-		setApiKey(input)
-		validateApiKey(input)
-	}
-
-	const validateApiKey = (value: string) => {
-		if (value.trim() === "") {
-			setApiKeyErrorMessage("API Key cannot be empty")
-		} else {
-			setApiKeyErrorMessage(undefined)
-		}
-	}
-
-	const handleMaxRequestsChange = (event: any) => {
-		const input = event.target.value
-		setMaxRequestsPerTask(input)
-		validateMaxRequests(input)
-	}
-
-	const validateMaxRequests = (value: string | undefined) => {
-		if (value?.trim()) {
-			const num = Number(value)
-			if (isNaN(num)) {
-				setMaxRequestsErrorMessage("Maximum requests must be a number")
-			} else if (num < 3 || num > 100) {
-				setMaxRequestsErrorMessage("Maximum requests must be between 3 and 100")
-			} else {
-				setMaxRequestsErrorMessage(undefined)
-			}
-		} else {
-			setMaxRequestsErrorMessage(undefined)
-		}
-	}
-
 	const handleSubmit = () => {
-		vscode.postMessage({ type: "apiKey", text: apiKey })
-		vscode.postMessage({ type: "maxRequestsPerTask", text: maxRequestsPerTask })
+		const apiValidationResult = validateApiConfiguration(apiConfiguration)
+		const maxRequestsValidationResult = validateMaxRequestsPerTask(maxRequestsPerTask)
 
-		onDone()
+		setApiErrorMessage(apiValidationResult)
+		setMaxRequestsErrorMessage(maxRequestsValidationResult)
+
+		if (!apiValidationResult && !maxRequestsValidationResult) {
+			vscode.postMessage({ type: "apiConfiguration", apiConfiguration })
+			vscode.postMessage({ type: "maxRequestsPerTask", text: maxRequestsPerTask })
+			vscode.postMessage({ type: "customInstructions", text: customInstructions })
+			onDone()
+		}
 	}
 
+	useEffect(() => {
+		setApiErrorMessage(undefined)
+	}, [apiConfiguration])
+
+	useEffect(() => {
+		setMaxRequestsErrorMessage(undefined)
+	}, [maxRequestsPerTask])
+
 	// validate as soon as the component is mounted
 	/*
 	useEffect will use stale values of variables if they are not included in the dependency array. so trying to use useEffect with a dependency array of only one value for example will use any other variables' old values. In most cases you don't want this, and should opt to use react-use hooks.
@@ -70,106 +63,115 @@ const SettingsView = ({ apiKey, setApiKey, maxRequestsPerTask, setMaxRequestsPer
 
 	If we only want to run code once on mount we can use react-use's useEffectOnce or useMount
 	*/
-	useEffectOnce(() => {
-		const timeoutId = setTimeout(() => {
-			validateApiKey(apiKey)
-			validateMaxRequests(maxRequestsPerTask)
-		}, 1000)
-
-		return () => clearTimeout(timeoutId)
-	})
 
 	return (
-		<div style={{ margin: "0 auto", paddingTop: "10px" }}>
-			<div
-				style={{
-					display: "flex",
-					justifyContent: "space-between",
-					alignItems: "center",
-					marginBottom: "17px",
-				}}>
-				<h3 style={{ color: "var(--vscode-foreground)", margin: 0 }}>Settings</h3>
-				<VSCodeButton onClick={handleSubmit} disabled={disableDoneButton}>
-					Done
-				</VSCodeButton>
-			</div>
+		<div
+			style={{
+				position: "fixed",
+				top: 0,
+				left: 0,
+				right: 0,
+				bottom: 0,
+				padding: "10px 18px 18px 20px",
+				display: "flex",
+				flexDirection: "column",
+				overflow: "hidden",
+			}}>
+			<div style={{ flexGrow: 1, overflow: "auto" }}>
+				<div
+					style={{
+						display: "flex",
+						justifyContent: "space-between",
+						alignItems: "center",
+						marginBottom: "17px",
+					}}>
+					<h3 style={{ color: "var(--vscode-foreground)", margin: 0 }}>Settings</h3>
+					<VSCodeButton onClick={handleSubmit}>Done</VSCodeButton>
+				</div>
 
-			<div style={{ marginBottom: "20px" }}>
-				<VSCodeTextField
-					value={apiKey}
-					style={{ width: "100%" }}
-					placeholder="Enter your Anthropic API Key"
-					onInput={handleApiKeyChange}>
-					<span style={{ fontWeight: "500" }}>Anthropic API Key</span>
-				</VSCodeTextField>
-				{apiKeyErrorMessage && (
+				<div style={{ marginBottom: 5 }}>
+					<ApiOptions
+						apiConfiguration={apiConfiguration}
+						setApiConfiguration={setApiConfiguration}
+						showModelOptions={true}
+					/>
+					{apiErrorMessage && (
+						<p
+							style={{
+								margin: "-5px 0 12px 0",
+								fontSize: "12px",
+								color: "var(--vscode-errorForeground)",
+							}}>
+							{apiErrorMessage}
+						</p>
+					)}
+				</div>
+
+				<div style={{ marginBottom: 15 }}>
+					<VSCodeTextArea
+						value={customInstructions}
+						style={{ width: "100%" }}
+						rows={4}
+						placeholder={
+							'e.g. "Run unit tests at the end", "Use TypeScript with async/await", "Speak in Spanish"'
+						}
+						onInput={(e: any) => setCustomInstructions(e.target?.value || "")}>
+						<span style={{ fontWeight: "500" }}>Custom Instructions</span>
+					</VSCodeTextArea>
 					<p
 						style={{
 							fontSize: "12px",
 							marginTop: "5px",
-							color: "var(--vscode-errorForeground)",
+							color: "var(--vscode-descriptionForeground)",
 						}}>
-						{apiKeyErrorMessage}
+						These instructions are added to the end of the system prompt sent with every request.
 					</p>
-				)}
-				<p
-					style={{
-						fontSize: "12px",
-						marginTop: "5px",
-						color: "var(--vscode-descriptionForeground)",
-					}}>
-					This key is not shared with anyone and only used to make API requests from the extension.
-					<VSCodeLink href="https://console.anthropic.com/" style={{ display: "inline" }}>
-						You can get an API key by signing up here.
-					</VSCodeLink>
-				</p>
-			</div>
+				</div>
 
-			<div style={{ marginBottom: "20px" }}>
-				<VSCodeTextField
-					value={maxRequestsPerTask}
-					style={{ width: "100%" }}
-					placeholder="20"
-					onInput={handleMaxRequestsChange}>
-					<span style={{ fontWeight: "500" }}>Maximum # Requests Per Task</span>
-				</VSCodeTextField>
-				{maxRequestsErrorMessage && (
+				<div style={{ marginBottom: 20 }}>
+					<VSCodeTextField
+						value={maxRequestsPerTask}
+						style={{ width: "100%" }}
+						placeholder="20"
+						onInput={(e: any) => setMaxRequestsPerTask(e.target?.value)}>
+						<span style={{ fontWeight: "500" }}>Maximum # Requests Per Task</span>
+					</VSCodeTextField>
 					<p
 						style={{
 							fontSize: "12px",
 							marginTop: "5px",
-							color: "var(--vscode-errorForeground)",
+							color: "var(--vscode-descriptionForeground)",
 						}}>
-						{maxRequestsErrorMessage}
+						If Claude Dev reaches this limit, it will pause and ask for your permission before making
+						additional requests.
 					</p>
-				)}
-				<p
-					style={{
-						fontSize: "12px",
-						marginTop: "5px",
-						color: "var(--vscode-descriptionForeground)",
-					}}>
-					If Claude Dev reaches this limit, it will pause and ask for your permission before making additional
-					requests.
-				</p>
+					{maxRequestsErrorMessage && (
+						<p
+							style={{
+								fontSize: "12px",
+								marginTop: "5px",
+								color: "var(--vscode-errorForeground)",
+							}}>
+							{maxRequestsErrorMessage}
+						</p>
+					)}
+				</div>
 			</div>
 
-			<VSCodeDivider />
-
 			<div
 				style={{
-					marginTop: "20px",
 					textAlign: "center",
 					color: "var(--vscode-descriptionForeground)",
 					fontSize: "12px",
 					lineHeight: "1.2",
-					fontStyle: "italic",
 				}}>
-				<p>
-					<VSCodeLink href="https://github.com/saoudrizwan/claude-dev">
+				<p style={{ wordWrap: "break-word", margin: 0, padding: 0 }}>
+					If you have any questions or feedback, feel free to open an issue at{" "}
+					<VSCodeLink href="https://github.com/saoudrizwan/claude-dev" style={{ display: "inline" }}>
 						https://github.com/saoudrizwan/claude-dev
 					</VSCodeLink>
 				</p>
+				<p style={{ fontStyle: "italic", margin: "10px 0 0 0", padding: 0 }}>v{version}</p>
 			</div>
 		</div>
 	)

webview-ui/src/components/TaskHeader.tsx

@@ -2,10 +2,12 @@ import { VSCodeButton } from "@vscode/webview-ui-toolkit/react"
 import React, { useEffect, useRef, useState } from "react"
 import TextTruncate from "react-text-truncate"
 import { useWindowSize } from "react-use"
-import { vscode } from "../utilities/vscode"
+import { ClaudeMessage } from "../../../src/shared/ExtensionMessage"
+import { vscode } from "../utils/vscode"
+import Thumbnails from "./Thumbnails"
 
 interface TaskHeaderProps {
-	taskText: string
+	task: ClaudeMessage
 	tokensIn: number
 	tokensOut: number
 	totalCost: number
@@ -13,7 +15,7 @@ interface TaskHeaderProps {
 	isHidden: boolean
 }
 
-const TaskHeader: React.FC<TaskHeaderProps> = ({ taskText, tokensIn, tokensOut, totalCost, onClose, isHidden }) => {
+const TaskHeader: React.FC<TaskHeaderProps> = ({ task, tokensIn, tokensOut, totalCost, onClose, isHidden }) => {
 	const [isExpanded, setIsExpanded] = useState(false)
 	const [textTruncateKey, setTextTruncateKey] = useState(0)
 	const textContainerRef = useRef<HTMLDivElement>(null)
@@ -116,7 +118,7 @@ const TaskHeader: React.FC<TaskHeaderProps> = ({ taskText, tokensIn, tokensOut,
 						line={isExpanded ? 0 : 3}
 						element="span"
 						truncateText="…"
-						text={taskText}
+						text={task.text}
 						textTruncateChild={
 							<span
 								style={{
@@ -141,16 +143,10 @@ const TaskHeader: React.FC<TaskHeaderProps> = ({ taskText, tokensIn, tokensOut,
 						</span>
 					)}
 				</div>
+				{task.images && task.images.length > 0 && <Thumbnails images={task.images} />}
 				<div style={{ display: "flex", flexDirection: "column", gap: "4px" }}>
 					<div style={{ display: "flex", alignItems: "center", gap: "4px" }}>
 						<span style={{ fontWeight: "bold" }}>Tokens:</span>
-						<span style={{ display: "flex", alignItems: "center", gap: "3px" }}>
-							<i
-								className="codicon codicon-arrow-down"
-								style={{ fontSize: "12px", fontWeight: "bold", marginBottom: "-2px" }}
-							/>
-							{tokensOut.toLocaleString()}
-						</span>
 						<span style={{ display: "flex", alignItems: "center", gap: "3px" }}>
 							<i
 								className="codicon codicon-arrow-up"
@@ -158,6 +154,13 @@ const TaskHeader: React.FC<TaskHeaderProps> = ({ taskText, tokensIn, tokensOut,
 							/>
 							{tokensIn.toLocaleString()}
 						</span>
+						<span style={{ display: "flex", alignItems: "center", gap: "3px" }}>
+							<i
+								className="codicon codicon-arrow-down"
+								style={{ fontSize: "12px", fontWeight: "bold", marginBottom: "-2px" }}
+							/>
+							{tokensOut.toLocaleString()}
+						</span>
 					</div>
 					<div
 						style={{

webview-ui/src/components/Thumbnails.tsx

@@ -0,0 +1,91 @@
+import React, { useState, useRef, useLayoutEffect } from "react"
+import { useWindowSize } from "react-use"
+
+interface ThumbnailsProps {
+	images: string[]
+	style?: React.CSSProperties
+	setImages?: React.Dispatch<React.SetStateAction<string[]>>
+	onHeightChange?: (height: number) => void
+}
+
+const Thumbnails: React.FC<ThumbnailsProps> = ({ images, style, setImages, onHeightChange }) => {
+	const [hoveredIndex, setHoveredIndex] = useState<number | null>(null)
+	const containerRef = useRef<HTMLDivElement>(null)
+	const { width } = useWindowSize()
+
+	useLayoutEffect(() => {
+		if (containerRef.current) {
+			let height = containerRef.current.clientHeight
+			// some browsers return 0 for clientHeight
+			if (!height) {
+				height = containerRef.current.getBoundingClientRect().height
+			}
+			onHeightChange?.(height)
+		}
+		setHoveredIndex(null)
+	}, [images, width, onHeightChange])
+
+	const handleDelete = (index: number) => {
+		setImages?.((prevImages) => prevImages.filter((_, i) => i !== index))
+	}
+
+	const isDeletable = setImages !== undefined
+
+	return (
+		<div
+			ref={containerRef}
+			style={{
+				display: "flex",
+				flexWrap: "wrap",
+				gap: 5,
+				rowGap: 3,
+				...style,
+			}}>
+			{images.map((image, index) => (
+				<div
+					key={index}
+					style={{ position: "relative" }}
+					onMouseEnter={() => setHoveredIndex(index)}
+					onMouseLeave={() => setHoveredIndex(null)}>
+					<img
+						src={image}
+						alt={`Thumbnail ${index + 1}`}
+						style={{
+							width: 34,
+							height: 34,
+							objectFit: "cover",
+							borderRadius: 4,
+						}}
+					/>
+					{isDeletable && hoveredIndex === index && (
+						<div
+							onClick={() => handleDelete(index)}
+							style={{
+								position: "absolute",
+								top: -4,
+								right: -4,
+								width: 13,
+								height: 13,
+								borderRadius: "50%",
+								backgroundColor: "var(--vscode-badge-background)",
+								display: "flex",
+								justifyContent: "center",
+								alignItems: "center",
+								cursor: "pointer",
+							}}>
+							<span
+								className="codicon codicon-close"
+								style={{
+									color: "var(--vscode-foreground)",
+									fontSize: 10,
+									fontWeight: "bold",
+								}}></span>
+						</div>
+					)}
+				</div>
+			))}
+		</div>
+	)
+}
+
+export default Thumbnails

webview-ui/src/components/WelcomeView.tsx

@@ -1,32 +1,27 @@
-import React, { useState, useEffect } from "react"
-import { VSCodeButton, VSCodeTextField, VSCodeLink, VSCodeDivider } from "@vscode/webview-ui-toolkit/react"
-import { vscode } from "../utilities/vscode"
+import { VSCodeButton, VSCodeLink } from "@vscode/webview-ui-toolkit/react"
+import React, { useEffect, useState } from "react"
+import { ApiConfiguration } from "../../../src/shared/api"
+import { validateApiConfiguration } from "../utils/validate"
+import { vscode } from "../utils/vscode"
+import ApiOptions from "./ApiOptions"
 
 interface WelcomeViewProps {
-	apiKey: string
-	setApiKey: React.Dispatch<React.SetStateAction<string>>
+	apiConfiguration?: ApiConfiguration
+	setApiConfiguration: React.Dispatch<React.SetStateAction<ApiConfiguration | undefined>>
 }
 
-const WelcomeView: React.FC<WelcomeViewProps> = ({ apiKey, setApiKey }) => {
-	const [apiKeyErrorMessage, setApiKeyErrorMessage] = useState<string | undefined>(undefined)
+const WelcomeView: React.FC<WelcomeViewProps> = ({ apiConfiguration, setApiConfiguration }) => {
+	const [apiErrorMessage, setApiErrorMessage] = useState<string | undefined>(undefined)
 
-	const disableLetsGoButton = apiKeyErrorMessage != null
-
-	const validateApiKey = (value: string) => {
-		if (value.trim() === "") {
-			setApiKeyErrorMessage("API Key cannot be empty")
-		} else {
-			setApiKeyErrorMessage(undefined)
-		}
-	}
+	const disableLetsGoButton = apiErrorMessage != null
 
 	const handleSubmit = () => {
-		vscode.postMessage({ type: "apiKey", text: apiKey })
+		vscode.postMessage({ type: "apiConfiguration", apiConfiguration })
 	}
 
 	useEffect(() => {
-		validateApiKey(apiKey)
-	}, [apiKey])
+		setApiErrorMessage(validateApiConfiguration(apiConfiguration))
+	}, [apiConfiguration])
 
 	return (
 		<div style={{ position: "fixed", top: 0, left: 0, right: 0, bottom: 0, padding: "0 20px" }}>
@@ -38,39 +33,22 @@ const WelcomeView: React.FC<WelcomeViewProps> = ({ apiKey, setApiKey }) => {
 					style={{ display: "inline" }}>
 					Claude 3.5 Sonnet's agentic coding capabilities.
 				</VSCodeLink>{" "}
-				I am prompted to think through tasks step-by-step and have access to tools that let me read & write
-				files, analyze project source code, and execute terminal commands (with your permission, of course).
+				I am prompted to think through tasks step-by-step and have access to tools that let me create & edit
+				files, explore complex projects, and execute terminal commands (with your permission, of course).
 			</p>
 
-			<b>To get started, this extension needs an Anthropic API key:</b>
-			<ol style={{ paddingLeft: "15px" }}>
-				<li>
-					Go to{" "}
-					<VSCodeLink href="https://console.anthropic.com" style={{ display: "inline" }}>
-						https://console.anthropic.com
-					</VSCodeLink>
-				</li>
-				<li>You may need to buy some credits (although Anthropic is offering $5 free credit for new users)</li>
-				<li>Click 'Get API Keys' and create a new key (you can delete it any time)</li>
-			</ol>
+			<b>To get started, this extension needs an API key for Claude 3.5 Sonnet:</b>
 
-			<VSCodeDivider />
-
-			<div style={{ marginTop: "20px", display: "flex", alignItems: "center" }}>
-				<VSCodeTextField
-					style={{ flexGrow: 1, marginRight: "10px" }}
-					placeholder="Enter API Key..."
-					value={apiKey}
-					onInput={(e: any) => setApiKey(e.target.value)}
+			<div style={{ marginTop: "15px" }}>
+				<ApiOptions
+					apiConfiguration={apiConfiguration}
+					setApiConfiguration={setApiConfiguration}
+					showModelOptions={false}
 				/>
-				<VSCodeButton onClick={handleSubmit} disabled={disableLetsGoButton}>
-					Submit
+				<VSCodeButton onClick={handleSubmit} disabled={disableLetsGoButton} style={{ marginTop: "3px" }}>
+					Let's go!
 				</VSCodeButton>
 			</div>
-
-			<p style={{ fontSize: "12px", marginTop: "10px", color: "var(--vscode-descriptionForeground)" }}>
-				Your API key is stored securely on your computer and used only for interacting with the Anthropic API.
-			</p>
 		</div>
 	)
 }

webview-ui/src/index.css

@@ -109,3 +109,31 @@ The above scrollbar styling uses some transparent background color magic to acco
 .code-block-scrollable::-webkit-scrollbar-corner {
 	background-color: transparent;
 }
+
+/*
+Dropdown label
+https://github.com/microsoft/vscode-webview-ui-toolkit/tree/main/src/dropdown#with-label
+*/
+.dropdown-container {
+	box-sizing: border-box;
+	display: flex;
+	flex-flow: column nowrap;
+	align-items: flex-start;
+	justify-content: flex-start;
+}
+.dropdown-container label {
+	display: block;
+	color: var(--vscode-foreground);
+	cursor: pointer;
+	font-size: var(--vscode-font-size);
+	line-height: normal;
+	margin-bottom: 2px;
+}
+
+/* Fix scrollbar in dropdown */
+
+vscode-dropdown::part(listbox) {
+	border-color: var(--vscode-scrollbarSlider-background);
+	transition: none;
+	scrollbar-color: var(--vscode-scrollbarSlider-background) transparent;
+}

webview-ui/src/utils/combineApiRequests.ts

@@ -1,15 +1,15 @@
-import { ClaudeMessage } from "@shared/ExtensionMessage"
+import { ClaudeMessage } from "../../../src/shared/ExtensionMessage"
 
 /**
  * Combines API request start and finish messages in an array of ClaudeMessages.
- * 
+ *
  * This function looks for pairs of 'api_req_started' and 'api_req_finished' messages.
  * When it finds a pair, it combines them into a single 'api_req_combined' message.
  * The JSON data in the text fields of both messages are merged.
- * 
+ *
  * @param messages - An array of ClaudeMessage objects to process.
  * @returns A new array of ClaudeMessage objects with API requests combined.
- * 
+ *
  * @example
  * const messages = [
  *   { type: "say", say: "api_req_started", text: '{"request":"GET /api/data"}', ts: 1000 },

webview-ui/src/utils/combineCommandSequences.ts

@@ -1,4 +1,4 @@
-import { ClaudeMessage } from "@shared/ExtensionMessage"
+import { ClaudeMessage } from "../../../src/shared/ExtensionMessage"
 
 /**
  * Combines sequences of command and command_output messages in an array of ClaudeMessages.
@@ -41,7 +41,11 @@ export function combineCommandSequences(messages: ClaudeMessage[]): ClaudeMessag
 						combinedText += `\n${COMMAND_OUTPUT_STRING}`
 						didAddOutput = true
 					}
-					combinedText += "\n" + (messages[j].text || "")
+					// handle cases where we receive empty command_output (ie when extension is relinquishing control over exit command button)
+					const output = messages[j].text || ""
+					if (output.length > 0) {
+						combinedText += "\n" + output
+					}
 				}
 				j++
 			}

webview-ui/src/utils/getApiMetrics.ts

@@ -1,4 +1,4 @@
-import { ClaudeMessage } from "@shared/ExtensionMessage"
+import { ClaudeMessage } from "../../../src/shared/ExtensionMessage"
 
 interface ApiMetrics {
 	totalTokensIn: number

webview-ui/src/utils/getLanguageFromPath.ts

@@ -85,6 +85,5 @@ const extensionToLanguage: { [key: string]: string } = {
 
 export function getLanguageFromPath(path: string): string | undefined {
 	const extension = path.split(".").pop()?.toLowerCase() || ""
-    console.log(`converted path to language: ${path} -> ${extensionToLanguage[extension]}`)
 	return extensionToLanguage[extension]
 }

webview-ui/src/utils/mockMessages.ts

@@ -1,4 +1,4 @@
-import { ClaudeMessage } from "@shared/ExtensionMessage";
+import { ClaudeMessage } from "../../../src/shared/ExtensionMessage"
 
 export const mockMessages: ClaudeMessage[] = [
 	{

webview-ui/src/utils/validate.ts

@@ -0,0 +1,34 @@
+import { ApiConfiguration } from "../../../src/shared/api"
+
+export function validateApiConfiguration(apiConfiguration?: ApiConfiguration): string | undefined {
+	if (apiConfiguration) {
+		switch (apiConfiguration.apiProvider) {
+			case "anthropic":
+				if (!apiConfiguration.apiKey) {
+					return "You must provide a valid API key or choose a different provider."
+				}
+				break
+			case "bedrock":
+				if (!apiConfiguration.awsAccessKey || !apiConfiguration.awsSecretKey || !apiConfiguration.awsRegion) {
+					return "You must provide a valid AWS access key, secret key, and region."
+				}
+				break
+			case "openrouter":
+				if (!apiConfiguration.openRouterApiKey) {
+					return "You must provide a valid API key or choose a different provider."
+				}
+				break
+		}
+	}
+	return undefined
+}
+
+export function validateMaxRequestsPerTask(maxRequestsPerTask?: string): string | undefined {
+	if (maxRequestsPerTask && maxRequestsPerTask.trim()) {
+		const num = Number(maxRequestsPerTask)
+		if (isNaN(num) || num < 3 || num > 100) {
+			return "Maximum requests must be between 3 and 100"
+		}
+	}
+	return undefined
+}

webview-ui/src/utils/vscode.ts

@@ -1,4 +1,4 @@
-import { WebviewMessage } from "@shared/WebviewMessage"
+import { WebviewMessage } from "../../../src/shared/WebviewMessage"
 import type { WebviewApi } from "vscode-webview"
 
 /**

webview-ui/tsconfig.json

@@ -14,10 +14,7 @@
 		"resolveJsonModule": true,
 		"isolatedModules": true,
 		"noEmit": true,
-		"jsx": "react-jsx",
-		"paths": {
-			"@shared/*": ["../src/shared/*"]
-		}
+		"jsx": "react-jsx"
 	},
 	"include": ["src", "../src/shared"]
 }