Compare commits

..
Author SHA1 Message Date
celestial-vault 5fbc5385e7 changeset 2025-03-31 18:54:19 -07:00
celestial-vault 9640caaba8 handle webview message for creating remote server 2025-03-31 18:53:53 -07:00
383 changed files with 5076 additions and 65064 deletions
-5
View File
@@ -1,5 +0,0 @@
---
"claude-dev": minor
---
add open ai cache to ui
+5
View File
@@ -0,0 +1,5 @@
---
"claude-dev": patch
---
fix bug in Requestly UX for model selection
-5
View File
@@ -1,5 +0,0 @@
---
"claude-dev": patch
---
Add confirmation dialog to Delete All History button
-5
View File
@@ -1,5 +0,0 @@
---
"claude-dev": minor
---
breakpoint just in system prompt for gemini for OR and cline provider
+5
View File
@@ -0,0 +1,5 @@
---
"claude-dev": minor
---
Add extension handler for adding remote server
+5
View File
@@ -0,0 +1,5 @@
---
"claude-dev": minor
---
Add remote server programatically
+5
View File
@@ -0,0 +1,5 @@
---
"claude-dev": patch
---
Add support for bad commitHashes already stored in ui_messages.json
-5
View File
@@ -1,5 +0,0 @@
---
"claude-dev": minor
---
updated gemini caching for OR and cline provider
-5
View File
@@ -1,5 +0,0 @@
---
"claude-dev": patch
---
adding quote reply support
-5
View File
@@ -1,5 +0,0 @@
---
"claude-dev": patch
---
Make Previous Updates in the Announcement a dropdown
-5
View File
@@ -1,5 +0,0 @@
---
"claude-dev": minor
---
add ui for windsurf and cursor rules
-5
View File
@@ -1,5 +0,0 @@
---
"claude-dev": patch
---
Batch selection and deletion of tasks in history
-5
View File
@@ -1,5 +0,0 @@
---
"claude-dev": minor
---
Migrate getting-started section to new docs
+5
View File
@@ -0,0 +1,5 @@
---
"claude-dev": patch
---
Escape html content for gemini when running commands
+43 -289
View File
@@ -8,18 +8,16 @@ Cline is a VSCode extension that provides AI assistance through a combination of
```mermaid
graph TB
subgraph VSCodeExtensionHost[VSCode Extension Host]
subgraph CoreExtension[Core Extension]
subgraph VSCode Extension Host
subgraph Core Extension
ExtensionEntry[Extension Entry<br/>src/extension.ts]
WebviewProvider[WebviewProvider<br/>src/core/webview/index.ts]
Controller[Controller<br/>src/core/controller/index.ts]
Task[Task<br/>src/core/task/index.ts]
ClineProvider[ClineProvider<br/>src/core/webview/ClineProvider.ts]
ClineClass[Cline Class<br/>src/core/Cline.ts]
GlobalState[VSCode Global State]
SecretsStorage[VSCode Secrets Storage]
McpHub[McpHub<br/>src/services/mcp/McpHub.ts]
end
subgraph WebviewUI[Webview UI]
subgraph Webview UI
WebviewApp[React App<br/>webview-ui/src/App.tsx]
ExtStateContext[ExtensionStateContext<br/>webview-ui/src/context/ExtensionStateContext.tsx]
ReactComponents[React Components]
@@ -29,101 +27,45 @@ graph TB
TaskStorage[Task Storage<br/>Per-Task Files & History]
CheckpointSystem[Git-based Checkpoints]
end
subgraph apiProviders[API Providers]
AnthropicAPI[Anthropic]
OpenRouterAPI[OpenRouter]
BedrockAPI[AWS Bedrock]
OtherAPIs[Other Providers]
end
subgraph MCPServers[MCP Servers]
ExternalMcpServers[External MCP Servers]
end
end
%% Core Extension Data Flow
ExtensionEntry --> WebviewProvider
WebviewProvider --> Controller
Controller --> Task
Controller --> McpHub
Task --> GlobalState
Task --> SecretsStorage
Task --> TaskStorage
Task --> CheckpointSystem
Task --> |API Requests| apiProviders
McpHub --> |Connects to| ExternalMcpServers
Task --> |Uses| McpHub
ExtensionEntry --> ClineProvider
ClineProvider --> ClineClass
ClineClass --> GlobalState
ClineClass --> SecretsStorage
ClineClass --> TaskStorage
ClineClass --> CheckpointSystem
%% Webview Data Flow
WebviewApp --> ExtStateContext
ExtStateContext --> ReactComponents
%% Bidirectional Communication
WebviewProvider <-->|postMessage| ExtStateContext
ClineProvider <-->|postMessage| ExtStateContext
style GlobalState fill:#f9f,stroke:#333,stroke-width:2px
style SecretsStorage fill:#f9f,stroke:#333,stroke-width:2px
style ExtStateContext fill:#bbf,stroke:#333,stroke-width:2px
style WebviewProvider fill:#bfb,stroke:#333,stroke-width:2px
style McpHub fill:#bfb,stroke:#333,stroke-width:2px
style apiProviders fill:#fdb,stroke:#333,stroke-width:2px
style ClineProvider fill:#bfb,stroke:#333,stroke-width:2px
```
## Definitions
- **Core Extension**: Anything inside the src folder, organized into modular components
- **Core Extension State**: Managed by the Controller class in src/core/controller/index.ts, which serves as the single source of truth for the extension's state. It manages multiple types of persistent storage (global state, workspace state, and secrets), handles state distribution to both the core extension and webview components, and coordinates state across multiple extension instances. This includes managing API configurations, task history, settings, and MCP configurations.
- **Webview**: Anything inside the webview-ui. All the react or view's seen by the user and user interaction components
- **Webview State**: Managed by ExtensionStateContext in webview-ui/src/context/ExtensionStateContext.tsx, which provides React components with access to the extension's state through a context provider pattern. It maintains local state for UI components, handles real-time updates through message events, manages partial message updates, and provides methods for state modifications. The context includes extension version, messages, task history, theme, API configurations, MCP servers, marketplace catalog, and workspace file paths. It synchronizes with the core extension through VSCode's message passing system and provides type-safe access to state through a custom hook (useExtensionState).
### Core Extension Architecture
The core extension follows a clear hierarchical structure:
1. **WebviewProvider** (src/core/webview/index.ts): Manages the webview lifecycle and communication
2. **Controller** (src/core/controller/index.ts): Handles webview messages and task management
3. **Task** (src/core/task/index.ts): Executes API requests and tool operations
This architecture provides clear separation of concerns:
- WebviewProvider focuses on VSCode webview integration
- Controller manages state and coordinates tasks
- Task handles the execution of AI requests and tool operations
### WebviewProvider Implementation
The WebviewProvider class in `src/core/webview/index.ts` is responsible for:
- Managing multiple active instances through a static set (`activeInstances`)
- Handling webview lifecycle events (creation, visibility changes, disposal)
- Implementing HTML content generation with proper CSP headers
- Supporting Hot Module Replacement (HMR) for development
- Setting up message listeners between the webview and extension
The WebviewProvider maintains a reference to the Controller and delegates message handling to it. It also handles the creation of both sidebar and tab panel webviews, allowing Cline to be used in different contexts within VSCode.
- core extension: Anything inside the src folder starting with the Cline.ts file
- core extension state: Managed by the ClineProvider class in src/core/webview/ClineProvider.ts, which serves as the single source of truth for the extension's state. It manages multiple types of persistent storage (global state, workspace state, and secrets), handles state distribution to both the core extension and webview components, and coordinates state across multiple extension instances. This includes managing API configurations, task history, settings, and MCP configurations.
- webview: Anything inside the webview-ui. All the react or view's seen by the user and user interaction compone
- webview state: Managed by ExtensionStateContext in webview-ui/src/context/ExtensionStateContext.tsx, which provides React components with access to the extension's state through a context provider pattern. It maintains local state for UI components, handles real-time updates through message events, manages partial message updates, and provides methods for state modifications. The context includes extension version, messages, task history, theme, API configurations, MCP servers, marketplace catalog, and workspace file paths. It synchronizes with the core extension through VSCode's message passing system and provides type-safe access to state through a custom hook (useExtensionState).
### Core Extension State
The `Controller` class manages multiple types of persistent storage:
The `ClineProvider` class manages multiple types of persistent storage:
- **Global State:** Stored across all VSCode instances. Used for settings and data that should persist globally.
- **Workspace State:** Specific to the current workspace. Used for task-specific data and settings.
- **Secrets:** Secure storage for sensitive information like API keys.
The `Controller` handles the distribution of state to both the core extension and webview components. It also coordinates state across multiple extension instances, ensuring consistency.
State synchronization between instances is handled through:
- File-based storage for task history and conversation data
- VSCode's global state API for settings and configuration
- Secrets storage for sensitive information
- Event listeners for file changes and configuration updates
The Controller implements methods for:
- Saving and loading task state
- Managing API configurations
- Handling user authentication
- Coordinating MCP server connections
- Managing task history and checkpoints
The `ClineProvider` handles the distribution of state to both the core extension and webview components. It also coordinates state across multiple extension instances, ensuring consistency.
### Webview State
@@ -140,65 +82,16 @@ The `ExtensionStateContext` in `webview-ui/src/context/ExtensionStateContext.tsx
It synchronizes with the core extension through VSCode's message passing system and provides type-safe access to the state via a custom hook (`useExtensionState`).
The ExtensionStateContext handles:
- Real-time updates through message events
- Partial message updates for streaming content
- State modifications through setter methods
- Type-safe access to state through a custom hook
## Core Extension (Cline.ts)
## API Provider System
Cline supports multiple AI providers through a modular API provider system. Each provider is implemented as a separate module in the `src/api/providers/` directory and follows a common interface.
### API Provider Architecture
The API system consists of:
1. **API Handlers**: Provider-specific implementations in `src/api/providers/`
2. **API Transformers**: Stream transformation utilities in `src/api/transform/`
3. **API Configuration**: User settings for API keys and endpoints
4. **API Factory**: Builder function to create the appropriate handler
Key providers include:
- **Anthropic**: Direct integration with Claude models
- **OpenRouter**: Meta-provider supporting multiple model providers
- **AWS Bedrock**: Integration with Amazon's AI services
- **Gemini**: Google's AI models
- **Ollama**: Local model hosting
- **LM Studio**: Local model hosting
- **VSCode LM**: VSCode's built-in language models
### API Configuration Management
API configurations are stored securely:
- API keys are stored in VSCode's secrets storage
- Model selections and non-sensitive settings are stored in global state
- The Controller manages switching between providers and updating configurations
The system supports:
- Secure storage of API keys
- Model selection and configuration
- Automatic retry and error handling
- Token usage tracking and cost calculation
- Context window management
### Plan/Act Mode API Configuration
Cline supports separate model configurations for Plan and Act modes:
- Different models can be used for planning vs. execution
- The system preserves model selections when switching modes
- The Controller handles the transition between modes and updates the API configuration accordingly
## Task Execution System
The Task class is responsible for executing AI requests and tool operations. Each task runs in its own instance of the Task class, ensuring isolation and proper state management.
The Cline class is the heart of the extension, managing task execution, state persistence, and tool coordination. Each task runs in its own instance of the Cline class, ensuring isolation and proper state management.
### Task Execution Loop
The core task execution loop follows this pattern:
```typescript
class Task {
class Cline {
async initiateTaskLoop(userContent: UserContent, isNewTask: boolean) {
while (!this.abort) {
// 1. Make API request and stream response
@@ -233,7 +126,7 @@ class Task {
The streaming system handles real-time updates and partial content:
```typescript
class Task {
class Cline {
async presentAssistantMessage() {
// Handle streaming locks to prevent race conditions
if (this.presentAssistantMessageLocked) {
@@ -268,7 +161,7 @@ class Task {
Tools follow a strict execution pattern:
```typescript
class Task {
class Cline {
async executeToolWithApproval(block: ToolBlock) {
// 1. Check auto-approval settings
if (this.shouldAutoApproveTool(block.name)) {
@@ -300,7 +193,7 @@ class Task {
The system includes robust error handling:
```typescript
class Task {
class Cline {
async handleError(action: string, error: Error) {
// 1. Check if task was abandoned
if (this.abandoned) return
@@ -323,23 +216,23 @@ class Task {
### API Request & Token Management
The Task class handles API requests with built-in retry, streaming, and token management:
The Cline class handles API requests with built-in retry, streaming, and token management:
```typescript
class Task {
class Cline {
async *attemptApiRequest(previousApiReqIndex: number): ApiStream {
// 1. Wait for MCP servers to connect
await pWaitFor(() => this.controllerRef.deref()?.mcpHub?.isConnecting !== true)
await pWaitFor(() => this.providerRef.deref()?.mcpHub?.isConnecting !== true)
// 2. Manage context window
const previousRequest = this.clineMessages[previousApiReqIndex]
if (previousRequest?.text) {
const { tokensIn, tokensOut } = JSON.parse(previousRequest.text || "{}")
const { tokensIn, tokensOut } = JSON.parse(previousRequest.text)
const totalTokens = (tokensIn || 0) + (tokensOut || 0)
// Truncate conversation if approaching context limit
if (totalTokens >= maxAllowedSize) {
this.conversationHistoryDeletedRange = this.contextManager.getNextTruncationRange(
this.conversationHistoryDeletedRange = getNextTruncationRange(
this.apiConversationHistory,
this.conversationHistoryDeletedRange,
totalTokens / 2 > maxAllowedSize ? "quarter" : "half"
@@ -406,32 +299,16 @@ Key features:
- Cost calculation
- Cache hit monitoring
### Context Management System
The Context Management System handles conversation history truncation to prevent context window overflow errors. Implemented in the `ContextManager` class, it ensures long-running conversations remain within model context limits while preserving critical context.
Key features:
1. **Model-Aware Sizing**: Dynamically adjusts based on different model context windows (64K for DeepSeek, 128K for most models, 200K for Claude).
2. **Proactive Truncation**: Monitors token usage and preemptively truncates conversations when approaching limits, maintaining buffers of 27K-40K tokens depending on the model.
3. **Intelligent Preservation**: Always preserves the original task message and maintains the user-assistant conversation structure when truncating.
4. **Adaptive Strategies**: Uses different truncation strategies based on context pressure - removing half of the conversation for moderate pressure or three-quarters for severe pressure.
5. **Error Recovery**: Includes specialized detection for context window errors from different providers with automatic retry and more aggressive truncation when needed.
### Task State & Resumption
The Task class provides robust task state management and resumption capabilities:
The Cline class provides robust task state management and resumption capabilities:
```typescript
class Task {
class Cline {
async resumeTaskFromHistory() {
// 1. Load saved state
this.clineMessages = await getSavedClineMessages(this.getContext(), this.taskId)
this.apiConversationHistory = await getSavedApiConversationHistory(this.getContext(), this.taskId)
this.clineMessages = await this.getSavedClineMessages()
this.apiConversationHistory = await this.getSavedApiConversationHistory()
// 2. Handle interrupted tool executions
const lastMessage = this.apiConversationHistory[this.apiConversationHistory.length - 1]
@@ -461,14 +338,14 @@ class Task {
private async saveTaskState() {
// Save conversation history
await saveApiConversationHistory(this.getContext(), this.taskId, this.apiConversationHistory)
await saveClineMessages(this.getContext(), this.taskId, this.clineMessages)
await this.saveApiConversationHistory()
await this.saveClineMessages()
// Create checkpoint
const commitHash = await this.checkpointTracker?.commit()
// Update task history
await this.controllerRef.deref()?.updateTaskHistory({
await this.providerRef.deref()?.updateTaskHistory({
id: this.taskId,
ts: lastMessage.ts,
task: taskMessage.text,
@@ -504,54 +381,11 @@ Key aspects of task state management:
- Resources are cleaned up properly
- User is notified of state changes
## Plan/Act Mode System
Cline implements a dual-mode system that separates planning from execution:
### Mode Architecture
The Plan/Act mode system consists of:
1. **Mode State**: Stored in `chatSettings.mode` in the Controller's state
2. **Mode Switching**: Handled by `togglePlanActModeWithChatSettings` in the Controller
3. **Mode-specific Models**: Optional configuration to use different models for each mode
4. **Mode-specific Prompting**: Different system prompts for planning vs. execution
### Mode Switching Process
When switching between modes:
1. The current model configuration is saved to mode-specific state
2. The previous mode's model configuration is restored
3. The Task instance is updated with the new mode
4. The webview is notified of the mode change
5. Telemetry events are captured for analytics
### Plan Mode
Plan mode is designed for:
- Information gathering and context building
- Asking clarifying questions
- Creating detailed execution plans
- Discussing approaches with the user
In Plan mode, the AI uses the `plan_mode_respond` tool to engage in conversational planning without executing actions.
### Act Mode
Act mode is designed for:
- Executing the planned actions
- Using tools to modify files, run commands, etc.
- Implementing the solution
- Providing results and completion feedback
In Act mode, the AI has access to all tools except `plan_mode_respond` and focuses on implementation rather than discussion.
## Data Flow & State Management
### Core Extension Role
The Controller acts as the single source of truth for all persistent state. It:
The core extension (ClineProvider) acts as the single source of truth for all persistent state. It:
- Manages VSCode global state and secrets storage
- Coordinates state updates between components
- Ensures state consistency across webview reloads
@@ -560,10 +394,10 @@ The Controller acts as the single source of truth for all persistent state. It:
### Terminal Management
The Task class manages terminal instances and command execution:
The Cline class manages terminal instances and command execution:
```typescript
class Task {
class Cline {
async executeCommandTool(command: string): Promise<[boolean, ToolResponse]> {
// 1. Get or create terminal
const terminalInfo = await this.terminalManager.getOrCreateTerminal(cwd)
@@ -619,10 +453,10 @@ Key features:
### Browser Session Management
The Task class handles browser automation through Puppeteer:
The Cline class handles browser automation through Puppeteer:
```typescript
class Task {
class Cline {
async executeBrowserAction(action: BrowserAction): Promise<BrowserActionResult> {
switch (action) {
case "launch":
@@ -659,93 +493,13 @@ Key aspects:
- Screenshot capture
- Error recovery
## MCP (Model Context Protocol) Integration
### MCP Architecture
The MCP system consists of:
1. **McpHub Class**: Central manager in `src/services/mcp/McpHub.ts`
2. **MCP Connections**: Manages connections to external MCP servers
3. **MCP Settings**: Configuration stored in a JSON file
4. **MCP Marketplace**: Online catalog of available MCP servers
5. **MCP Tools & Resources**: Capabilities exposed by connected servers
The McpHub class:
- Manages the lifecycle of MCP server connections
- Handles server configuration through a settings file
- Provides methods for calling tools and accessing resources
- Implements auto-approval settings for MCP tools
- Monitors server health and handles reconnection
### MCP Server Types
Cline supports two types of MCP server connections:
- **Stdio**: Command-line based servers that communicate via standard I/O
- **SSE**: HTTP-based servers that communicate via Server-Sent Events
### MCP Server Management
The McpHub class provides methods for:
- Discovering and connecting to MCP servers
- Monitoring server health and status
- Restarting servers when needed
- Managing server configurations
- Setting timeouts and auto-approval rules
### MCP Tool Integration
MCP tools are integrated into the Task execution system:
- Tools are discovered and registered at connection time
- The Task class can call MCP tools through the McpHub
- Tool results are streamed back to the AI
- Auto-approval settings can be configured per tool
### MCP Marketplace
The MCP Marketplace provides:
- A catalog of available MCP servers
- One-click installation
- README previews
- Server status monitoring
The Controller class manages MCP servers through the McpHub service:
```typescript
class Controller {
mcpHub?: McpHub
constructor(context: vscode.ExtensionContext, outputChannel: vscode.OutputChannel, webviewProvider: WebviewProvider) {
this.mcpHub = new McpHub(this)
}
async downloadMcp(mcpId: string) {
// Fetch server details from marketplace
const response = await axios.post<McpDownloadResponse>(
"https://api.cline.bot/v1/mcp/download",
{ mcpId },
{
headers: { "Content-Type": "application/json" },
timeout: 10000,
}
)
// Create task with context from README
const task = `Set up the MCP server from ${mcpDetails.githubUrl}...`
// Initialize task and show chat view
await this.initClineWithTask(task)
}
}
```
## Conclusion
This guide provides a comprehensive overview of the Cline extension architecture, with special focus on state management, data persistence, and code organization. Following these patterns ensures robust feature implementation with proper state handling across the extension's components.
Remember:
- Always persist important state in the extension
- The core extension follows a WebviewProvider -> Controller -> Task flow
- The core extension exists in the src/ folder
- Use proper typing for all state and messages
- Handle errors and edge cases
- Test state persistence across webview reloads
-6
View File
@@ -1,6 +0,0 @@
[codespell]
# Ref: https://github.com/codespell-project/codespell#using-a-config-file
skip = .git*,*.svg,package-lock.json,*.css,.codespellrc,locales
check-hidden = true
ignore-regex = (\b(optIn|isTaller)\b|https://\S+)
# ignore-words-list =
+1 -1
View File
@@ -1 +1 @@
* @saoudrizwan @ocasta181 @NightTrek @pashpashpash @dcbartlett @saito-sv @Garoth
* @saoudrizwan @ocasta181 @NightTrek @pashpashpash @dcbartlett @saito-sv
-8
View File
@@ -31,14 +31,6 @@ body:
label: Relevant API REQUEST output
description: Please copy and paste any relevant output. This will be automatically formatted into code, so no need for backticks.
render: shell
- type: input
id: provider-model
attributes:
label: Provider/Model
description: What provider and model were you using when the issue occurred?
placeholder: "e.g., cline:anthropic/claude-3.7-sonnet, gemini:gemini-2.5-pro-exp-03-25"
validations:
required: true
- type: input
id: operating-system
attributes:
-3
View File
@@ -13,10 +13,7 @@
- [ ] 🐛 Bug fix (non-breaking change which fixes an issue)
- [ ] ✨ New feature (non-breaking change which adds functionality)
- [ ] 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] ♻️ Refactor Changes
- [ ] 💅 Cosmetic Changes
- [ ] 📚 Documentation update
- [ ] 🏃 Workflow Changes
### Pre-flight Checklist
@@ -1,19 +0,0 @@
"""
Coverage utility package for GitHub Actions workflows.
This package handles extracting coverage percentages, comparing them, and generating PR comments.
"""
# Import external dependencies
import requests
# Import main function for CLI usage
from .__main__ import main
# Import functions from extraction module
from .extraction import extract_coverage, compare_coverage, run_coverage, set_verbose
# Import functions from github_api module
from .github_api import generate_comment, post_comment, set_github_output
# Import functions from workflow module
from .workflow import process_coverage_workflow
-154
View File
@@ -1,154 +0,0 @@
"""
Main module.
This module provides the CLI interface for the coverage utility script.
"""
import sys
import argparse
from .extraction import extract_coverage, compare_coverage, run_coverage, set_verbose
from .github_api import generate_comment, post_comment, set_github_output
from .workflow import process_coverage_workflow
from .util import log
def setup_verbose_mode(args):
"""
Set up verbose mode based on command line arguments.
Args:
args: Parsed command line arguments
"""
if getattr(args, 'verbose', False):
set_verbose(True)
log("Verbose mode enabled")
def main():
# Create parent parser with common arguments
parent_parser = argparse.ArgumentParser(add_help=False)
parent_parser.add_argument('-v', '--verbose', action='store_true', help='Enable verbose output')
# Create main parser that inherits common arguments
parser = argparse.ArgumentParser(description='Coverage utility script for GitHub Actions workflows', parents=[parent_parser])
subparsers = parser.add_subparsers(dest='command', help='Command to run')
# extract-coverage command - used directly in workflow
extract_parser = subparsers.add_parser('extract-coverage', help='Extract coverage percentage from a file', parents=[parent_parser])
extract_parser.add_argument('file_path', help='Path to the coverage report file')
extract_parser.add_argument('--type', choices=['extension', 'webview'], default='extension',
help='Type of coverage report')
extract_parser.add_argument('--github-output', action='store_true', help='Output in GitHub Actions format')
# compare-coverage command - used by process-workflow
compare_parser = subparsers.add_parser('compare-coverage', help='Compare coverage percentages', parents=[parent_parser])
compare_parser.add_argument('base_cov', help='Base branch coverage percentage')
compare_parser.add_argument('pr_cov', help='PR branch coverage percentage')
compare_parser.add_argument('--output-prefix', default='', help='Prefix for GitHub Actions output variables')
compare_parser.add_argument('--github-output', action='store_true', help='Output in GitHub Actions format')
# generate-comment command - used by process-workflow
comment_parser = subparsers.add_parser('generate-comment', help='Generate PR comment with coverage comparison', parents=[parent_parser])
comment_parser.add_argument('base_ext_cov', help='Base branch extension coverage')
comment_parser.add_argument('pr_ext_cov', help='PR branch extension coverage')
comment_parser.add_argument('ext_decreased', help='Whether extension coverage decreased (true/false)')
comment_parser.add_argument('ext_diff', help='Extension coverage difference')
comment_parser.add_argument('base_web_cov', help='Base branch webview coverage')
comment_parser.add_argument('pr_web_cov', help='PR branch webview coverage')
comment_parser.add_argument('web_decreased', help='Whether webview coverage decreased (true/false)')
comment_parser.add_argument('web_diff', help='Webview coverage difference')
# post-comment command - used by process-workflow
post_parser = subparsers.add_parser('post-comment', help='Post a comment to a GitHub PR', parents=[parent_parser])
post_parser.add_argument('comment_path', help='Path to the file containing the comment text')
post_parser.add_argument('pr_number', help='PR number')
post_parser.add_argument('repo', help='Repository in the format "owner/repo"')
post_parser.add_argument('--token', help='GitHub token')
# run-coverage command - used by process-workflow
run_parser = subparsers.add_parser('run-coverage', help='Run a coverage command and extract the coverage percentage', parents=[parent_parser])
run_parser.add_argument('coverage_cmd', help='Command to run')
run_parser.add_argument('output_file', help='File to save the output to')
run_parser.add_argument('--type', choices=['extension', 'webview'], default='extension',
help='Type of coverage report')
run_parser.add_argument('--github-output', action='store_true', help='Output in GitHub Actions format')
# process-workflow command - used directly in workflow
workflow_parser = subparsers.add_parser('process-workflow', help='Process the entire coverage workflow', parents=[parent_parser])
workflow_parser.add_argument('--base-branch', required=True, help='Base branch name')
workflow_parser.add_argument('--pr-number', help='PR number')
workflow_parser.add_argument('--repo', help='Repository in the format "owner/repo"')
workflow_parser.add_argument('--token', help='GitHub token')
# set-github-output command - used by process-workflow
output_parser = subparsers.add_parser('set-github-output', help='Set GitHub Actions output variable', parents=[parent_parser])
output_parser.add_argument('name', help='Output variable name')
output_parser.add_argument('value', help='Output variable value')
args = parser.parse_args()
# Set up verbose mode
setup_verbose_mode(args)
if args.command == 'extract-coverage':
log(f"Extracting coverage from file: {args.file_path} (type: {args.type})")
coverage_pct = extract_coverage(args.file_path, args.type)
if args.github_output:
set_github_output(f"{args.type}_coverage", coverage_pct)
else:
log(f"Coverage: {coverage_pct}%")
elif args.command == 'compare-coverage':
log(f"Comparing coverage: base={args.base_cov}%, PR={args.pr_cov}%")
decreased, diff = compare_coverage(args.base_cov, args.pr_cov)
if args.github_output:
prefix = args.output_prefix
set_github_output(f"{prefix}decreased", str(decreased).lower())
set_github_output(f"{prefix}diff", diff)
log(f"Coverage difference: {diff}%")
log(f"Coverage decreased: {decreased}")
else:
log(f"decreased={str(decreased).lower()}")
log(f"diff={diff}")
elif args.command == 'generate-comment':
log("Generating coverage comparison comment")
comment = generate_comment(
args.base_ext_cov, args.pr_ext_cov, args.ext_decreased, args.ext_diff,
args.base_web_cov, args.pr_web_cov, args.web_decreased, args.web_diff
)
# Output the comment to stdout
log(comment)
elif args.command == 'post-comment':
log(f"Posting comment from {args.comment_path} to PR #{args.pr_number} in {args.repo}")
post_comment(args.comment_path, args.pr_number, args.repo, args.token)
elif args.command == 'run-coverage':
log(f"Running coverage command: {args.coverage_cmd}")
log(f"Output file: {args.output_file}")
log(f"Coverage type: {args.type}")
coverage_pct = run_coverage(args.coverage_cmd, args.output_file, args.type)
if args.github_output:
set_github_output(f"{args.type}_coverage", coverage_pct)
else:
log(f"Coverage: {coverage_pct}%")
elif args.command == 'process-workflow':
log("Processing coverage workflow")
log(f"Base branch: {args.base_branch}")
if args.pr_number:
log(f"PR number: {args.pr_number}")
if args.repo:
log(f"Repository: {args.repo}")
process_coverage_workflow(args)
elif args.command == 'set-github-output':
log(f"Setting GitHub output: {args.name}={args.value}")
set_github_output(args.name, args.value)
else:
log("No command specified")
parser.print_help()
sys.exit(1)
if __name__ == "__main__":
main()
@@ -1,265 +0,0 @@
"""
Coverage extraction module.
This module handles extracting coverage percentages from coverage report files.
"""
import os
import re
import sys
import shlex
import subprocess
import traceback
from .util import log, file_exists, get_file_size, list_directory, is_safe_command, run_command
# Global verbose flag
verbose = False
def set_verbose(value):
"""Set the global verbose flag."""
global verbose
verbose = value
def print_debug_output(content, coverage_type):
"""
Print debug information about the coverage output.
Args:
content: The content of the coverage file
coverage_type: Type of coverage report (extension or webview)
"""
if not verbose:
return
# Extract and print only the coverage summary section
if coverage_type == "extension":
# Look for the coverage summary section
summary_match = re.search(r'=============================== Coverage summary ===============================\n(.*?)\n=+', content, re.DOTALL)
if summary_match:
sys.stdout.write("\n##[group]EXTENSION COVERAGE SUMMARY\n")
sys.stdout.write("=============================== Coverage summary ===============================\n")
sys.stdout.write(summary_match.group(1) + "\n")
sys.stdout.write("================================================================================\n")
sys.stdout.write("##[endgroup]\n")
sys.stdout.flush()
else:
sys.stdout.write("\n##[warning]No coverage summary found in extension coverage file\n")
sys.stdout.flush()
else: # webview
# Look for the coverage table - specifically the "All files" row
table_match = re.search(r'% Coverage report from v8.*?-+\|.*?\n.*?\n(All files.*?)(?:\n[^\n]*\|)', content, re.DOTALL)
if table_match:
sys.stdout.write("\n##[group]WEBVIEW COVERAGE SUMMARY\n")
sys.stdout.write("% Coverage report from v8\n")
sys.stdout.write("-------------------|---------|----------|---------|---------|-------------------\n")
sys.stdout.write("File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s \n")
sys.stdout.write("-------------------|---------|----------|---------|---------|-------------------\n")
sys.stdout.write(table_match.group(1) + "\n")
sys.stdout.write("-------------------|---------|----------|---------|---------|-------------------\n")
sys.stdout.write("##[endgroup]\n")
sys.stdout.flush()
else:
sys.stdout.write("\n##[warning]No coverage table found in webview coverage file\n")
sys.stdout.flush()
def extract_coverage(file_path, coverage_type="extension"):
"""
Extract coverage percentage from a coverage report file.
Args:
file_path: Path to the coverage report file
coverage_type: Type of coverage report (extension or webview)
Returns:
Coverage percentage as a float
"""
# Always print file path for debugging
log(f"Checking coverage file: {file_path}")
# Check if file exists and get its size
if not file_exists(file_path):
sys.stdout.write(f"\n##[error]File {file_path} does not exist\n")
sys.stdout.flush()
log(f"Error: File {file_path} does not exist")
# Check if the directory exists
dir_path = os.path.dirname(file_path)
if not os.path.exists(dir_path):
sys.stdout.write(f"\n##[error]Directory {dir_path} does not exist\n")
sys.stdout.flush()
log(f"Error: Directory {dir_path} does not exist")
else:
# List directory contents for debugging
log(f"Directory {dir_path} exists, listing contents:")
try:
dir_contents = list_directory(dir_path)
for name, size in dir_contents:
log(f" {name} - {size}")
sys.stdout.write(f" {name} - {size}\n")
sys.stdout.flush()
except Exception as e:
log(f"Error listing directory: {e}")
return 0.0
file_size = get_file_size(file_path)
log(f"File size: {file_size} bytes")
sys.stdout.write(f"\n##[info]Coverage file {file_path} exists, size: {file_size} bytes\n")
sys.stdout.flush()
if file_size == 0:
sys.stdout.write(f"\n##[warning]File {file_path} is empty\n")
sys.stdout.flush()
log(f"Warning: File {file_path} is empty")
return 0.0
# List directory contents for debugging
dir_path = os.path.dirname(file_path)
log(f"Directory contents of {dir_path}:")
try:
dir_contents = list_directory(dir_path)
for name, size in dir_contents:
log(f" {name} - {size}")
except Exception as e:
log(f"Error listing directory: {e}")
with open(file_path, 'r') as f:
content = f.read()
# Print debug information if verbose
print_debug_output(content, coverage_type)
# Extract coverage percentage based on coverage type
if coverage_type == "extension":
# Extract the percentage from the "Lines" row in the coverage summary
# Pattern: Lines : xx.xx% ( xxxxxxx/xxxxxxx )
lines_match = re.search(r'Lines\s*:\s*(\d+\.\d+)%', content)
if lines_match:
coverage_pct = float(lines_match.group(1))
if verbose:
sys.stdout.write(f"Pattern matched (Lines percentage): {coverage_pct}\n")
sys.stdout.flush()
return coverage_pct
else:
# No coverage data found, log full content for debugging
log("No coverage data found. Full file content:")
log("=== Full file content ===")
log(content)
log("=== End file content ===")
else: # webview
# Extract the percentage from the "% Lines" column in the "All files" row
# Pattern: All files | xx.xx | xx.xx | xx.xx | xx.xx |
all_files_match = re.search(r'All files\s+\|\s+\d+\.\d+\s+\|\s+\d+\.\d+\s+\|\s+\d+\.\d+\s+\|\s+(\d+\.\d+)', content)
if all_files_match:
coverage_pct = float(all_files_match.group(1))
if verbose:
sys.stdout.write(f"Pattern matched (All files % Lines): {coverage_pct}\n")
sys.stdout.flush()
return coverage_pct
else:
# No coverage data found, log full content for debugging
log("No coverage data found. Full file content:")
log("=== Full file content ===")
log(content)
log("=== End file content ===")
# If no match found, return 0.0
return 0.0
def compare_coverage(base_cov, pr_cov):
"""
Compare coverage percentages between base and PR branches.
Args:
base_cov: Base branch coverage percentage
pr_cov: PR branch coverage percentage
Returns:
Tuple of (decreased, diff)
"""
try:
base_cov = float(base_cov)
pr_cov = float(pr_cov)
except ValueError:
sys.stdout.write(f"Error: Invalid coverage values - base: {base_cov}, PR: {pr_cov}\n")
sys.stdout.flush()
return False, 0
diff = pr_cov - base_cov
decreased = diff < 0
return decreased, abs(diff)
def run_coverage(command, output_file, coverage_type="extension"):
"""
Run a coverage command and extract the coverage percentage.
Args:
command: Command to run
output_file: File to save the output to
coverage_type: Type of coverage report (extension or webview)
Returns:
Coverage percentage as a float
Raises:
SystemExit: If the output file is not created or is empty
"""
try:
# Run the command and capture output
if not is_safe_command(command):
error_msg = f"ERROR: Unsafe command detected: {command}"
log(error_msg)
sys.stdout.write(f"\n##[error]{error_msg}\n")
sys.stdout.flush()
sys.exit(1)
# Run command using safe execution from util
returncode, stdout, stderr = run_command(command)
# Log command result
log(f"Command exit code: {returncode}")
log(f"Command stdout length: {len(stdout)} bytes")
log(f"Command stderr length: {len(stderr)} bytes")
# Save output to file
log(f"Saving command output to {output_file}")
with open(output_file, 'w') as f:
f.write(stdout)
if stderr:
f.write("\n\n=== STDERR ===\n")
f.write(stderr)
# Verify file was created and has content
if not file_exists(output_file):
error_msg = f"ERROR: Output file {output_file} was not created"
log(error_msg)
sys.stdout.write(f"\n##[error]{error_msg}\n")
sys.stdout.flush()
sys.exit(1) # Exit with error code to fail the workflow
file_size = get_file_size(output_file)
if file_size == 0:
error_msg = f"ERROR: Output file {output_file} is empty"
log(error_msg)
sys.stdout.write(f"\n##[error]{error_msg}\n")
sys.stdout.flush()
sys.exit(1) # Exit with error code to fail the workflow
log(f"Output file size: {file_size} bytes")
# Extract coverage percentage
coverage_pct = extract_coverage(output_file, coverage_type)
log(f"{coverage_type.capitalize()} coverage: {coverage_pct}%")
return coverage_pct
except Exception as e:
error_msg = f"Error running coverage command: {e}"
log(error_msg)
sys.stdout.write(f"\n##[error]{error_msg}\n")
sys.stdout.flush()
# Print stack trace for debugging
log(traceback.format_exc())
sys.exit(1) # Exit with error code to fail the workflow
@@ -1,177 +0,0 @@
"""
GitHub API module.
This module handles interactions with the GitHub API for posting comments to PRs.
"""
import os
import requests
from .util import log, file_exists
def generate_comment(base_ext_cov, pr_ext_cov, ext_decreased, ext_diff,
base_web_cov, pr_web_cov, web_decreased, web_diff):
"""
Generate a PR comment with coverage comparison.
Args:
base_ext_cov: Base branch extension coverage
pr_ext_cov: PR branch extension coverage
ext_decreased: Whether extension coverage decreased
ext_diff: Extension coverage difference
base_web_cov: Base branch webview coverage
pr_web_cov: PR branch webview coverage
web_decreased: Whether webview coverage decreased
web_diff: Webview coverage difference
Returns:
Comment text
"""
from datetime import datetime
# Convert string inputs to appropriate types
try:
base_ext_cov = float(base_ext_cov)
pr_ext_cov = float(pr_ext_cov)
# Handle ext_decreased as either string or boolean
if isinstance(ext_decreased, str):
ext_decreased = ext_decreased.lower() == 'true'
else:
ext_decreased = bool(ext_decreased)
ext_diff = float(ext_diff)
base_web_cov = float(base_web_cov)
pr_web_cov = float(pr_web_cov)
# Handle web_decreased as either string or boolean
if isinstance(web_decreased, str):
web_decreased = web_decreased.lower() == 'true'
else:
web_decreased = bool(web_decreased)
web_diff = float(web_diff)
except ValueError as e:
log(f"Error converting input values: {e}")
return ""
# Add a unique identifier to find this comment later
comment = '<!-- COVERAGE_REPORT -->\n'
comment += '## Coverage Report\n\n'
# Extension coverage
comment += '### Extension Coverage\n\n'
comment += f'Base branch: {base_ext_cov:.0f}%\n\n'
comment += f'PR branch: {pr_ext_cov:.0f}%\n\n'
if ext_decreased:
comment += f'⚠️ **Warning: Coverage decreased by {ext_diff:.2f}%**\n\n'
comment += 'Consider adding tests to cover your changes.\n\n'
else:
comment += '✅ Coverage increased or remained the same\n\n'
# Webview coverage
comment += '### Webview Coverage\n\n'
comment += f'Base branch: {base_web_cov:.0f}%\n\n'
comment += f'PR branch: {pr_web_cov:.0f}%\n\n'
if web_decreased:
comment += f'⚠️ **Warning: Coverage decreased by {web_diff:.2f}%**\n\n'
comment += 'Consider adding tests to cover your changes.\n\n'
else:
comment += '✅ Coverage increased or remained the same\n\n'
# Overall assessment
comment += '### Overall Assessment\n\n'
if ext_decreased or web_decreased:
comment += '⚠️ **Test coverage has decreased in this PR**\n\n'
comment += 'Please consider adding tests to maintain or improve coverage.\n\n'
else:
comment += '✅ **Test coverage has been maintained or improved**\n\n'
# Add timestamp
comment += f'\n\n<sub>Last updated: {datetime.now().isoformat()}</sub>'
return comment
def post_comment(comment_path, pr_number, repo, token=None):
"""
Post a comment to a GitHub PR.
Args:
comment_path: Path to the file containing the comment text
pr_number: PR number
repo: Repository in the format "owner/repo"
token: GitHub token
"""
if not file_exists(comment_path):
log(f"Error: Comment file {comment_path} does not exist")
return
with open(comment_path, 'r') as f:
comment_body = f.read()
if not token:
token = os.environ.get('GITHUB_TOKEN')
if not token:
log("Error: GitHub token not provided")
return
# Find existing comment
headers = {
'Authorization': f'token {token}',
'Accept': 'application/vnd.github.v3+json'
}
# Get all comments
comments_url = f'https://api.github.com/repos/{repo}/issues/{pr_number}/comments'
log(f"Getting comments from: {comments_url}")
response = requests.get(comments_url, headers=headers)
if response.status_code != 200:
log(f"Error getting comments: {response.status_code} - {response.text}")
return
comments = response.json()
log(f"Found {len(comments)} existing comments")
# Find comment with our identifier
comment_id = None
for comment in comments:
if '<!-- COVERAGE_REPORT -->' in comment['body']:
comment_id = comment['id']
log(f"Found existing coverage report comment with ID: {comment_id}")
break
if comment_id:
# Update existing comment
update_url = f'https://api.github.com/repos/{repo}/issues/comments/{comment_id}'
log(f"Updating existing comment at: {update_url}")
response = requests.patch(update_url, headers=headers, json={'body': comment_body})
if response.status_code == 200:
log(f"Successfully updated existing comment: {comment_id}")
else:
log(f"Error updating comment: {response.status_code} - {response.text}")
else:
# Create new comment
log(f"Creating new comment at: {comments_url}")
response = requests.post(comments_url, headers=headers, json={'body': comment_body})
if response.status_code == 201:
log("Successfully created new comment")
else:
log(f"Error creating comment: {response.status_code} - {response.text}")
def set_github_output(name, value):
"""
Set GitHub Actions output variable.
Args:
name: Output variable name
value: Output variable value
"""
# Write to the GitHub output file if available
if 'GITHUB_OUTPUT' in os.environ:
with open(os.environ['GITHUB_OUTPUT'], 'a') as f:
f.write(f"{name}={value}\n")
else:
# Fallback to the deprecated method for backward compatibility
log(f"::set-output name={name}::{value}")
# Also print for human readability
log(f"{name}: {value}")
-245
View File
@@ -1,245 +0,0 @@
"""
Utility module.
This module provides utility functions used across the coverage check scripts.
"""
import os
import sys
import re
import shlex
import subprocess
import traceback
from typing import List, Tuple, Dict, Any, Optional, Union
# List of allowed commands and their arguments
ALLOWED_COMMANDS = {
'xvfb-run': ['-a'],
'npm': ['run', 'test:coverage', 'ci', 'install', '--no-save', '@vitest/coverage-v8', 'check-types', 'lint', 'format', 'compile'],
'cd': ['webview-ui'],
'python': ['-m', 'coverage_check'],
'git': ['fetch', 'checkout', 'origin'],
}
def is_safe_command(command: Union[str, List[str]]) -> bool:
"""
Check if a command is safe to execute.
Args:
command: Command to check (string or list)
Returns:
True if command is safe, False otherwise
"""
# Convert string command to list
if isinstance(command, str):
try:
cmd_parts = shlex.split(command)
except ValueError:
return False
else:
cmd_parts = command
if not cmd_parts:
return False
# Get base command
base_cmd = os.path.basename(cmd_parts[0])
# Check if command is in allowed list
if base_cmd not in ALLOWED_COMMANDS:
return False
# For each argument, check for suspicious patterns
for arg in cmd_parts[1:]:
# Check for shell metacharacters
if re.search(r'[;&|`$]', arg):
return False
# Check for path traversal
if '..' in arg and not (base_cmd == 'npm' and arg.startswith('@')):
return False
return True
def log(message: str) -> None:
"""
Write a message to stdout and flush.
Args:
message: The message to write
"""
sys.stdout.write(f"{message}\n")
sys.stdout.flush()
def file_exists(file_path: str) -> bool:
"""
Check if a file exists.
Args:
file_path: Path to the file
Returns:
True if the file exists, False otherwise
"""
return os.path.exists(file_path) and os.path.isfile(file_path)
def get_file_size(file_path: str) -> int:
"""
Get the size of a file in bytes.
Args:
file_path: Path to the file
Returns:
Size of the file in bytes, or 0 if the file doesn't exist
"""
if file_exists(file_path):
return os.path.getsize(file_path)
return 0
def list_directory(dir_path: str) -> List[Tuple[str, Union[int, str]]]:
"""
List the contents of a directory.
Args:
dir_path: Path to the directory
Returns:
List of (name, size) tuples for each file/directory in the directory
"""
if not os.path.exists(dir_path) or not os.path.isdir(dir_path):
return []
contents = []
for item in os.listdir(dir_path):
item_path = os.path.join(dir_path, item)
if os.path.isfile(item_path):
contents.append((item, os.path.getsize(item_path)))
else:
contents.append((item, "DIR"))
return contents
def read_file_content(file_path: str, default: str = "") -> str:
"""
Read file content with error handling.
Args:
file_path: Path to the file
default: Default value to return if file cannot be read
Returns:
File content or default value
"""
if not file_exists(file_path):
log(f"File does not exist: {file_path}")
return default
try:
with open(file_path, 'r') as f:
return f.read()
except Exception as e:
log(f"Error reading file {file_path}: {e}")
return default
def write_file_content(file_path: str, content: str) -> bool:
"""
Write content to file with error handling.
Args:
file_path: Path to the file
content: Content to write
Returns:
True if successful, False otherwise
"""
try:
# Create directory if it doesn't exist
os.makedirs(os.path.dirname(file_path), exist_ok=True)
with open(file_path, 'w') as f:
f.write(content)
return True
except Exception as e:
log(f"Error writing to file {file_path}: {e}")
return False
def run_command(command: Union[str, List[str]], capture_output: bool = True) -> Tuple[int, str, str]:
"""
Run a command and return the result.
Args:
command: Command to run (string or list)
capture_output: Whether to capture stdout/stderr
Returns:
Tuple of (returncode, stdout, stderr)
"""
if not is_safe_command(command):
error_msg = f"Unsafe command detected: {command}"
log(error_msg)
return 1, "", error_msg
log(f"Running command: {command}")
try:
# Convert string command to list
if isinstance(command, str):
cmd_list = shlex.split(command)
else:
cmd_list = command
result = subprocess.run(
cmd_list,
shell=False, # Never use shell=True for security
capture_output=capture_output,
text=True
)
log(f"Command exit code: {result.returncode}")
return result.returncode, result.stdout, result.stderr
except Exception as e:
log(f"Error running command: {e}")
log(traceback.format_exc())
return 1, "", str(e)
def find_pattern(content: str, pattern: str, group: int = 0,
default: Optional[str] = None) -> Optional[str]:
"""
Find a pattern in content and return the specified group.
Args:
content: Text content to search
pattern: Regex pattern to search for
group: Group number to return (default: 0 for entire match)
default: Default value to return if pattern not found
Returns:
Matched text or default value
"""
match = re.search(pattern, content, re.DOTALL)
if match:
return match.group(group)
return default
def get_env_var(name: str, default: Optional[str] = None) -> Optional[str]:
"""
Get environment variable with default value.
Args:
name: Environment variable name
default: Default value if not set
Returns:
Environment variable value or default
"""
return os.environ.get(name, default)
def format_exception(e: Exception) -> str:
"""
Format an exception with traceback for logging.
Args:
e: Exception to format
Returns:
Formatted exception string
"""
return f"{type(e).__name__}: {str(e)}\n{traceback.format_exc()}"
-432
View File
@@ -1,432 +0,0 @@
"""
Workflow module.
This module handles the main workflow logic for running coverage tests and processing results.
"""
import os
import re
import sys
import subprocess
import traceback
from .extraction import run_coverage, compare_coverage, extract_coverage
from .github_api import generate_comment, post_comment, set_github_output
from .util import log, file_exists, get_file_size, list_directory, run_command
def is_valid_branch_name(branch_name: str) -> bool:
"""
Validate a git branch name.
Args:
branch_name: Branch name to validate
Returns:
True if valid, False otherwise
"""
# Check for common branch name patterns
if not re.match(r'^[a-zA-Z0-9_\-./]+$', branch_name):
return False
# Check for path traversal
if '..' in branch_name:
return False
# Check for shell metacharacters
if re.search(r'[;&|`$]', branch_name):
return False
return True
def checkout_branch(branch_name: str) -> None:
"""
Checkout a branch for testing.
Args:
branch_name: Branch name to checkout
Raises:
RuntimeError: If branch checkout fails
ValueError: If branch name is invalid
"""
if not is_valid_branch_name(branch_name):
raise ValueError(f"Invalid branch name: {branch_name}")
log(f"=== Checking out branch: {branch_name} ===")
# Fetch the branch
returncode, stdout, stderr = run_command(['git', 'fetch', 'origin', branch_name])
if returncode != 0:
log(f"ERROR: Failed to fetch branch {branch_name}")
log(f"Error details: {stderr}")
raise RuntimeError(f"Git fetch failed: {stderr}")
# Checkout the branch
returncode, stdout, stderr = run_command(['git', 'checkout', branch_name])
if returncode != 0:
log(f"ERROR: Failed to checkout branch {branch_name}")
log(f"Error details: {stderr}")
raise RuntimeError(f"Git checkout failed: {stderr}")
log(f"Successfully checked out branch: {branch_name}")
def extract_extension_coverage_from_file(file_path):
"""Extract extension coverage from file when run_coverage returns 0."""
if not file_exists(file_path):
log(f"File {file_path} does not exist, cannot extract extension coverage")
return 0.0
file_size = get_file_size(file_path)
if file_size == 0:
log(f"File {file_path} is empty, cannot extract extension coverage")
return 0.0
log(f"Extension coverage is 0.0, trying to read from file directly: {file_path} (size: {file_size} bytes)")
with open(file_path, 'r') as f:
content = f.read()
# Extract the percentage from the "Lines" row in the coverage summary
# Pattern: Lines : xx.xx% ( xxxxxxx/xxxxxxx )
lines_match = re.search(r'Lines\s*:\s*(\d+\.\d+)%', content)
if lines_match:
coverage = float(lines_match.group(1))
log(f"Found extension coverage in file: {coverage}%")
return coverage
return 0.0
def extract_webview_coverage_from_file(file_path):
"""Extract webview coverage from file when run_coverage returns 0."""
if not file_exists(file_path):
log(f"File {file_path} does not exist, cannot extract webview coverage")
return 0.0
file_size = get_file_size(file_path)
if file_size == 0:
log(f"File {file_path} is empty, cannot extract webview coverage")
return 0.0
log(f"Webview coverage is 0.0, trying to read from file directly: {file_path} (size: {file_size} bytes)")
with open(file_path, 'r') as f:
content = f.read()
# Extract the percentage from the "% Lines" column in the "All files" row
# Pattern: All files | xx.xx | xx.xx | xx.xx | xx.xx |
all_files_match = re.search(r'All files\s+\|\s+\d+\.\d+\s+\|\s+\d+\.\d+\s+\|\s+\d+\.\d+\s+\|\s+(\d+\.\d+)', content)
if all_files_match:
coverage = float(all_files_match.group(1))
log(f"Found webview coverage in file: {coverage}%")
return coverage
return 0.0
def run_extension_coverage(branch_name=None):
"""Run extension coverage tests and extract results."""
prefix = 'base_' if branch_name else ''
file_path = f"{prefix}extension_coverage.txt"
# Run coverage tests
ext_cov = run_coverage(
["xvfb-run", "-a", "npm", "run", "test:coverage"],
file_path,
"extension"
)
# If coverage is 0.0, try to extract from file directly
if ext_cov == 0.0:
ext_cov = extract_extension_coverage_from_file(file_path)
return ext_cov
def run_webview_coverage(branch_name=None):
"""Run webview coverage tests and extract results."""
prefix = 'base_' if branch_name else ''
file_path = f"{prefix}webview_coverage.txt"
# Save current directory
original_dir = os.getcwd()
try:
# Change to webview-ui directory
os.chdir('webview-ui')
# Install coverage dependency
returncode, stdout, stderr = run_command(["npm", "install", "--no-save", "@vitest/coverage-v8"])
if returncode != 0:
log(f"Failed to install coverage dependency: {stderr}")
return 0.0
# Run coverage tests from webview-ui directory
web_cov = run_coverage(
["npm", "run", "test:coverage"],
os.path.join('..', file_path),
"webview"
)
finally:
# Always change back to original directory
os.chdir(original_dir)
# If coverage is 0.0, try to extract from file directly
if web_cov == 0.0:
web_cov = extract_webview_coverage_from_file(file_path)
return web_cov
def run_branch_coverage(branch_name=None):
"""
Run coverage tests for a branch.
Args:
branch_name: Name of the branch to checkout before running tests (optional)
Returns:
Tuple of (extension_coverage, webview_coverage)
"""
# Checkout branch if specified
if branch_name:
checkout_branch(branch_name)
# Run coverage tests
log(f"=== Running coverage tests{' for ' + branch_name if branch_name else ''} ===")
# Run extension and webview coverage
ext_cov = run_extension_coverage(branch_name)
web_cov = run_webview_coverage(branch_name)
return ext_cov, web_cov
def find_potential_coverage_files():
"""Find potential coverage files in the current directory and webview-ui."""
log("Searching for potential coverage files...")
# Find files in current directory
current_dir_files = list_directory('.')
for name, size in current_dir_files:
if 'coverage' in name.lower() and size != "DIR":
log(f"Found potential coverage file: {name} (size: {size} bytes)")
# Find files in webview-ui directory
if os.path.exists('webview-ui') and os.path.isdir('webview-ui'):
webview_files = list_directory('webview-ui')
for name, size in webview_files:
if 'coverage' in name.lower() and size != "DIR":
log(f"Found potential webview coverage file: webview-ui/{name} (size: {size} bytes)")
else:
log("webview-ui directory not found")
def generate_warnings(base_ext_cov, pr_ext_cov, ext_decreased, ext_diff,
base_web_cov, pr_web_cov, web_decreased, web_diff):
"""Generate warnings for coverage decreases."""
if not (ext_decreased or web_decreased):
return []
warnings = [
"Test coverage has decreased in this PR",
f"Extension coverage: {base_ext_cov}% -> {pr_ext_cov}% (Diff: {ext_diff}%)",
f"Webview coverage: {base_web_cov}% -> {pr_web_cov}% (Diff: {web_diff}%)"
]
# Additional warning for significant decrease (more than 1%)
if ext_decreased and ext_diff > 1.0:
warnings.append(f"Extension coverage decreased by more than 1% ({ext_diff}%). Consider adding tests to cover your changes.")
if web_decreased and web_diff > 1.0:
warnings.append(f"Webview coverage decreased by more than 1% ({web_diff}%). Consider adding tests to cover your changes.")
return warnings
def output_warnings(warnings):
"""Output warnings to GitHub step summary and console."""
if not warnings:
return
# Get the GitHub step summary file path from environment variable
github_step_summary = os.environ.get('GITHUB_STEP_SUMMARY')
# Write to GitHub step summary if available
if github_step_summary:
with open(github_step_summary, 'a') as f:
f.write("## Coverage Warnings\n\n")
for warning in warnings:
f.write(f"⚠️ {warning}\n\n")
# Also output to console with ::warning:: syntax for backward compatibility
for warning in warnings:
log(f"::warning::{warning}")
def output_github_results(pr_ext_cov, pr_web_cov, base_ext_cov, base_web_cov,
ext_decreased, ext_diff, web_decreased, web_diff):
"""Output results for GitHub Actions."""
set_github_output("pr_extension_coverage", pr_ext_cov)
set_github_output("pr_webview_coverage", pr_web_cov)
set_github_output("base_extension_coverage", base_ext_cov)
set_github_output("base_webview_coverage", base_web_cov)
set_github_output("extension_decreased", str(ext_decreased).lower())
set_github_output("extension_diff", ext_diff)
set_github_output("webview_decreased", str(web_decreased).lower())
set_github_output("webview_diff", web_diff)
def extract_pr_coverage_from_artifacts():
"""
Extract PR branch coverage from artifact files.
Returns:
Tuple of (extension_coverage, webview_coverage)
Raises:
SystemExit: If the coverage files don't exist
"""
log("=== Extracting PR branch coverage from artifacts ===")
# Check if the coverage files exist
ext_file_path = "extension_coverage.txt"
web_file_path = "webview-ui/webview_coverage.txt"
# Extract extension coverage
log(f"Extracting extension coverage from {ext_file_path}")
if not file_exists(ext_file_path):
error_msg = f"ERROR: PR extension coverage file {ext_file_path} not found"
log(error_msg)
# List directory contents for debugging
log("Current directory contents:")
try:
dir_contents = list_directory('.')
for name, size in dir_contents:
log(f" {name} - {size}\n")
except Exception as e:
log(f"Error listing directory: {e}")
sys.exit(1) # Exit with error code to fail the workflow
ext_cov = extract_extension_coverage_from_file(ext_file_path)
log(f"PR extension coverage from artifact: {ext_cov}%")
# Extract webview coverage
log(f"Extracting webview coverage from {web_file_path}")
if not file_exists(web_file_path):
error_msg = f"ERROR: PR webview coverage file {web_file_path} not found"
log(error_msg)
# Check if the webview-ui directory exists
if not os.path.exists('webview-ui'):
log("ERROR: webview-ui directory not found")
else:
# List webview-ui directory contents for debugging
log("webview-ui directory contents:")
try:
dir_contents = list_directory('webview-ui')
for name, size in dir_contents:
log(f" {name} - {size}")
except Exception as e:
log(f"Error listing directory: {e}")
sys.exit(1) # Exit with error code to fail the workflow
web_cov = extract_webview_coverage_from_file(web_file_path)
log(f"PR webview coverage from artifact: {web_cov}%")
return ext_cov, web_cov
def process_coverage_workflow(args):
"""
Process the entire coverage workflow.
Args:
args: Command line arguments
"""
# Initialize all variables at the start
pr_ext_cov = 0.0
pr_web_cov = 0.0
base_ext_cov = 0.0
base_web_cov = 0.0
ext_decreased = False
ext_diff = 0.0
web_decreased = False
web_diff = 0.0
try:
# Validate branch name
if not is_valid_branch_name(args.base_branch):
raise ValueError(f"Invalid base branch name: {args.base_branch}")
# Check if we're running in GitHub Actions
is_github_actions = 'GITHUB_ACTIONS' in os.environ
if is_github_actions:
log("Running in GitHub Actions environment")
# Extract PR branch coverage from artifacts (from test job)
pr_ext_cov, pr_web_cov = extract_pr_coverage_from_artifacts()
# Verify PR coverage values
if pr_ext_cov == 0.0:
log("WARNING: PR extension coverage is 0.0, this may indicate an issue with the coverage report")
find_potential_coverage_files()
if pr_web_cov == 0.0:
log("WARNING: PR webview coverage is 0.0, this may indicate an issue with the coverage report")
find_potential_coverage_files()
# Run base branch coverage
log(f"=== Running base branch coverage for {args.base_branch} ===")
base_ext_cov, base_web_cov = run_branch_coverage(args.base_branch)
# Verify base coverage values
if base_ext_cov == 0.0:
log("WARNING: Base extension coverage is 0.0, this may indicate an issue with the coverage report")
if base_web_cov == 0.0:
log("WARNING: Base webview coverage is 0.0, this may indicate an issue with the coverage report")
# Compare coverage
log("=== Comparing extension coverage ===")
ext_decreased, ext_diff = compare_coverage(base_ext_cov, pr_ext_cov)
log("=== Comparing webview coverage ===")
web_decreased, web_diff = compare_coverage(base_web_cov, pr_web_cov)
# Print summary of coverage values
log("\n=== Coverage Summary ===")
log(f"PR extension coverage: {pr_ext_cov}%")
log(f"Base extension coverage: {base_ext_cov}%")
log(f"Extension coverage change: {'+' if not ext_decreased else '-'}{ext_diff}%")
log(f"PR webview coverage: {pr_web_cov}%")
log(f"Base webview coverage: {base_web_cov}%")
log(f"Webview coverage change: {'+' if not web_decreased else '-'}{web_diff}%")
# Generate and output warnings
warnings = generate_warnings(
base_ext_cov, pr_ext_cov, ext_decreased, ext_diff,
base_web_cov, pr_web_cov, web_decreased, web_diff
)
output_warnings(warnings)
# Generate comment
log("=== Generating comment ===")
comment = generate_comment(
base_ext_cov, pr_ext_cov, str(ext_decreased).lower(), ext_diff,
base_web_cov, pr_web_cov, str(web_decreased).lower(), web_diff
)
# Save comment to file
with open("coverage_comment.md", "w") as f:
f.write(comment)
# Post comment if PR number is provided
if args.pr_number:
log(f"=== Posting comment to PR #{args.pr_number} ===")
post_comment("coverage_comment.md", args.pr_number, args.repo, args.token)
# Output results for GitHub Actions
output_github_results(
pr_ext_cov, pr_web_cov, base_ext_cov, base_web_cov,
ext_decreased, ext_diff, web_decreased, web_diff
)
except Exception as e:
log(f"ERROR in process_coverage_workflow: {e}")
traceback.print_exc()
# Try to output results even if there was an error
try:
output_github_results(
pr_ext_cov, pr_web_cov, base_ext_cov, base_web_cov,
ext_decreased, ext_diff, web_decreased, web_diff
)
except Exception as e2:
log(f"ERROR outputting GitHub results: {e2}")
@@ -1,282 +0,0 @@
#!/usr/bin/env python3
"""
Tests for coverage_check script.
"""
import os
import sys
import unittest
import subprocess
import tempfile
from unittest.mock import patch, MagicMock, call, mock_open
# Add parent directory to path so we can import coverage modules
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
from coverage_check import extract_coverage, compare_coverage, set_verbose, generate_comment, post_comment, set_github_output
from coverage_check.util import log, file_exists, get_file_size, list_directory
class TestCoverage(unittest.TestCase):
# Class variables to store coverage files
temp_dir = None
extension_coverage_file = None
webview_coverage_file = None
@classmethod
def setUpClass(cls):
"""Set up test environment once for all tests."""
# Create temporary directory for test files
cls.temp_dir = tempfile.TemporaryDirectory()
cls.extension_coverage_file = os.path.join(cls.temp_dir.name, 'extension_coverage.txt')
cls.webview_coverage_file = os.path.join(cls.temp_dir.name, 'webview_coverage.txt')
# Run actual tests to generate coverage reports
cls.generate_coverage_reports()
# Verify files exist and are not empty
assert os.path.exists(cls.extension_coverage_file), \
f"Extension coverage file {cls.extension_coverage_file} does not exist"
assert os.path.getsize(cls.extension_coverage_file) > 0, \
f"Extension coverage file {cls.extension_coverage_file} is empty"
assert os.path.exists(cls.webview_coverage_file), \
f"Webview coverage file {cls.webview_coverage_file} does not exist"
assert os.path.getsize(cls.webview_coverage_file) > 0, \
f"Webview coverage file {cls.webview_coverage_file} is empty"
@classmethod
def tearDownClass(cls):
"""Clean up test environment after all tests."""
if cls.temp_dir:
cls.temp_dir.cleanup()
@classmethod
def generate_coverage_reports(cls):
"""Generate real coverage reports by running tests."""
log("Generating coverage reports (this may take a while)...")
# Run extension tests with coverage
try:
# Get absolute paths
root_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '../../..'))
webview_dir = os.path.join(root_dir, 'webview-ui')
# Use xvfb-run on Linux
if sys.platform.startswith('linux'):
cmd = f"cd {root_dir} && xvfb-run -a npm run test:coverage > {cls.extension_coverage_file} 2>&1"
else:
cmd = f"cd {root_dir} && npm run test:coverage > {cls.extension_coverage_file} 2>&1"
log("Running extension tests...")
log(f"Command: {cmd}")
result = subprocess.run(cmd, shell=True, check=False, capture_output=True, text=True)
log(f"Extension tests exit code: {result.returncode}")
# Run webview tests with coverage
log("Running webview tests...")
cmd = f"cd {webview_dir} && npm run test:coverage > {cls.webview_coverage_file} 2>&1"
log(f"Command: {cmd}")
result = subprocess.run(cmd, shell=True, check=False, capture_output=True, text=True)
log(f"Webview tests exit code: {result.returncode}")
# Verify files were created
if file_exists(cls.extension_coverage_file):
ext_size = get_file_size(cls.extension_coverage_file)
log(f"Extension coverage file created: {cls.extension_coverage_file} (size: {ext_size} bytes)")
else:
log(f"WARNING: Extension coverage file was not created: {cls.extension_coverage_file}")
if file_exists(cls.webview_coverage_file):
web_size = get_file_size(cls.webview_coverage_file)
log(f"Webview coverage file created: {cls.webview_coverage_file} (size: {web_size} bytes)")
else:
log(f"WARNING: Webview coverage file was not created: {cls.webview_coverage_file}")
log("Coverage reports generation completed.")
except Exception as e:
log(f"Error generating coverage reports: {e}")
import traceback
log(traceback.format_exc())
# Create empty files if tests fail
log("Creating fallback coverage files...")
with open(cls.extension_coverage_file, 'w') as f:
f.write("No coverage data available")
with open(cls.webview_coverage_file, 'w') as f:
f.write("No coverage data available")
def test_extract_coverage(self):
"""Test extract_coverage function with both extension and webview coverage."""
# Check if verbose mode is enabled
if '-v' in sys.argv or '--verbose' in sys.argv:
set_verbose(True)
# Verify files exist before testing
self.assertTrue(file_exists(self.extension_coverage_file),
f"Extension coverage file does not exist: {self.extension_coverage_file}")
self.assertTrue(file_exists(self.webview_coverage_file),
f"Webview coverage file does not exist: {self.webview_coverage_file}")
# Log file sizes
ext_size = get_file_size(self.extension_coverage_file)
web_size = get_file_size(self.webview_coverage_file)
log(f"Extension coverage file size: {ext_size} bytes")
log(f"Webview coverage file size: {web_size} bytes")
# Test extension coverage
log("Testing extension coverage extraction...")
ext_coverage_pct = extract_coverage(self.extension_coverage_file, 'extension')
# Check that coverage percentage is a float
self.assertIsInstance(ext_coverage_pct, float)
# Check that coverage percentage is between 0 and 100
self.assertGreaterEqual(ext_coverage_pct, 0)
self.assertLessEqual(ext_coverage_pct, 100)
# Log coverage percentage for debugging
log(f"Extension coverage: {ext_coverage_pct}%")
# Test webview coverage
log("Testing webview coverage extraction...")
web_coverage_pct = extract_coverage(self.webview_coverage_file, 'webview')
# Convert to float if it's an integer
if isinstance(web_coverage_pct, int):
web_coverage_pct = float(web_coverage_pct)
# Check that coverage percentage is a float
self.assertIsInstance(web_coverage_pct, float)
# Check that coverage percentage is between 0 and 100
self.assertGreaterEqual(web_coverage_pct, 0)
self.assertLessEqual(web_coverage_pct, 100)
# Log coverage percentage for debugging
log(f"Webview coverage: {web_coverage_pct}%")
def test_compare_coverage(self):
"""Test compare_coverage function."""
# Test with coverage increase
decreased, diff = compare_coverage(80, 90)
self.assertFalse(decreased)
self.assertEqual(diff, 10)
# Test with coverage decrease
decreased, diff = compare_coverage(90, 80)
self.assertTrue(decreased)
self.assertEqual(diff, 10)
# Test with no change
decreased, diff = compare_coverage(80, 80)
self.assertFalse(decreased)
self.assertEqual(diff, 0)
def test_generate_comment(self):
"""Test generate_comment function."""
comment = generate_comment(
80, 90, 'false', 10,
70, 75, 'false', 5
)
# Check that comment contains expected sections
self.assertIn('Coverage Report', comment)
self.assertIn('Extension Coverage', comment)
self.assertIn('Webview Coverage', comment)
self.assertIn('Overall Assessment', comment)
# Check that comment contains coverage percentages
self.assertIn('Base branch: 80%', comment)
self.assertIn('PR branch: 90%', comment)
self.assertIn('Base branch: 70%', comment)
self.assertIn('PR branch: 75%', comment)
# Check that comment contains correct assessment
self.assertIn('Coverage increased or remained the same', comment)
self.assertIn('Test coverage has been maintained or improved', comment)
@patch('coverage_check.requests.get')
@patch('coverage_check.requests.post')
@patch('coverage_check.requests.patch')
def test_post_comment_new(self, mock_patch, mock_post, mock_get):
"""Test post_comment function when creating a new comment."""
# Create a temporary comment file
comment_file = os.path.join(self.temp_dir.name, 'comment.md')
with open(comment_file, 'w') as f:
f.write('<!-- COVERAGE_REPORT -->\nTest comment')
# Mock the API responses
mock_get.return_value = MagicMock(status_code=200, json=lambda: [])
mock_post.return_value = MagicMock(status_code=201)
# Test post_comment function
post_comment(comment_file, '123', 'owner/repo', 'token')
# Check that the correct API calls were made
mock_get.assert_called_once()
mock_post.assert_called_once()
mock_patch.assert_not_called()
@patch('coverage_check.requests.get')
@patch('coverage_check.requests.post')
@patch('coverage_check.requests.patch')
def test_post_comment_update(self, mock_patch, mock_post, mock_get):
"""Test post_comment function when updating an existing comment."""
# Create a temporary comment file
comment_file = os.path.join(self.temp_dir.name, 'comment.md')
with open(comment_file, 'w') as f:
f.write('<!-- COVERAGE_REPORT -->\nTest comment')
# Mock the API responses
mock_get.return_value = MagicMock(
status_code=200,
json=lambda: [{'id': 456, 'body': '<!-- COVERAGE_REPORT -->\nOld comment'}]
)
mock_patch.return_value = MagicMock(status_code=200)
# Test post_comment function
post_comment(comment_file, '123', 'owner/repo', 'token')
# Check that the correct API calls were made
mock_get.assert_called_once()
mock_patch.assert_called_once()
mock_post.assert_not_called()
def test_set_github_output(self):
"""Test set_github_output function."""
# Capture stdout
with patch('sys.stdout', new=MagicMock()) as mock_stdout:
# Mock environment without GITHUB_OUTPUT
with patch.dict('os.environ', {}, clear=True):
set_github_output('test_name', 'test_value')
# Check that the correct output was printed to stdout
mock_stdout.assert_has_calls([
# GitHub Actions output format (deprecated method)
call.write('::set-output name=test_name::test_value\n'),
call.flush(),
# Human readable format
call.write('test_name: test_value\n'),
call.flush()
], any_order=False)
# Reset mock for next test
mock_stdout.reset_mock()
# Test with GITHUB_OUTPUT environment variable
with patch.dict('os.environ', {'GITHUB_OUTPUT': '/tmp/github_output'}), \
patch('builtins.open', mock_open()) as mock_file:
set_github_output('test_name', 'test_value')
# Check that file was written to
mock_file.assert_called_once_with('/tmp/github_output', 'a')
mock_file().write.assert_called_once_with('test_name=test_value\n')
# Check that human readable output was printed
mock_stdout.assert_has_calls([
call.write('test_name: test_value\n'),
call.flush()
], any_order=False)
if __name__ == '__main__':
unittest.main()
+5 -26
View File
@@ -2,7 +2,6 @@ name: Changeset Converter
run-name: Changeset Conversion
on:
workflow_dispatch:
pull_request:
types: [closed]
@@ -14,36 +13,16 @@ env:
jobs:
# Job 1: Create version bump PR when changesets are merged to main
changeset-pr-version-bump:
if: |
github.event_name == 'workflow_dispatch' ||
(
github.event_name == 'pull_request' &&
github.event.pull_request.merged == true &&
github.event.pull_request.base.ref == 'main' &&
github.actor != 'github-actions'
)
if: >
github.event_name == 'pull_request' &&
github.event.pull_request.merged == true &&
github.event.pull_request.base.ref == 'main' &&
github.actor != 'github-actions'
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- name: Check user for team affiliation
id: team_check
if: github.event_name == 'workflow_dispatch'
uses: morfien101/actions-authorized-user@4a3cfbf0bcb3cafe4a71710a278920c5d94bb38b
with:
username: ${{ github.actor }}
team: "deployer"
github_token: ${{ secrets.GITHUB_TOKEN }}
- name: Check if user is authorized
if: github.event_name == 'workflow_dispatch'
run: |
if [ "${{ steps.team_check.outputs.authorized }}" != "true" ]; then
echo "User is not authorized to run this workflow."
exit 1
fi
- name: Git Checkout
uses: actions/checkout@v4
with:
-28
View File
@@ -1,28 +0,0 @@
# Codespell configuration is within .codespellrc
---
name: Codespell
on:
push:
branches: [main]
pull_request:
branches: [main]
permissions:
contents: read
jobs:
codespell:
if: false
name: Check for spelling errors
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Annotate locations with typos
uses: codespell-project/codespell-problem-matcher@v1
- name: Codespell
uses: codespell-project/actions-codespell@v2
with:
only_warn: 1
+4 -4
View File
@@ -34,7 +34,7 @@ jobs:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "lts/*"
node-version: 20.15.1
# Cache root dependencies - only reuse if package-lock.json exactly matches
- name: Cache root dependencies
@@ -54,14 +54,14 @@ jobs:
- name: Install root dependencies
if: steps.root-cache.outputs.cache-hit != 'true'
run: npm ci --include=optional
run: npm ci
- name: Install webview-ui dependencies
if: steps.webview-cache.outputs.cache-hit != 'true'
run: cd webview-ui && npm ci --include=optional
run: cd webview-ui && npm ci
- name: Install Publishing Tools
run: npm install -g @vscode/vsce ovsx
run: npm install -g vsce ovsx
- name: Get Version
id: get_version
+3 -139
View File
@@ -23,18 +23,7 @@ jobs:
- name: Setup Node.js environment
uses: actions/setup-node@v4
with:
node-version: 22
# Setup Python for coverage script
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: "3.10"
- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
pip install requests
node-version: 20.15.1
# Cache root dependencies - only reuse if package-lock.json exactly matches
- name: Cache root dependencies
@@ -69,130 +58,5 @@ jobs:
- name: Prettier / Format Check
run: npm run format
# Build the extension before running tests
- name: Build Tests and Extension
run: npm run pretest
- name: Unit Tests
run: npm run test:unit
# Run extension tests with coverage
- name: Extension Tests with Coverage
id: extension_coverage
continue-on-error: true
run: |
xvfb-run -a npm run test:coverage > extension_coverage.txt 2>&1
PYTHONPATH=.github/scripts python -m coverage_check extract-coverage extension_coverage.txt --type=extension --github-output --verbose
# Run webview tests with coverage
- name: Webview Tests with Coverage
id: webview_coverage
continue-on-error: true
run: |
cd webview-ui
# Ensure coverage dependency is installed
npm install --no-save @vitest/coverage-v8
npm run test:coverage > webview_coverage.txt 2>&1 || true
cd ..
PYTHONPATH=.github/scripts python -m coverage_check extract-coverage webview-ui/webview_coverage.txt --type=webview --github-output --verbose
# Save coverage reports as artifacts (workflow-scoped)
- name: Save Coverage Reports
uses: actions/upload-artifact@v4
with:
name: pr-coverage-reports
path: |
extension_coverage.txt
webview-ui/webview_coverage.txt
retention-period: workflow # Artifacts are automatically deleted when the workflow completes
# Set the check as failed if any of the tests failed
- name: Check for test failures
run: |
# Check if any of the test steps failed
# https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/accessing-contextual-information-about-workflow-runs#steps-context
if [ "${{ steps.extension_coverage.outcome }}" != "success" ] || [ "${{ steps.webview_coverage.outcome }}" != "success" ]; then
echo "Tests failed."
cat extension_coverage.txt
cat webview-ui/webview_coverage.txt
exit 1
fi
coverage:
needs: test
runs-on: ubuntu-latest
# Only run on PRs to main branch
if: github.event_name == 'pull_request' && github.base_ref == 'main'
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0 # Fetch all history for accurate comparison
# Setup Python for coverage script
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: "3.10"
- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
pip install requests
- name: Setup Node.js environment
uses: actions/setup-node@v4
with:
node-version: 22
# Cache root dependencies - only reuse if package-lock.json exactly matches
- name: Cache root dependencies
uses: actions/cache@v4
id: root-cache
with:
path: node_modules
key: ${{ runner.os }}-npm-${{ hashFiles('package-lock.json') }}
# Cache webview-ui dependencies - only reuse if package-lock.json exactly matches
- name: Cache webview-ui dependencies
uses: actions/cache@v4
id: webview-cache
with:
path: webview-ui/node_modules
key: ${{ runner.os }}-npm-webview-${{ hashFiles('webview-ui/package-lock.json') }}
- name: Install root dependencies
if: steps.root-cache.outputs.cache-hit != 'true'
run: npm ci
- name: Install webview-ui dependencies
if: steps.webview-cache.outputs.cache-hit != 'true'
run: cd webview-ui && npm ci
# Build the extension before running tests
- name: Build Extension
run: npm run compile
# Download coverage artifacts from test job
- name: Download Coverage Reports
uses: actions/download-artifact@v4
with:
name: pr-coverage-reports
path: . # Download to root directory to match expected paths
# Process coverage workflow
- name: Process coverage workflow
id: coverage
run: |
# Extract PR number from GITHUB_REF
PR_NUMBER=$(echo "$GITHUB_REF" | sed -e 's/refs\/pull\///' -e 's/\/merge//')
# Run the coverage workflow from root directory
PYTHONPATH=.github/scripts python -m coverage_check process-workflow \
--base-branch ${{ github.base_ref }} \
--pr-number $PR_NUMBER \
--repo $GITHUB_REPOSITORY \
--token ${{ secrets.GITHUB_TOKEN }} \
--verbose
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Extension Tests
run: xvfb-run -a npm run test
+2 -10
View File
@@ -1,4 +1,5 @@
out
coverage
dist
node_modules
tmp
@@ -9,13 +10,4 @@ tmp
pnpm-lock.yaml
.clineignore
.venv
.actrc
# Ignore coverage directories and files
coverage
# But don't ignore the coverage scripts in .github/scripts/
!.github/scripts/coverage/
*evals.env
.clineignore
-6
View File
@@ -1,6 +0,0 @@
{
"extension": ["ts"],
"spec": "src/**/__tests__/*.ts",
"require": ["ts-node/register", "source-map-support/register", "./src/test/requires.ts"],
"recursive": true
}
-4
View File
@@ -6,10 +6,6 @@ export default defineConfig({
mocha: {
ui: "bdd",
timeout: 20000, // Maximum time (in ms) that a test can run before failing
/** Set up alias path resolution during tests
* @See {@link file://./test-setup.js}
*/
require: ["./test-setup.js"],
},
workspaceFolder: "test-workspace",
version: "stable",
+1 -1
View File
@@ -9,7 +9,7 @@
"name": "Run Extension",
"type": "extensionHost",
"request": "launch",
"args": ["--extensionDevelopmentPath=${workspaceFolder}", "--disable-workspace-trust", "${workspaceFolder}"],
"args": ["--extensionDevelopmentPath=${workspaceFolder}"],
"outFiles": ["${workspaceFolder}/dist/**/*.js"],
"preLaunchTask": "${defaultBuildTask}",
"env": {
+12 -85
View File
@@ -3,46 +3,17 @@
{
"version": "2.0.0",
"tasks": [
{
"label": "npm: protos",
"type": "npm",
"script": "protos",
"problemMatcher": [],
"isBackground": false,
"presentation": {
"reveal": "always"
},
"options": {
"env": {
"IS_DEV": "true"
}
}
},
{
"label": "watch",
"dependsOn": ["npm: protos", "npm: build:webview", "npm: dev:webview", "npm: watch:tsc", "npm: watch:esbuild"],
"dependsOn": ["npm: build:webview", "npm: dev:webview", "npm: watch:tsc", "npm: watch:esbuild"],
"presentation": {
"reveal": "always"
"reveal": "never"
},
"group": {
"kind": "build",
"isDefault": true
}
},
{
"label": "watch:test",
"dependsOn": [
"npm: protos",
"npm: build:webview:test",
"npm: dev:webview",
"npm: watch:tsc",
"npm: watch:esbuild:test"
],
"presentation": {
"reveal": "always"
},
"group": "build"
},
{
"type": "npm",
"script": "build:webview",
@@ -50,10 +21,10 @@
"problemMatcher": [],
"isBackground": true,
"label": "npm: build:webview",
"dependsOn": ["npm: protos"],
"presentation": {
"group": "watch",
"reveal": "always"
"reveal": "never",
"close": true
},
"options": {
"env": {
@@ -61,25 +32,6 @@
}
}
},
{
"type": "npm",
"script": "build:webview:test",
"group": "build",
"problemMatcher": [],
"isBackground": true,
"label": "npm: build:webview:test",
"dependsOn": ["npm: protos"],
"presentation": {
"group": "watch",
"reveal": "always"
},
"options": {
"env": {
"IS_DEV": "true",
"IS_TEST": "true"
}
}
},
{
"type": "npm",
"script": "dev:webview",
@@ -103,10 +55,10 @@
],
"isBackground": true,
"label": "npm: dev:webview",
"dependsOn": ["npm: protos"],
"presentation": {
"group": "watch",
"reveal": "always"
"reveal": "never",
"close": true
},
"options": {
"env": {
@@ -121,34 +73,10 @@
"problemMatcher": "$esbuild-watch",
"isBackground": true,
"label": "npm: watch:esbuild",
"dependsOn": ["npm: protos"],
"presentation": {
"group": "watch",
"reveal": "always"
},
"options": {
"env": {
"IS_DEV": "true"
}
}
},
{
"type": "npm",
"script": "watch:esbuild:test",
"group": "build",
"problemMatcher": "$esbuild-watch",
"isBackground": true,
"label": "npm: watch:esbuild:test",
"dependsOn": ["npm: protos"],
"presentation": {
"group": "watch",
"reveal": "always"
},
"options": {
"env": {
"IS_DEV": "true",
"IS_TEST": "true"
}
"reveal": "never",
"close": true
}
},
{
@@ -158,10 +86,10 @@
"problemMatcher": "$tsc-watch",
"isBackground": true,
"label": "npm: watch:tsc",
"dependsOn": ["npm: protos"],
"presentation": {
"group": "watch",
"reveal": "always"
"reveal": "never",
"close": true
}
},
{
@@ -169,16 +97,15 @@
"script": "watch-tests",
"problemMatcher": "$tsc-watch",
"isBackground": true,
"dependsOn": ["npm: protos"],
"presentation": {
"reveal": "always",
"reveal": "never",
"group": "watchers"
},
"group": "build"
},
{
"label": "tasks: watch-tests",
"dependsOn": ["npm: protos", "npm: watch", "npm: watch-tests"],
"dependsOn": ["npm: watch", "npm: watch-tests"],
"problemMatcher": []
},
{
+1 -5
View File
@@ -37,12 +37,8 @@ docs/**
!node_modules/@vscode/codicons/dist/codicon.css
!node_modules/@vscode/codicons/dist/codicon.ttf
# Include KaTeX CSS and fonts for LaTeX rendering
!webview-ui/node_modules/katex/dist/katex.min.css
!webview-ui/node_modules/katex/dist/fonts/**
# Include default themes JSON files used in getTheme
!src/integrations/theme/default-themes/**
# Include icons
!assets/icons/**
!assets/icons/**
+7 -166
View File
@@ -1,162 +1,5 @@
# Changelog
## [3.14.0]
- Add support for custom model ID in AWS Bedrock provider, enabling use of Application Inference Profile (Thanks @clicube!)
- Add more robust caching & cache tracking for gemini & vertex providers
- Add support for LaTeX rendering
- Add support for custom API request timeout. Timeouts were 15-30s, but can now be configured via settings for OpenRouter/Cline & Ollama (Thanks @WingsDrafterwork!)
- Add truncation notice when truncating manually
- Add a timeout setting for the terminal connection, allowing users to set a time to wait for terminal startup
- Add copy button to code blocks
- Add copy button to markdown blocks (Thanks @weshoke!)
- Add checkpoints to more messages
- Add slash command to create a new rules file (/newrule)
- Add cache ui for open router and cline provider
- Add Amazon Nova Premier model to Bedrock (Thanks @watany!)
- Add support for cursorrules and windsurfrules
- Add support for batch history deletion (Thanks @danix800!)
- Improve Drag & Drop experience
- Create clinerules folder when creating new rule if it's needed
- Enable pricing calculation for gemini and vertex providers
- Refactor message handling to not show the MCP View of the server modal
- Migrate the addRemoteServer to protobus (Thanks @DaveFres!)
- Update task header to be expanded by default
- Update Gemini cache TTL time to 15 minutes
- Fix race condition in terminal command usage
- Fix to correctly handle `import.meta.url`, avoiding leading slash in pathname for Windows (Thanks @DaveFres!)
- Fix @withRetry() decoration syntax error when running extension locally (Thanks @DaveFres!)
- Fix for git commit mentions in repos with no git commits
- Fix cost calculation (Thanks @BarreiroT!)
## [3.13.3]
- Add download counts to MCP marketplace items
- Add `/compact` command
- Add prompt caching to gemini models in cline / openrouter providers
- Add tooltips to bottom row menu
## [3.13.2]
- Add Gemini 2.5 Flash model to Vertex and Gemini Providers (Thanks monotykamary!)
- Add Caching to gemini provider (Thanks arafatkatze!)
- Add thinking budget support to Gemini Models (Thanks monotykamary!)
- Add !include .file directive support for .clineignore (Thanks watany-dev!)
- Improve slash command functionality
- Improve prompting for new task tool
- Fix o1 temperature being passed to the azure api (Thanks treeleaves30760!)
- Fix to make "add new rule file" button functional
- Fix Ollama provider timeout, allowing for a larger loading time (Thanks suvarchal!)
- Fix Non-UTF-8 File Handling: Improve Encoding Detection to Prevent Garbled Text and Binary Misclassification (Thanks yt3trees!)
- Fix settings to not reset by changing providers
- Fix terminal outputs missing commas
- Fix terminal errors caused by starting non-alphanumeric outputs
- Fix auto approve settings becoming unset
- Fix Mermaid syntax error in documentation (Thanks tuki0918!)
- Remove supportsComputerUse restriction and support browser use through any model that supports images (Thanks arafatkatze!)
## [3.13.1]
- Fix bug where task cancellation during thinking stream would result in error state
## [3.13.0]
- Add Cline rules popover under the chat field, allowing you to easily add, enable & disable workspace level or global rule files
- Add new slash command menu letting you type “/“ to do quick actions like creating new tasks
- Add ability to edit past messages, with options to restore your workspace back to that point
- Allow sending a message when selecting an option provided by the question or plan tool
- Add command to jump to Cline's chat input
- Add support for OpenAI o3 & 4o-mini (Thanks @PeterDaveHello and @arafatkatze!)
- Add baseURL option for Google Gemini provider (Thanks @owengo and @olivierhub!)
- Add support for Azure's DeepSeek model. (Thanks @yt3trees!)
- Add ability for models that support it to receive image responses from MCP servers (Thanks @rikaaa0928!)
- Improve search and replace diff editing by making it more flexible with models that fail to follow structured output instructions. (Thanks @chi-cat!)
- Add detection of Ctrl+C termination in terminal, improving output reading issues
- Fix issue where some commands with large output would cause UI to freeze
- Fix token usage tracking issues with vertex provider (Thanks @mzsima!)
- Fix issue with xAI reasoning content not being parsed (Thanks @mrubens!)
## [3.12.3]
- Add copy button to MermaidBlock component (Thanks @cacosub7!)
- Add the ability to fetch from global cline rules files
- Add icon to indicate when a file outside of the users workspace is edited
## [3.12.2]
- Add gpt-4.1
## [3.12.1]
- Use visual checkpoint indicator to make it clear when checkpoints are created
- Big shoutout to @samuel871211 for numerous code quality improvements, refactoring contributions, and webview performance improvements!
- Use improved context manager
## [3.12.0]
- Add favorite toggles for models when using the Cline & OpenRouter providers
- Add auto-approve options for edits/reads outside of the workspace
- Improve diff editing animation for large files
- Add indicator showing number of diff edits when Cline edits a file
- Add streaming support and reasoning effort option to xAI's Grok 3 Mini
- Add settings button to MCP popover to easily modify installed servers
- Fix bug where browser tool actions would show unparsed results in the chat view
- Fix issue with new checkpoints popover hiding too quickly
- Fix duplicate checkpoints bug
- Improve Ollama provider with retry mechanism, timeout handling, and improved error handling (thanks suvarchal!)
## [3.11.0]
- Redesign checkpoint UI to declutter chat view by using a subtle indicator line that expands to a popover on hover, with a new date indicator for when it was created
- Add support for xAI's provider's Grok 3 models
- Add more robust error tracking for users opted in to telemetry (thank you for helping us make Cline better!)
## [3.10.1]
- Add CMD+' keyboard shortcut to add selected text to Cline
- Cline now auto focuses the text field when using 'Add to Cline' shortcut
- Add new 'Create New Task' tool to let Cline start a new task autonomously!
- Fix Mermaid diagram issues
- Fix Gemini provider cost calculation to take new tiered pricing structure into account
## [3.10.0]
- Add setting to let browser tool use local Chrome via remote debugging, enabling session-based browsing. Replaces sessionless Chromium, unlocking debugging and productivity workflows tied to your real browser state.
- Add new auto-approve option to approve _ALL_ commands (use at your own risk!)
- Add modal in the chat area to more easily enable or disable MCP servers
- Add drag and drop of file/folders into cline chat (Thanks eljapi!)
- Add prompt caching for LiteLLM + Claude (Thanks sammcj!)
- Add Improved context management
- Fix MCP auto approve toggle issues being out of sync with settings
## [3.9.2]
- Add recommended models for Cline provider
- Add ability to detect when user edits files manually so Cline knows to re-read, leading to reduced diff edit errors
- Add improvements to file mention searching for faster searching
- Add scoring logic to file mentions to sort and exclude results based on relevance
- Add Support for Bytedance Doubao (Thanks Tunixer!)
- Fix to prevent duplicate BOM (Thanks bamps53!)
## [3.9.1]
- Add Gemini 2.5 Pro Preview 03-25 to Google Provider
## [3.9.0]
- Add Enable extended thinking for LiteLLM provider (Thanks @jorgegarciarey!)
- Add a tab for configuring local MCP Servers
- Fix issue with DeepSeek API provider token counting + context management
- Fix issues with checkpoints hanging under certain conditions
## [3.8.6]
- Add UI for adding remote servers
- Add Mentions Feature Guide and update related documentation
- Fix bug where menu would open in sidebar and open tab
- Fix issue with Cline accounts not showing user info in popout tabs
- Fix bug where menu buttons wouldn't open view in sidebar
## [3.8.5]
- Add support for remote MCP Servers using SSE
@@ -165,8 +8,6 @@
- Add task feedback telemetry (thumbs up/down on task completion)
- Add toggle disabled for remote servers
- Move the MCP Restart and Delete buttons and add an auto-approve all toggle
- Update Requestly UX for model selection (thanks @arafatkatze!)
- Add escape for html content for gemini when running commands
- Improve search and replace edit failure behaviors
## [3.8.4]
@@ -379,8 +220,8 @@
## [3.1.0]
- Added checkpoints: Snapshots of workspace are automatically created whenever Cline uses a tool
- Compare changes: Hover over any tool use to see a diff between the snapshot and current workspace state
- Restore options: Choose to restore just the task state, just the workspace files, or both
- Compare changes: Hover over any tool use to see a diff between the snapshot and current workspace state
- Restore options: Choose to restore just the task state, just the workspace files, or both
- New 'See new changes' button appears after task completion, providing an overview of all workspace changes
- Task header now shows disk space usage with a delete button to help manage snapshot storage
@@ -562,10 +403,10 @@
## [1.8.0]
- You can now use '@' in the textarea to add context!
- @url: Paste in a URL for the extension to fetch and convert to markdown, useful when you want to give Claude the latest docs!
- @problems: Add workspace errors and warnings for Claude to fix, no more back-and-forth about debugging
- @file: Adds a file's contents so you don't have to waste API requests approving read file (+ type to search files)
- @folder: Adds folder's files all at once to speed up your workflow even more
- @url: Paste in a URL for the extension to fetch and convert to markdown, useful when you want to give Claude the latest docs!
- @problems: Add workspace errors and warnings for Claude to fix, no more back-and-forth about debugging
- @file: Adds a file's contents so you don't have to waste API requests approving read file (+ type to search files)
- @folder: Adds folder's files all at once to speed up your workflow even more
## [1.7.0]
@@ -612,7 +453,7 @@
- Adds "Always allow read-only operations" setting to let Claude read files and view directories without needing approval (off by default)
- Implement sliding window context management to keep tasks going past 200k tokens
- Adds Google Cloud Vertex AI support and updates Claude 3.5 Sonnet max output to 8192 tokens for all providers.
- Improves system prompt to guard against lazy edits (less "//rest of code here")
- Improves system prompt to gaurd against lazy edits (less "//rest of code here")
## [1.3.0]
+1 -1
View File
@@ -24,7 +24,7 @@ English | <a href="https://github.com/cline/cline/blob/main/locales/es/README.md
<a href="https://github.com/cline/cline/discussions/categories/feature-requests?discussions_q=is%3Aopen+category%3A%22Feature+Requests%22+sort%3Atop" target="_blank"><strong>Feature Requests</strong></a>
</td>
<td align="center">
<a href="https://docs.cline.bot/getting-started/for-new-coders" target="_blank"><strong>Getting Started</strong></a>
<a href="https://docs.cline.bot/getting-started/getting-started-new-coders" target="_blank"><strong>Getting Started</strong></a>
</td>
</tbody>
</table>
-1
View File
@@ -18,7 +18,6 @@ Welcome to the Cline documentation - your comprehensive guide to using and exten
- **Understand Cline's capabilities:**
- [Cline Tools Guide](tools/cline-tools-guide.md)
- [Mentions Feature Guide](tools/mentions-guide.md)
- **Extend Cline with MCP Servers:**
- [MCP Overview](mcp/README.md)
@@ -0,0 +1,41 @@
graph TB
subgraph VSCode Extension Host
subgraph Core Extension
ExtensionEntry[Extension Entry<br/>src/extension.ts]
ClineProvider[ClineProvider<br/>src/core/webview/ClineProvider.ts]
ClineClass[Cline Class<br/>src/core/Cline.ts]
GlobalState[VSCode Global State]
SecretsStorage[VSCode Secrets Storage]
end
subgraph Webview UI
WebviewApp[React App<br/>webview-ui/src/App.tsx]
ExtStateContext[ExtensionStateContext<br/>webview-ui/src/context/ExtensionStateContext.tsx]
ReactComponents[React Components]
end
subgraph Storage
TaskStorage[Task Storage<br/>Per-Task Files & History]
CheckpointSystem[Git-based Checkpoints]
end
end
%% Core Extension Data Flow
ExtensionEntry --> ClineProvider
ClineProvider --> ClineClass
ClineClass --> GlobalState
ClineClass --> SecretsStorage
ClineClass --> TaskStorage
ClineClass --> CheckpointSystem
%% Webview Data Flow
WebviewApp --> ExtStateContext
ExtStateContext --> ReactComponents
%% Bidirectional Communication
ClineProvider <-->|postMessage| ExtStateContext
style GlobalState fill:#ff0066,stroke:#333,stroke-width:2px,color:#ffffff
style SecretsStorage fill:#ff0066,stroke:#333,stroke-width:2px,color:#ffffff
style ExtStateContext fill:#0066ff,stroke:#333,stroke-width:2px,color:#ffffff
style ClineProvider fill:#00cc66,stroke:#333,stroke-width:2px,color:#ffffff
Binary file not shown.

Before

Width:  |  Height:  |  Size: 902 B

Binary file not shown.

Before

Width:  |  Height:  |  Size: 666 B

-95
View File
@@ -1,95 +0,0 @@
{
"$schema": "https://mintlify.com/docs.json",
"theme": "linden",
"name": "Cline",
"description": "AI-powered coding assistant for VSCode",
"colors": {
"primary": "#9D4EDD",
"light": "#F0E6FF",
"dark": "#000000"
},
"logo": {
"light": "/assets/robot_panel_light.png",
"dark": "/assets/robot_panel_dark.png"
},
"favicon": {
"light": "/assets/robot_panel_light.png",
"dark": "/assets/robot_panel_dark.png"
},
"background": {
"color": {
"light": "#F0E6FF",
"dark": "#000000"
},
"decoration": "gradient"
},
"styling": {
"eyebrows": "breadcrumbs",
"codeblocks": "system"
},
"appearance": {
"default": "system",
"strict": false
},
"fonts": {
"family": "Roboto",
"weight": 400
},
"navbar": {
"links": [
{
"label": "GitHub",
"href": "https://github.com/cline/cline"
},
{
"label": "Discord",
"href": "https://discord.gg/cline"
}
],
"primary": {
"type": "button",
"label": "Install Cline",
"href": "https://cline.bot/install?utm_source=website&utm_medium=header"
}
},
"navigation": {
"groups": [
{
"group": "Getting Started",
"pages": [
"getting-started/for-new-coders",
"getting-started/installing-cline",
"getting-started/installing-dev-essentials",
"getting-started/model-selection-guide",
"getting-started/our-favorite-tech-stack",
"getting-started/understanding-context-management",
"getting-started/what-is-cline"
]
},
{
"group": "Exploring Cline's Tools",
"pages": [
"exploring-clines-tools/cline-tools-guide",
"exploring-clines-tools/plan-and-act-modes-a-guide-to-effective-ai-development",
"exploring-clines-tools/checkpoints",
"exploring-clines-tools/new-task-tool",
"exploring-clines-tools/remote-browser-support",
"exploring-clines-tools/slash-commands"
]
}
]
},
"footer": {
"socials": {
"x": "https://x.com/cline",
"github": "https://github.com/cline/cline",
"discord": "https://discord.gg/cline"
}
},
"search": {
"prompt": "Search Cline documentation..."
},
"contextual": {
"options": ["copy"]
}
}
-108
View File
@@ -1,108 +0,0 @@
---
title: "Checkpoints and Messages"
description: "When working with AI coding assistants, it's easy to lose control as they make rapid changes to your codebase. That's why we built Checkpoints - your safety net for experimenting confidently."
---
Checkpoints automatically save snapshots of your workspace after each step in a task. This powerful feature lets you:
- Track and review changes made during a task
- Roll back to any previous point if needed
- Experiment confidently with auto-approve mode
- Maintain full control over your workspace
### ⚙️ How Checkpoints Work
Cline creates a checkpoint after each tool use (file edits, commands, etc.). These checkpoints:
- Work alongside your Git workflow without interference
- Maintain context between restores
- Use a shadow Git repository to track changes
For example, if you're working on a feature and Cline makes multiple file changes, each change creates a checkpoint. This means you can review each modification and, if needed, roll back to any point without affecting your main Git repository.
#### Viewing Changes & Restoring to Checkpoint
After each tool use, you can:
1. Click the "Compare" button to see modified files
2. Click the "Restore" button to open restore options
<Frame>
<img src="/assets/robot_panel_dark.png" alt="Checkpoint comparison and restore options" />
</Frame>
#### Rolling Back
To restore to a previous point:
1. Click the "Restore" button next to any step
2. Choose from three options:
- **Restore Task and Workspace**: Reset both codebase and task to that point
- **Restore Task Only**: Keep codebase changes but revert task context
- **Restore Workspace Only**: Reset codebase while preserving task context
Example: If Cline makes changes you don't like while styling a component, you can use "Restore Workspace Only" to revert the code changes while keeping the conversation context, allowing you to try a different approach.
### 💡 Use Cases
Checkpoints let you be more experimental with Cline. While human coding is often methodical and iterative, AI can make substantial changes quickly. Checkpoints help you track these changes and revert if needed.
#### 1. Using Auto-Approve Mode
- Provides safety net for rapid iterations
- Makes it easy to undo unexpected results
#### 2. Testing Different Approaches
- Try multiple solutions confidently
- Compare different implementations
- Quickly revert to working states
- Ideal for exploring different design patterns or architectural approaches
<Frame caption="In this case, I didn't like the changes Cline made to my robot dog-walking website (still working on the robots) and I wanted to revert both the codebase and the task to before any changes were made so I could start fresh.">
<img src="/assets/robot_panel_dark.png" alt="Checkpoint restore demo" />
</Frame>
### ✨ Best Practices
1. Use checkpoints as safety nets when experimenting
2. Leverage auto-approve mode more confidently, knowing you can always roll back
3. Restore selectively based on needs:
- Use "Restore Task and Workspace" for a fresh start, reversing changes to files and the task conversation.
- Use "Restore Task Only" to try different prompts, but leave all files as they exist
- Use "Restore Workspace Only" to attempt different implementations, or prune context from the task
🛟 Checkpoints are your safety net when working with Cline, enabling you to experiment freely while maintaining full control over your codebase. Whether you're refactoring a complex component, trying different implementation approaches, or using auto-approve mode for rapid development, checkpoints ensure you can always review changes and roll back if needed.
#### 🗑️ Deleting Checkpoints
You can delete all checkpoints by using the **"Delete All History"** button in the task history menu. Note that this will also delete all tasks. Checkpoints are stored in VS Code's globalStorage.
---
## Editing Messages
Cline allows you to edit chat messages in a task after they've been submitted (with the exception of the message that started the task).
Perhaps you didn't get the results you wanted, thought of a better way to phrase your request, or need to add more information. Editing your message allows you to re-submit a request without starting over or restoring your files or workspace with checkpoints. There are two Restore options:
- **"Restore Chat"** restores just the task state and re-submits an API request to your provider with your edited message.
- **"Restore All"** restores both the task state and workspace state before re-submitting an API request. "Workspace state" refers to the condition of your workspace (files, content, etc.) at different points in the conversation.
**Interactive Editing:**
- Messages can be clicked to enter edit mode
- Cline automatically selects all text when entering edit mode
**Keyboard Shortcuts:**
- Escape: Exit edit mode
- Enter: Restore just the task
- Cmd/Ctrl + Enter: Restore the task and workspace
- Shift + Enter: Insert new line / line break
<Frame>
<img src="/assets/robot_panel_dark.png" alt="Message editing interface" />
</Frame>
@@ -1,139 +0,0 @@
---
title: "Cline Tools Reference Guide"
---
## What Can Cline Do?
Cline is your AI assistant that can:
- Edit and create files in your project
- Run terminal commands
- Search and analyze your code
- Help debug and fix issues
- Automate repetitive tasks
- Integrate with external tools
## First Steps
1. **Start a Task**
- Type your request in the chat
- Example: "Create a new React component called Header"
2. **Provide Context**
- Use @ mentions to add files, folders, or URLs
- Example: "@file:src/components/App.tsx"
3. **Review Changes**
- Cline will show diffs before making changes
- You can edit or reject changes
## Key Features
1. **File Editing**
- Create new files
- Modify existing code
- Search and replace across files
2. **Terminal Commands**
- Run npm commands
- Start development servers
- Install dependencies
3. **Code Analysis**
- Find and fix errors
- Refactor code
- Add documentation
4. **Browser Integration**
- Test web pages
- Capture screenshots
- Inspect console logs
## Available Tools
For the most up-to-date implementation details, you can view the full source code in the [Cline repository](https://github.com/cline/cline/blob/main/src/core/Cline.ts).
Cline has access to the following tools for various tasks:
1. **File Operations**
- `write_to_file`: Create or overwrite files
- `read_file`: Read file contents
- `replace_in_file`: Make targeted edits to files
- `search_files`: Search files using regex
- `list_files`: List directory contents
2. **Terminal Operations**
- `execute_command`: Run CLI commands
- `list_code_definition_names`: List code definitions
3. **MCP Tools**
- `use_mcp_tool`: Use tools from MCP servers
- `access_mcp_resource`: Access MCP server resources
- Users can create custom MCP tools that Cline can then access
- Example: Create a weather API tool that Cline can use to fetch forecasts
4. **Interaction Tools**
- `ask_followup_question`: Ask user for clarification
- `attempt_completion`: Present final results
Each tool has specific parameters and usage patterns. Here are some examples:
- Create a new file (write_to_file):
```xml
<write_to_file>
<path>src/components/Header.tsx</path>
<content>
// Header component code
</content>
</write_to_file>
```
- Search for a pattern (search_files):
```xml
<search_files>
<path>src</path>
<regex>function\s+\w+\(</regex>
<file_pattern>*.ts</file_pattern>
</search_files>
```
- Run a command (execute_command):
```xml
<execute_command>
<command>npm install axios</command>
<requires_approval>false</requires_approval>
</execute_command>
```
## Common Tasks
1. **Create a New Component**
- "Create a new React component called Footer"
2. **Fix a Bug**
- "Fix the error in src/utils/format.ts"
3. **Refactor Code**
- "Refactor the Button component to use TypeScript"
4. **Run Commands**
- "Run npm install to add axios"
## Getting Help
- [Join the Discord community](https://discord.gg/cline)
- Check the documentation
- Provide feedback to improve Cline
@@ -1,381 +0,0 @@
---
title: "New Task Tool"
---
### The `new_task` Tool & Context Management Strategies
#### Overview
Cline includes a powerful internal tool, `new_task`, designed to help manage workflow continuity and context preservation, especially during complex or long-running tasks. This tool, combined with Cline's awareness of its own context window usage and the flexibility of `.clinerules`, enables sophisticated strategies for breaking down work and ensuring seamless transitions between task sessions.
Understanding the core capabilities and how they interact with custom rules is key to leveraging this feature effectively.
#### Core Capabilities
Two fundamental capabilities enable advanced context management:
1. **The `new_task` Tool:**
- **Function:** Allows Cline, upon user approval, to end the current task session and immediately start a new one.
- **Context Preloading:** Crucially, Cline can **preload** this new task session with specific context provided within the tool's `<context>` block. This context can be anything Cline or a `.clinerules` file defines summaries, code snippets, next steps, project state, etc.
2. **Context Window Awareness:**
- **Tracking:** Cline internally tracks the percentage of its available context window currently being used during a task.
- **Visibility:** This information is visible in the `environment_details` provided to Cline in its prompt.
#### Using the `/newtask` Slash Command
As a quick alternative to Cline suggesting the `newtask` tool or defining complex rules, you can directly initiate the process using a Slash Command.
- **How:** Simply type `/newtask` in the chat input field.
- **Action:** Cline will propose creating a new task, typically suggesting context based on the current session (similar to its default behavior when using the tool). You will still get the `ask_followup_question` prompt to confirm and potentially modify the context before the new task is created.
- **Benefit:** Provides a fast, user-initiated way to leverage the `new_task` functionality for branching explorations or managing long sessions without waiting for Cline to suggest it.
#### Default Behavior (Without `.clinerules`)
By default, without specific `.clinerules` dictating its behavior:
- **Tool Availability:** The `new_task` tool exists, and Cline _can_ choose to use it.
- **Context Awareness:** Cline _is_ aware of its context usage percentage.
- **No Automatic Trigger:** Cline **will not** automatically initiate a task handoff _solely_ based on context usage reaching a specific percentage (like 50%). The decision to suggest using `new_task` comes from the AI model's reasoning based on the overall task progress and prompt instructions.
- **Basic Context Preloading:** If `new_task` is used without specific rules defining the `<context>` block structure, Cline will attempt to preload relevant information based on its current understanding (e.g., a basic summary of progress and next steps), but this may be less comprehensive than a rule-driven approach.
#### The Power of `.clinerules`: Enabling Custom Workflows
While the core capabilities exist by default, the true power, automation, and customization emerge when you combine `new_task` and context awareness with custom workflows defined in `.clinerules`. This allows you to precisely control _when_ and _how_ Cline manages context and task continuity.
Key benefits of using `.clinerules` with `new_task`:
- **Automated Context Management:** Define rules to automatically trigger handoffs at specific context percentages (e.g., >50%, >70%) or token counts, ensuring optimal performance and preventing context loss.
- **Model-Specific Optimization:** Tailor handoff triggers based on known thresholds for different LLMs (e.g., trigger earlier for models known to degrade past a certain token count).
- **Intelligent Breakpoints:** Instruct Cline via rules to find logical stopping points (e.g., after completing a function or test) _after_ a context threshold is passed, ensuring cleaner handoffs.
- **Structured Task Decomposition:** Use Plan Mode to define subtasks, then use `.clinerules` to have Cline automatically create a new task via `new_task` upon completing each subtask, preloading the context for the _next_ subtask.
- **Custom Context Packaging:** Mandate the exact structure and content of the `<context>` block in `.clinerules` for highly detailed and consistent handoffs (see example below).
- **Improved Memory Persistence:** Use `new_task` context blocks as a primary, integrated way to persist information across sessions, potentially replacing or supplementing file-based memory systems.
- **Workflow Automation:** Define rules for specific scenarios, like always preloading certain setup instructions or project boilerplate when starting tasks of a particular type.
#### Example Rule-Driven Workflow: Task Handoff Process
A common workflow, **driven by specific `.clinerules` like the example below**, involves these steps:
1. **Trigger Identification (Rule-Based):** Cline monitors for handoff points defined in the rules (e.g., context usage > 50%, task completion).
2. **User Confirmation:** Cline uses `ask_followup_question` to propose creating a new task, often showing the intended context defined by the rules.
```xml
<ask_followup_question>
<question>I've completed [specific accomplishment] and context usage is high (XX%). Would you like me to create a new task to continue with [remaining work], preloading the following context?</question>
<options>["Yes, create new task", "Modify context first", "No, continue this session"]</options>
</ask_followup_question>
```
3. **User Control:** You can approve, deny, or ask Cline to modify the context before the new task is created.
4. **Context Packaging (`new_task` Tool):** If approved, Cline uses `new_task`, packaging the context according to the structure mandated by the `.clinerules`.
5. **New Task Creation:** The current task ends, and a new session begins immediately, preloaded with the specified context.
#### The Handoff Context Block (Rule-Defined Structure)
The effectiveness of rule-driven handoffs depends heavily on how `.clinerules` define the `<context>` block. A comprehensive structure often includes:
- **`## Completed Work`**: Detailed list of accomplishments, files modified/created, key decisions.
- **`## Current State`**: Project status, running processes, key file states.
- **`## Next Steps`**: Clear, prioritized list of remaining tasks, implementation details, known challenges.
- **`## Reference Information`**: Links, code snippets, patterns, user preferences.
- **Actionable Start:** A clear instruction for the immediate next action.
#### Potential Use Cases & Workflows
The flexibility of `new_task` combined with `.clinerules` opens up many possibilities:
- **Proactive Context Window Management:** Automatically trigger handoffs at specific percentages (e.g., 50%, 70%) or token counts to maintain optimal performance.
- **Intelligent Breakpoints:** Instruct Cline to find logical stopping points (e.g., after completing a function or test) _after_ a context threshold is passed, ensuring cleaner handoffs.
- **Structured Task Decomposition:** Use Plan Mode to define subtasks, then use `.clinerules` to have Cline automatically create a new task via `new_task` upon completing each subtask.
- **Automated Session Summaries:** Configure the `<context>` block to always include a summary of the previous session's key discussion points.
- **Preloading Boilerplate/Setup:** Start new tasks related to specific projects preloaded with standard setup instructions or file templates.
- **"Memory Bank" Alternative:** Use `new_task` context blocks as the primary way to persist information across sessions, potentially replacing file-based memory systems.
Experimenting with `.clinerules` is encouraged to discover workflows that best suit your needs!
#### Example `.clinerules`: Task Handoff Strategy Guide
Below is an example `.clinerules` file focused specifically on using `new_task` for context window management. **Remember, this is just one specific strategy; the core `new_task` tool can be used differently with other custom rules.**
````markdown
# You MUST use the `new_task` tool: Task Handoff Strategy Guide
**⚠️ CRITICAL INSTRUCTIONS - YOU MUST FOLLOW THESE GUIDELINES ⚠️**
This guide provides **MANDATORY** instructions for effectively breaking down complex tasks and implementing a smooth handoff process between tasks. You **MUST** follow these guidelines to ensure continuity, context preservation, and efficient task completion.
## ⚠️ CONTEXT WINDOW MONITORING - MANDATORY ACTION REQUIRED ⚠️
You **MUST** monitor the context window usage displayed in the environment details. When usage exceeds 50% of the available context window, you **MUST** initiate a task handoff using the `new_task` tool.
Example of context window usage over 50% with a 200K context window:
\`\`\`text
# Context Window Usage
105,000 / 200,000 tokens (53%)
Model: anthropic/claude-3.7-sonnet (200K context window)
\`\`\`
**IMPORTANT**: When you see context window usage at or above 50%, you MUST:
1. Complete your current logical step
2. Use the `ask_followup_question` tool to offer creating a new task
3. If approved, use the `new_task` tool with comprehensive handoff instructions
## Task Breakdown in Plan Mode - REQUIRED PROCESS
Plan Mode is specifically designed for analyzing complex tasks and breaking them into manageable subtasks. When in Plan Mode, you **MUST**:
### 1. Initial Task Analysis - REQUIRED
- **MUST** begin by thoroughly understanding the full scope of the user's request
- **MUST** identify all major components and dependencies of the task
- **MUST** consider potential challenges, edge cases, and prerequisites
### 2. Strategic Task Decomposition - REQUIRED
- **MUST** break the overall task into logical, discrete subtasks
- **MUST** prioritize subtasks based on dependencies (what must be completed first)
- **MUST** aim for subtasks that can be completed within a single session (15-30 minutes of work)
- **MUST** consider natural breaking points where context switching makes sense
### 3. Creating a Task Roadmap - REQUIRED
- **MUST** present a clear, numbered list of subtasks to the user
- **MUST** explain dependencies between subtasks
- **MUST** provide time estimates for each subtask when possible
- **MUST** use Mermaid diagrams to visualize task flow and dependencies when helpful
\`\`\`mermaid
graph TD
A[Main Task] --> B[Subtask 1: Setup]
A --> C[Subtask 2: Core Implementation]
A --> D[Subtask 3: Testing]
A --> E[Subtask 4: Documentation]
B --> C
C --> D
\`\`\`
### 4. Getting User Approval - REQUIRED
- **MUST** ask for user feedback on the proposed task breakdown
- **MUST** adjust the plan based on user priorities or additional requirements
- **MUST** confirm which subtask to begin with
- **MUST** request the user to toggle to Act Mode when ready to implement
## Task Implementation and Handoff Process - MANDATORY PROCEDURES
When implementing tasks in Act Mode, you **MUST** follow these guidelines for effective task handoff:
### 1. Focused Implementation - REQUIRED
- **MUST** focus on completing the current subtask fully
- **MUST** document progress clearly through comments and commit messages
- **MUST** create checkpoints at logical completion points
### 2. Recognizing Completion Points - CRITICAL
You **MUST** identify natural handoff points when:
- The current subtask is fully completed
- You've reached a logical stopping point in a larger subtask
- The implementation is taking longer than expected and can be continued later
- The task scope has expanded beyond the original plan
- **CRITICAL**: The context window usage exceeds 50% (e.g., 100,000+ tokens for a 200K context window)
### 3. Initiating the Handoff Process - MANDATORY ACTION
When you've reached a completion point, you **MUST**:
1. Summarize what has been accomplished so far
2. Clearly state what remains to be done
3. **MANDATORY**: Use the `ask_followup_question` tool to offer creating a new task:
\`\`\`xml
<ask_followup_question>
<question>I've completed [specific accomplishment]. Would you like me to create a new task to continue with [remaining work]?</question>
<options>["Yes, create a new task", "No, continue in this session", "Let me think about it"]</options>
</ask_followup_question>
\`\`\`
### 4. Creating a New Task with Context - REQUIRED ACTION
If the user agrees to create a new task, you **MUST** use the `new_task` tool with comprehensive handoff instructions:
\`\`\`xml
<new_task>
<context>
# Task Continuation: [Brief Task Title]
## Completed Work
- [Detailed list of completed items]
- [Include specific files modified/created]
- [Note any important decisions made]
## Current State
- [Description of the current state of the project]
- [Any running processes or environment setup]
- [Key files and their current state]
## Next Steps
- [Detailed list of remaining tasks]
- [Specific implementation details to address]
- [Any known challenges to be aware of]
## Reference Information
- [Links to relevant documentation]
- [Important code snippets or patterns to follow]
- [Any user preferences noted during the current session]
Please continue the implementation by [specific next action].
</context>
</new_task>
\`\`\`
### 5. Detailed Context Transfer - MANDATORY COMPONENTS
When creating a new task, you **MUST** always include:
#### Project Context - REQUIRED
- **MUST** include the overall goal and purpose of the project
- **MUST** include key architectural decisions and patterns
- **MUST** include technology stack and dependencies
#### Implementation Details - REQUIRED
- **MUST** list files created or modified in the current session
- **MUST** describe specific functions, classes, or components implemented
- **MUST** explain design patterns being followed
- **MUST** outline testing approach
#### Progress Tracking - REQUIRED
- **MUST** provide checklist of completed items
- **MUST** provide checklist of remaining items
- **MUST** note any blockers or challenges encountered
#### User Preferences - REQUIRED
- **MUST** note coding style preferences mentioned by the user
- **MUST** document specific approaches requested by the user
- **MUST** highlight priority areas identified by the user
## Best Practices for Effective Handoffs - MANDATORY GUIDELINES
### 1. Maintain Continuity - REQUIRED
- **MUST** use consistent terminology between tasks
- **MUST** reference previous decisions and their rationale
- **MUST** maintain the same architectural approach unless explicitly changing direction
### 2. Preserve Context - REQUIRED
- **MUST** include relevant code snippets in the handoff
- **MUST** summarize key discussions from the previous session
- **MUST** reference specific files and line numbers when applicable
### 3. Set Clear Next Actions - REQUIRED
- **MUST** begin the handoff with a clear, actionable next step
- **MUST** prioritize remaining tasks
- **MUST** highlight any decisions that need to be made
### 4. Document Assumptions - REQUIRED
- **MUST** clearly state any assumptions made during implementation
- **MUST** note areas where user input might be needed
- **MUST** identify potential alternative approaches
### 5. Optimize for Resumability - REQUIRED
- **MUST** structure the handoff so the next session can begin working immediately
- **MUST** include setup instructions if environment configuration is needed
- **MUST** provide a quick summary at the top for rapid context restoration
## Example Task Handoff
### Example #1 of an effective task handoff:
\`\`\`xml
<new_task>
<context>
# Task Continuation: Implement User Authentication System
## Completed Work
- Created basic Express.js server structure
- Implemented MongoDB connection and user schema
- Completed user registration endpoint with password hashing
- Added input validation using Joi
- Created initial test suite for registration endpoint
## Current State
- Server runs successfully on port 3000
- MongoDB connection is established
- Registration endpoint (/api/users/register) is fully functional
- Test suite passes for all registration scenarios
## Next Steps
1. Implement login endpoint (/api/users/login)
- Use bcrypt to compare passwords
- Generate JWT token upon successful login
- Add proper error handling for invalid credentials
2. Create authentication middleware
- Verify JWT tokens
- Extract user information
- Handle expired tokens
3. Add protected routes that require authentication
4. Implement password reset functionality
## Reference Information
- JWT secret should be stored in .env file
- Follow the existing error handling pattern in routes/users.js
- User schema is defined in models/User.js
- Test patterns are established in tests/auth.test.js
Please continue by implementing the login endpoint following the same patterns established in the registration endpoint.
</context>
</new_task>
\`\`\`
### Example #2 of an ineffective task handoff:
_(Note: The example provided in the original rules showing "YOLO MODE Implementation" seems less like a direct handoff context block and more like a general status update with future considerations. A true ineffective handoff might lack detail in 'Current State' or 'Next Steps')._
## When to Use Task Handoffs - MANDATORY TRIGGERS
You **MUST** initiate task handoffs in these scenarios:
1. **CRITICAL**: When context window usage exceeds 50% (e.g., 100,000+ tokens for a 200K context window)
2. **Long-running projects** that exceed a single session
3. **Complex implementations** with multiple distinct phases
4. **When context window limitations** are approaching
5. **When switching focus areas** within a larger project
6. **When different expertise** might be beneficial for different parts of the task
**⚠️ FINAL REMINDER - CRITICAL INSTRUCTION ⚠️**
You **MUST** monitor the context window usage in the environment details section. When it exceeds 50% (e.g., "105,000 / 200,000 tokens (53%)"), you **MUST** proactively initiate the task handoff process using the `ask_followup_question` tool followed by the `new_task` tool. You MUST use the `new_task` tool.
By strictly following these guidelines, you'll ensure smooth transitions between tasks, maintain project momentum, and provide the best possible experience for users working on complex, multi-session projects.
```markdown
## User Interaction & Workflow Considerations
- **Linear Flow:** Currently, using `new_task` creates a linear sequence. The old task ends, and the new one begins. The old task history remains accessible for backtracking.
- **User Approval:** You always have control, approving the handoff and having the chance to modify the context Cline proposes to carry forward.
- **Flexibility:** The core `new_task` tool is a flexible building block. Experiment with `.clinerules` to create workflows that best suit your needs, whether for strict context management, task decomposition, or other creative uses.
```
````
@@ -1,136 +0,0 @@
---
title: "Plan & Act Modes: A Guide to Effective AI Development"
---
## Overview
Plan & Act modes represent Cline's approach to structured AI development, emphasizing thoughtful planning before implementation. This dual-mode system helps developers create more maintainable, accurate code while reducing iteration time.
<Frame>
<img src="/assets/robot_panel_dark.png" alt="Use Plan to gather context before using Act to implement the plan" />
</Frame>
### Understanding the Modes
#### Plan Mode
- Optimized for context gathering and strategy
- Cannot make changes to your codebase
- Focused on understanding requirements and creating implementation plans
- Enables full file reading for comprehensive project understanding
#### Act Mode
- Streamlined for implementation based on established plans
- Has access to all of Cline's building capabilities
- Maintains context from the planning phase
- Can execute changes to your codebase
<Frame>
<img src="/assets/robot_panel_dark.png" alt="Act mode capabilities" />
</Frame>
### Workflow Guide
#### 1. Start with Plan Mode
Begin every significant development task in Plan mode:
In this mode:
<Frame>
<img src="/assets/robot_panel_dark.png" alt="Plan mode workflow" />
</Frame>
- Share your requirements
- Let Cline analyze relevant files
- Engage in dialogue to clarify objectives
- Develop implementation strategy
<Frame>
<img src="/assets/robot_panel_dark.png" alt="Planning phase" />
</Frame>
#### 2. Switch to Act Mode
Once you have a clear plan, switch to Act mode:
<Frame>
<img src="/assets/robot_panel_dark.png" alt="Switching to Act mode" />
</Frame>
Act mode allows Cline to:
- Execute against the agreed plan
- Make changes to your codebase
- Maintain context from planning phase
#### 3. Iterate as Needed
Complex projects often require multiple plan-act cycles:
- Return to Plan mode when encountering unexpected complexity
- Use Act mode for implementing solutions
- Maintain development momentum while ensuring quality
### Best Practices
#### Planning Phase
1. Be comprehensive with requirements
2. Share relevant context upfront
3. Point Cline to relevant files if he hasn't read them
4. Validate approach before implementation
#### Implementation Phase
1. Follow the established plan
2. Monitor progress against objectives
3. Track changes and their impact
4. Document significant decisions
<Frame>
<img src="/assets/robot_panel_dark.png" alt="Implementation best practices" />
</Frame>
### Power User Tips
#### Enhancing Planning
- Use Plan mode to explore edge cases before implementation
- Switch back to Plan when encountering unexpected complexity
- Leverage file reading to validate assumptions early
- Have Cline write markdown files of the plan for future reference
### Common Patterns
#### When to Use Plan Mode
- Starting new features
- Debugging complex issues
- Architectural decisions
- Requirements analysis
#### When to Use Act Mode
- Implementing agreed solutions
- Making routine changes
- Following established patterns
- Executing test cases
<Frame>
<img src="/assets/robot_panel_dark.png" alt="Mode usage patterns" />
</Frame>
### Contributing
Share your experiences and improvements:
- Join our [Discord community](https://discord.gg/cline)
- Participate in discussions
- Submit feature requests
- Report issues
---
Remember: The time invested in planning pays dividends in implementation quality and maintenance efficiency
@@ -1,114 +0,0 @@
---
title: "Remote Browser Support"
description: "Remote browser support allows Cline to utilize a remote Chrome instance, leveraging authentication tokens and session cookies relevant to certain web development test cases."
icon: globe-pointer
---
The Remote Browser feature in Cline allows the AI assistant to interact with web content directly through a controlled browser instance. This enables several powerful capabilities:
- Viewing and interacting with websites
- Testing locally running web applications
- Monitoring console logs and errors
- Performing browser actions like clicking, typing, and scrolling
## Remote Browser in Cline
### What is Remote Browser?
Remote Browser allows Cline to view and interact with websites directly. This feature enables Cline to:
- Visit websites and view their content
- Test your locally running web applications
- Fill out forms and click on elements
- Capture screenshots of what it sees
- Scroll through pages to see more content
### How to Use Remote Browser
#### Basic Commands
You can ask Cline to use the browser with simple instructions:
- **Open a website**: "Use the browser to check the website at [https://example.com](https://example.com/)"
- **Click on elements**: "Click the login button"
- **Type text**: "Type 'Hello world' in the search box"
- **Scroll the page**: "Scroll down to see more content"
- **Close the browser**: "Close the browser now"
#### Example Workflows
**Testing a Web Application:**
```javascript
Can you start my React app with "npm start" and then check if it's working properly at http://localhost:3000?
```
**Analyzing a Website:**
```javascript
Can you visit https://example.com and tell me what you think about its design and layout?
```
**Filling Out a Form:**
```javascript
Please go to https://example.com/contact, fill out the contact form with some test data, and submit it.
```
### Important Things to Know
#### One Browser at a Time
Cline can only use one browser at a time. If you want to visit a different website, you can either:
- Ask Cline to navigate to a new URL within the same browser session
- Ask Cline to close the current browser and open a new one
#### Browser Must Be Closed Before Using Other Tools
If you want Cline to edit files or run commands after using the browser, you must first ask it to close the browser:
```javascript
Close the browser and then update the CSS file to fix the alignment issue we saw.
```
#### What Cline Sees
The browser has a fixed viewport size (900x600 pixels by default), similar to a small laptop screen. Cline will share screenshots after each action so you can see exactly what it sees.
#### Console Logs
Cline captures browser console logs, which can be helpful for debugging web applications. These logs are included with each screenshot.
### Common Use Cases
- **Web Development**: Test your websites and web applications
- **UI/UX Review**: Get feedback on website design and usability
- **Content Research**: Have Cline browse websites to gather information
- **Form Testing**: Verify that forms work correctly
- **Responsive Design Testing**: Check how websites look at different screen sizes
### Troubleshooting
- **If a website doesn't load**: Try providing a direct URL with the http:// or https:// prefix
- **If clicking doesn't work**: Try describing the location of the element more precisely
- **If the browser seems stuck**: Ask Cline to close the browser and try again
### Using Remote Browser with VS Code in WSL
When running VS Code in WSL, you'll need to configure Windows to allow WSL to connect to Chrome. Follow these steps:
#### Open PowerShell as Administrator and Run:
```powershell
# Allow WSL to connect to Chrome's debugging port
New-NetFirewallRule -DisplayName "WSL Chrome Debug" -Direction Inbound -LocalPort 9222 -Protocol TCP -Action Allow
```
#### Configure Cline in VS Code:
1. Open VS Code settings
2. Search for "Cline: Chrome Executable Path"
3. Set the value to the path of your Chrome executable (e.g., `C:\Program Files\Google\Chrome\Application\chrome.exe`)
Cline should now be able to use the Remote Browser feature from within WSL.
@@ -1,57 +0,0 @@
---
title: "Slash Commands"
---
#### Overview
Cline provides slash commands as a quick way to invoke specific tools or actions directly from the chat input, offering shortcuts for common operations. This page details the available slash commands and their usage.
#### /newtask
The `/newtask` slash command provides a fast, user-initiated way to leverage the `new_task` tool's functionality for branching explorations or managing long sessions without waiting for Cline to suggest it.
**Functionality:**
1. **Initiation:** Typing `/newtask` in the chat input signals Cline to prepare for starting a new task session.
2. **Context Proposal:** Cline proposes creating a new task and typically suggests context to preload based on the current session (summarizing key aspects like current work, technical concepts, relevant files, problems solved, and next steps).
3. **User Confirmation:** You will receive a confirmation prompt (via the `ask_followup_question` tool) displaying the proposed context. You can approve it directly or modify the context before the new task begins.
4. **New Session:** Upon confirmation, Cline ends the current task session and immediately starts a new one, preloaded with the approved context.
**Benefit:** Allows you to cleanly branch your work or start a new phase while carrying over essential background information ("knowledge transfer") without manual copying or losing the thread of the previous session.
#### /smol (alias /compact)
The `/smol` slash command (with `/compact` as an alias) allows you to condense the chat history **within your current task**. This is useful when a conversation becomes very long, potentially impacting performance or making it harder for the model to maintain focus.
**Functionality:**
1. **Initiation:** Typing `/smol` or `/compact` tells Cline you want to condense the current chat history. You can optionally add instructions after the command to guide the summarization process (e.g., `/smol focus only on the database changes` or `/smol be concise, use bullet points`).
2. **Summarization:** Cline analyzes the conversation history, considering any additional instructions provided, and generates a summary focusing on key elements: recent discussion points, important decisions, technical concepts, relevant files, problems solved, and planned next steps. Cline determines the appropriate length and detail for the summary. It retains the beginning and very recent parts of the chat while summarizing the middle sections.
3. **User Confirmation:** Cline presents this generated summary to you via a confirmation prompt and asks if it accurately reflects the essential context.
4. **Condensing:** If you approve the summary, Cline replaces the summarized middle portion of the chat history in its active context with the generated summary. This reduces the overall token count for subsequent interactions within the _same task_.
5. **Feedback:** If you reject the summary or provide feedback, Cline will retain the original history and incorporate your feedback for future actions.
**Benefit:** Helps maintain focus and manage token usage during very long, continuous tasks (like deep debugging or extended feature development) without needing to start an entirely new task session. Allows user guidance on the summarization focus.
#### When to Use Which?
Choosing between `/newtask` and `/smol` depends on your goal:
- Use `/smol` (or `/compact`) when:
- You want to continue the **same task**, but the chat history has become very long or costly.
- You need to reduce token usage for upcoming interactions within the current workflow.
- Example: Deep debugging session where you want to summarize previous steps before continuing.
- Use `/newtask` when:
- You have finished one phase of work and want to start a **fresh, related task**.
- You want to branch your exploration while preserving key context from the previous session.
- Example: Moving from developing Feature A to starting work on Feature B, carrying over relevant architectural decisions.
#### Why Manage Context?
While Cline supports large context windows, actively managing context using tools and commands like `/newtask` and `/smol` is often beneficial:
- **Performance:** Large language models can sometimes experience performance degradation or lose focus when context windows become extremely full (e.g., over 50-75% capacity, depending on the model). Condensing or resetting context can help maintain optimal performance.
- **Relevance:** Summarizing or starting fresh ensures the most relevant information is prioritized in the context window.
- **Cost:** Reducing the number of tokens sent to the model in each turn can help manage costs, especially with more expensive models.
Using `/newtask` and `/smol` provides you with direct control over the conversation context, allowing for more efficient and effective interaction with Cline.
-68
View File
@@ -1,68 +0,0 @@
---
title: "For New Coders"
description: "Welcome to Cline, your AI-powered coding companion! This guide will help you quickly set up your development environment and begin your coding journey with ease."
---
> 💡 **Tip:** If you're completely new to coding, take your time with each step. There's no rush — Cline is here to guide you!
### 🚀 Getting Started
Before you jump into coding, make sure you have these essentials ready:
#### 1. **VS Code**
A popular, free, and powerful code editor.
- [Download VS Code](https://code.visualstudio.com/)
📺 **Recommended YouTube Tutorial:** [How to Install VS Code](https://www.youtube.com/watch?v=MlIzFUI1QGA)
> ✅ **Pro Tip:** Install VS Code in your Applications folder (macOS) or Program Files (Windows) for easy access from your dock or start menu.
#### 2. **Essential Development Tools**
Basic software required for coding efficiently:
- Homebrew (macOS)
- Node.js
- Git
👉 Follow our detailed guide on Installing Essential Development Tools with step-by-step help from Cline.
📺 **Recommended YouTube Tutorials:**
- **For macOS:**
- [Install Homebrew on Mac](https://www.youtube.com/watch?v=hwGNgVbqasc)
- [Install Git on MacOS 2024](https://www.youtube.com/watch?v=B4qsvQ5IqWk)
- [Install Node.js on Mac (M1 | M2 | M3)](https://www.youtube.com/watch?v=I8H4wolRFBk)
- **For Windows:**
- [Install Git on Windows 10/11 (2024)](https://www.youtube.com/watch?v=yjxv1HuRQy0)
- [Install Node.js in Windows 10/11](https://www.youtube.com/watch?v=uCgAuOYpJd0)
> ⚠️ **Note:** If you run into permission issues during installation, try running your terminal or command prompt as an administrator.
#### 3. **Organize Your Projects**
Create a dedicated folder named `Cline` in your Documents folder for all your coding projects:
- **macOS:** `/Users/[your-username]/Documents/Cline`
- **Windows:** `C:\Users\[your-username]\Documents\Cline`
Inside your `Cline` folder, structure projects clearly:
- `Documents/Cline/workout-app` _(e.g., for a fitness tracking app)_
- `Documents/Cline/portfolio-website` _(e.g., to showcase your work)_
> 💡 **Tip:** Keeping your projects organized from the start will save you time and confusion later!
#### 4. **Install the Cline VS Code Extension**
Enhance your coding workflow by installing the Cline extension directly within VS Code:
- Get Started with Cline Extension Tutorial
📺 **Recommended YouTube Tutorial:** [How To Install Extensions in VS Code](https://www.youtube.com/watch?v=E7trgwZa-mk)
> ✅ **Pro Tip:** After installing, reload VS Code to ensure the extension is activated properly.
🎉 You're all set! Dive in and start coding smarter and faster with **Cline**.
-81
View File
@@ -1,81 +0,0 @@
---
title: "Installing Cline"
description: "Cline is a VS Code extension that brings AI-powered coding assistance directly
to your editor. Install using one of these methods:"
---
### Installation Options
- **VS Code Marketplace (Recommended):** Fastest method for standard VS Code and Cursor users.
- **Open VSX Registry:** For VS Code-compatible editors like VSCodium.
### 🛠️ VS Code Marketplace: Step-by-Step Setup
Follow these steps to get Cline up and running:
1. **Open VS Code:** Launch the VS Code application.
> ⚠️ **Note:** If VS Code shows "Running extensions might...", click "Allow".
2. **Open Your Cline Folder:** In VS Code, open the Cline folder you created in Documents.
3. **Navigate to Extensions:** Click on the Extensions icon in the Activity Bar on the side of VS Code (`Ctrl + Shift + X` or `Cmd + Shift + X`).
4. **Search for 'Cline':** In the Extensions search bar, type `Cline`.
<Frame caption="VS Code marketplace with Cline extension ready to install">
<img src="/assets/robot_panel_dark.png" alt="VS Code marketplace showing Cline extension" />
</Frame>
1. **Install the Extension:** Click the "Install" button next to the Cline extension.
2. **Open Cline:**
- Click the Cline icon in the Activity Bar.
- Or, use the command palette (`Ctrl/Cmd + Shift + P`) and type "Cline: Open In New Tab" for a better view.
3. **Troubleshooting:** If you don't see the Cline icon, try restarting VS Code.
> ✅ **Pro Tip:** You should see the Cline chat window appear in your VS Code editor!
### 🌐 Open VSX Registry
For VS Code-compatible editors without Marketplace access (like VSCodium and Windsurf):
1. Open your editor.
2. Access the Extensions view.
3. Search for "Cline".
4. Select "Cline" by saoudrizwan and click **Install**.
5. Reload if prompted.
### 👤 Creating Your Cline Account
Now that you have Cline installed, let's get you set up with your account:
1. **Sign In to Cline:**
- Click the **Sign In** button in the Cline extension.
- You'll be taken to [app.cline.bot](https://app.cline.bot) to create your account.
2. **Start with Free Credits:**
- No credit card needed!
3. **Available AI Models:**
- Anthropic Claude 3.5-Sonnet (recommended for coding)
- DeepSeek Chat (cost-effective alternative)
- Google Gemini 2.0 Flash
- And more — all through your Cline account.
4. -
### 💻 Your First Interaction with Cline
You're ready to start building! Copy and paste this prompt into the Cline chat window:
```
Hey Cline! Could you help me create a new project folder called "hello-world" in my Cline directory and make a simple webpage that says "Hello World" in big blue text?
```
> ✅ **Pro Tip:** Cline will help you create the project folder and set up your first webpage!
### 🧩 Tips for Working with Cline
- **Ask Questions:** If you're unsure about something, ask Cline!
- **Use Screenshots:** Cline can understand images — show him what you're working on.
- **Copy and Paste Errors:** Share error messages in the chat for solutions.
- **Speak Plainly:** Use your own words — Cline will translate them into code.
### 🫂 Still Struggling?
Join our Discord community and engage with our team and other Cline users directly.
@@ -1,111 +0,0 @@
---
title: "Installing Dev Essentials"
description: >-
When you start coding, you'll need some essential development tools installed
on your computer. Cline can help you install everything you need in a safe,
guided way.
---
### 🧰 The Essential Tools
Here are the core tools you'll need for development:
- **Node.js & npm:** Required for JavaScript and web development
- **Git:** For tracking changes in your code and collaborating with others
- **Package Managers:** Tools that make it easy to install other development tools
- Homebrew for macOS
- Chocolatey for Windows
- apt/yum for Linux
> 💡 **Tip:** These tools are the foundation of your developer toolkit. Installing them properly will set you up for success!
### 🚀 Let Cline Install Everything
Copy one of these prompts based on your operating system and paste it into **Cline**:
#### For macOS
```
Hello Cline! I need help setting up my Mac for software development. Could you please help me install the essential development tools like Homebrew, Node.js, Git, and any other core utilities that are commonly needed for coding? I'd like you to guide me through the process step-by-step.
```
#### For Windows
```
Hello Cline! I need help setting up my Windows PC for software development. Could you please help me install the essential development tools like Node.js, Git, and any other core utilities that are commonly needed for coding? I'd like you to guide me through the process step-by-step.
```
#### For Linux
```
Hello Cline! I need help setting up my Linux system for software development. Could you please help me install the essential development tools like Node.js, Git, and any other core utilities that are commonly needed for coding? I'd like you to guide me through the process step-by-step.
```
> ✅ **Pro Tip:** Cline will show you each command before running it. You stay in control the entire time!
### 🔍 What Will Happen
Cline will guide you through the following steps:
1. Installing the appropriate package manager for your system
2. Using the package manager to install Node.js and Git
3. Showing you the exact command before it runs (you approve each step!)
4. Verifying each installation is successful
> ⚠️ **Note:** You might need to enter your computer's password for some installations. This is normal!
### 💡 Why These Tools Are Important
- **Node.js & npm:**
- Build websites with frameworks like React or Next.js
- Run JavaScript code
- Install JavaScript packages
- **Git:**
- Save different versions of your code
- Collaborate with other developers
- Back up your work
- **Package Managers:**
- Quickly install and update development tools
- Keep your environment organized and up to date
### 🧩 Notes
> 💡 **Tip:** The installation process is interactive — Cline will guide you step by step!
- All commands are shown to you for approval before they run.
- If you run into any issues, Cline will help troubleshoot them.
- You may need to enter your computer's password for certain steps.
### 🧑‍💻 Additional Tips for New Coders
#### Understanding the Terminal
The Terminal is an application where you can type commands to interact with your computer.
- **macOS:** Open it by searching for "Terminal" in Spotlight.
- **Example:**
```
$ open -a Terminal
```
#### Understanding VS Code Features
- **Terminal in VS Code:** Run commands directly from within VS Code!
- Go to **View > Terminal** or press \`Ctrl + \`\`.
- Example:
```
$ node -v
v16.14.0
```
- **Document View:** Where you edit your code files.
- Open files from the Explorer panel on the left.
- **Problems Section:** View errors or warnings in your code.
- Access it by clicking the lightbulb icon or **View > Problems**.
#### Common Features
- **Command Line Interface (CLI):** A powerful tool for running commands.
- **Permissions:** You might need to grant permissions to certain commands — this keeps your system secure.
@@ -1,123 +0,0 @@
---
title: "Model Selection Guide"
description: "Last updated: Feb 5, 2025."
---
## Understanding Context Windows
Think of a context window as your AI assistant's working memory - similar to RAM in a computer. It determines how much information the model can "remember" and process at once during your conversation. This includes:
- Your code files and conversations
- The assistant's responses
- Any documentation or additional context provided
Context windows are measured in tokens (roughly 3/4 of a word in English). Different models have different context window sizes:
- Claude 3.5 Sonnet: 200K tokens
- DeepSeek Models: 128K tokens
- Gemini Flash 2.0: 1M tokens
- Gemini 1.5 Pro: 2M tokens
When you reach the limit of your context window, older information needs to be removed to make room for new information - just like clearing RAM to run new programs. This is why sometimes AI assistants might seem to "forget" earlier parts of your conversation.
Cline helps you manage this limitation with its Context Window Progress Bar, which shows:
- Input tokens (what you've sent to the model)
- Output tokens (what the model has generated)
- A visual representation of how much of your context window you've used
- The total capacity for your chosen model
<Frame caption="Visual representation of the context window usage in Cline">
<img src="/assets/robot_panel_light.png" alt="Context window progress bar example" />
</Frame>
This visibility helps you work more effectively with Cline by letting you know when you might need to start fresh or break tasks into smaller chunks.
### Model Comparison
## LLM Model Comparison for Cline (Feb 2025)
| Model | Input Cost\* | Output Cost\* | Context Window | Best For |
| ----------------- | ------------ | ------------- | -------------- | ----------------------------------- |
| Claude 3.5 Sonnet | $3.00 | $15.00 | 200K | Best code implementation & tool use |
| DeepSeek R1 | $0.55 | $2.19 | 128K | Planning & reasoning champion |
| DeepSeek V3 | $0.14 | $0.28 | 128K | Value code implementation |
| o3-mini | $1.10 | $4.40 | 200K | Flexible use, strong planning |
| Gemini Flash 2.0 | $0.00 | $0.00 | 1M | Strong all-rounder |
| Gemini 1.5 Pro | $0.00 | $0.00 | 2M | Large context processing |
\*Costs per million tokens
### Top Picks for 2025
1. **Claude 3.5 Sonnet**
- Best overall code implementation
- Most reliable tool usage
- Expensive but worth it for critical code
2. **DeepSeek R1**
- Exceptional planning & reasoning
- Great value pricing
3. **o3-mini**
- Strong for planning with adjustable reasoning
- Three reasoning modes for different needs
- Requires OpenAI Tier 3 API access
- 200K context window
4. **DeepSeek V3**
- Reliable code implementation
- Great for daily coding
- Cost-effective for implementation
5. **Gemini Flash 2.0**
- Massive 1M context window
- Improved speed and performance
- Good all-around capabilities
### Best Models by Mode (Plan or Act)
#### Planning
1. **DeepSeek R1**
- Best reasoning capabilities in class
- Excellent at breaking down complex tasks
- Strong math/algorithm planning
- MoE architecture helps with reasoning
2. **o3-mini (high reasoning)**
- Three reasoning levels:
- High: Complex planning
- Medium: Daily tasks
- Low: Quick ideas
- 200K context helps with large projects
3. **Gemini Flash 2.0**
- Massive context window for complex planning
- Strong reasoning capabilities
- Good with multi-step tasks
#### Acting (coding)
1. **Claude 3.5 Sonnet**
- Best code quality
- Most reliable with Cline tools
- Worth the premium for critical code
2. **DeepSeek V3**
- Nearly Sonnet-level code quality
- Better API stability than R1
- Great for daily coding
- Strong tool usage
3. **Gemini 1.5 Pro**
- 2M context window
- Good with complex codebases
- Reliable API
- Strong multi-file understanding
### A Note on Local Models
While running models locally might seem appealing for cost savings, we currently don't recommend any local models for use with Cline. [Local models are significantly less reliable](https://docs.cline.bot/running-models-locally/read-me-first) at using Cline's essential tools and typically retain only 1-26% of the original model's capabilities. The full cloud version of DeepSeek-R1, for example, is 671B parameters - local versions are drastically simplified copies that struggle with complex tasks and tool usage. Even with high-end hardware (RTX 3070+, 32GB+ RAM), you'll experience slower responses, less reliable tool execution, and reduced capabilities. For the best development experience, we recommend sticking with the cloud models listed above.
### Key Takeaways
1. **Plan vs Act Matters**: Choose models based on task type
2. **Real Performance > Benchmarks**: Focus on actual Cline performance
3. **Mix & Match**: Use different models for planning and implementation
4. **Cost vs Quality**: Premium models worth it for critical code
5. **Keep Backups**: Have alternatives ready for API issues
_\*Note: Based on real usage patterns and community feedback rather than just benchmarks. Your experience may vary. This is not an exhaustive list of all the models available for use within Cline._
@@ -1,238 +0,0 @@
---
title: "Our Favorite Tech Stack"
description: "A curated list of our recommended technologies and tools for building modern web applications with Cline."
---
## Recommended Stack for New Cline Users (2025)
### Your Complete Development Environment
#### Development Tools
- **VS Code** - Your code editor, [download here](https://code.visualstudio.com/)
- **GitHub** - Where your code lives, [sign up here](https://github.com)
#### Frontend
- **Next.js 14+** - React framework with App Router
- **Tailwind CSS** - Beautiful styling without writing CSS
- **TypeScript** - JavaScript, but safer and smarter
#### Backend
- **Supabase** - Your complete backend solution, [sign up with GitHub](https://supabase.com)
- PostgreSQL database
- Authentication
- File storage
- Real-time updates
#### Deployment
- **Vercel** - Where your app runs, [sign up with GitHub](https://vercel.com)
- Automatic deployments from GitHub
- Preview deployments for testing
- Production-ready CDN
#### AI Development
Choose your AI assistant based on your needs:
| Model | Input Cost (per 1M tokens) | Output Cost (per 1M tokens) | Best For |
| ----------------- | -------------------------- | --------------------------- | ------------------------------ |
| Claude 3.5 Sonnet | $3.00 | $15.00 | Production apps, complex tasks |
| DeepSeek R1 | $1.00 | $3.00 | Budget-conscious production |
| DeepSeek V3 | $0.14 | $2.20 | Budget-conscious development |
#### Free Tier Benefits
**Vercel (Hobby)**
- 100 GB data transfer/month
- 100k serverless function invocations
- 100 MB deployment size
- Automatic HTTPS & CI/CD
**Supabase (Free)**
- 500 MB database storage
- 1 GB file storage
- 50k monthly active users
- 2M real-time messages/month
**GitHub (Free)**
- Unlimited public repositories
- GitHub Actions CI/CD
- Project management tools
- Collaboration features
### Getting Started
1. Install the development essentials:
- Follow our [Development Essentials Installation Guide](https://docs.cline.bot/getting-started/getting-started-new-coders/installing-dev-essentials)
2. Set up Cline's Memory Bank:
- Follow the [Memory Bank setup instructions](https://docs.cline.bot/improving-your-prompting-skills/custom-instructions-library/cline-memory-bank)
- Create an empty `cline_docs` folder in your project root
- Create `projectBrief.md` in the `cline_docs` folder (see example below)
- Tell Cline to "initialize memory bank"
3. Add our recommended stack configuration:
- Create `.clinerules` file (see template below)
- Let Cline handle the rest!
#### Example Project Brief
```markdown
# Project Brief
## Overview
Building a [type of application] that will [main purpose].
## Core Features
- Feature 1
- Feature 2
- Feature 3
## Target Users
[Describe who will use your application]
## Technical Preferences (optional)
- Any specific technologies you want to use
- Any specific requirements or constraints
```
### .clinerules Template
```markdown
# Project Configuration
## Tech Stack
- Next.js 14+ with App Router
- Tailwind CSS for styling
- Supabase for backend
- Vercel for deployment
- GitHub for version control
## Project Structure
/src
/app # Next.js App Router pages
/components # React components
/lib # Utility functions
/types # TypeScript types
/supabase
/migrations # SQL migration files
/seed # Seed data files
/public # Static assets
## Database Migrations
SQL files in /supabase/migrations should:
- Use sequential numbering: 001, 002, etc.
- Include descriptive names
- Be reviewed by Cline before execution
Example: 001_create_users_table.sql
## Development Workflow
- Cline helps write and review code changes
- Vercel automatically deploys from main branch
- Database migrations reviewed by Cline before execution
## Security
DO NOT read or modify:
- .env files
- \*_/config/secrets._
- Any file containing API keys or credentials
```
### Learning Resources (2025)
Want to learn more about the technologies we're using? Here are some great resources:
#### Next.js and React
- [Official Learn Next.js Course](https://nextjs.org/learn) - Interactive tutorial
- [NextJS App Router: Modern Web Dev in 1 Hour](https://www.youtube.com/nextjs-modern) - Quick overview
- [Building Real-World Apps with Next.js](https://www.youtube.com/nextjs-real-world) - Practical examples
#### Supabase
- [Supabase From Scratch](https://www.udemy.com/supabase-scratch) - Comprehensive course
- [Official Quickstart Guides](https://supabase.com/docs/guides/getting-started)
- [Real-Time Apps with Next.js and Supabase](https://www.newline.co/courses/supabase-nextjs)
#### Tailwind CSS
- [Tailwind CSS Tutorial for Beginners](https://www.youtube.com/tailwind-2025)
- [Official Tailwind Documentation](https://tailwindcss.com/docs)
- Interactive course at [Scrimba Tailwind CSS Course](https://scrimba.com/learn/tailwind)
### Other Things to Know
#### Working with Git & GitHub
Git helps you track changes in your code and collaborate with others. Here are the essential commands you'll use:
**Daily Development**
```bash
# Save your changes (do this often!)
git add . # Stage all changed files
git commit -m "Add login page" # Save changes with a clear message
# Share your changes
git push origin main # Upload to GitHub
```
**Common Workflow**
1. **Start of day**: Get latest changes
```bash
bashCopygit pull origin main # Download latest code
```
2. **During development**: Save work regularly
```bash
bashCopygit add .
git commit -m "Clear message about changes"
```
3. **End of day**: Share your progress
```bash
bashCopygit push origin main # Upload to GitHub
```
**Best Practices**
- Commit often with clear messages
- Pull before starting new work
- Push completed work to share with others
- Use `.gitignore` to avoid committing sensitive files
> **Tip**: Vercel automatically deploys when you push to main!
#### Environment Variables
- Store secrets in `.env.local` for development
- Add them to Vercel project settings for production
- Never commit `.env` files to Git
#### Getting Help
1. Use `/help` in Cline chat for immediate assistance
2. Check [Cline Documentation](https://docs.cline.bot)
3. Join our [Discord Community](https://discord.gg/cline)
4. Search GitHub issues for common problems
Remember: Cline is here to help at every step. Just ask for guidance or clarification when needed!
@@ -1,156 +0,0 @@
---
title: "Context Management"
description: "Context is key to getting the most out of Cline"
---
> 💡 **Quick Reference**
>
> - Context = The information Cline knows about your project
> - Context Window = How much information Cline can hold at once
> - Use context files to maintain project knowledge
> - Reset when the context window gets full
## Understanding Context & Context Windows
<Frame caption="In a world of infinite context, the context window is what Cline currently has available">
<img
src="/assets/robot_panel_dark.png"
alt="In a world of infinite context, the context window is what Cline currently has available"
/>
</Frame>
Think of working with Cline like collaborating with a thorough, proactive teammate:
### How Context is Built
Cline actively builds context in two ways:
1. **Automatic Context Gathering (i.e. Cline-driven)**
- Proactively reads related files
- Explores project structure
- Analyzes patterns and relationships
- Maps dependencies and imports
- Asks clarifying questions
2. **User-Guided Context**
- Share specific files
- Provide documentation
- Answer Cline's questions
- Guide focus areas
- Share design thoughts and requirements
💡 **Key Point**: Cline isn't passive - it actively seeks to understand your project. You can either let it explore or guide its focus, especially in [Plan](https://docs.cline.bot/exploring-clines-tools/plan-and-act-modes-a-guide-to-effective-ai-development) mode.
### Context & Context Windows
Think of context like a whiteboard you and Cline share:
- **Context** is all the information available:
- What Cline has discovered
- What you've shared
- Your conversation history
- Project requirements
- Previous decisions
- **Context Window** is the size of the whiteboard itself:
- Measured in tokens (1 token ≈ 3/4 of an English word)
- Each model has a fixed size:
- Claude 3.5 Sonnet: 200,000 tokens
- DeepSeek: 64,000 tokens
- When the whiteboard is full, you need to erase (clear context) to write more
- [How Cline manages context under the hood](https://cline.bot/blog/understanding-the-new-context-window-progress-bar-in-cline)
⚠️ **Important**: Having a large context window (like Claude's 200k tokens) doesn't mean you should fill it completely. Just like a cluttered whiteboard, too much information can make it harder to focus on what's important.
## Understanding the Context Window Progress Bar
Cline provides a visual way to monitor your context window usage through a progress bar:
<Frame caption="Visual representation of the context window usage">
<img src="/assets/robot_panel_light.png" alt="Context window progress bar" />
</Frame>
### Reading the Bar
- ↑ shows input tokens (what you've sent to the LLM)
- ↓ shows output tokens (what the LLM has generated)
- The progress bar visualizes how much of your context window you've used
- The total shows your model's maximum capacity (e.g., 200k for Claude 3.5-Sonnet)
### When to Watch the Bar
- During long coding sessions
- When working with multiple files
- Before starting complex tasks
- When Cline seems to lose context
💡 **Tip**: Consider starting a fresh session when usage reaches 70-80% to maintain optimal performance.
## Working with Context Files
Context files help maintain understanding across sessions. They serve as documentation specifically designed to help AI assistants understand your project.
#### Approaches to Context Files
1. **Evergreen Project Context (i.e.** [**Memory Bank**](https://docs.cline.bot/improving-your-prompting-skills/custom-instructions-library/cline-memory-bank)**)**
- Living documentation that evolves with your project
- Updated as architecture and patterns emerge
- Example: The Memory Bank pattern maintains files like `techContext.md` and `systemPatterns.md`
- Useful for long-running projects and teams
2. **Task-Specific Context (i.e.** [**Structured Approach**](https://cline.bot/blog/building-advanced-software-with-cline-a-structured-approach)**)**
- Created for specific implementation tasks
- Document requirements, constraints, and decisions
- Example:
```markdown
# auth-system-implementation.md
## Requirements
- OAuth2 implementation
- Support for Google and GitHub
- Rate limiting on auth endpoints
## Technical Decisions
- Using Passport.js for provider integration
- JWT for session management
- Redis for rate limiting
```
3. **Knowledge Transfer Docs**
- Switch to plan mode and ask Cline to document everything you've accomplished so far, along with the remaining steps, in a markdown file.
- Copy the contents of the markdown file.
- Start a new task using that content as context.
#### Using Context Files Effectively
1. **Structure and Format**
- Use clear, consistent organization
- Include relevant examples
- Link related concepts
- Keep information focused
2. **Maintenance**
- Update after significant changes
- Version control your context files
- Remove outdated information
- Document key decisions
## Practical Tips
1. **Starting New Projects**
- Let Cline explore the codebase
- Answer its questions about structure and patterns
- Consider setting up basic context files
- Document key design decisions
2. **Ongoing Development**
- Update context files with significant changes
- Share relevant documentation
- Use Plan mode for complex discussions
- Start fresh sessions when needed
3. **Team Projects**
- Share common context files (consider using [.clinerules](https://docs.cline.bot/improving-your-prompting-skills/prompting) files in project roots)
- Document architectural decisions
- Maintain consistent patterns
- Keep documentation current
Remember: The goal is to help Cline maintain consistent understanding of your project across sessions.
-8
View File
@@ -1,8 +0,0 @@
---
title: "What is Cline?"
description: "An introduction to Cline, your AI-powered development assistant in VS Code."
---
Cline is an AI development assistant which integrates with Microsoft Visual Studio Code. It provides an interface between your IDE and LLMs facilitating code development, increasing productivity and lowering the barrier to entry for new coders. Depending on permissions, Cline can read/write files, execute commands, use your web browser, and expand its capabilities with Model Context Protocol servers.
What makes Cline distinctive is its thoughtful approach to code generation and its extensive integration capabilities. Rather than simply generating code snippets, Cline collaborates with developers by planning solutions step-by-step, maintaining awareness of the entire development environment, and requiring explicit approval for all changes. It can understand large codebases, accelerate onboarding for new engineers, and connect with hundreds of tools through its Model Context Protocol Marketplace, enabling everything from streamlined project deployments to automated incident response—all through natural language commands.
@@ -20,9 +20,8 @@ Cline is your AI assistant that can:
2. **Provide Context**
- Use @ mentions to add files, folders, URLs, diagnostics, terminal output, and more
- Example: "@/src/components/App.tsx"
- See the [Mentions Feature Guide](./mentions-guide.md) for details
- Use @ mentions to add files, folders, or URLs
- Example: "@file:src/components/App.tsx"
3. **Review Changes**
- Cline will show diffs before making changes
@@ -55,7 +54,7 @@ Cline is your AI assistant that can:
## Available Tools
For the most up-to-date implementation details, you can view the full source code in the [Cline repository](https://github.com/cline/cline/blob/main/src/core/task/index.ts).
For the most up-to-date implementation details, you can view the full source code in the [Cline repository](https://github.com/cline/cline/blob/main/src/core/Cline.ts).
Cline has access to the following tools for various tasks:
@@ -82,7 +81,6 @@ Cline has access to the following tools for various tasks:
4. **Interaction Tools**
- `ask_followup_question`: Ask user for clarification
- `attempt_completion`: Present final results
- `new_task`: Start a new task with preloaded context
Each tool has specific parameters and usage patterns. Here are some examples:
@@ -115,21 +113,6 @@ Each tool has specific parameters and usage patterns. Here are some examples:
</execute_command>
```
- Start a new task with context (new_task):
```xml
<new_task>
<context>
We've completed the backend API with these endpoints:
- GET /api/tasks
- POST /api/tasks
- PUT /api/tasks/:id
- DELETE /api/tasks/:id
Now we need to implement the React frontend.
</context>
</new_task>
```
## Common Tasks
1. **Create a New Component**
-59
View File
@@ -8,60 +8,6 @@ const watch = process.argv.includes("--watch")
/**
* @type {import('esbuild').Plugin}
*/
const aliasResolverPlugin = {
name: "alias-resolver",
setup(build) {
const aliases = {
"@": path.resolve(__dirname, "src"),
"@api": path.resolve(__dirname, "src/api"),
"@core": path.resolve(__dirname, "src/core"),
"@integrations": path.resolve(__dirname, "src/integrations"),
"@services": path.resolve(__dirname, "src/services"),
"@shared": path.resolve(__dirname, "src/shared"),
"@utils": path.resolve(__dirname, "src/utils"),
"@packages": path.resolve(__dirname, "src/packages"),
}
// For each alias entry, create a resolver
Object.entries(aliases).forEach(([alias, aliasPath]) => {
const aliasRegex = new RegExp(`^${alias}($|/.*)`)
build.onResolve({ filter: aliasRegex }, (args) => {
const importPath = args.path.replace(alias, aliasPath)
// First, check if the path exists as is
if (fs.existsSync(importPath)) {
const stats = fs.statSync(importPath)
if (stats.isDirectory()) {
// If it's a directory, try to find index files
const extensions = [".ts", ".tsx", ".js", ".jsx"]
for (const ext of extensions) {
const indexFile = path.join(importPath, `index${ext}`)
if (fs.existsSync(indexFile)) {
return { path: indexFile }
}
}
} else {
// It's a file that exists, so return it
return { path: importPath }
}
}
// If the path doesn't exist, try appending extensions
const extensions = [".ts", ".tsx", ".js", ".jsx"]
for (const ext of extensions) {
const pathWithExtension = `${importPath}${ext}`
if (fs.existsSync(pathWithExtension)) {
return { path: pathWithExtension }
}
}
// If nothing worked, return the original path and let esbuild handle the error
return { path: importPath }
})
})
},
}
const esbuildProblemMatcherPlugin = {
name: "esbuild-problem-matcher",
@@ -122,13 +68,8 @@ const extensionConfig = {
minify: production,
sourcemap: !production,
logLevel: "silent",
define: {
"process.env.IS_DEV": JSON.stringify(!production),
},
tsconfig: path.resolve(__dirname, "tsconfig.json"),
plugins: [
copyWasmFiles,
aliasResolverPlugin,
/* add to the end of plugins array */
esbuildProblemMatcherPlugin,
{
-3
View File
@@ -1,3 +0,0 @@
repositories
results/evals.db
-186
View File
@@ -1,186 +0,0 @@
# Cline Evaluation System
This directory contains the evaluation system for benchmarking Cline against various coding evaluation frameworks.
## Overview
The Cline Evaluation System allows you to:
1. Run Cline against standardized coding benchmarks
2. Collect comprehensive metrics on performance
3. Generate detailed reports on evaluation results
4. Compare performance across different models and benchmarks
## Architecture
The evaluation system consists of two main components:
1. **Test Server**: Enhanced HTTP server in `src/services/test/TestServer.ts` that provides detailed task results
2. **CLI Tool**: Command-line interface in `evals/cli/` for orchestrating evaluations
## Directory Structure
```
cline-repo/
├── src/
│ ├── services/
│ │ ├── test/
│ │ │ ├── TestServer.ts # Enhanced HTTP server for task execution
│ │ │ ├── GitHelper.ts # Git utilities for file tracking
│ │ │ └── ...
│ │ └── ...
│ └── ...
├── evals/ # Main directory for evaluation system
│ ├── cli/ # CLI tool for orchestrating evaluations
│ │ ├── src/
│ │ │ ├── index.ts # CLI entry point
│ │ │ ├── commands/ # CLI commands (setup, run, report)
│ │ │ ├── adapters/ # Benchmark adapters
│ │ │ ├── db/ # Database management
│ │ │ └── utils/ # Utility functions
│ │ ├── package.json
│ │ └── tsconfig.json
│ ├── repositories/ # Cloned benchmark repositories
│ │ ├── exercism/ # Modified Exercism (from pashpashpash/evals)
│ │ ├── swe-bench/ # SWE-Bench repository
│ │ ├── swelancer/ # SWELancer repository
│ │ └── multi-swe/ # Multi-SWE-Bench repository
│ ├── results/ # Evaluation results storage
│ │ ├── runs/ # Individual run results
│ │ └── reports/ # Generated reports
│ └── README.md # This file
└── ...
```
## Getting Started
### Prerequisites
- Node.js 16+
- VSCode with Cline extension installed
- Git
### Activation Mechanism
The evaluation system uses an `evals.env` file approach to activate test mode in the Cline extension. When an evaluation is run:
1. The CLI creates an `evals.env` file in the workspace directory
2. The Cline extension activates due to the `workspaceContains:evals.env` activation event
3. The extension detects this file and automatically enters test mode
4. After evaluation completes, the file is automatically removed
This approach eliminates the need for environment variables during the build process and allows for targeted activation only when needed for evaluations. The extension remains dormant during normal use, only activating when an evals.env file is present. For more details, see [Evals Env Activation](./docs/evals-env-activation.md).
### Installation
1. Build the CLI tool:
```bash
cd evals/cli
npm install
npm run build
```
### Usage
#### Setting Up Benchmarks
```bash
cd evals/cli
node dist/index.js setup
```
This will clone and set up all benchmark repositories. You can specify specific benchmarks:
```bash
node dist/index.js setup --benchmarks exercism
```
#### Running Evaluations
```bash
node dist/index.js run --model claude-3-opus-20240229 --benchmark exercism
```
Options:
- `--model`: The model to evaluate (default: claude-3-opus-20240229)
- `--benchmark`: Specific benchmark to run (default: all)
- `--count`: Number of tasks to run (default: all)
#### Generating Reports
```bash
node dist/index.js report
```
Options:
- `--format`: Report format (json, markdown) (default: markdown)
- `--output`: Output path for the report
#### Managing Test Mode Activation
The CLI provides a command to manually manage the evals.env file for test mode activation:
```bash
node dist/index.js evals-env create # Create evals.env file in current directory
node dist/index.js evals-env remove # Remove evals.env file from current directory
node dist/index.js evals-env check # Check if evals.env file exists in current directory
```
Options:
- `--directory`: Specify a directory other than the current one
## Benchmarks
### Exercism
Modified Exercism exercises from the [pashpashpash/evals](https://github.com/pashpashpash/evals) repository. These are small, focused programming exercises in various languages.
### SWE-Bench (Coming Soon)
Real-world software engineering tasks from the [SWE-bench](https://github.com/SWE-bench/SWE-bench) repository.
### SWELancer (Coming Soon)
Freelance-style programming tasks from the SWELancer benchmark.
### Multi-SWE-Bench (Coming Soon)
Multi-file software engineering tasks from the Multi-SWE-Bench repository.
## Metrics
The evaluation system collects the following metrics:
- **Token Usage**: Input and output tokens
- **Cost**: Estimated cost of API calls
- **Duration**: Time taken to complete tasks
- **Tool Usage**: Number of tool calls and failures
- **Success Rate**: Percentage of tasks completed successfully
- **Functional Correctness**: Percentage of tests passed
## Reports
Reports are generated in Markdown or JSON format and include:
- Overall summary
- Benchmark-specific results
- Model-specific results
- Tool usage statistics
- Charts and visualizations
## Development
### Adding a New Benchmark
1. Create a new adapter in `evals/cli/src/adapters/`
2. Implement the `BenchmarkAdapter` interface
3. Register the adapter in `evals/cli/src/adapters/index.ts`
### Extending Metrics
To add new metrics:
1. Update the database schema in `evals/cli/src/db/schema.ts`
2. Add collection logic in `evals/cli/src/utils/results.ts`
3. Update report generation in `evals/cli/src/commands/report.ts`
-2455
View File
File diff suppressed because it is too large Load Diff
-39
View File
@@ -1,39 +0,0 @@
{
"name": "cline-evaluation-cli",
"version": "0.1.0",
"description": "CLI tool for orchestrating Cline evaluations across multiple benchmarks",
"main": "dist/index.js",
"scripts": {
"build": "tsc",
"start": "node dist/index.js",
"dev": "ts-node src/index.ts",
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [
"cline",
"evaluation",
"benchmark"
],
"author": "",
"license": "MIT",
"dependencies": {
"better-sqlite3": "^8.0.0",
"chalk": "^4.1.2",
"commander": "^9.4.1",
"execa": "^5.1.1",
"node-fetch": "^2.7.0",
"ora": "^5.4.1",
"sqlite": "^4.1.2",
"uuid": "^9.0.0",
"yargs": "^17.6.2"
},
"devDependencies": {
"@types/better-sqlite3": "^7.6.3",
"@types/node": "^18.11.18",
"@types/node-fetch": "^2.6.12",
"@types/uuid": "^9.0.0",
"@types/yargs": "^17.0.19",
"ts-node": "^10.9.1",
"typescript": "^4.9.4"
}
}
-194
View File
@@ -1,194 +0,0 @@
import * as path from "path"
import * as fs from "fs"
import execa from "execa"
import { BenchmarkAdapter, Task, VerificationResult } from "./types"
const EVALS_DIR = path.resolve(__dirname, "../../../")
/**
* Adapter for the modified Exercism benchmark
*/
export class ExercismAdapter implements BenchmarkAdapter {
name = "exercism"
/**
* Set up the Exercism benchmark repository
*/
async setup(): Promise<void> {
// Clone repository if needed
const exercismDir = path.join(EVALS_DIR, "repositories", "exercism")
if (!fs.existsSync(exercismDir)) {
console.log(`Cloning Exercism repository to ${exercismDir}...`)
await execa("git", ["clone", "https://github.com/pashpashpash/evals.git", exercismDir])
console.log("Exercism repository cloned successfully")
} else {
console.log(`Exercism repository already exists at ${exercismDir}`)
// Pull latest changes
console.log("Pulling latest changes...")
await execa("git", ["pull"], { cwd: exercismDir })
console.log("Repository updated successfully")
}
}
/**
* List all available tasks in the Exercism benchmark
*/
async listTasks(): Promise<Task[]> {
const tasks: Task[] = []
const exercisesDir = path.join(EVALS_DIR, "repositories", "exercism")
// Ensure the repository exists
if (!fs.existsSync(exercisesDir)) {
throw new Error(`Exercism repository not found at ${exercisesDir}. Run setup first.`)
}
// Read language directories
const languages = fs
.readdirSync(exercisesDir)
.filter((dir) => fs.statSync(path.join(exercisesDir, dir)).isDirectory())
.filter((dir) => !dir.startsWith(".") && !["node_modules", ".git"].includes(dir))
for (const language of languages) {
const languageDir = path.join(exercisesDir, language)
// Read exercise directories
const exercises = fs.readdirSync(languageDir).filter((dir) => fs.statSync(path.join(languageDir, dir)).isDirectory())
for (const exercise of exercises) {
const exerciseDir = path.join(languageDir, exercise)
// Read instructions
let description = ""
const instructionsPath = path.join(exerciseDir, "docs", "instructions.md")
if (fs.existsSync(instructionsPath)) {
description = fs.readFileSync(instructionsPath, "utf-8")
}
// Determine test commands based on language
let testCommands: string[] = []
switch (language) {
case "javascript":
testCommands = ["npm install", "npm test"]
break
case "python":
testCommands = ["python -m pytest -o markers=task *_test.py"]
break
case "go":
testCommands = ["go test"]
break
case "java":
testCommands = ["./gradlew test"]
break
case "rust":
testCommands = ["cargo test"]
break
default:
testCommands = []
}
tasks.push({
id: `exercism-${language}-${exercise}`,
name: exercise,
description,
workspacePath: exerciseDir,
setupCommands: [],
verificationCommands: testCommands,
metadata: {
language,
type: "exercism",
},
})
}
}
return tasks
}
/**
* Prepare a specific task for execution
* @param taskId The ID of the task to prepare
*/
async prepareTask(taskId: string): Promise<Task> {
const tasks = await this.listTasks()
const task = tasks.find((t) => t.id === taskId)
if (!task) {
throw new Error(`Task ${taskId} not found`)
}
// Check if Git repository is already initialized
const gitDirExists = fs.existsSync(path.join(task.workspacePath, ".git"))
try {
// Initialize Git repository if needed
if (!gitDirExists) {
await execa("git", ["init"], { cwd: task.workspacePath })
}
// Create a dummy file to ensure there's something to commit
const dummyFilePath = path.join(task.workspacePath, ".eval-timestamp")
fs.writeFileSync(dummyFilePath, new Date().toISOString())
// Add all files and commit
await execa("git", ["add", "."], { cwd: task.workspacePath })
try {
await execa("git", ["commit", "-m", "Initial commit"], { cwd: task.workspacePath })
} catch (error: any) {
// If commit fails because there are no changes, that's okay
if (!error.stderr?.includes("nothing to commit")) {
throw error
}
}
} catch (error: any) {
console.warn(`Warning: Git operations failed: ${error.message}`)
console.warn("Continuing without Git initialization")
}
return task
}
/**
* Verify the result of a task execution
* @param task The task that was executed
* @param result The result of the task execution
*/
async verifyResult(task: Task, result: any): Promise<VerificationResult> {
// Run verification commands
let success = true
let output = ""
for (const command of task.verificationCommands) {
try {
const [cmd, ...args] = command.split(" ")
const { stdout } = await execa(cmd, args, { cwd: task.workspacePath })
output += stdout + "\n"
} catch (error: any) {
success = false
if (error.stdout) {
output += error.stdout + "\n"
}
if (error.stderr) {
output += error.stderr + "\n"
}
}
}
// Parse test results
const testsPassed = (output.match(/PASS/g) || []).length
const testsFailed = (output.match(/FAIL/g) || []).length
const testsTotal = testsPassed + testsFailed
return {
success,
metrics: {
testsPassed,
testsFailed,
testsTotal,
functionalCorrectness: testsTotal > 0 ? testsPassed / testsTotal : 0,
},
}
}
}
-47
View File
@@ -1,47 +0,0 @@
import { BenchmarkAdapter } from "./types"
import { ExercismAdapter } from "./exercism"
import { SWEBenchAdapter } from "./swe-bench"
import { SWELancerAdapter } from "./swelancer"
import { MultiSWEAdapter } from "./multi-swe"
// Registry of all available adapters
const adapters: Record<string, BenchmarkAdapter> = {
// Exercism is the primary adapter with real implementation
exercism: new ExercismAdapter(),
// Dummy adapters for testing
"swe-bench": new SWEBenchAdapter(),
swelancer: new SWELancerAdapter(),
"multi-swe": new MultiSWEAdapter(),
}
/**
* Get a specific adapter by name
* @param name The name of the adapter to get
* @returns The requested adapter
* @throws Error if the adapter is not found
*/
export function getAdapter(name: string): BenchmarkAdapter {
const adapter = adapters[name]
if (!adapter) {
throw new Error(`Adapter for benchmark '${name}' not found`)
}
return adapter
}
/**
* Get all available adapters
* @returns Array of all registered adapters
*/
export function getAllAdapters(): BenchmarkAdapter[] {
return Object.values(adapters)
}
/**
* Register a new adapter
* @param name The name to register the adapter under
* @param adapter The adapter to register
*/
export function registerAdapter(name: string, adapter: BenchmarkAdapter): void {
adapters[name] = adapter
}
-192
View File
@@ -1,192 +0,0 @@
import * as path from "path"
import * as fs from "fs"
import execa from "execa"
import { BenchmarkAdapter, Task, VerificationResult } from "./types"
const EVALS_DIR = path.resolve(__dirname, "../../../")
/**
* Dummy adapter for the Multi-SWE-Bench benchmark
*/
export class MultiSWEAdapter implements BenchmarkAdapter {
name = "multi-swe"
/**
* Set up the Multi-SWE-Bench benchmark repository (dummy implementation)
*/
async setup(): Promise<void> {
console.log("Multi-SWE-Bench dummy setup completed")
// Create repositories directory if it doesn't exist
const repoDir = path.join(EVALS_DIR, "repositories", "multi-swe")
if (!fs.existsSync(repoDir)) {
fs.mkdirSync(repoDir, { recursive: true })
console.log(`Created dummy Multi-SWE-Bench directory at ${repoDir}`)
}
}
/**
* List all available tasks in the Multi-SWE-Bench benchmark (dummy implementation)
*/
async listTasks(): Promise<Task[]> {
return [
{
id: "multi-swe-task-1",
name: "Multi-Language API Integration",
description:
"Implement a system that integrates a Python backend with a TypeScript frontend and a Rust processing service.",
workspacePath: path.join(EVALS_DIR, "repositories", "multi-swe"),
setupCommands: [],
verificationCommands: [],
metadata: {
languages: ["python", "typescript", "rust"],
complexity: "high",
type: "multi-swe",
},
},
{
id: "multi-swe-task-2",
name: "Cross-Platform Mobile App",
description: "Create a cross-platform mobile app using React Native with native modules in Swift and Kotlin.",
workspacePath: path.join(EVALS_DIR, "repositories", "multi-swe"),
setupCommands: [],
verificationCommands: [],
metadata: {
languages: ["javascript", "swift", "kotlin"],
complexity: "medium",
type: "multi-swe",
},
},
{
id: "multi-swe-task-3",
name: "Microservice Architecture",
description: "Design and implement a microservice architecture with services written in Go, Node.js, and Java.",
workspacePath: path.join(EVALS_DIR, "repositories", "multi-swe"),
setupCommands: [],
verificationCommands: [],
metadata: {
languages: ["go", "javascript", "java"],
complexity: "high",
type: "multi-swe",
},
},
]
}
/**
* Prepare a specific task for execution (dummy implementation)
* @param taskId The ID of the task to prepare
*/
async prepareTask(taskId: string): Promise<Task> {
const tasks = await this.listTasks()
const task = tasks.find((t) => t.id === taskId)
if (!task) {
throw new Error(`Task ${taskId} not found`)
}
// Create a dummy workspace for the task
const taskDir = path.join(task.workspacePath, taskId)
if (!fs.existsSync(taskDir)) {
fs.mkdirSync(taskDir, { recursive: true })
// Create a dummy file for the task
fs.writeFileSync(
path.join(taskDir, "README.md"),
`# ${task.name}\n\n${task.description}\n\nThis is a dummy task for testing purposes.`,
)
// Create additional dummy files based on task type
if (task.id === "multi-swe-task-1") {
// Python backend
fs.mkdirSync(path.join(taskDir, "backend"), { recursive: true })
fs.writeFileSync(
path.join(taskDir, "backend", "app.py"),
`# TODO: Implement Python backend\nfrom flask import Flask\n\napp = Flask(__name__)\n\n@app.route('/')\ndef hello():\n return "Hello, World!"\n`,
)
// TypeScript frontend
fs.mkdirSync(path.join(taskDir, "frontend"), { recursive: true })
fs.writeFileSync(
path.join(taskDir, "frontend", "app.ts"),
`// TODO: Implement TypeScript frontend\nconsole.log('Frontend starting...');\n`,
)
// Rust processing service
fs.mkdirSync(path.join(taskDir, "processor"), { recursive: true })
fs.writeFileSync(
path.join(taskDir, "processor", "main.rs"),
`// TODO: Implement Rust processing service\nfn main() {\n println!("Processor starting...");\n}\n`,
)
} else if (task.id === "multi-swe-task-2") {
// React Native app
fs.mkdirSync(path.join(taskDir, "app"), { recursive: true })
fs.writeFileSync(
path.join(taskDir, "app", "App.js"),
`// TODO: Implement React Native app\nimport React from 'react';\nimport { View, Text } from 'react-native';\n\nexport default function App() {\n return (\n <View>\n <Text>Hello, World!</Text>\n </View>\n );\n}\n`,
)
// Swift native module
fs.mkdirSync(path.join(taskDir, "ios"), { recursive: true })
fs.writeFileSync(
path.join(taskDir, "ios", "NativeModule.swift"),
`// TODO: Implement Swift native module\nimport Foundation\n\n@objc(NativeModule)\nclass NativeModule: NSObject {\n @objc\n func hello() -> String {\n return "Hello from Swift"\n }\n}\n`,
)
// Kotlin native module
fs.mkdirSync(path.join(taskDir, "android"), { recursive: true })
fs.writeFileSync(
path.join(taskDir, "android", "NativeModule.kt"),
`// TODO: Implement Kotlin native module\npackage com.example.app\n\nclass NativeModule {\n fun hello(): String {\n return "Hello from Kotlin"\n }\n}\n`,
)
} else if (task.id === "multi-swe-task-3") {
// Go service
fs.mkdirSync(path.join(taskDir, "service-go"), { recursive: true })
fs.writeFileSync(
path.join(taskDir, "service-go", "main.go"),
`// TODO: Implement Go service\npackage main\n\nimport "fmt"\n\nfunc main() {\n\tfmt.Println("Go service starting...")\n}\n`,
)
// Node.js service
fs.mkdirSync(path.join(taskDir, "service-node"), { recursive: true })
fs.writeFileSync(
path.join(taskDir, "service-node", "server.js"),
`// TODO: Implement Node.js service\nconsole.log('Node.js service starting...');\n`,
)
// Java service
fs.mkdirSync(path.join(taskDir, "service-java"), { recursive: true })
fs.writeFileSync(
path.join(taskDir, "service-java", "Main.java"),
`// TODO: Implement Java service\npublic class Main {\n public static void main(String[] args) {\n System.out.println("Java service starting...");\n }\n}\n`,
)
}
}
// Update the task's workspace path to the task-specific directory
return {
...task,
workspacePath: taskDir,
}
}
/**
* Verify the result of a task execution (dummy implementation)
* @param task The task that was executed
* @param result The result of the task execution
*/
async verifyResult(task: Task, result: any): Promise<VerificationResult> {
// Always return success for dummy implementation
return {
success: true,
metrics: {
testsPassed: 1,
testsFailed: 0,
testsTotal: 1,
functionalCorrectness: 1.0,
crossLanguageIntegration: 0.9, // Dummy metric specific to Multi-SWE
architectureQuality: 0.85, // Dummy metric specific to Multi-SWE
},
}
}
}
-125
View File
@@ -1,125 +0,0 @@
import * as path from "path"
import * as fs from "fs"
import execa from "execa"
import { BenchmarkAdapter, Task, VerificationResult } from "./types"
const EVALS_DIR = path.resolve(__dirname, "../../../")
/**
* Dummy adapter for the SWE-Bench benchmark
*/
export class SWEBenchAdapter implements BenchmarkAdapter {
name = "swe-bench"
/**
* Set up the SWE-Bench benchmark repository (dummy implementation)
*/
async setup(): Promise<void> {
console.log("SWE-Bench dummy setup completed")
// Create repositories directory if it doesn't exist
const repoDir = path.join(EVALS_DIR, "repositories", "swe-bench")
if (!fs.existsSync(repoDir)) {
fs.mkdirSync(repoDir, { recursive: true })
console.log(`Created dummy SWE-Bench directory at ${repoDir}`)
}
}
/**
* List all available tasks in the SWE-Bench benchmark (dummy implementation)
*/
async listTasks(): Promise<Task[]> {
return [
{
id: "swe-bench-task-1",
name: "Fix React Component Bug",
description: "Fix a bug in a React component where the state is not properly updated.",
workspacePath: path.join(EVALS_DIR, "repositories", "swe-bench"),
setupCommands: [],
verificationCommands: [],
metadata: {
repository: "facebook/react",
issue: "#12345",
type: "swe-bench",
},
},
{
id: "swe-bench-task-2",
name: "Optimize Database Query",
description: "Optimize a slow database query in a Django application.",
workspacePath: path.join(EVALS_DIR, "repositories", "swe-bench"),
setupCommands: [],
verificationCommands: [],
metadata: {
repository: "django/django",
issue: "#6789",
type: "swe-bench",
},
},
{
id: "swe-bench-task-3",
name: "Fix Memory Leak",
description: "Fix a memory leak in a Node.js application.",
workspacePath: path.join(EVALS_DIR, "repositories", "swe-bench"),
setupCommands: [],
verificationCommands: [],
metadata: {
repository: "nodejs/node",
issue: "#9876",
type: "swe-bench",
},
},
]
}
/**
* Prepare a specific task for execution (dummy implementation)
* @param taskId The ID of the task to prepare
*/
async prepareTask(taskId: string): Promise<Task> {
const tasks = await this.listTasks()
const task = tasks.find((t) => t.id === taskId)
if (!task) {
throw new Error(`Task ${taskId} not found`)
}
// Create a dummy workspace for the task
const taskDir = path.join(task.workspacePath, taskId)
if (!fs.existsSync(taskDir)) {
fs.mkdirSync(taskDir, { recursive: true })
// Create a dummy file for the task
fs.writeFileSync(
path.join(taskDir, "README.md"),
`# ${task.name}\n\n${task.description}\n\nThis is a dummy task for testing purposes.`,
)
}
// Update the task's workspace path to the task-specific directory
return {
...task,
workspacePath: taskDir,
}
}
/**
* Verify the result of a task execution (dummy implementation)
* @param task The task that was executed
* @param result The result of the task execution
*/
async verifyResult(task: Task, result: any): Promise<VerificationResult> {
// Always return success for dummy implementation
return {
success: true,
metrics: {
testsPassed: 1,
testsFailed: 0,
testsTotal: 1,
functionalCorrectness: 1.0,
performanceImprovement: 0.25, // Dummy metric specific to SWE-Bench
codeQuality: 0.9, // Dummy metric specific to SWE-Bench
},
}
}
}
-143
View File
@@ -1,143 +0,0 @@
import * as path from "path"
import * as fs from "fs"
import execa from "execa"
import { BenchmarkAdapter, Task, VerificationResult } from "./types"
const EVALS_DIR = path.resolve(__dirname, "../../../")
/**
* Dummy adapter for the SWELancer benchmark
*/
export class SWELancerAdapter implements BenchmarkAdapter {
name = "swelancer"
/**
* Set up the SWELancer benchmark repository (dummy implementation)
*/
async setup(): Promise<void> {
console.log("SWELancer dummy setup completed")
// Create repositories directory if it doesn't exist
const repoDir = path.join(EVALS_DIR, "repositories", "swelancer")
if (!fs.existsSync(repoDir)) {
fs.mkdirSync(repoDir, { recursive: true })
console.log(`Created dummy SWELancer directory at ${repoDir}`)
}
}
/**
* List all available tasks in the SWELancer benchmark (dummy implementation)
*/
async listTasks(): Promise<Task[]> {
return [
{
id: "swelancer-task-1",
name: "Create Landing Page",
description: "Create a responsive landing page for a new product using HTML, CSS, and JavaScript.",
workspacePath: path.join(EVALS_DIR, "repositories", "swelancer"),
setupCommands: [],
verificationCommands: [],
metadata: {
client: "TechStartup Inc.",
difficulty: "medium",
type: "swelancer",
},
},
{
id: "swelancer-task-2",
name: "Build REST API",
description: "Create a RESTful API for a blog application using Node.js and Express.",
workspacePath: path.join(EVALS_DIR, "repositories", "swelancer"),
setupCommands: [],
verificationCommands: [],
metadata: {
client: "BlogCo",
difficulty: "hard",
type: "swelancer",
},
},
{
id: "swelancer-task-3",
name: "Fix CSS Layout Issues",
description: "Fix layout issues in a responsive website across different screen sizes.",
workspacePath: path.join(EVALS_DIR, "repositories", "swelancer"),
setupCommands: [],
verificationCommands: [],
metadata: {
client: "DesignAgency",
difficulty: "easy",
type: "swelancer",
},
},
]
}
/**
* Prepare a specific task for execution (dummy implementation)
* @param taskId The ID of the task to prepare
*/
async prepareTask(taskId: string): Promise<Task> {
const tasks = await this.listTasks()
const task = tasks.find((t) => t.id === taskId)
if (!task) {
throw new Error(`Task ${taskId} not found`)
}
// Create a dummy workspace for the task
const taskDir = path.join(task.workspacePath, taskId)
if (!fs.existsSync(taskDir)) {
fs.mkdirSync(taskDir, { recursive: true })
// Create a dummy file for the task
fs.writeFileSync(
path.join(taskDir, "README.md"),
`# ${task.name}\n\n${task.description}\n\nThis is a dummy task for testing purposes.`,
)
// Create additional dummy files based on task type
if (task.id === "swelancer-task-1") {
fs.writeFileSync(
path.join(taskDir, "index.html"),
`<!DOCTYPE html>\n<html>\n<head>\n <title>Landing Page</title>\n</head>\n<body>\n <!-- TODO: Implement landing page -->\n</body>\n</html>`,
)
} else if (task.id === "swelancer-task-2") {
fs.writeFileSync(
path.join(taskDir, "server.js"),
`// TODO: Implement REST API\nconsole.log('Server starting...');`,
)
} else if (task.id === "swelancer-task-3") {
fs.writeFileSync(
path.join(taskDir, "styles.css"),
`/* TODO: Fix layout issues */\nbody {\n margin: 0;\n padding: 0;\n}`,
)
}
}
// Update the task's workspace path to the task-specific directory
return {
...task,
workspacePath: taskDir,
}
}
/**
* Verify the result of a task execution (dummy implementation)
* @param task The task that was executed
* @param result The result of the task execution
*/
async verifyResult(task: Task, result: any): Promise<VerificationResult> {
// Always return success for dummy implementation
return {
success: true,
metrics: {
testsPassed: 1,
testsFailed: 0,
testsTotal: 1,
functionalCorrectness: 1.0,
clientSatisfaction: 0.95, // Dummy metric specific to SWELancer
timeEfficiency: 0.85, // Dummy metric specific to SWELancer
},
}
}
}
-31
View File
@@ -1,31 +0,0 @@
/**
* Represents a task to be executed
*/
export interface Task {
id: string
name: string
description: string
workspacePath: string
setupCommands: string[]
verificationCommands: string[]
metadata: Record<string, any>
}
/**
* Result of verifying a task execution
*/
export interface VerificationResult {
success: boolean
metrics: Record<string, any>
}
/**
* Interface for benchmark adapters
*/
export interface BenchmarkAdapter {
name: string
setup(): Promise<void>
listTasks(): Promise<Task[]>
prepareTask(taskId: string): Promise<Task>
verifyResult(task: Task, result: any): Promise<VerificationResult>
}
-53
View File
@@ -1,53 +0,0 @@
import * as path from "path"
import chalk from "chalk"
import { createEvalsEnvFile, removeEvalsEnvFile, checkEvalsEnvFile } from "../utils/evals-env"
interface EvalsEnvOptions {
action: "create" | "remove" | "check"
directory?: string
}
/**
* Handler for the evals-env command
* @param options Command options
*/
export async function evalsEnvHandler(options: EvalsEnvOptions): Promise<void> {
// Determine the directory to use - default to repository root instead of current directory
const currentDir = process.cwd()
const repoRoot = path.resolve(currentDir, "..", "..") // Navigate up from evals/cli to root
const directory = options.directory || repoRoot
console.log(chalk.blue(`Working with directory: ${directory}`))
// Perform the requested action
switch (options.action) {
case "create":
console.log(chalk.blue("Creating evals.env file..."))
createEvalsEnvFile(directory)
console.log(chalk.green("The Cline extension should now detect this file and enter test mode."))
console.log(chalk.yellow("Note: You may need to reload VSCode for the changes to take effect."))
break
case "remove":
console.log(chalk.blue("Removing evals.env file..."))
removeEvalsEnvFile(directory)
console.log(chalk.green("The Cline extension should now exit test mode."))
console.log(chalk.yellow("Note: You may need to reload VSCode for the changes to take effect."))
break
case "check":
console.log(chalk.blue("Checking for evals.env file..."))
const exists = checkEvalsEnvFile(directory)
if (exists) {
console.log(chalk.green("The Cline extension should be in test mode."))
} else {
console.log(chalk.yellow("The Cline extension should not be in test mode."))
}
break
default:
console.error(chalk.red(`Unknown action: ${options.action}`))
console.log(chalk.yellow("Valid actions are: create, remove, check"))
break
}
}
-237
View File
@@ -1,237 +0,0 @@
import * as fs from "fs"
import * as path from "path"
import chalk from "chalk"
import ora from "ora"
import { ResultsDatabase } from "../db"
import { generateMarkdownReport } from "../utils/markdown"
interface ReportOptions {
format?: "json" | "markdown"
output?: string
}
/**
* Handler for the report command
* @param options Command options
*/
export async function reportHandler(options: ReportOptions): Promise<void> {
const format = options.format || "markdown"
const db = new ResultsDatabase()
try {
const spinner = ora("Generating report...").start()
// Get all runs
const runs = db.getRuns()
console.log(chalk.blue(`Found ${runs.length} evaluation runs`))
if (runs.length === 0) {
spinner.fail("No evaluation runs found")
return
}
// Generate summary report
const summary = {
runs: runs.length,
models: [...new Set(runs.map((run) => run.model))],
benchmarks: [...new Set(runs.map((run) => run.benchmark))],
tasks: 0,
successRate: 0,
averageTokens: 0,
averageCost: 0,
averageDuration: 0,
totalToolCalls: 0,
totalToolFailures: 0,
toolSuccessRate: 0,
toolUsage: {} as Record<string, { calls: number; failures: number }>,
}
let totalTasks = 0
let successfulTasks = 0
let totalTokens = 0
let totalCost = 0
let totalDuration = 0
let totalToolCalls = 0
let totalToolFailures = 0
for (const run of runs) {
const tasks = db.getRunTasks(run.id)
totalTasks += tasks.length
for (const task of tasks) {
if (task.success) {
successfulTasks++
}
const metrics = db.getTaskMetrics(task.id)
const tokensIn = metrics.find((m) => m.name === "tokensIn")?.value || 0
const tokensOut = metrics.find((m) => m.name === "tokensOut")?.value || 0
totalTokens += tokensIn + tokensOut
totalCost += metrics.find((m) => m.name === "cost")?.value || 0
totalDuration += metrics.find((m) => m.name === "duration")?.value || 0
// Collect tool call metrics
totalToolCalls += task.total_tool_calls || 0
totalToolFailures += task.total_tool_failures || 0
// Get detailed tool usage
const toolCalls = db.getTaskToolCalls(task.id)
for (const toolCall of toolCalls) {
if (!summary.toolUsage[toolCall.tool_name]) {
summary.toolUsage[toolCall.tool_name] = {
calls: 0,
failures: 0,
}
}
summary.toolUsage[toolCall.tool_name].calls += toolCall.call_count
summary.toolUsage[toolCall.tool_name].failures += toolCall.failure_count
}
}
}
// Calculate tool success rate
summary.totalToolCalls = totalToolCalls
summary.totalToolFailures = totalToolFailures
summary.toolSuccessRate = totalToolCalls > 0 ? 1 - totalToolFailures / totalToolCalls : 1.0
summary.tasks = totalTasks
summary.successRate = totalTasks > 0 ? successfulTasks / totalTasks : 0
summary.averageTokens = totalTasks > 0 ? totalTokens / totalTasks : 0
summary.averageCost = totalTasks > 0 ? totalCost / totalTasks : 0
summary.averageDuration = totalTasks > 0 ? totalDuration / totalTasks : 0
// Generate benchmark-specific reports
const benchmarkReports: Record<string, any> = {}
for (const benchmark of summary.benchmarks) {
const benchmarkRuns = runs.filter((run) => run.benchmark === benchmark)
const benchmarkSummary = {
runs: benchmarkRuns.length,
models: [...new Set(benchmarkRuns.map((run) => run.model))],
tasks: 0,
successRate: 0,
averageTokens: 0,
averageCost: 0,
averageDuration: 0,
}
let benchmarkTasks = 0
let benchmarkSuccessfulTasks = 0
let benchmarkTotalTokens = 0
let benchmarkTotalCost = 0
let benchmarkTotalDuration = 0
for (const run of benchmarkRuns) {
const tasks = db.getRunTasks(run.id)
benchmarkTasks += tasks.length
for (const task of tasks) {
if (task.success) {
benchmarkSuccessfulTasks++
}
const metrics = db.getTaskMetrics(task.id)
const tokensIn = metrics.find((m) => m.name === "tokensIn")?.value || 0
const tokensOut = metrics.find((m) => m.name === "tokensOut")?.value || 0
benchmarkTotalTokens += tokensIn + tokensOut
benchmarkTotalCost += metrics.find((m) => m.name === "cost")?.value || 0
benchmarkTotalDuration += metrics.find((m) => m.name === "duration")?.value || 0
}
}
benchmarkSummary.tasks = benchmarkTasks
benchmarkSummary.successRate = benchmarkTasks > 0 ? benchmarkSuccessfulTasks / benchmarkTasks : 0
benchmarkSummary.averageTokens = benchmarkTasks > 0 ? benchmarkTotalTokens / benchmarkTasks : 0
benchmarkSummary.averageCost = benchmarkTasks > 0 ? benchmarkTotalCost / benchmarkTasks : 0
benchmarkSummary.averageDuration = benchmarkTasks > 0 ? benchmarkTotalDuration / benchmarkTasks : 0
benchmarkReports[benchmark] = benchmarkSummary
}
// Generate model-specific reports
const modelReports: Record<string, any> = {}
for (const model of summary.models) {
const modelRuns = runs.filter((run) => run.model === model)
const modelSummary = {
runs: modelRuns.length,
benchmarks: [...new Set(modelRuns.map((run) => run.benchmark))],
tasks: 0,
successRate: 0,
averageTokens: 0,
averageCost: 0,
averageDuration: 0,
}
let modelTasks = 0
let modelSuccessfulTasks = 0
let modelTotalTokens = 0
let modelTotalCost = 0
let modelTotalDuration = 0
for (const run of modelRuns) {
const tasks = db.getRunTasks(run.id)
modelTasks += tasks.length
for (const task of tasks) {
if (task.success) {
modelSuccessfulTasks++
}
const metrics = db.getTaskMetrics(task.id)
const tokensIn = metrics.find((m) => m.name === "tokensIn")?.value || 0
const tokensOut = metrics.find((m) => m.name === "tokensOut")?.value || 0
modelTotalTokens += tokensIn + tokensOut
modelTotalCost += metrics.find((m) => m.name === "cost")?.value || 0
modelTotalDuration += metrics.find((m) => m.name === "duration")?.value || 0
}
}
modelSummary.tasks = modelTasks
modelSummary.successRate = modelTasks > 0 ? modelSuccessfulTasks / modelTasks : 0
modelSummary.averageTokens = modelTasks > 0 ? modelTotalTokens / modelTasks : 0
modelSummary.averageCost = modelTasks > 0 ? modelTotalCost / modelTasks : 0
modelSummary.averageDuration = modelTasks > 0 ? modelTotalDuration / modelTasks : 0
modelReports[model] = modelSummary
}
// Save reports
const reportDir = path.join(path.resolve(__dirname, "../../../"), "results", "reports")
fs.mkdirSync(reportDir, { recursive: true })
const timestamp = new Date().toISOString().replace(/:/g, "-")
if (format === "json") {
// Save JSON reports
fs.writeFileSync(path.join(reportDir, `summary-${timestamp}.json`), JSON.stringify(summary, null, 2))
fs.writeFileSync(path.join(reportDir, `benchmarks-${timestamp}.json`), JSON.stringify(benchmarkReports, null, 2))
fs.writeFileSync(path.join(reportDir, `models-${timestamp}.json`), JSON.stringify(modelReports, null, 2))
spinner.succeed(`JSON reports generated in ${reportDir}`)
} else {
// Generate markdown report
const outputPath = options.output || path.join(reportDir, `report-${timestamp}.md`)
generateMarkdownReport(summary, benchmarkReports, modelReports, outputPath)
spinner.succeed(`Markdown report generated at ${outputPath}`)
}
} catch (error: any) {
console.error(chalk.red(`Error generating report: ${error.message}`))
console.error(error.stack)
} finally {
db.close()
}
}
-133
View File
@@ -1,133 +0,0 @@
import * as path from "path"
import { v4 as uuidv4 } from "uuid"
import chalk from "chalk"
import ora from "ora"
import { getAdapter } from "../adapters"
import { ResultsDatabase } from "../db"
import { spawnVSCode, cleanupVSCode } from "../utils/vscode"
import { sendTaskToServer } from "../utils/task"
import { storeTaskResult } from "../utils/results"
interface RunOptions {
benchmark?: string
model: string
count?: number
apiKey?: string
}
/**
* Handler for the run command
* @param options Command options
*/
export async function runHandler(options: RunOptions): Promise<void> {
// Determine which benchmarks to run
const benchmarks = options.benchmark ? [options.benchmark] : ["exercism"] // Default to exercism for now
const model = options.model
const count = options.count || Infinity
console.log(chalk.blue(`Running evaluations for model: ${model}`))
console.log(chalk.blue(`Benchmarks: ${benchmarks.join(", ")}`))
// Create a run for each benchmark
for (const benchmark of benchmarks) {
const runId = uuidv4()
const db = new ResultsDatabase()
console.log(chalk.green(`\nStarting run for benchmark: ${benchmark}`))
// Create run in database
db.createRun(runId, model, benchmark)
// Get adapter for this benchmark
try {
const adapter = getAdapter(benchmark)
// List tasks
const spinner = ora("Listing tasks...").start()
const tasks = await adapter.listTasks()
spinner.succeed(`Found ${tasks.length} tasks for ${benchmark}`)
// Limit number of tasks if specified
const tasksToRun = tasks.slice(0, count)
console.log(chalk.blue(`Running ${tasksToRun.length} tasks...`))
// Run each task
for (let i = 0; i < tasksToRun.length; i++) {
const task = tasksToRun[i]
console.log(chalk.cyan(`\nTask ${i + 1}/${tasksToRun.length}: ${task.name}`))
// Prepare task
const prepareSpinner = ora("Preparing task...").start()
const preparedTask = await adapter.prepareTask(task.id)
prepareSpinner.succeed("Task prepared")
// Spawn VSCode
console.log("Spawning VSCode...")
await spawnVSCode(preparedTask.workspacePath)
// Send task to server
const sendSpinner = ora("Sending task to server...").start()
try {
const result = await sendTaskToServer(preparedTask.description, options.apiKey)
sendSpinner.succeed("Task completed")
// Verify result
const verifySpinner = ora("Verifying result...").start()
const verification = await adapter.verifyResult(preparedTask, result)
if (verification.success) {
verifySpinner.succeed(
`Verification successful: ${verification.metrics.testsPassed}/${verification.metrics.testsTotal} tests passed`,
)
} else {
verifySpinner.fail(
`Verification failed: ${verification.metrics.testsPassed}/${verification.metrics.testsTotal} tests passed`,
)
}
// Store result
const storeSpinner = ora("Storing result...").start()
await storeTaskResult(runId, preparedTask, result, verification)
storeSpinner.succeed("Result stored")
console.log(chalk.green(`Task completed. Success: ${verification.success}`))
// Clean up VS Code and temporary files
const cleanupSpinner = ora("Cleaning up...").start()
try {
await cleanupVSCode(preparedTask.workspacePath)
cleanupSpinner.succeed("Cleanup completed")
} catch (cleanupError: any) {
cleanupSpinner.fail(`Cleanup failed: ${cleanupError.message}`)
console.error(chalk.yellow(cleanupError.stack))
}
} catch (error: any) {
sendSpinner.fail(`Task failed: ${error.message}`)
console.error(chalk.red(error.stack))
// Clean up VS Code and temporary files even if the task failed
const cleanupSpinner = ora("Cleaning up...").start()
try {
await cleanupVSCode(preparedTask.workspacePath)
cleanupSpinner.succeed("Cleanup completed")
} catch (cleanupError: any) {
cleanupSpinner.fail(`Cleanup failed: ${cleanupError.message}`)
console.error(chalk.yellow(cleanupError.stack))
}
}
}
// Mark run as complete
db.completeRun(runId)
console.log(chalk.green(`\nRun complete for benchmark: ${benchmark}`))
} catch (error: any) {
console.error(chalk.red(`Error running benchmark ${benchmark}: ${error.message}`))
console.error(error.stack)
}
}
console.log(chalk.green("\nAll evaluations complete"))
}
-72
View File
@@ -1,72 +0,0 @@
import * as path from "path"
import * as fs from "fs"
import execa from "execa"
import chalk from "chalk"
import ora from "ora"
import { getAllAdapters } from "../adapters/index"
import { BenchmarkAdapter } from "../adapters/types"
interface SetupOptions {
benchmarks: string
}
/**
* Handler for the setup command
* @param options Command options
*/
export async function setupHandler(options: SetupOptions): Promise<void> {
const benchmarks = options.benchmarks.split(",")
console.log(chalk.blue(`Setting up benchmarks: ${benchmarks.join(", ")}`))
// Create directories
const evalsDir = path.resolve(__dirname, "../../../")
const reposDir = path.join(evalsDir, "repositories")
const resultsDir = path.join(evalsDir, "results")
const spinner = ora("Creating directory structure").start()
try {
fs.mkdirSync(reposDir, { recursive: true })
fs.mkdirSync(resultsDir, { recursive: true })
fs.mkdirSync(path.join(resultsDir, "runs"), { recursive: true })
fs.mkdirSync(path.join(resultsDir, "reports"), { recursive: true })
spinner.succeed("Directory structure created")
} catch (error) {
spinner.fail(`Failed to create directory structure: ${(error as Error).message}`)
throw error
}
// Set up each benchmark
try {
const adapters = getAllAdapters().filter((adapter: BenchmarkAdapter) => benchmarks.includes(adapter.name))
if (adapters.length === 0) {
console.warn(chalk.yellow("No valid benchmarks specified. Available benchmarks:"))
console.warn(
chalk.yellow(
getAllAdapters()
.map((a: BenchmarkAdapter) => a.name)
.join(", "),
),
)
return
}
for (const adapter of adapters) {
const setupSpinner = ora(`Setting up ${adapter.name}...`).start()
try {
await adapter.setup()
setupSpinner.succeed(`${adapter.name} setup complete`)
} catch (error) {
setupSpinner.fail(`Failed to set up ${adapter.name}: ${(error as Error).message}`)
throw error
}
}
console.log(chalk.green("Setup complete"))
} catch (error) {
console.error(chalk.red(`Setup failed: ${(error as Error).message}`))
throw error
}
}
-211
View File
@@ -1,211 +0,0 @@
import * as path from "path"
import * as fs from "fs"
import Database from "better-sqlite3"
import { SCHEMA } from "./schema"
const EVALS_DIR = path.resolve(__dirname, "../../../")
/**
* Database class for storing evaluation results
*/
export class ResultsDatabase {
db: Database.Database
constructor() {
// Ensure results directory exists
const resultsDir = path.join(EVALS_DIR, "results")
fs.mkdirSync(resultsDir, { recursive: true })
// Create database file
const dbPath = path.join(resultsDir, "evals.db")
this.db = new Database(dbPath)
// Initialize schema
this.initSchema()
}
/**
* Initialize the database schema
*/
private initSchema(): void {
this.db.exec(SCHEMA)
}
/**
* Create a new evaluation run
* @param id Run ID
* @param model Model name
* @param benchmark Benchmark name
*/
createRun(id: string, model: string, benchmark: string): void {
const stmt = this.db.prepare(`
INSERT INTO runs (id, timestamp, model, benchmark)
VALUES (?, ?, ?, ?)
`)
stmt.run(id, Date.now(), model, benchmark)
}
/**
* Mark a run as completed
* @param id Run ID
*/
completeRun(id: string): void {
const stmt = this.db.prepare(`
UPDATE runs SET completed = 1 WHERE id = ?
`)
stmt.run(id)
}
/**
* Create a new task
* @param id Task ID
* @param runId Run ID
* @param taskId Original task ID
*/
createTask(id: string, runId: string, taskId: string): void {
const stmt = this.db.prepare(`
INSERT INTO tasks (id, run_id, task_id, timestamp)
VALUES (?, ?, ?, ?)
`)
stmt.run(id, runId, taskId, Date.now())
}
/**
* Mark a task as completed
* @param id Task ID
* @param success Whether the task was successful
* @param toolCalls Total tool calls
* @param toolFailures Total tool failures
*/
completeTask(id: string, success: boolean, toolCalls: number = 0, toolFailures: number = 0): void {
const stmt = this.db.prepare(`
UPDATE tasks
SET success = ?, total_tool_calls = ?, total_tool_failures = ?
WHERE id = ?
`)
stmt.run(success ? 1 : 0, toolCalls, toolFailures, id)
}
/**
* Add a metric to a task
* @param taskId Task ID
* @param name Metric name
* @param value Metric value
*/
addMetric(taskId: string, name: string, value: number): void {
const stmt = this.db.prepare(`
INSERT INTO metrics (task_id, name, value)
VALUES (?, ?, ?)
`)
stmt.run(taskId, name, value)
}
/**
* Add a tool call record
* @param taskId Task ID
* @param toolName Tool name
* @param callCount Number of calls
* @param failureCount Number of failures
*/
addToolCall(taskId: string, toolName: string, callCount: number, failureCount: number): void {
const stmt = this.db.prepare(`
INSERT INTO tool_calls (task_id, tool_name, call_count, failure_count)
VALUES (?, ?, ?, ?)
`)
stmt.run(taskId, toolName, callCount, failureCount)
}
/**
* Add a file record
* @param taskId Task ID
* @param filePath File path
* @param status File status (created, modified, deleted)
*/
addFile(taskId: string, filePath: string, status: "created" | "modified" | "deleted"): void {
const stmt = this.db.prepare(`
INSERT INTO files (task_id, path, status)
VALUES (?, ?, ?)
`)
stmt.run(taskId, filePath, status)
}
/**
* Get all runs
* @returns Array of runs
*/
getRuns(): any[] {
const stmt = this.db.prepare(`
SELECT * FROM runs ORDER BY timestamp DESC
`)
return stmt.all()
}
/**
* Get all tasks for a run
* @param runId Run ID
* @returns Array of tasks
*/
getRunTasks(runId: string): any[] {
const stmt = this.db.prepare(`
SELECT * FROM tasks WHERE run_id = ? ORDER BY timestamp ASC
`)
return stmt.all(runId)
}
/**
* Get all metrics for a task
* @param taskId Task ID
* @returns Array of metrics
*/
getTaskMetrics(taskId: string): any[] {
const stmt = this.db.prepare(`
SELECT name, value FROM metrics WHERE task_id = ?
`)
return stmt.all(taskId)
}
/**
* Get all tool calls for a task
* @param taskId Task ID
* @returns Array of tool calls
*/
getTaskToolCalls(taskId: string): any[] {
const stmt = this.db.prepare(`
SELECT tool_name, call_count, failure_count
FROM tool_calls
WHERE task_id = ?
`)
return stmt.all(taskId)
}
/**
* Get all files for a task
* @param taskId Task ID
* @returns Array of files
*/
getTaskFiles(taskId: string): any[] {
const stmt = this.db.prepare(`
SELECT path, status FROM files WHERE task_id = ?
`)
return stmt.all(taskId)
}
/**
* Close the database connection
*/
close(): void {
this.db.close()
}
}
-48
View File
@@ -1,48 +0,0 @@
/**
* SQL schema for the evaluation database
*/
export const SCHEMA = `
CREATE TABLE IF NOT EXISTS runs (
id TEXT PRIMARY KEY,
timestamp INTEGER NOT NULL,
model TEXT NOT NULL,
benchmark TEXT NOT NULL,
completed INTEGER NOT NULL DEFAULT 0
);
CREATE TABLE IF NOT EXISTS tasks (
id TEXT PRIMARY KEY,
run_id TEXT NOT NULL,
task_id TEXT NOT NULL,
timestamp INTEGER NOT NULL,
success INTEGER NOT NULL DEFAULT 0,
total_tool_calls INTEGER DEFAULT 0,
total_tool_failures INTEGER DEFAULT 0,
FOREIGN KEY (run_id) REFERENCES runs(id)
);
CREATE TABLE IF NOT EXISTS metrics (
id INTEGER PRIMARY KEY AUTOINCREMENT,
task_id TEXT NOT NULL,
name TEXT NOT NULL,
value REAL NOT NULL,
FOREIGN KEY (task_id) REFERENCES tasks(id)
);
CREATE TABLE IF NOT EXISTS tool_calls (
id INTEGER PRIMARY KEY AUTOINCREMENT,
task_id TEXT NOT NULL,
tool_name TEXT NOT NULL,
call_count INTEGER NOT NULL,
failure_count INTEGER NOT NULL,
FOREIGN KEY (task_id) REFERENCES tasks(id)
);
CREATE TABLE IF NOT EXISTS files (
id INTEGER PRIMARY KEY AUTOINCREMENT,
task_id TEXT NOT NULL,
path TEXT NOT NULL,
status TEXT NOT NULL,
FOREIGN KEY (task_id) REFERENCES tasks(id)
);
`
-86
View File
@@ -1,86 +0,0 @@
#!/usr/bin/env node
import { Command } from "commander"
import chalk from "chalk"
import { setupHandler } from "./commands/setup"
import { runHandler } from "./commands/run"
import { reportHandler } from "./commands/report"
import { evalsEnvHandler } from "./commands/evals-env"
// Create the CLI program
const program = new Command()
// Set up CLI metadata
program.name("cline-eval").description("CLI tool for orchestrating Cline evaluations across multiple benchmarks").version("0.1.0")
// Setup command
program
.command("setup")
.description("Clone and set up benchmark repositories")
.option(
"-b, --benchmarks <benchmarks>",
"Comma-separated list of benchmarks to set up",
"exercism,swe-bench,swelancer,multi-swe",
)
.action(async (options) => {
try {
await setupHandler(options)
} catch (error) {
console.error(chalk.red(`Error during setup: ${error instanceof Error ? error.message : String(error)}`))
process.exit(1)
}
})
// Run command
program
.command("run")
.description("Run evaluations")
.option("-b, --benchmark <benchmark>", "Specific benchmark to run")
.option("-m, --model <model>", "Model to evaluate", "claude-3-opus-20240229")
.option("-c, --count <count>", "Number of tasks to run", parseInt)
.option("-k, --api-key <apiKey>", "Cline API key to use for evaluations")
.action(async (options) => {
try {
await runHandler(options)
} catch (error) {
console.error(chalk.red(`Error during run: ${error instanceof Error ? error.message : String(error)}`))
process.exit(1)
}
})
// Report command
program
.command("report")
.description("Generate reports")
.option("-f, --format <format>", "Report format (json, markdown)", "markdown")
.option("-o, --output <path>", "Output path for the report")
.action(async (options) => {
try {
await reportHandler(options)
} catch (error) {
console.error(chalk.red(`Error generating report: ${error instanceof Error ? error.message : String(error)}`))
process.exit(1)
}
})
// Evals-env command
program
.command("evals-env")
.description("Manage evals.env files for test mode activation")
.argument("<action>", "Action to perform: create, remove, or check")
.option("-d, --directory <directory>", "Directory to create/remove/check evals.env file in (defaults to current directory)")
.action(async (action, options) => {
try {
await evalsEnvHandler({ action, ...options })
} catch (error) {
console.error(chalk.red(`Error managing evals.env file: ${error instanceof Error ? error.message : String(error)}`))
process.exit(1)
}
})
// Parse command line arguments
program.parse(process.argv)
// If no arguments provided, show help
if (process.argv.length === 2) {
program.help()
}
-79
View File
@@ -1,79 +0,0 @@
import * as fs from "fs"
import * as path from "path"
import chalk from "chalk"
/**
* Creates an evals.env file in the specified directory
* @param directory The directory where the evals.env file should be created
* @returns True if the file was created, false if it already exists
*/
export function createEvalsEnvFile(directory: string): boolean {
const evalsEnvPath = path.join(directory, "evals.env")
// Check if the file already exists
if (fs.existsSync(evalsEnvPath)) {
console.log(chalk.yellow(`evals.env file already exists at ${evalsEnvPath}`))
return false
}
// Create the file
try {
const content = `# This file activates Cline test mode
# Created at: ${new Date().toISOString()}
#
# This file is automatically detected by the Cline extension
# and enables test mode for automated evaluations.
#
# Delete this file to deactivate test mode.
`
fs.writeFileSync(evalsEnvPath, content)
console.log(chalk.green(`Created evals.env file at ${evalsEnvPath}`))
return true
} catch (error) {
console.error(chalk.red(`Error creating evals.env file: ${error}`))
return false
}
}
/**
* Removes an evals.env file from the specified directory
* @param directory The directory where the evals.env file should be removed
* @returns True if the file was removed, false if it doesn't exist
*/
export function removeEvalsEnvFile(directory: string): boolean {
const evalsEnvPath = path.join(directory, "evals.env")
// Check if the file exists
if (!fs.existsSync(evalsEnvPath)) {
console.log(chalk.yellow(`No evals.env file found at ${evalsEnvPath}`))
return false
}
// Remove the file
try {
fs.unlinkSync(evalsEnvPath)
console.log(chalk.green(`Removed evals.env file from ${evalsEnvPath}`))
return true
} catch (error) {
console.error(chalk.red(`Error removing evals.env file: ${error}`))
return false
}
}
/**
* Checks if an evals.env file exists in the specified directory
* @param directory The directory to check for an evals.env file
* @returns True if the file exists, false otherwise
*/
export function checkEvalsEnvFile(directory: string): boolean {
const evalsEnvPath = path.join(directory, "evals.env")
const exists = fs.existsSync(evalsEnvPath)
if (exists) {
console.log(chalk.green(`evals.env file found at ${evalsEnvPath}`))
} else {
console.log(chalk.yellow(`No evals.env file found at ${evalsEnvPath}`))
}
return exists
}
-131
View File
@@ -1,131 +0,0 @@
import execa from "execa"
import * as fs from "fs"
import * as path from "path"
import * as os from "os"
/**
* List of VSCode extensions to install for evaluation environments
* These extensions provide language support and other useful features
*/
export const REQUIRED_EXTENSIONS = [
"golang.go", // Go language support
"dbaeumer.vscode-eslint", // ESLint support
"redhat.java", // Java support
"ms-python.python", // Python support
"rust-lang.rust-analyzer", // Rust support
"ms-vscode.cpptools", // C/C++ support
]
/**
* Install required VSCode extensions in the specified extensions directory
* @param extensionsDir The directory where extensions should be installed
* @returns Promise that resolves when all extensions are installed
*/
export async function installRequiredExtensions(extensionsDir: string): Promise<void> {
console.log("Installing required VSCode extensions...")
// Create the extensions directory if it doesn't exist
if (!fs.existsSync(extensionsDir)) {
fs.mkdirSync(extensionsDir, { recursive: true })
}
// Install each extension
for (const extension of REQUIRED_EXTENSIONS) {
try {
console.log(`Installing extension: ${extension}...`)
await execa("code", ["--extensions-dir", extensionsDir, "--install-extension", extension, "--force"])
console.log(`✅ Extension ${extension} installed successfully`)
} catch (error: any) {
console.warn(`⚠️ Failed to install extension ${extension}: ${error.message}`)
// Continue with other extensions even if one fails
}
}
console.log("✅ All required extensions installed")
}
/**
* Check if a VSCode extension is installed in the specified directory
* @param extensionsDir The directory to check for installed extensions
* @param extensionId The ID of the extension to check
* @returns True if the extension is installed, false otherwise
*/
export function isExtensionInstalled(extensionsDir: string, extensionId: string): boolean {
// Extensions are installed in directories named publisher.name-version
// We need to check if any directory starts with the extensionId
const extensionPrefix = extensionId.toLowerCase() + "-"
try {
const files = fs.readdirSync(extensionsDir)
return files.some((file) => {
const lowerCaseFile = file.toLowerCase()
return lowerCaseFile === extensionId.toLowerCase() || lowerCaseFile.startsWith(extensionPrefix)
})
} catch (error) {
return false
}
}
/**
* Get the path to the VSCode settings file in the specified user data directory
* @param userDataDir The VSCode user data directory
* @returns The path to the settings.json file
*/
export function getSettingsPath(userDataDir: string): string {
const settingsDir = path.join(userDataDir, "User")
fs.mkdirSync(settingsDir, { recursive: true })
return path.join(settingsDir, "settings.json")
}
/**
* Configure extension settings in the VSCode user data directory
* @param userDataDir The VSCode user data directory
*/
export function configureExtensionSettings(userDataDir: string): void {
const settingsPath = getSettingsPath(userDataDir)
// Read existing settings if they exist
let settings = {}
if (fs.existsSync(settingsPath)) {
try {
settings = JSON.parse(fs.readFileSync(settingsPath, "utf8"))
} catch (error) {
console.warn(`Error reading settings file: ${error}`)
}
}
// Add or update extension-specific settings
const updatedSettings = {
...settings,
// Go extension settings
"go.toolsManagement.autoUpdate": false,
"go.survey.prompt": false,
// ESLint settings
"eslint.enable": true,
"eslint.run": "onSave",
// Java settings
"java.configuration.checkProjectSettingsExclusions": false,
"java.configure.checkForOutdatedExtensions": false,
"java.help.firstView": false,
// Python settings
"python.experiments.enabled": false,
"python.showStartPage": false,
// Rust settings
"rust-analyzer.checkOnSave.command": "check",
// C/C++ settings
"C_Cpp.intelliSenseEngine": "default",
// General extension settings
"extensions.autoUpdate": false,
"extensions.ignoreRecommendations": true,
}
// Write updated settings
fs.writeFileSync(settingsPath, JSON.stringify(updatedSettings, null, 2))
console.log("✅ Extension settings configured")
}
-109
View File
@@ -1,109 +0,0 @@
import * as fs from "fs"
import * as path from "path"
/**
* Generate a markdown report from evaluation results
* @param summary Overall summary
* @param benchmarkReports Benchmark-specific reports
* @param modelReports Model-specific reports
* @param outputPath Output file path
*/
export function generateMarkdownReport(
summary: any,
benchmarkReports: Record<string, any>,
modelReports: Record<string, any>,
outputPath: string,
): void {
let markdown = `# Cline Evaluation Report\n\n`
// Generate summary section
markdown += `## Summary\n\n`
markdown += `- **Total Runs:** ${summary.runs}\n`
markdown += `- **Models:** ${summary.models.join(", ")}\n`
markdown += `- **Benchmarks:** ${summary.benchmarks.join(", ")}\n`
markdown += `- **Total Tasks:** ${summary.tasks}\n`
markdown += `- **Success Rate:** ${(summary.successRate * 100).toFixed(2)}%\n`
markdown += `- **Average Tokens:** ${Math.round(summary.averageTokens)}\n`
markdown += `- **Average Cost:** $${summary.averageCost.toFixed(4)}\n`
markdown += `- **Average Duration:** ${(summary.averageDuration / 1000).toFixed(2)}s\n`
markdown += `- **Total Tool Calls:** ${summary.totalToolCalls}\n`
markdown += `- **Tool Success Rate:** ${(summary.toolSuccessRate * 100).toFixed(2)}%\n\n`
// Generate tool usage section
markdown += `## Tool Usage\n\n`
markdown += `| Tool | Calls | Failures | Success Rate |\n`
markdown += `| ---- | ----- | -------- | ------------ |\n`
for (const [toolName, metrics] of Object.entries(summary.toolUsage)) {
const calls = (metrics as any).calls
const failures = (metrics as any).failures
const successRate = calls > 0 ? (1 - failures / calls) * 100 : 100
markdown += `| ${toolName} | ${calls} | ${failures} | ${successRate.toFixed(2)}% |\n`
}
// Generate benchmark results section
markdown += `\n## Benchmark Results\n\n`
for (const [benchmark, report] of Object.entries(benchmarkReports)) {
markdown += `### ${benchmark}\n\n`
markdown += `- **Runs:** ${report.runs}\n`
markdown += `- **Models:** ${report.models.join(", ")}\n`
markdown += `- **Tasks:** ${report.tasks}\n`
markdown += `- **Success Rate:** ${(report.successRate * 100).toFixed(2)}%\n`
markdown += `- **Average Tokens:** ${Math.round(report.averageTokens)}\n`
markdown += `- **Average Cost:** $${report.averageCost.toFixed(4)}\n`
markdown += `- **Average Duration:** ${(report.averageDuration / 1000).toFixed(2)}s\n\n`
}
// Generate model results section
markdown += `## Model Results\n\n`
for (const [model, report] of Object.entries(modelReports)) {
markdown += `### ${model}\n\n`
markdown += `- **Runs:** ${report.runs}\n`
markdown += `- **Benchmarks:** ${report.benchmarks.join(", ")}\n`
markdown += `- **Tasks:** ${report.tasks}\n`
markdown += `- **Success Rate:** ${(report.successRate * 100).toFixed(2)}%\n`
markdown += `- **Average Tokens:** ${Math.round(report.averageTokens)}\n`
markdown += `- **Average Cost:** $${report.averageCost.toFixed(4)}\n`
markdown += `- **Average Duration:** ${(report.averageDuration / 1000).toFixed(2)}s\n\n`
}
// Add charts using Mermaid
markdown += `## Charts\n\n`
// Success rate by benchmark chart
markdown += `### Success Rate by Benchmark\n\n`
markdown += "```mermaid\n"
markdown += "graph TD\n"
markdown += " title[Success Rate by Benchmark]\n"
markdown += " style title fill:none,stroke:none\n\n"
for (const [benchmark, report] of Object.entries(benchmarkReports)) {
const successRate = (report.successRate * 100).toFixed(2)
markdown += ` ${benchmark}[${benchmark}: ${successRate}%]\n`
}
markdown += "```\n\n"
// Success rate by model chart
markdown += `### Success Rate by Model\n\n`
markdown += "```mermaid\n"
markdown += "graph TD\n"
markdown += " title[Success Rate by Model]\n"
markdown += " style title fill:none,stroke:none\n\n"
for (const [model, report] of Object.entries(modelReports)) {
const successRate = (report.successRate * 100).toFixed(2)
markdown += ` ${model.replace(/[-\.]/g, "_")}[${model}: ${successRate}%]\n`
}
markdown += "```\n\n"
// Add timestamp
markdown += `\n\n---\n\nReport generated on ${new Date().toISOString()}\n`
// Write markdown to file
fs.writeFileSync(outputPath, markdown)
}
-79
View File
@@ -1,79 +0,0 @@
import { v4 as uuidv4 } from "uuid"
import { ResultsDatabase } from "../db"
import { Task } from "../adapters/types"
/**
* Store task result in the database
* @param runId The run ID
* @param task The task that was executed
* @param result The result from the test server
* @param verification The verification result
*/
export async function storeTaskResult(runId: string, task: Task, result: any, verification: any): Promise<void> {
const db = new ResultsDatabase()
const taskId = uuidv4()
try {
// Extract metrics from the result
const { metrics } = result
const totalToolCalls = metrics?.totalToolCalls || 0
const totalToolFailures = metrics?.totalToolFailures || 0
// Create task with tool metrics
db.createTask(taskId, runId, task.id)
db.completeTask(taskId, verification.success, totalToolCalls, totalToolFailures)
// Store metrics
if (metrics) {
// Store token metrics
if (metrics.tokensIn) db.addMetric(taskId, "tokensIn", metrics.tokensIn)
if (metrics.tokensOut) db.addMetric(taskId, "tokensOut", metrics.tokensOut)
if (metrics.cost) db.addMetric(taskId, "cost", metrics.cost)
if (metrics.duration) db.addMetric(taskId, "duration", metrics.duration)
// Store tool call metrics
if (metrics.toolCalls) {
for (const [toolName, callCount] of Object.entries(metrics.toolCalls)) {
const failureCount = metrics.toolFailures?.[toolName] || 0
db.addToolCall(taskId, toolName, callCount as number, failureCount)
}
}
}
// Store verification metrics
if (verification.metrics) {
for (const [key, value] of Object.entries(verification.metrics)) {
if (typeof value === "number") {
db.addMetric(taskId, key, value)
}
}
}
// Store file changes
if (result.files) {
// Store created files
if (result.files.created) {
for (const file of result.files.created) {
db.addFile(taskId, file, "created")
}
}
// Store modified files
if (result.files.modified) {
for (const file of result.files.modified) {
db.addFile(taskId, file, "modified")
}
}
// Store deleted files
if (result.files.deleted) {
for (const file of result.files.deleted) {
db.addFile(taskId, file, "deleted")
}
}
}
} finally {
// Close the database connection
db.close()
}
}
-52
View File
@@ -1,52 +0,0 @@
import fetch from "node-fetch"
import chalk from "chalk"
/**
* Send a task to the Cline test server
* @param task The task description to send
* @param apiKey Optional Cline API key to use for the task
* @returns The result of the task execution
*/
export async function sendTaskToServer(task: string, apiKey?: string): Promise<any> {
const SERVER_URL = "http://localhost:9876/task"
try {
console.log(chalk.blue(`Sending task to server: ${task.substring(0, 100)}${task.length > 100 ? "..." : ""}`))
const response = await fetch(SERVER_URL, {
method: "POST",
headers: {
"Content-Type": "application/json",
},
body: JSON.stringify({
task,
apiKey,
}),
})
if (!response.ok) {
const errorText = await response.text()
throw new Error(`Server responded with status ${response.status}: ${errorText}`)
}
const result = await response.json()
if (!result.success) {
throw new Error(`Task execution failed: ${result.error || "Unknown error"}`)
}
if (result.timeout) {
throw new Error("Task execution timed out")
}
return result
} catch (error: any) {
if (error.code === "ECONNREFUSED") {
throw new Error(
"Could not connect to the test server. Make sure VSCode is running with the Cline extension and the test server is active.",
)
}
throw error
}
}
-615
View File
@@ -1,615 +0,0 @@
import execa from "execa"
import * as path from "path"
import * as fs from "fs"
import fetch from "node-fetch"
import * as os from "os"
import { installRequiredExtensions, configureExtensionSettings } from "./extensions"
// Store temporary directories for cleanup
interface VSCodeResources {
tempUserDataDir: string
tempExtensionsDir: string
vscodePid?: number
}
// Global map to track resources for each workspace
const workspaceResources = new Map<string, VSCodeResources>()
/**
* Spawn a VSCode instance with the Cline extension
* @param workspacePath The workspace path to open
* @param vsixPath Optional path to a VSIX file to install
* @returns The resources created for this VS Code instance
*/
export async function spawnVSCode(workspacePath: string, vsixPath?: string): Promise<VSCodeResources> {
// Ensure the workspace path exists
if (!fs.existsSync(workspacePath)) {
throw new Error(`Workspace path does not exist: ${workspacePath}`)
}
// If no VSIX path is provided, build one with IS_TEST=true
if (!vsixPath) {
try {
// Build the VSIX (no longer need to set IS_TEST=true as we'll use evals.env file)
console.log("Building VSIX...")
const clineRoot = path.resolve(process.cwd(), "..", "..")
await execa("npx", ["vsce", "package"], {
cwd: clineRoot,
stdio: "inherit",
})
// Find the generated VSIX file(s)
const files = fs.readdirSync(clineRoot)
const vsixFiles = files.filter((file) => file.endsWith(".vsix"))
if (vsixFiles.length > 0) {
// Get file stats to find the most recent one
const vsixFilesWithStats = vsixFiles.map((file) => {
const filePath = path.join(clineRoot, file)
return {
file,
path: filePath,
mtime: fs.statSync(filePath).mtime,
}
})
// Sort by modification time (most recent first)
vsixFilesWithStats.sort((a, b) => b.mtime.getTime() - a.mtime.getTime())
// Use the most recent VSIX
vsixPath = vsixFilesWithStats[0].path
console.log(`Using most recent VSIX: ${vsixPath} (modified ${vsixFilesWithStats[0].mtime.toISOString()})`)
// Log all found VSIX files for debugging
if (vsixFiles.length > 1) {
console.log(`Found ${vsixFiles.length} VSIX files:`)
vsixFilesWithStats.forEach((f) => {
console.log(` - ${f.file} (modified ${f.mtime.toISOString()})`)
})
}
} else {
console.warn("Could not find generated VSIX file")
}
} catch (error) {
console.warn("Failed to build test VSIX:", error)
}
}
// Create a temporary user data directory for this VS Code instance
const tempUserDataDir = path.join(os.tmpdir(), `vscode-cline-eval-${Date.now()}`)
fs.mkdirSync(tempUserDataDir, { recursive: true })
console.log(`Created temporary user data directory: ${tempUserDataDir}`)
// Create a temporary extensions directory to ensure no other extensions are loaded
const tempExtensionsDir = path.join(os.tmpdir(), `vscode-cline-eval-ext-${Date.now()}`)
fs.mkdirSync(tempExtensionsDir, { recursive: true })
console.log(`Created temporary extensions directory: ${tempExtensionsDir}`)
// Create evals.env file in the workspace to trigger test mode
console.log(`Creating evals.env file in workspace: ${workspacePath}`)
const evalsEnvPath = path.join(workspacePath, "evals.env")
fs.writeFileSync(
evalsEnvPath,
`# This file activates Cline test mode
# Created at: ${new Date().toISOString()}
#
# This file is automatically detected by the Cline extension
# and enables test mode for automated evaluations.
#
# Delete this file to deactivate test mode.
`,
)
// Create settings.json in the temporary user data directory to disable workspace trust
// and configure Cline to auto-open on startup
const settingsDir = path.join(tempUserDataDir, "User")
fs.mkdirSync(settingsDir, { recursive: true })
const settingsPath = path.join(settingsDir, "settings.json")
const settings = {
// Disable workspace trust
"security.workspace.trust.enabled": false,
"security.workspace.trust.startupPrompt": "never",
"security.workspace.trust.banner": "never",
"security.workspace.trust.emptyWindow": true,
// Configure startup behavior
"workbench.startupEditor": "none",
// Auto-open Cline on startup
"cline.autoOpenOnStartup": true,
// Show the activity bar and sidebar
"workbench.activityBar.visible": true,
"workbench.sideBar.visible": true,
"workbench.view.extension.saoudrizwan.claude-dev-ActivityBar.visible": true,
"workbench.view.alwaysShowHeaderActions": true,
"workbench.editor.openSideBySideDirection": "right",
// Disable GitLens from opening automatically
"gitlens.views.repositories.autoReveal": false,
"gitlens.views.fileHistory.autoReveal": false,
"gitlens.views.lineHistory.autoReveal": false,
"gitlens.views.compare.autoReveal": false,
"gitlens.views.search.autoReveal": false,
"gitlens.showWelcomeOnInstall": false,
"gitlens.showWhatsNewAfterUpgrades": false,
// Disable other extensions that might compete for startup focus
"extensions.autoUpdate": false,
}
fs.writeFileSync(settingsPath, JSON.stringify(settings, null, 2))
console.log(`Created settings.json to disable workspace trust and auto-open Cline`)
// Create keybindings.json to automatically open Cline on startup
const keybindingsPath = path.join(settingsDir, "keybindings.json")
const keybindings = [
{
key: "alt+c",
command: "workbench.view.extension.saoudrizwan.claude-dev-ActivityBar",
when: "viewContainer.workbench.view.extension.saoudrizwan.claude-dev-ActivityBar.enabled",
},
{
key: "alt+shift+c",
command: "cline.openInNewTab",
when: "viewContainer.workbench.view.extension.saoudrizwan.claude-dev-ActivityBar.enabled",
},
]
fs.writeFileSync(keybindingsPath, JSON.stringify(keybindings, null, 2))
console.log(`Created keybindings.json to help with Cline activation`)
// Build the command arguments with custom user data directory
const args = [
// Use a custom user data directory to isolate this instance
"--user-data-dir",
tempUserDataDir,
// Use a custom extensions directory to ensure only our extension is loaded
"--extensions-dir",
tempExtensionsDir,
// Disable workspace trust
"--disable-workspace-trust",
"-n",
workspacePath,
// Force the extension to be activated on startup
"--start-up-extension",
"saoudrizwan.claude-dev",
// Run a command on startup to open Cline
"--command",
"workbench.view.extension.saoudrizwan.claude-dev-ActivityBar",
// Additional flags to help with extension activation
"--disable-gpu=false",
"--max-memory=4096",
]
// Create a startup script to run commands after VS Code launches
const startupScriptPath = path.join(settingsDir, "startup.js")
const startupScript = `
// This script will be executed when VS Code starts
setTimeout(() => {
// Try to open Cline in the sidebar
require('vscode').commands.executeCommand('workbench.view.extension.saoudrizwan.claude-dev-ActivityBar');
// Also try to open Cline in a tab as a fallback
setTimeout(() => {
require('vscode').commands.executeCommand('cline.openInNewTab');
}, 5000);
}, 5000);
`
fs.writeFileSync(startupScriptPath, startupScript)
console.log(`Created startup script to activate Cline`)
// If a VSIX is provided, install it
if (vsixPath) {
if (!fs.existsSync(vsixPath)) {
throw new Error(`VSIX file does not exist: ${vsixPath}`)
}
args.unshift("--install-extension", vsixPath)
}
// Install required extensions
console.log("Installing required VSCode extensions...")
await installRequiredExtensions(tempExtensionsDir)
// Configure extension settings
console.log("Configuring extension settings...")
configureExtensionSettings(tempUserDataDir)
// Execute the command
try {
// We don't need to install extensions globally anymore since we're using a custom user data directory
// The VSIX will be installed in the isolated environment if provided in the args
// Launch VS Code
console.log("Launching VS Code...")
await execa("code", args, {
stdio: "inherit",
})
// Wait longer for VSCode to initialize and extension to load
console.log("Waiting for VS Code to initialize...")
await new Promise((resolve) => setTimeout(resolve, 30000))
// Create a JavaScript file that will be loaded as a VS Code extension
const extensionDir = path.join(tempExtensionsDir, "cline-activator")
fs.mkdirSync(extensionDir, { recursive: true })
// Create package.json for the extension
const packageJsonPath = path.join(extensionDir, "package.json")
const packageJson = {
name: "cline-activator",
displayName: "Cline Activator",
description: "Activates Cline and starts the test server",
version: "0.0.1",
engines: {
vscode: "^1.60.0",
},
main: "./extension.js",
activationEvents: ["*"],
contributes: {
commands: [
{
command: "cline-activator.activate",
title: "Activate Cline",
},
],
},
}
fs.writeFileSync(packageJsonPath, JSON.stringify(packageJson, null, 2))
// Create extension.js
const extensionJsPath = path.join(extensionDir, "extension.js")
const extensionJs = `
const vscode = require('vscode');
/**
* @param {vscode.ExtensionContext} context
*/
function activate(context) {
console.log('Cline Activator is now active!');
// Register the command to activate Cline
let disposable = vscode.commands.registerCommand('cline-activator.activate', async function () {
try {
// Make sure the Cline extension is activated
const extension = vscode.extensions.getExtension('saoudrizwan.claude-dev');
if (!extension) {
console.error('Cline extension not found');
return;
}
if (!extension.isActive) {
console.log('Activating Cline extension...');
await extension.activate();
}
// Show the Cline sidebar
console.log('Opening Cline sidebar...');
await vscode.commands.executeCommand('workbench.view.extension.saoudrizwan.claude-dev-ActivityBar');
// Wait a moment for the sidebar to initialize
await new Promise(resolve => setTimeout(resolve, 2000));
// Also open Cline in a tab as a fallback
console.log('Opening Cline in a tab...');
await vscode.commands.executeCommand('cline.openInNewTab');
// Wait a moment for the tab to initialize
await new Promise(resolve => setTimeout(resolve, 2000));
// Create the test server if it doesn't exist
console.log('Creating test server...');
// Get the visible webview instance
const clineRootPath = '${path.resolve(process.cwd(), "..", "..")}';
const visibleWebview = require(path.join(clineRootPath, 'src', 'core', 'webview')).WebviewProvider.getVisibleInstance();
if (visibleWebview) {
require(path.join(clineRootPath, 'src', 'services', 'test', 'TestServer')).createTestServer(visibleWebview);
console.log('Test server created successfully');
} else {
console.error('No visible webview instance found');
}
} catch (error) {
console.error('Error activating Cline:', error);
}
});
context.subscriptions.push(disposable);
// Automatically run the command after a delay
setTimeout(() => {
vscode.commands.executeCommand('cline-activator.activate');
}, 5000);
}
function deactivate() {}
module.exports = {
activate,
deactivate
}
`
fs.writeFileSync(extensionJsPath, extensionJs)
console.log(`Created Cline Activator extension`)
// Try multiple approaches to activate the extension
let serverStarted = false
// Create an activation script to run in VS Code
const activationScriptPath = path.join(settingsDir, "activate-cline.js")
const activationScript = `
// This script will be executed to activate Cline and start the test server
const vscode = require('vscode');
// Execute the cline-activator.activate command
vscode.commands.executeCommand('cline-activator.activate');
`
fs.writeFileSync(activationScriptPath, activationScript)
console.log(`Created activation script to run in VS Code`)
// Execute the activation script
try {
console.log("Executing activation script to start Cline and test server...")
await execa(
"code",
[
"--user-data-dir",
tempUserDataDir,
"--extensions-dir",
tempExtensionsDir,
"--folder-uri",
`file://${workspacePath}`,
"--execute",
activationScriptPath,
],
{
stdio: "inherit",
},
)
// Wait for the test server to start
console.log("Waiting for test server to start...")
for (let i = 0; i < 30; i++) {
try {
// Try to connect to the test server
const response = await fetch("http://localhost:9876/task", {
method: "OPTIONS",
headers: {
"Content-Type": "application/json",
},
})
if (response.status === 204) {
console.log("Test server is running!")
serverStarted = true
break
}
} catch (error) {
// Server not started yet, wait and try again
await new Promise((resolve) => setTimeout(resolve, 1000))
}
}
} catch (error) {
console.warn("Failed to execute activation script:", error)
}
if (!serverStarted) {
console.warn("Test server did not start after multiple attempts")
console.log("You may need to manually open the Cline extension in VS Code")
}
// Store the resources for this workspace
const resources: VSCodeResources = {
tempUserDataDir,
tempExtensionsDir,
}
// Store in the global map
workspaceResources.set(workspacePath, resources)
// Return the resources
return resources
} catch (error: any) {
throw new Error(`Failed to spawn VSCode: ${error.message}`)
}
}
/**
* Clean up VS Code resources and shut down the test server
* @param workspacePath The workspace path to clean up resources for
*/
export async function cleanupVSCode(workspacePath: string): Promise<void> {
console.log(`Cleaning up VS Code resources for workspace: ${workspacePath}`)
// Get the resources for this workspace
const resources = workspaceResources.get(workspacePath)
if (!resources) {
console.log(`No resources found for workspace: ${workspacePath}`)
return
}
// Try to shut down the test server
try {
console.log("Shutting down test server...")
await fetch("http://localhost:9876/shutdown", {
method: "POST",
headers: {
"Content-Type": "application/json",
},
}).catch(() => {
// Ignore errors, the server might already be down
})
} catch (error) {
console.warn(`Error shutting down test server: ${error}`)
}
// Try to gracefully close VS Code instead of killing it
try {
console.log("Attempting to gracefully close VS Code...")
// Create a settings file that will disable the crash reporter and the exit confirmation dialog
const settingsDir = path.join(resources.tempUserDataDir, "User")
const settingsPath = path.join(settingsDir, "settings.json")
// Read existing settings if they exist
let settings = {}
if (fs.existsSync(settingsPath)) {
try {
settings = JSON.parse(fs.readFileSync(settingsPath, "utf8"))
} catch (error) {
console.warn(`Error reading settings file: ${error}`)
}
}
// Update settings to disable crash reporter and exit confirmation
settings = {
...settings,
"window.confirmBeforeClose": "never",
"telemetry.enableCrashReporter": false,
"window.restoreWindows": "none",
"window.newWindowDimensions": "default",
}
// Write updated settings
fs.writeFileSync(settingsPath, JSON.stringify(settings, null, 2))
// On macOS, use AppleScript to quit VS Code gracefully
if (process.platform === "darwin") {
try {
// First try AppleScript to quit VS Code gracefully
await execa("osascript", ["-e", 'tell application "Visual Studio Code" to quit'])
// Wait a moment for VS Code to close
await new Promise((resolve) => setTimeout(resolve, 2000))
} catch (appleScriptError) {
console.warn(`Error using AppleScript to quit VS Code: ${appleScriptError}`)
}
} else if (process.platform === "win32") {
// On Windows, try to use taskkill without /F first
try {
await execa("taskkill", ["/IM", "code.exe"])
// Wait a moment for VS Code to close
await new Promise((resolve) => setTimeout(resolve, 2000))
} catch (taskkillError) {
console.warn(`Error using taskkill to quit VS Code: ${taskkillError}`)
}
} else {
// On Linux, try to use SIGTERM first
try {
// Find VS Code processes
const { stdout } = await execa("ps", ["aux"])
const lines = stdout.split("\n")
for (const line of lines) {
if (line.includes(resources.tempUserDataDir)) {
const parts = line.trim().split(/\s+/)
const pid = parseInt(parts[1])
if (pid && !isNaN(pid)) {
console.log(`Sending SIGTERM to VS Code process with PID: ${pid}`)
try {
// Use SIGTERM instead of SIGKILL for a graceful shutdown
process.kill(pid, "SIGTERM")
} catch (killError) {
console.warn(`Failed to terminate process ${pid}: ${killError}`)
}
}
}
}
// Wait a moment for VS Code to close
await new Promise((resolve) => setTimeout(resolve, 2000))
} catch (psError) {
console.warn(`Error listing processes: ${psError}`)
}
}
// If graceful methods failed, fall back to forceful termination as a last resort
// Check if VS Code is still running with the temp user data dir
let vsCodeStillRunning = false
if (process.platform !== "win32") {
try {
const { stdout } = await execa("ps", ["aux"])
vsCodeStillRunning = stdout.split("\n").some((line) => line.includes(resources.tempUserDataDir))
} catch (error) {
console.warn(`Error checking if VS Code is still running: ${error}`)
}
} else {
try {
const { stdout } = await execa("tasklist", ["/FI", `IMAGENAME eq code.exe`])
vsCodeStillRunning = stdout.includes("code.exe")
} catch (error) {
console.warn(`Error checking if VS Code is still running: ${error}`)
}
}
// If VS Code is still running, use forceful termination as a last resort
if (vsCodeStillRunning) {
console.log("Graceful shutdown failed, falling back to forceful termination...")
if (process.platform === "win32") {
try {
await execa("taskkill", ["/IM", "code.exe", "/F"])
} catch (error) {
console.warn(`Error forcefully terminating VS Code: ${error}`)
}
} else {
try {
const { stdout } = await execa("ps", ["aux"])
const lines = stdout.split("\n")
for (const line of lines) {
if (line.includes(resources.tempUserDataDir)) {
const parts = line.trim().split(/\s+/)
const pid = parseInt(parts[1])
if (pid && !isNaN(pid)) {
console.log(`Forcefully killing VS Code process with PID: ${pid}`)
try {
process.kill(pid, "SIGKILL")
} catch (killError) {
console.warn(`Failed to kill process ${pid}: ${killError}`)
}
}
}
}
} catch (error) {
console.warn(`Error forcefully terminating VS Code: ${error}`)
}
}
}
} catch (error) {
console.warn(`Error closing VS Code: ${error}`)
}
// Clean up temporary directories and evals.env file
try {
console.log(`Removing temporary user data directory: ${resources.tempUserDataDir}`)
fs.rmSync(resources.tempUserDataDir, { recursive: true, force: true })
} catch (error) {
console.warn(`Error removing temporary user data directory: ${error}`)
}
try {
console.log(`Removing temporary extensions directory: ${resources.tempExtensionsDir}`)
fs.rmSync(resources.tempExtensionsDir, { recursive: true, force: true })
} catch (error) {
console.warn(`Error removing temporary extensions directory: ${error}`)
}
// Remove the evals.env file
try {
const evalsEnvPath = path.join(workspacePath, "evals.env")
if (fs.existsSync(evalsEnvPath)) {
console.log(`Removing evals.env file: ${evalsEnvPath}`)
fs.unlinkSync(evalsEnvPath)
}
} catch (error) {
console.warn(`Error removing evals.env file: ${error}`)
}
// Remove from the global map
workspaceResources.delete(workspacePath)
console.log("Cleanup completed")
}
-17
View File
@@ -1,17 +0,0 @@
{
"compilerOptions": {
"target": "ES2020",
"module": "commonjs",
"lib": ["ES2020"],
"declaration": true,
"outDir": "dist",
"rootDir": "src",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist"]
}
+1 -1
View File
@@ -2,7 +2,7 @@
العربية | <a href="https://github.com/cline/cline/blob/main/locales/es/README.md" target="_blank">الإسبانية</a> | <a href="https://github.com/cline/cline/blob/main/locales/de/README.md" target="_blank">الألمانية</a> | <a href="https://github.com/cline/cline/blob/main/locales/ja/README.md" target="_blank">اليابانية</a> | <a href="https://github.com/cline/cline/blob/main/locales/zh-cn/README.md" target="_blank">الصينية المبسطة</a> | <a href="https://github.com/cline/cline/blob/main/locales/zh-tw/README.md" target="_blank">الصينية التقليدية</a> | <a href="https://github.com/cline/cline/blob/main/locales/pt-BR/README.md" target="_blank">البرتغالية</a>
</sub></div>
# Cline
# Cline \#1 على OpenRouter
<p align="center">
<img src="https://media.githubusercontent.com/media/cline/cline/main/assets/docs/demo.gif" width="100%" />
+1 -1
View File
@@ -1,4 +1,4 @@
# Cline
# Cline \#1 auf OpenRouter
<p align="center">
<img src="https://media.githubusercontent.com/media/cline/cline/main/assets/docs/demo.gif" width="100%" />
+1 -1
View File
@@ -1,4 +1,4 @@
# Cline
# Cline #1 en OpenRouter
<p align="center">
<img src="https://media.githubusercontent.com/media/cline/cline/main/assets/docs/demo.gif" width="100%" />
+1 -1
View File
@@ -1,4 +1,4 @@
# Cline
# Clineへの貢献
Clineへの貢献に興味をお持ちいただきありがとうございます。
+1 -1
View File
@@ -1,4 +1,4 @@
# Cline
# Cline OpenRouterでのナンバーワン
<p align="center">
<img src="https://media.githubusercontent.com/media/cline/cline/main/assets/docs/demo.gif" width="100%" />

Some files were not shown because too many files have changed in this diff Show More