mirror of
https://github.com/cline/cline.git
synced 2026-09-15 13:02:17 +08:00
Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9038a0330b | ||
|
|
570728b5f8 | ||
|
|
944a4626e4 |
@@ -0,0 +1,5 @@
|
||||
---
|
||||
"claude-dev": patch
|
||||
---
|
||||
|
||||
Add voice dictation feature for Cline account users
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
"claude-dev": patch
|
||||
---
|
||||
|
||||
Default the credits balance to dashes on the account page, making it clear that the balance is not zero
|
||||
@@ -1,13 +1,5 @@
|
||||
# Changelog
|
||||
|
||||
## [3.18.14]
|
||||
|
||||
- Fix bug where Cline account users logged in with invalid token would not be shown as logged out in webview presentation layer
|
||||
|
||||
## [3.18.13]
|
||||
|
||||
- Fix authentication issue where Cline accounts users would keep getting logged out or seeing 'Unexpected API response' errors
|
||||
|
||||
## [3.18.12]
|
||||
|
||||
- Fix flaky organization switching behavior in Cline provider that caused UI inconsistencies and double loading
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
---
|
||||
title: Voice Recording
|
||||
description: Record audio messages and have them transcribed using OpenAI Whisper
|
||||
---
|
||||
|
||||
# Voice Recording
|
||||
|
||||
Cline supports voice recording functionality that allows you to record audio messages directly in the chat interface. Your voice is automatically transcribed using OpenAI's Whisper model.
|
||||
|
||||
## How It Works
|
||||
|
||||
1. **Enable voice recording** in General Settings (enabled by default)
|
||||
2. **Click the microphone button** in the chat input area
|
||||
3. **Speak your message** - the button will show a red recording indicator
|
||||
4. **Click the stop button** when finished
|
||||
5. **Wait for transcription** - you'll see "[Transcribing...]" in the input field
|
||||
6. **Review and send** - the transcribed text appears in the input field
|
||||
|
||||
## Settings
|
||||
|
||||
### Enable/Disable Voice Recording
|
||||
|
||||
You can toggle voice recording on or off in the General Settings:
|
||||
|
||||
1. Open Cline settings (gear icon)
|
||||
2. Go to "General Settings"
|
||||
3. Toggle "Enable Voice Recording" checkbox
|
||||
4. The microphone button will appear/disappear based on this setting
|
||||
|
||||
## Requirements
|
||||
|
||||
### OpenAI API Key
|
||||
|
||||
Voice transcription requires an OpenAI API key to use the Whisper model. The voice feature will automatically search for any configured OpenAI key in your settings, regardless of which provider you're using for chat.
|
||||
|
||||
You can configure an OpenAI key in any of these ways:
|
||||
|
||||
- As your main chat provider (OpenAI or OpenAI Native)
|
||||
- Just having an OpenAI key saved in settings (even if using a different chat provider)
|
||||
|
||||
### Audio Recording Tools
|
||||
|
||||
Cline uses FFmpeg to capture your voice across all platforms:
|
||||
|
||||
- **macOS**: FFmpeg (via Homebrew: `brew install ffmpeg`)
|
||||
- **Linux**: FFmpeg (via apt: `sudo apt-get install ffmpeg`)
|
||||
- **Windows**: FFmpeg (via winget: `winget install Gyan.FFmpeg`)
|
||||
|
||||
If you don't have FFmpeg installed, Cline will automatically detect this and prompt you to install it with a single click.
|
||||
|
||||
## Technical Details
|
||||
|
||||
### Independent from Chat Provider
|
||||
|
||||
The voice transcription feature is completely independent from your chat provider selection. This means:
|
||||
|
||||
- You can use Claude, GPT-4, or any other model for chat
|
||||
- Voice transcription will always use OpenAI's Whisper model
|
||||
- As long as you have an OpenAI API key configured somewhere, voice will work
|
||||
|
||||
### Audio Format
|
||||
|
||||
- Records in WebM format with Opus codec for optimal compression
|
||||
- Mono channel for optimal speech recognition
|
||||
- 16kHz sample rate optimized for voice
|
||||
- 32kbps bitrate for efficient file sizes
|
||||
|
||||
### Privacy & Security
|
||||
|
||||
- Audio is recorded locally on your machine
|
||||
- Only the audio file is sent to OpenAI for transcription
|
||||
- No audio is stored after transcription
|
||||
- Temporary files are automatically cleaned up
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "No OpenAI API key found"
|
||||
|
||||
Make sure you have configured an OpenAI API key in Cline's settings. You don't need to switch to OpenAI as your chat provider - just having the key saved is enough.
|
||||
|
||||
### "Failed to start recording"
|
||||
|
||||
If you see this error, it means that the audio recording tools are not installed on your system. Cline will prompt you to install them automatically. Follow the on-screen instructions to install the required tools.
|
||||
|
||||
### "Transcription failed"
|
||||
|
||||
Check that:
|
||||
|
||||
- Your OpenAI API key is valid
|
||||
- You have sufficient OpenAI API credits
|
||||
- Your internet connection is stable
|
||||
|
||||
## API Usage
|
||||
|
||||
Voice transcription uses the OpenAI Whisper API, which is billed separately from chat completions. Check OpenAI's pricing page for current rates.
|
||||
Generated
+2
-17
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "claude-dev",
|
||||
"version": "3.18.14",
|
||||
"version": "3.18.12",
|
||||
"lockfileVersion": 2,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "claude-dev",
|
||||
"version": "3.18.14",
|
||||
"version": "3.18.12",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"@anthropic-ai/sdk": "^0.37.0",
|
||||
@@ -51,7 +51,6 @@
|
||||
"image-size": "^2.0.2",
|
||||
"isbinaryfile": "^5.0.2",
|
||||
"jschardet": "^3.1.4",
|
||||
"jwt-decode": "^4.0.0",
|
||||
"mammoth": "^1.8.0",
|
||||
"monaco-vscode-textmate-theme-converter": "^0.1.7",
|
||||
"nice-grpc": "^2.1.12",
|
||||
@@ -15786,15 +15785,6 @@
|
||||
"safe-buffer": "^5.0.1"
|
||||
}
|
||||
},
|
||||
"node_modules/jwt-decode": {
|
||||
"version": "4.0.0",
|
||||
"resolved": "https://registry.npmjs.org/jwt-decode/-/jwt-decode-4.0.0.tgz",
|
||||
"integrity": "sha512-+KJGIyHgkGuIq3IEBNftfhW/LfWhXUIY6OmyVWjliu5KH1y0fw7VQ8YndE2O4qZdMSd9SqbnC8GOcZEy0Om7sA==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/katex": {
|
||||
"version": "0.16.22",
|
||||
"resolved": "https://registry.npmjs.org/katex/-/katex-0.16.22.tgz",
|
||||
@@ -35238,11 +35228,6 @@
|
||||
"safe-buffer": "^5.0.1"
|
||||
}
|
||||
},
|
||||
"jwt-decode": {
|
||||
"version": "4.0.0",
|
||||
"resolved": "https://registry.npmjs.org/jwt-decode/-/jwt-decode-4.0.0.tgz",
|
||||
"integrity": "sha512-+KJGIyHgkGuIq3IEBNftfhW/LfWhXUIY6OmyVWjliu5KH1y0fw7VQ8YndE2O4qZdMSd9SqbnC8GOcZEy0Om7sA=="
|
||||
},
|
||||
"katex": {
|
||||
"version": "0.16.22",
|
||||
"resolved": "https://registry.npmjs.org/katex/-/katex-0.16.22.tgz",
|
||||
|
||||
+1
-2
@@ -2,7 +2,7 @@
|
||||
"name": "claude-dev",
|
||||
"displayName": "Cline",
|
||||
"description": "Autonomous coding agent right in your IDE, capable of creating/editing files, running commands, using the browser, and more with your permission every step of the way.",
|
||||
"version": "3.18.14",
|
||||
"version": "3.18.12",
|
||||
"icon": "assets/icons/icon.png",
|
||||
"engines": {
|
||||
"vscode": "^1.84.0"
|
||||
@@ -449,7 +449,6 @@
|
||||
"image-size": "^2.0.2",
|
||||
"isbinaryfile": "^5.0.2",
|
||||
"jschardet": "^3.1.4",
|
||||
"jwt-decode": "^4.0.0",
|
||||
"mammoth": "^1.8.0",
|
||||
"monaco-vscode-textmate-theme-converter": "^0.1.7",
|
||||
"nice-grpc": "^2.1.12",
|
||||
|
||||
@@ -16,6 +16,7 @@ export const serviceNameMap = {
|
||||
models: "cline.ModelsService",
|
||||
slash: "cline.SlashService",
|
||||
ui: "cline.UiService",
|
||||
voice: "cline.VoiceService",
|
||||
// Add new services here - no other code changes needed!
|
||||
}
|
||||
|
||||
@@ -27,6 +28,5 @@ export const hostServiceNameMap = {
|
||||
workspace: "host.WorkspaceService",
|
||||
env: "host.EnvService",
|
||||
window: "host.WindowService",
|
||||
diff: "host.DiffService",
|
||||
// Add new host services here
|
||||
}
|
||||
|
||||
@@ -1,25 +0,0 @@
|
||||
syntax = "proto3";
|
||||
|
||||
package host;
|
||||
option java_package = "bot.cline.host.proto";
|
||||
option java_multiple_files = true;
|
||||
|
||||
import "common.proto";
|
||||
|
||||
// Provides methods for diff views.
|
||||
service DiffService {
|
||||
// Open the diff view/editor.
|
||||
rpc openDiff(OpenDiffRequest) returns (OpenDiffResponse);
|
||||
}
|
||||
|
||||
message OpenDiffRequest {
|
||||
optional cline.Metadata metadata = 1;
|
||||
// The absolute path of the document being edited.
|
||||
optional string path = 2;
|
||||
// The new content for the file.
|
||||
optional string content = 3;
|
||||
}
|
||||
|
||||
message OpenDiffResponse {
|
||||
// TODO(sfortune) the host needs to return a unique id for the diff editor.
|
||||
}
|
||||
@@ -58,6 +58,11 @@ message ChatSettings {
|
||||
optional string open_ai_reasoning_effort = 3;
|
||||
}
|
||||
|
||||
message DictationSettings {
|
||||
bool voice_recording_enabled = 1;
|
||||
string dictation_language = 2;
|
||||
}
|
||||
|
||||
message ChatContent {
|
||||
optional string message = 1;
|
||||
repeated string images = 2;
|
||||
@@ -114,6 +119,7 @@ message UpdateSettingsRequest {
|
||||
optional bool mcp_responses_collapsed = 10;
|
||||
optional bool mcp_rich_display_enabled = 11;
|
||||
optional int64 terminal_output_line_limit = 12;
|
||||
optional DictationSettings dictation_settings = 13;
|
||||
}
|
||||
|
||||
// Complete API Configuration message
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
syntax = "proto3";
|
||||
|
||||
import "common.proto";
|
||||
|
||||
package cline;
|
||||
option java_package = "bot.cline.proto";
|
||||
option java_multiple_files = true;
|
||||
|
||||
service VoiceService {
|
||||
rpc startRecording(StartRecordingRequest) returns (RecordingResult);
|
||||
rpc stopRecording(StopRecordingRequest) returns (RecordedAudio);
|
||||
rpc getRecordingStatus(GetRecordingStatusRequest) returns (RecordingStatus);
|
||||
rpc transcribeAudio(TranscribeAudioRequest) returns (Transcription);
|
||||
}
|
||||
|
||||
// Request messages
|
||||
message StartRecordingRequest {
|
||||
Metadata metadata = 1;
|
||||
// Could add options later
|
||||
}
|
||||
|
||||
message StopRecordingRequest {
|
||||
Metadata metadata = 1;
|
||||
}
|
||||
|
||||
message GetRecordingStatusRequest {
|
||||
Metadata metadata = 1;
|
||||
}
|
||||
|
||||
message TranscribeAudioRequest {
|
||||
Metadata metadata = 1;
|
||||
string audio_base64 = 2;
|
||||
string language = 3; // optional language hint
|
||||
}
|
||||
|
||||
// Plain, reusable response types
|
||||
message RecordingResult {
|
||||
bool success = 1;
|
||||
string error = 2;
|
||||
}
|
||||
|
||||
message RecordedAudio {
|
||||
bool success = 1;
|
||||
string audio_base64 = 2;
|
||||
string error = 3;
|
||||
}
|
||||
|
||||
message RecordingStatus {
|
||||
bool is_recording = 1;
|
||||
double duration_seconds = 2;
|
||||
string error = 3;
|
||||
}
|
||||
|
||||
message Transcription {
|
||||
string text = 1;
|
||||
string error = 2;
|
||||
}
|
||||
@@ -27,7 +27,7 @@ export class ClineHandler implements ApiHandler {
|
||||
private _authService: AuthService
|
||||
private client: OpenAI | undefined
|
||||
// TODO: replace this with a global API Host
|
||||
private readonly _baseUrl = "https://api.cline.bot"
|
||||
private readonly _baseUrl = "http://localhost:7777"
|
||||
// private readonly _baseUrl = "https://core-api.staging.int.cline.bot"
|
||||
// private readonly _baseUrl = "http://localhost:7777"
|
||||
lastGenerationId?: string
|
||||
@@ -182,7 +182,7 @@ export class ClineHandler implements ApiHandler {
|
||||
}
|
||||
} catch (error) {
|
||||
if (error.code === "ERR_BAD_REQUEST" || error.status === 401) {
|
||||
throw new Error("Unauthorized: Please sign in to Cline before trying again.") // match with webview-ui/src/components/chat/ChatRow.tsx
|
||||
throw new Error("Unauthorized: Please sign in to Cline before trying again.")
|
||||
} else if (error.code === "insufficient_credits" || error.status === 402) {
|
||||
throw new Error(error.error ? JSON.stringify(error.error) : "Insufficient credits or unknown error.")
|
||||
}
|
||||
|
||||
@@ -6,7 +6,7 @@ import * as path from "path"
|
||||
import { FileContextTracker } from "./FileContextTracker"
|
||||
import * as diskModule from "@core/storage/disk"
|
||||
import type { TaskMetadata, FileMetadataEntry } from "./ContextTrackerTypes"
|
||||
import type { DiffViewProviderCreator, WebviewProviderCreator } from "@/hosts/host-providers"
|
||||
import type { WebviewProviderCreator } from "@/hosts/host-providers"
|
||||
import * as hostProviders from "@hosts/host-providers"
|
||||
import { vscodeHostBridgeClient } from "@/hosts/vscode/client/host-grpc-client"
|
||||
|
||||
@@ -53,11 +53,7 @@ describe("FileContextTracker", () => {
|
||||
mockTaskMetadata = { files_in_context: [], model_usage: [] }
|
||||
getTaskMetadataStub = sandbox.stub(diskModule, "getTaskMetadata").resolves(mockTaskMetadata)
|
||||
saveTaskMetadataStub = sandbox.stub(diskModule, "saveTaskMetadata").resolves()
|
||||
hostProviders.initializeHostProviders(
|
||||
((_) => {}) as WebviewProviderCreator,
|
||||
(() => {}) as DiffViewProviderCreator,
|
||||
vscodeHostBridgeClient,
|
||||
)
|
||||
hostProviders.initializeHostProviders(((_) => {}) as WebviewProviderCreator, vscodeHostBridgeClient)
|
||||
|
||||
// Create tracker instance
|
||||
taskId = "test-task-id"
|
||||
|
||||
@@ -74,7 +74,7 @@ export class Controller {
|
||||
)
|
||||
this.accountService = ClineAccountService.getInstance()
|
||||
this.authService = AuthService.getInstance(context)
|
||||
this.authService.restoreRefreshTokenAndRetrieveAuthInfo()
|
||||
this.authService.restoreAuthToken()
|
||||
|
||||
// Clean up legacy checkpoints
|
||||
cleanupLegacyCheckpoints(this.context.globalStorageUri.fsPath, this.outputChannel).catch((error) => {
|
||||
@@ -847,6 +847,7 @@ export class Controller {
|
||||
welcomeViewCompleted,
|
||||
mcpResponsesCollapsed,
|
||||
terminalOutputLineLimit,
|
||||
dictationSettings,
|
||||
} = await getAllExtensionState(this.context)
|
||||
|
||||
// Get current mode using helper function
|
||||
@@ -905,6 +906,7 @@ export class Controller {
|
||||
welcomeViewCompleted: welcomeViewCompleted as boolean, // Can be undefined but is set to either true or false by the migration that runs on extension launch in extension.ts
|
||||
mcpResponsesCollapsed,
|
||||
terminalOutputLineLimit,
|
||||
dictationSettings,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@ import { updateApiConfiguration } from "../../storage/state"
|
||||
import { buildApiHandler } from "../../../api"
|
||||
import { convertProtoApiConfigurationToApiConfiguration } from "../../../shared/proto-conversions/state/settings-conversion"
|
||||
import { convertProtoChatSettingsToChatSettings } from "../../../shared/proto-conversions/state/chat-settings-conversion"
|
||||
import { convertProtoDictationSettingsToDictationSettings } from "../../../shared/proto-conversions/state/dictation-settings-conversion"
|
||||
import { TelemetrySetting } from "@/shared/TelemetrySetting"
|
||||
|
||||
/**
|
||||
@@ -73,6 +74,12 @@ export async function updateSettings(controller: Controller, request: UpdateSett
|
||||
}
|
||||
}
|
||||
|
||||
// Update dictation settings
|
||||
if (request.dictationSettings) {
|
||||
const dictationSettings = convertProtoDictationSettingsToDictationSettings(request.dictationSettings)
|
||||
await controller.context.globalState.update("dictationSettings", dictationSettings)
|
||||
}
|
||||
|
||||
// Update terminal timeout setting
|
||||
if (request.shellIntegrationTimeout !== undefined) {
|
||||
await controller.context.globalState.update("shellIntegrationTimeout", Number(request.shellIntegrationTimeout))
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
import { RecordingStatus } from "@shared/proto/voice"
|
||||
import { audioRecordingService } from "@/services/dictation/AudioRecordingService"
|
||||
import { VoiceMethodHandler } from "./index"
|
||||
|
||||
/**
|
||||
* Gets the current recording status
|
||||
* @returns RecordingStatus with current status
|
||||
*/
|
||||
export const getRecordingStatus: VoiceMethodHandler = async (): Promise<RecordingStatus> => {
|
||||
try {
|
||||
const status = audioRecordingService.getRecordingStatus()
|
||||
|
||||
return RecordingStatus.create({
|
||||
isRecording: status.isRecording,
|
||||
durationSeconds: status.durationSeconds,
|
||||
error: status.error || "",
|
||||
})
|
||||
} catch (error) {
|
||||
console.error("Error getting recording status:", error)
|
||||
return RecordingStatus.create({
|
||||
isRecording: false,
|
||||
durationSeconds: 0,
|
||||
error: error instanceof Error ? error.message : "Unknown error occurred",
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
import { Controller } from ".."
|
||||
import { StartRecordingRequest, RecordingResult } from "@shared/proto/voice"
|
||||
import { audioRecordingService } from "@/services/dictation/AudioRecordingService"
|
||||
import { telemetryService } from "@services/posthog/telemetry/TelemetryService"
|
||||
import { VoiceMethodHandler } from "./index"
|
||||
import * as vscode from "vscode"
|
||||
|
||||
/**
|
||||
* Starts audio recording using the Extension Host
|
||||
* @param controller The controller instance
|
||||
* @param request StartRecordingRequest
|
||||
* @returns RecordingResult with success status
|
||||
*/
|
||||
export const startRecording: VoiceMethodHandler = async (
|
||||
controller: Controller,
|
||||
_request: StartRecordingRequest,
|
||||
): Promise<RecordingResult> => {
|
||||
const taskId = controller.task?.taskId
|
||||
|
||||
try {
|
||||
const result = await audioRecordingService.startRecording()
|
||||
|
||||
// Capture telemetry for recording start
|
||||
if (result.success) {
|
||||
telemetryService.captureVoiceRecordingStarted(taskId, process.platform)
|
||||
}
|
||||
|
||||
return RecordingResult.create({
|
||||
success: result.success,
|
||||
error: result.error || "",
|
||||
})
|
||||
} catch (error) {
|
||||
console.error("Error starting recording:", error)
|
||||
const errorMessage = error instanceof Error ? error.message : "Unknown error occurred"
|
||||
vscode.window.showErrorMessage(`Voice recording error: ${errorMessage}`)
|
||||
return RecordingResult.create({
|
||||
success: false,
|
||||
error: errorMessage,
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
import { Controller } from ".."
|
||||
import { StopRecordingRequest, RecordedAudio } from "@shared/proto/voice"
|
||||
import { audioRecordingService } from "@/services/dictation/AudioRecordingService"
|
||||
import { telemetryService } from "@services/posthog/telemetry/TelemetryService"
|
||||
import { VoiceMethodHandler } from "./index"
|
||||
|
||||
/**
|
||||
* Stops audio recording and returns the recorded audio
|
||||
* @param controller The controller instance
|
||||
* @param request StopRecordingRequest
|
||||
* @returns RecordedAudio with audio data
|
||||
*/
|
||||
export const stopRecording: VoiceMethodHandler = async (
|
||||
controller: Controller,
|
||||
_request: StopRecordingRequest,
|
||||
): Promise<RecordedAudio> => {
|
||||
const taskId = controller.task?.taskId
|
||||
const recordingStatus = audioRecordingService.getRecordingStatus()
|
||||
const recordingDuration = recordingStatus.durationSeconds * 1000 // Convert to milliseconds
|
||||
|
||||
try {
|
||||
const result = await audioRecordingService.stopRecording()
|
||||
|
||||
// Capture telemetry for recording stop
|
||||
telemetryService.captureVoiceRecordingStopped(taskId, recordingDuration, result.success, process.platform)
|
||||
|
||||
// Calculate audio size if available
|
||||
|
||||
return RecordedAudio.create({
|
||||
success: result.success,
|
||||
audioBase64: result.audioBase64 || "",
|
||||
error: result.error || "",
|
||||
})
|
||||
} catch (error) {
|
||||
console.error("Error stopping recording:", error)
|
||||
|
||||
// Capture telemetry for recording failure
|
||||
telemetryService.captureVoiceRecordingStopped(taskId, recordingDuration, false, process.platform)
|
||||
|
||||
return RecordedAudio.create({
|
||||
success: false,
|
||||
audioBase64: "",
|
||||
error: error instanceof Error ? error.message : "Unknown error occurred",
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
import { Controller } from ".."
|
||||
import { TranscribeAudioRequest, Transcription } from "@shared/proto/voice"
|
||||
import { voiceTranscriptionService } from "@/services/dictation/VoiceTranscriptionService"
|
||||
import { telemetryService } from "@services/posthog/telemetry/TelemetryService"
|
||||
import { VoiceMethodHandler } from "./index"
|
||||
import * as vscode from "vscode"
|
||||
|
||||
/**
|
||||
* Transcribes audio using Cline transcription service
|
||||
* @param controller The controller instance
|
||||
* @param request TranscribeAudioRequest containing base64 audio data
|
||||
* @returns Transcription with transcribed text or error
|
||||
*/
|
||||
export const transcribeAudio: VoiceMethodHandler = async (
|
||||
controller: Controller,
|
||||
request: TranscribeAudioRequest,
|
||||
): Promise<Transcription> => {
|
||||
const taskId = controller.task?.taskId
|
||||
const startTime = Date.now()
|
||||
|
||||
// Calculate audio size from base64
|
||||
const audioSizeBytes = Math.ceil((request.audioBase64.length * 3) / 4)
|
||||
|
||||
// Capture telemetry for transcription start
|
||||
telemetryService.captureVoiceTranscriptionStarted(taskId, audioSizeBytes, request.language || "en")
|
||||
|
||||
try {
|
||||
// Transcribe the audio
|
||||
const result = await voiceTranscriptionService.transcribeAudio(request.audioBase64, request.language || undefined)
|
||||
const durationMs = Date.now() - startTime
|
||||
|
||||
// Handle transcription result
|
||||
if (result.error) {
|
||||
// Determine error type for telemetry
|
||||
let errorType = "api_error"
|
||||
if (result.error.includes("Authentication failed")) {
|
||||
errorType = "invalid_jwt_token"
|
||||
} else if (result.error.includes("Insufficient credits")) {
|
||||
errorType = "insufficient_credits"
|
||||
} else if (result.error.includes("Invalid audio format")) {
|
||||
errorType = "invalid_audio_format"
|
||||
} else if (result.error.includes("Cannot connect")) {
|
||||
errorType = "connection_error"
|
||||
} else if (result.error.includes("Network error")) {
|
||||
errorType = "network_error"
|
||||
}
|
||||
|
||||
// Capture telemetry for transcription error
|
||||
telemetryService.captureVoiceTranscriptionError(taskId, errorType, result.error, durationMs)
|
||||
|
||||
// Show error notification if transcription failed
|
||||
if (result.error.includes("Authentication failed")) {
|
||||
vscode.window.showErrorMessage("Authentication failed. ")
|
||||
} else if (result.error.includes("Insufficient credits")) {
|
||||
vscode.window.showWarningMessage("Insufficient credits for transcription service.")
|
||||
} else if (result.error.includes("Cannot connect")) {
|
||||
vscode.window.showErrorMessage("Cannot connect to transcription service. ")
|
||||
} else {
|
||||
vscode.window.showErrorMessage(`Voice transcription failed: ${result.error}`)
|
||||
}
|
||||
} else if (result.text) {
|
||||
// Capture telemetry for successful transcription
|
||||
telemetryService.captureVoiceTranscriptionCompleted(taskId, result.text.length, durationMs, request.language || "en")
|
||||
}
|
||||
|
||||
// Return the response
|
||||
return Transcription.create({
|
||||
text: result.text || "",
|
||||
error: result.error || "",
|
||||
})
|
||||
} catch (error) {
|
||||
console.error("Error transcribing audio:", error)
|
||||
const durationMs = Date.now() - startTime
|
||||
const errorMessage = error instanceof Error ? error.message : "Unknown error occurred"
|
||||
|
||||
// Capture telemetry for unexpected error
|
||||
telemetryService.captureVoiceTranscriptionError(taskId, "unexpected_error", errorMessage, durationMs)
|
||||
|
||||
return Transcription.create({
|
||||
text: "",
|
||||
error: errorMessage,
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -80,6 +80,7 @@ export type GlobalStateKey =
|
||||
| "claudeCodePath"
|
||||
// Settings around plan/act and ephemeral model configuration
|
||||
| "chatSettings"
|
||||
| "dictationSettings"
|
||||
| "mode"
|
||||
// Current active model configuration (per workspace)
|
||||
| "apiProvider"
|
||||
|
||||
@@ -2,12 +2,14 @@ import * as vscode from "vscode"
|
||||
import { DEFAULT_CHAT_SETTINGS } from "@shared/ChatSettings"
|
||||
import { DEFAULT_BROWSER_SETTINGS } from "@shared/BrowserSettings"
|
||||
import { DEFAULT_AUTO_APPROVAL_SETTINGS } from "@shared/AutoApprovalSettings"
|
||||
import { DEFAULT_DICTATION_SETTINGS } from "@shared/DictationSettings"
|
||||
import { GlobalStateKey, LocalStateKey, SecretKey } from "./state-keys"
|
||||
import { ApiConfiguration, ApiProvider, BedrockModelId, ModelInfo } from "@shared/api"
|
||||
import { HistoryItem } from "@shared/HistoryItem"
|
||||
import { AutoApprovalSettings } from "@shared/AutoApprovalSettings"
|
||||
import { BrowserSettings } from "@shared/BrowserSettings"
|
||||
import { StoredChatSettings } from "@shared/ChatSettings"
|
||||
import { DictationSettings } from "@shared/DictationSettings"
|
||||
import { TelemetrySetting } from "@shared/TelemetrySetting"
|
||||
import { UserInfo } from "@shared/UserInfo"
|
||||
import { ClineRulesToggles } from "@shared/cline-rules"
|
||||
@@ -293,6 +295,7 @@ export async function getAllExtensionState(context: vscode.ExtensionContext) {
|
||||
previousModeAwsBedrockCustomModelBaseId,
|
||||
previousModeSapAiCoreModelId,
|
||||
sapAiCoreModelId,
|
||||
dictationSettings,
|
||||
] = await Promise.all([
|
||||
getGlobalState(context, "chatSettings") as Promise<StoredChatSettings | undefined>,
|
||||
getGlobalState(context, "mode") as Promise<"plan" | "act" | undefined>,
|
||||
@@ -325,6 +328,7 @@ export async function getAllExtensionState(context: vscode.ExtensionContext) {
|
||||
getGlobalState(context, "previousModeAwsBedrockCustomModelBaseId") as Promise<BedrockModelId | undefined>,
|
||||
getGlobalState(context, "previousModeSapAiCoreModelId") as Promise<string | undefined>,
|
||||
getGlobalState(context, "sapAiCoreModelId") as Promise<string | undefined>,
|
||||
getGlobalState(context, "dictationSettings") as Promise<DictationSettings | undefined>,
|
||||
])
|
||||
|
||||
const processingStart = performance.now()
|
||||
@@ -474,6 +478,7 @@ export async function getAllExtensionState(context: vscode.ExtensionContext) {
|
||||
terminalOutputLineLimit: terminalOutputLineLimit ?? 500,
|
||||
defaultTerminalProfile: defaultTerminalProfile ?? "default",
|
||||
globalWorkflowToggles: globalWorkflowToggles || {},
|
||||
dictationSettings: dictationSettings || DEFAULT_DICTATION_SETTINGS,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -82,7 +82,6 @@ import { MessageStateHandler } from "./message-state"
|
||||
import { TaskState } from "./TaskState"
|
||||
import { ToolExecutor } from "./ToolExecutor"
|
||||
import { formatErrorWithStatusCode, updateApiReqMsg } from "./utils"
|
||||
import { createDiffViewProvider, getHostBridgeProvider } from "@/hosts/host-providers"
|
||||
|
||||
export const USE_EXPERIMENTAL_CLAUDE4_FEATURES = false
|
||||
|
||||
@@ -175,7 +174,7 @@ export class Task {
|
||||
this.urlContentFetcher = new UrlContentFetcher(context)
|
||||
this.browserSession = new BrowserSession(context, browserSettings)
|
||||
this.contextManager = new ContextManager()
|
||||
this.diffViewProvider = createDiffViewProvider()
|
||||
this.diffViewProvider = new DiffViewProvider()
|
||||
this.autoApprovalSettings = autoApprovalSettings
|
||||
this.browserSettings = browserSettings
|
||||
this.chatSettings = chatSettings
|
||||
@@ -1724,7 +1723,7 @@ export class Task {
|
||||
await this.messageStateHandler.updateClineMessage(lastApiReqStartedIndex, {
|
||||
text: JSON.stringify({
|
||||
...currentApiReqInfo, // Spread the modified info (with retryStatus removed)
|
||||
// cancelReason: "retries_exhausted", // Indicate that automatic retries failed
|
||||
cancelReason: "retries_exhausted", // Indicate that automatic retries failed
|
||||
streamingFailedMessage: errorMessage,
|
||||
} satisfies ClineApiReqInfo),
|
||||
})
|
||||
|
||||
+2
-6
@@ -37,7 +37,6 @@ import { VscodeWebviewProvider } from "./core/webview/VscodeWebviewProvider"
|
||||
import { ExtensionContext } from "vscode"
|
||||
import { AuthService } from "./services/auth/AuthService"
|
||||
import { writeTextToClipboard, readTextFromClipboard } from "@/utils/env"
|
||||
import { VscodeDiffViewProvider } from "./integrations/editor/VscodeDiffViewProvider"
|
||||
import { getHostBridgeProvider } from "@hosts/host-providers"
|
||||
import { ShowMessageRequest, ShowMessageType } from "./shared/proto/host/window"
|
||||
/*
|
||||
@@ -716,7 +715,7 @@ export async function activate(context: vscode.ExtensionContext) {
|
||||
context.subscriptions.push(
|
||||
context.secrets.onDidChange((event) => {
|
||||
if (event.key === "clineAccountId") {
|
||||
AuthService.getInstance(context)?.restoreRefreshTokenAndRetrieveAuthInfo()
|
||||
AuthService.getInstance(context)?.restoreAuthToken()
|
||||
}
|
||||
}),
|
||||
)
|
||||
@@ -730,10 +729,7 @@ function maybeSetupHostProviders(context: ExtensionContext) {
|
||||
const createWebview = function (type: WebviewProviderType) {
|
||||
return new VscodeWebviewProvider(context, outputChannel, type)
|
||||
}
|
||||
const createDiffView = function () {
|
||||
return new VscodeDiffViewProvider()
|
||||
}
|
||||
hostProviders.initializeHostProviders(createWebview, createDiffView, vscodeHostBridgeClient)
|
||||
hostProviders.initializeHostProviders(createWebview, vscodeHostBridgeClient)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
// AUTO-GENERATED FILE - DO NOT MODIFY DIRECTLY
|
||||
// Generated by proto/build-proto.js
|
||||
|
||||
import { createChannel, createClientFactory, Client } from "nice-grpc"
|
||||
import { Channel } from "@grpc/grpc-js"
|
||||
|
||||
const channel = createChannel("localhost:50051") // Replace with your server address
|
||||
|
||||
export function createGrpcClient<T extends {}>(definition: any): Client<T> {
|
||||
return createClientFactory().create(definition, channel)
|
||||
}
|
||||
@@ -3,7 +3,6 @@ import {
|
||||
WorkspaceServiceClientInterface,
|
||||
EnvServiceClientInterface,
|
||||
WindowServiceClientInterface,
|
||||
DiffServiceClientInterface,
|
||||
} from "@generated/hosts/host-bridge-client-types"
|
||||
|
||||
/**
|
||||
@@ -14,7 +13,6 @@ export interface HostBridgeClientProvider {
|
||||
workspaceClient: WorkspaceServiceClientInterface
|
||||
envClient: EnvServiceClientInterface
|
||||
windowClient: WindowServiceClientInterface
|
||||
diffClient: DiffServiceClientInterface
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -1,28 +1,22 @@
|
||||
import { WebviewProvider } from "@core/webview"
|
||||
import { HostBridgeClientProvider } from "./host-provider-types"
|
||||
import { WebviewProviderType } from "@/shared/webview/types"
|
||||
import { DiffViewProvider } from "@/integrations/editor/DiffViewProvider"
|
||||
|
||||
/**
|
||||
* A function that creates WebviewProvider instances
|
||||
*/
|
||||
export type WebviewProviderCreator = (providerType: WebviewProviderType) => WebviewProvider
|
||||
|
||||
export type DiffViewProviderCreator = () => DiffViewProvider
|
||||
|
||||
let _webviewProviderCreator: WebviewProviderCreator | undefined
|
||||
let _diffViewProviderCreator: DiffViewProviderCreator | undefined
|
||||
let _hostBridgeProvider: HostBridgeClientProvider | undefined
|
||||
|
||||
export var isSetup: boolean = false
|
||||
|
||||
export function initializeHostProviders(
|
||||
webviewProviderCreator: WebviewProviderCreator,
|
||||
diffViewProviderCreator: DiffViewProviderCreator,
|
||||
hostBridgeProvider: HostBridgeClientProvider,
|
||||
) {
|
||||
_webviewProviderCreator = webviewProviderCreator
|
||||
_diffViewProviderCreator = diffViewProviderCreator
|
||||
_hostBridgeProvider = hostBridgeProvider
|
||||
isSetup = true
|
||||
}
|
||||
@@ -34,13 +28,6 @@ export function createWebviewProvider(providerType: WebviewProviderType): Webvie
|
||||
return _webviewProviderCreator(providerType)
|
||||
}
|
||||
|
||||
export function createDiffViewProvider(): DiffViewProvider {
|
||||
if (!_diffViewProviderCreator) {
|
||||
throw Error("Host providers not initialized")
|
||||
}
|
||||
return _diffViewProviderCreator()
|
||||
}
|
||||
|
||||
export function getHostBridgeProvider(): HostBridgeClientProvider {
|
||||
if (!_hostBridgeProvider) {
|
||||
throw Error("Host providers not initialized")
|
||||
|
||||
@@ -7,5 +7,4 @@ export const vscodeHostBridgeClient: HostBridgeClientProvider = {
|
||||
workspaceClient: createGrpcClient(host.WorkspaceServiceDefinition),
|
||||
envClient: createGrpcClient(host.EnvServiceDefinition),
|
||||
windowClient: createGrpcClient(host.WindowServiceDefinition),
|
||||
diffClient: createGrpcClient(host.DiffServiceDefinition),
|
||||
}
|
||||
|
||||
@@ -1,5 +0,0 @@
|
||||
import { OpenDiffRequest, OpenDiffResponse } from "@/shared/proto/index.host"
|
||||
|
||||
export async function openDiff(_request: OpenDiffRequest): Promise<OpenDiffResponse> {
|
||||
throw new Error("diffService.openDiff is not supported. Use the VscodeDiffViewProvider.")
|
||||
}
|
||||
@@ -10,30 +10,29 @@ import { diagnosticsToProblemsString, getNewDiagnostics } from "../diagnostics"
|
||||
import { detectEncoding } from "../misc/extract-text"
|
||||
import * as iconv from "iconv-lite"
|
||||
import { getHostBridgeProvider } from "@/hosts/host-providers"
|
||||
import { ShowTextDocumentRequest, ShowTextDocumentOptions } from "@/shared/proto/host/window"
|
||||
import { ShowTextDocumentRequest, ShowTextDocumentOptions, TextEditorInfo } from "@/shared/proto/host/window"
|
||||
|
||||
export const DIFF_VIEW_URI_SCHEME = "cline-diff"
|
||||
|
||||
export abstract class DiffViewProvider {
|
||||
export class DiffViewProvider {
|
||||
editType?: "create" | "modify"
|
||||
isEditing = false
|
||||
originalContent: string | undefined
|
||||
private createdDirs: string[] = []
|
||||
protected documentWasOpen = false
|
||||
protected relPath?: string
|
||||
protected absolutePath?: string
|
||||
protected fileEncoding: string = "utf8"
|
||||
private streamedLines: string[] = []
|
||||
private documentWasOpen = false
|
||||
private relPath?: string
|
||||
private absolutePath?: string
|
||||
private newContent?: string
|
||||
|
||||
protected activeDiffEditor?: vscode.TextEditor
|
||||
protected fadedOverlayController?: DecorationController
|
||||
protected activeLineController?: DecorationController
|
||||
protected preDiagnostics: [vscode.Uri, vscode.Diagnostic[]][] = []
|
||||
private activeDiffEditor?: vscode.TextEditor
|
||||
private fadedOverlayController?: DecorationController
|
||||
private activeLineController?: DecorationController
|
||||
private streamedLines: string[] = []
|
||||
private preDiagnostics: [vscode.Uri, vscode.Diagnostic[]][] = []
|
||||
private fileEncoding: string = "utf8"
|
||||
|
||||
constructor() {}
|
||||
|
||||
public async open(relPath: string): Promise<void> {
|
||||
async open(relPath: string): Promise<void> {
|
||||
this.isEditing = true
|
||||
this.relPath = relPath
|
||||
this.absolutePath = path.resolve(await getCwd(), relPath)
|
||||
@@ -66,20 +65,6 @@ export abstract class DiffViewProvider {
|
||||
this.streamedLines = []
|
||||
}
|
||||
|
||||
/**
|
||||
* Opens a diff editor or viewer for the current file.
|
||||
*
|
||||
* This abstract method must be implemented by subclasses to create and display
|
||||
* a diff editor or viewer that shows the difference between the original and
|
||||
* modified content.
|
||||
*
|
||||
* Called automatically by the `open` method after ensuring the file exists and
|
||||
* creating any necessary directories.
|
||||
*
|
||||
* @returns A promise that resolves when the diff editor is open and ready
|
||||
*/
|
||||
protected abstract openDiffEditor(): Promise<void>
|
||||
|
||||
async update(
|
||||
accumulatedContent: string,
|
||||
isFinal: boolean,
|
||||
@@ -342,6 +327,88 @@ export abstract class DiffViewProvider {
|
||||
}
|
||||
}
|
||||
|
||||
private async openDiffEditor(): Promise<void> {
|
||||
if (!this.absolutePath) {
|
||||
throw new Error("No file path set")
|
||||
}
|
||||
// get diagnostics before editing the file, we'll compare to diagnostics after editing to see if cline needs to fix anything
|
||||
this.preDiagnostics = vscode.languages.getDiagnostics()
|
||||
|
||||
// if the file was already open, close it (must happen after showing the diff view since if it's the only tab the column will close)
|
||||
this.documentWasOpen = false
|
||||
// close the tab if it's open (it's already been saved)
|
||||
const tabs = vscode.window.tabGroups.all
|
||||
.map((tg) => tg.tabs)
|
||||
.flat()
|
||||
.filter((tab) => tab.input instanceof vscode.TabInputText && arePathsEqual(tab.input.uri.fsPath, this.absolutePath))
|
||||
for (const tab of tabs) {
|
||||
if (!tab.isDirty) {
|
||||
await vscode.window.tabGroups.close(tab)
|
||||
}
|
||||
this.documentWasOpen = true
|
||||
}
|
||||
|
||||
const uri = vscode.Uri.file(this.absolutePath)
|
||||
// If this diff editor is already open (ie if a previous write file was interrupted) then we should activate that instead of opening a new diff
|
||||
const diffTab = vscode.window.tabGroups.all
|
||||
.flatMap((group) => group.tabs)
|
||||
.find(
|
||||
(tab) =>
|
||||
tab.input instanceof vscode.TabInputTextDiff &&
|
||||
tab.input?.original?.scheme === DIFF_VIEW_URI_SCHEME &&
|
||||
arePathsEqual(tab.input.modified.fsPath, uri.fsPath),
|
||||
)
|
||||
if (diffTab && diffTab.input instanceof vscode.TabInputTextDiff) {
|
||||
const editorInfo = await getHostBridgeProvider().windowClient.showTextDocument(
|
||||
ShowTextDocumentRequest.create({
|
||||
path: diffTab.input.modified.fsPath,
|
||||
options: ShowTextDocumentOptions.create({
|
||||
preserveFocus: true,
|
||||
}),
|
||||
}),
|
||||
)
|
||||
// Find the editor that matches the returned path
|
||||
const editor = vscode.window.visibleTextEditors.find((e) => e.document.uri.fsPath === editorInfo.documentPath)
|
||||
if (!editor) {
|
||||
throw new Error("Failed to find opened text editor")
|
||||
}
|
||||
this.activeDiffEditor = editor
|
||||
} else {
|
||||
// Open new diff editor
|
||||
this.activeDiffEditor = await new Promise<vscode.TextEditor>((resolve, reject) => {
|
||||
const fileName = path.basename(uri.fsPath)
|
||||
const fileExists = this.editType === "modify"
|
||||
const disposable = vscode.window.onDidChangeActiveTextEditor((editor) => {
|
||||
if (editor && arePathsEqual(editor.document.uri.fsPath, uri.fsPath)) {
|
||||
disposable.dispose()
|
||||
resolve(editor)
|
||||
}
|
||||
})
|
||||
vscode.commands.executeCommand(
|
||||
"vscode.diff",
|
||||
vscode.Uri.parse(`${DIFF_VIEW_URI_SCHEME}:${fileName}`).with({
|
||||
query: Buffer.from(this.originalContent ?? "").toString("base64"),
|
||||
}),
|
||||
uri,
|
||||
`${fileName}: ${fileExists ? "Original ↔ Cline's Changes" : "New File"} (Editable)`,
|
||||
{
|
||||
preserveFocus: true,
|
||||
},
|
||||
)
|
||||
// This may happen on very slow machines ie project idx
|
||||
setTimeout(() => {
|
||||
disposable.dispose()
|
||||
reject(new Error("Failed to open diff editor, please try again..."))
|
||||
}, 10_000)
|
||||
})
|
||||
}
|
||||
|
||||
this.fadedOverlayController = new DecorationController("fadedOverlay", this.activeDiffEditor)
|
||||
this.activeLineController = new DecorationController("activeLine", this.activeDiffEditor)
|
||||
// Apply faded overlay to all lines initially
|
||||
this.fadedOverlayController.addLines(0, this.activeDiffEditor.document.lineCount)
|
||||
}
|
||||
|
||||
private scrollEditorToLine(line: number) {
|
||||
if (this.activeDiffEditor) {
|
||||
const scrollLine = line + 4
|
||||
|
||||
@@ -1,80 +0,0 @@
|
||||
import { arePathsEqual } from "@/utils/path"
|
||||
import * as path from "path"
|
||||
import * as vscode from "vscode"
|
||||
import { DecorationController } from "./DecorationController"
|
||||
import { DIFF_VIEW_URI_SCHEME, DiffViewProvider } from "./DiffViewProvider"
|
||||
|
||||
export class VscodeDiffViewProvider extends DiffViewProvider {
|
||||
override async openDiffEditor(): Promise<void> {
|
||||
if (!this.absolutePath) {
|
||||
throw new Error("No file path set")
|
||||
}
|
||||
// get diagnostics before editing the file, we'll compare to diagnostics after editing to see if cline needs to fix anything
|
||||
this.preDiagnostics = vscode.languages.getDiagnostics()
|
||||
|
||||
// if the file was already open, close it (must happen after showing the diff view since if it's the only tab the column will close)
|
||||
this.documentWasOpen = false
|
||||
// close the tab if it's open (it's already been saved)
|
||||
const tabs = vscode.window.tabGroups.all
|
||||
.map((tg) => tg.tabs)
|
||||
.flat()
|
||||
.filter((tab) => tab.input instanceof vscode.TabInputText && arePathsEqual(tab.input.uri.fsPath, this.absolutePath))
|
||||
for (const tab of tabs) {
|
||||
if (!tab.isDirty) {
|
||||
await vscode.window.tabGroups.close(tab)
|
||||
}
|
||||
this.documentWasOpen = true
|
||||
}
|
||||
|
||||
const uri = vscode.Uri.file(this.absolutePath)
|
||||
// If this diff editor is already open (ie if a previous write file was interrupted) then we should activate that instead of opening a new diff
|
||||
const diffTab = vscode.window.tabGroups.all
|
||||
.flatMap((group) => group.tabs)
|
||||
.find(
|
||||
(tab) =>
|
||||
tab.input instanceof vscode.TabInputTextDiff &&
|
||||
tab.input?.original?.scheme === DIFF_VIEW_URI_SCHEME &&
|
||||
arePathsEqual(tab.input.modified.fsPath, uri.fsPath),
|
||||
)
|
||||
|
||||
if (diffTab && diffTab.input instanceof vscode.TabInputTextDiff) {
|
||||
// Use already open diff editor.
|
||||
this.activeDiffEditor = await vscode.window.showTextDocument(diffTab.input.modified, {
|
||||
preserveFocus: true,
|
||||
})
|
||||
} else {
|
||||
// Open new diff editor.
|
||||
this.activeDiffEditor = await new Promise<vscode.TextEditor>((resolve, reject) => {
|
||||
const fileName = path.basename(uri.fsPath)
|
||||
const fileExists = this.editType === "modify"
|
||||
const disposable = vscode.window.onDidChangeActiveTextEditor((editor) => {
|
||||
if (editor && arePathsEqual(editor.document.uri.fsPath, uri.fsPath)) {
|
||||
disposable.dispose()
|
||||
resolve(editor)
|
||||
}
|
||||
})
|
||||
vscode.commands.executeCommand(
|
||||
"vscode.diff",
|
||||
vscode.Uri.parse(`${DIFF_VIEW_URI_SCHEME}:${fileName}`).with({
|
||||
query: Buffer.from(this.originalContent ?? "").toString("base64"),
|
||||
}),
|
||||
uri,
|
||||
`${fileName}: ${fileExists ? "Original ↔ Cline's Changes" : "New File"} (Editable)`,
|
||||
{
|
||||
preserveFocus: true,
|
||||
},
|
||||
)
|
||||
// This may happen on very slow machines ie project idx
|
||||
setTimeout(() => {
|
||||
disposable.dispose()
|
||||
reject(new Error("Failed to open diff editor, please try again..."))
|
||||
}, 10_000)
|
||||
})
|
||||
}
|
||||
|
||||
this.fadedOverlayController = new DecorationController("fadedOverlay", this.activeDiffEditor)
|
||||
this.activeLineController = new DecorationController("activeLine", this.activeDiffEditor)
|
||||
// Apply faded overlay to all lines initially
|
||||
this.fadedOverlayController.addLines(0, this.activeDiffEditor.document.lineCount)
|
||||
}
|
||||
}
|
||||
@@ -13,7 +13,7 @@ export class ClineAccountService {
|
||||
private static instance: ClineAccountService
|
||||
private _authService: AuthService
|
||||
// TODO: replace this with a global API Host
|
||||
private readonly _baseUrl = "https://api.cline.bot"
|
||||
private readonly _baseUrl = "http://localhost:7777"
|
||||
// private readonly _baseUrl = "https://core-api.staging.int.cline.bot"
|
||||
// private readonly _baseUrl = "http://localhost:7777"
|
||||
|
||||
@@ -274,8 +274,26 @@ export class ClineAccountService {
|
||||
console.error("Error switching account:", error)
|
||||
throw error
|
||||
} finally {
|
||||
// After user switches account, we will force a refresh of the id token by calling this function that restores the refresh token and retrieves new auth info
|
||||
await this._authService.restoreRefreshTokenAndRetrieveAuthInfo()
|
||||
// Request a new authentication token
|
||||
await this._authService.refreshAuth()
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Transcribes audio using the Cline transcription service
|
||||
* @param audioBase64 - Base64 encoded audio data
|
||||
* @param language - Optional language hint for transcription
|
||||
* @returns Promise with transcribed text or error
|
||||
*/
|
||||
async transcribeAudio(audioBase64: string, language?: string): Promise<{ text: string }> {
|
||||
const response = await this.authenticatedRequest<{ text: string }>(`/api/v1/chat/transcriptions`, {
|
||||
method: "POST",
|
||||
data: {
|
||||
audioData: audioBase64,
|
||||
language: language || "en",
|
||||
},
|
||||
})
|
||||
|
||||
return response
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
import vscode from "vscode"
|
||||
import crypto from "crypto"
|
||||
import { EmptyRequest, String } from "../../shared/proto/common"
|
||||
import { AuthState, UserInfo } from "../../shared/proto/account"
|
||||
import { AuthState } from "../../shared/proto/account"
|
||||
import { StreamingResponseHandler, getRequestRegistry } from "@/core/controller/grpc-handler"
|
||||
import { FirebaseAuthProvider } from "./providers/FirebaseAuthProvider"
|
||||
import { Controller } from "@/core/controller"
|
||||
import { storeSecret } from "@/core/storage/state"
|
||||
|
||||
const DefaultClineAccountURI = "https://app.cline.bot/auth"
|
||||
const DefaultClineAccountURI = "http://localhost:3000/auth"
|
||||
// const DefaultClineAccountURI = "https://staging-app.cline.bot/auth"
|
||||
// const DefaultClineAccountURI = "http://localhost:3000/auth"
|
||||
let authProviders: any[] = []
|
||||
@@ -22,35 +22,14 @@ const availableAuthProviders = {
|
||||
// Add other providers here as needed
|
||||
}
|
||||
|
||||
export interface ClineAuthInfo {
|
||||
idToken: string
|
||||
userInfo: ClineAccountUserInfo
|
||||
}
|
||||
|
||||
export interface ClineAccountUserInfo {
|
||||
createdAt: string
|
||||
displayName: string
|
||||
email: string
|
||||
id: string
|
||||
organizations: ClineAccountOrganization[]
|
||||
}
|
||||
|
||||
export interface ClineAccountOrganization {
|
||||
active: boolean
|
||||
memberId: string
|
||||
name: string
|
||||
organizationId: string
|
||||
roles: string[]
|
||||
}
|
||||
|
||||
// TODO: Add logic to handle multiple webviews getting auth updates.
|
||||
|
||||
export class AuthService {
|
||||
private static instance: AuthService | null = null
|
||||
private _config: ServiceConfig
|
||||
private _authenticated: boolean = false
|
||||
private _clineAuthInfo: ClineAuthInfo | null = null
|
||||
private _provider: { provider: FirebaseAuthProvider } | null = null
|
||||
private _user: any = null
|
||||
private _provider: any = null
|
||||
private readonly _authNonce = crypto.randomBytes(32).toString("hex")
|
||||
private _activeAuthStatusUpdateSubscriptions = new Set<[Controller, StreamingResponseHandler]>()
|
||||
private _context: vscode.ExtensionContext
|
||||
@@ -72,14 +51,14 @@ export class AuthService {
|
||||
const authProvidersConfigs = [
|
||||
{
|
||||
name: "firebase",
|
||||
config: {
|
||||
apiKey: "AIzaSyC5rx59Xt8UgwdU3PCfzUF7vCwmp9-K2vk",
|
||||
authDomain: "cline-prod.firebaseapp.com",
|
||||
projectId: "cline-prod",
|
||||
storageBucket: "cline-prod.firebasestorage.app",
|
||||
messagingSenderId: "941048379330",
|
||||
appId: "1:941048379330:web:45058eedeefc5cdfcc485b",
|
||||
},
|
||||
// config: {
|
||||
// apiKey: "AIzaSyC5rx59Xt8UgwdU3PCfzUF7vCwmp9-K2vk",
|
||||
// authDomain: "cline-prod.firebaseapp.com",
|
||||
// projectId: "cline-prod",
|
||||
// storageBucket: "cline-prod.firebasestorage.app",
|
||||
// messagingSenderId: "941048379330",
|
||||
// appId: "1:941048379330:web:45058eedeefc5cdfcc485b",
|
||||
// },
|
||||
// Uncomment for staging environment
|
||||
// config: {
|
||||
// apiKey: "AIzaSyASSwkwX1kSO8vddjZkE5N19QU9cVQ0CIk",
|
||||
@@ -90,14 +69,15 @@ export class AuthService {
|
||||
// appId: "1:853479478430:web:2de0dba1c63c3262d4578f",
|
||||
// },
|
||||
// Uncomment for local development environment
|
||||
// config: {
|
||||
// apiKey: "AIzaSyASSwkwX1kSO8vddjZkE5N19QU9cVQ0CIk",
|
||||
// authDomain: "cline-staging.firebaseapp.com",
|
||||
// projectId: "cline-staging",
|
||||
// storageBucket: "cline-staging.firebasestorage.app",
|
||||
// messagingSenderId: "853479478430",
|
||||
// appId: "1:853479478430:web:2de0dba1c63c3262d4578f",
|
||||
// },
|
||||
config: {
|
||||
apiKey: "AIzaSyD8wtkd1I-EICuAg6xgAQpRdwYTvwxZG2w",
|
||||
authDomain: "cline-preview.firebaseapp.com",
|
||||
projectId: "cline-preview",
|
||||
storageBucket: "cline-preview.firebasestorage.app",
|
||||
messagingSenderId: "654681443338",
|
||||
appId: "1:654681443338:web:93bfe710626a573d9123f3",
|
||||
measurementId: "G-QL8CKK5TNJ",
|
||||
},
|
||||
// config: {
|
||||
// apiKey: "AIzaSyD8wtkd1I-EICuAg6xgAQpRdwYTvwxZG2w",
|
||||
// authDomain: "cline-preview.firebaseapp.com",
|
||||
@@ -163,19 +143,13 @@ export class AuthService {
|
||||
}
|
||||
|
||||
async getAuthToken(): Promise<string | null> {
|
||||
if (!this._clineAuthInfo) {
|
||||
if (!this._user) {
|
||||
return null
|
||||
}
|
||||
const idToken = this._clineAuthInfo.idToken
|
||||
const shouldRefreshIdToken = await this._provider?.provider.shouldRefreshIdToken(idToken)
|
||||
if (shouldRefreshIdToken) {
|
||||
// Retrieves the stored id token and refreshes it, then updates this._clineAuthInfo
|
||||
await this.restoreRefreshTokenAndRetrieveAuthInfo()
|
||||
if (!this._clineAuthInfo) {
|
||||
return null
|
||||
}
|
||||
}
|
||||
return this._clineAuthInfo.idToken
|
||||
|
||||
// TODO: This may need to be dependant on the auth provider
|
||||
// Return the ID token from the user object
|
||||
return this._provider.provider.getAuthToken(this._user)
|
||||
}
|
||||
|
||||
private _setProvider(providerName: string): void {
|
||||
@@ -188,17 +162,9 @@ export class AuthService {
|
||||
}
|
||||
|
||||
getInfo(): AuthState {
|
||||
// TODO: this logic should be cleaner, but this will determine the authentication state for the webview -- if a user object is returned then the webview assumes authenticated, otherwise it assumes logged out (we previously returned a UserInfo object with empty fields, and this represented a broken logged in state)
|
||||
let user: any = null
|
||||
if (this._clineAuthInfo && this._authenticated) {
|
||||
const userInfo = this._clineAuthInfo.userInfo
|
||||
user = UserInfo.create({
|
||||
// TODO: create proto for new user info type
|
||||
uid: userInfo?.id,
|
||||
displayName: userInfo?.displayName,
|
||||
email: userInfo?.email,
|
||||
photoUrl: undefined,
|
||||
})
|
||||
let user = null
|
||||
if (this._user && this._authenticated) {
|
||||
user = this._provider.provider.convertUserData(this._user)
|
||||
}
|
||||
|
||||
return AuthState.create({
|
||||
@@ -235,7 +201,8 @@ export class AuthService {
|
||||
}
|
||||
|
||||
try {
|
||||
this._clineAuthInfo = null
|
||||
await this._provider.provider.signOut()
|
||||
this._user = null
|
||||
this._authenticated = false
|
||||
this.sendAuthStatusUpdate()
|
||||
} catch (error) {
|
||||
@@ -250,11 +217,12 @@ export class AuthService {
|
||||
}
|
||||
|
||||
try {
|
||||
this._clineAuthInfo = await this._provider.provider.signIn(this._context, token, provider)
|
||||
this._user = await this._provider.provider.signIn(this._context, token, provider)
|
||||
this._authenticated = true
|
||||
|
||||
await this.sendAuthStatusUpdate()
|
||||
// return this._clineAuthInfo
|
||||
this.setupAutoRefreshAuth()
|
||||
return this._user
|
||||
} catch (error) {
|
||||
console.error("Error signing in with custom token:", error)
|
||||
throw error
|
||||
@@ -273,29 +241,59 @@ export class AuthService {
|
||||
* Restores the authentication token from the extension's storage.
|
||||
* This is typically called when the extension is activated.
|
||||
*/
|
||||
async restoreRefreshTokenAndRetrieveAuthInfo(): Promise<void> {
|
||||
async restoreAuthToken(): Promise<void> {
|
||||
if (!this._provider || !this._provider.provider) {
|
||||
throw new Error("Auth provider is not set")
|
||||
}
|
||||
|
||||
try {
|
||||
this._clineAuthInfo = await this._provider.provider.retrieveClineAuthInfo(this._context)
|
||||
if (this._clineAuthInfo) {
|
||||
this._user = await this._provider.provider.restoreAuthCredential(this._context)
|
||||
if (this._user) {
|
||||
this._authenticated = true
|
||||
await this.sendAuthStatusUpdate()
|
||||
this.setupAutoRefreshAuth()
|
||||
// Setup auto-refresh for the auth token
|
||||
} else {
|
||||
console.warn("No user found after restoring auth token")
|
||||
this._authenticated = false
|
||||
this._clineAuthInfo = null
|
||||
this._user = null
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Error restoring auth token:", error)
|
||||
this._authenticated = false
|
||||
this._clineAuthInfo = null
|
||||
this._user = null
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Refreshes the authentication status and sends an update to all subscribers.
|
||||
*/
|
||||
async refreshAuth(): Promise<void> {
|
||||
if (!this._user) {
|
||||
console.warn("No user is authenticated, skipping auth refresh")
|
||||
return
|
||||
}
|
||||
|
||||
await this._provider.provider.refreshAuthToken()
|
||||
this.sendAuthStatusUpdate()
|
||||
}
|
||||
|
||||
private setupAutoRefreshAuth(): void {
|
||||
// Set timeoutDuration to refresh the auth token 5 minutes before it expires
|
||||
const timeoutDuration = Math.floor(this._user.stsTokenManager.expirationTime - 5 * 60000 - Date.now()) // Milliseconds until 5 minutes before expiration
|
||||
setTimeout(() => this._autoRefreshAuth(), timeoutDuration)
|
||||
}
|
||||
|
||||
private async _autoRefreshAuth(): Promise<void> {
|
||||
if (!this._user) {
|
||||
console.warn("No user is authenticated, skipping auth refresh")
|
||||
return
|
||||
}
|
||||
await this.refreshAuth()
|
||||
this.setupAutoRefreshAuth() // Reschedule the next auto-refresh
|
||||
}
|
||||
|
||||
/**
|
||||
* Subscribe to authStatusUpdate events
|
||||
* @param controller The controller instance
|
||||
|
||||
@@ -1,11 +1,18 @@
|
||||
import { getSecret, storeSecret } from "@/core/storage/state"
|
||||
import { ErrorService } from "@/services/error/ErrorService"
|
||||
import axios from "axios"
|
||||
import { initializeApp } from "firebase/app"
|
||||
import { GithubAuthProvider, GoogleAuthProvider, User, getAuth, signInWithCredential } from "firebase/auth"
|
||||
import {
|
||||
AuthCredential,
|
||||
GoogleAuthProvider,
|
||||
GithubAuthProvider,
|
||||
OAuthCredential,
|
||||
User,
|
||||
UserCredential,
|
||||
getAuth,
|
||||
signInWithCredential,
|
||||
signOut,
|
||||
} from "firebase/auth"
|
||||
import { ExtensionContext } from "vscode"
|
||||
import { ClineAccountUserInfo, ClineAuthInfo } from "../AuthService"
|
||||
import { jwtDecode } from "jwt-decode"
|
||||
|
||||
export class FirebaseAuthProvider {
|
||||
private _config: any
|
||||
@@ -22,16 +29,80 @@ export class FirebaseAuthProvider {
|
||||
this._config = value
|
||||
}
|
||||
|
||||
async shouldRefreshIdToken(existingIdToken: string): Promise<boolean> {
|
||||
const decodedToken = jwtDecode(existingIdToken)
|
||||
const exp = decodedToken.exp || 0 // 1752297633
|
||||
const expirationTime = exp * 1000
|
||||
const currentTime = Date.now()
|
||||
const fiveMinutesInMs = 5 * 60 * 1000
|
||||
if (currentTime > expirationTime - fiveMinutesInMs) {
|
||||
return true // id token is expired or about to be expired
|
||||
/**
|
||||
* Gets the authentication token of the current user.
|
||||
* @returns {Promise<string | null>} A promise that resolves to the authentication token of the current user, or null if no user is signed in.
|
||||
*/
|
||||
async getAuthToken(): Promise<string | null> {
|
||||
const user = getAuth().currentUser
|
||||
const idToken = user ? await user.getIdToken() : null
|
||||
return idToken
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets the refresh token of the current user.
|
||||
* @returns {Promise<string | null>} A promise that resolves to the refresh token of the current user, or null if no user is signed in.
|
||||
*/
|
||||
async getRefreshToken(): Promise<string | null> {
|
||||
const user = getAuth().currentUser
|
||||
const refreshToken = user ? user.refreshToken : null
|
||||
return refreshToken
|
||||
}
|
||||
|
||||
/**
|
||||
* Refreshes the authentication token of the current user.
|
||||
* @returns {Promise<string | null>} A promise that resolves to the refreshed authentication token of the current user, or null if no user is signed in.
|
||||
*/
|
||||
async refreshAuthToken(): Promise<string | null> {
|
||||
const user = getAuth().currentUser
|
||||
const idToken = user ? await user.getIdToken(true) : null
|
||||
return idToken
|
||||
}
|
||||
|
||||
/**
|
||||
* Converts Firebase User object to a generic user object.
|
||||
* @param user - The Firebase User object.
|
||||
* @returns {User} A generic user object.
|
||||
*/
|
||||
convertUserData(user: User) {
|
||||
return {
|
||||
uid: user.uid,
|
||||
email: user.email,
|
||||
displayName: user.displayName,
|
||||
photoUrl: user.photoURL,
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Signs out the current user from Firebase.
|
||||
* @returns {Promise<void>} A promise that resolves when the user is signed out.
|
||||
*/
|
||||
async signOut(): Promise<void> {
|
||||
signOut(getAuth(initializeApp(Object.assign({}, this._config))))
|
||||
.then(() => {
|
||||
console.log("User signed out successfully.")
|
||||
})
|
||||
.catch((error) => {
|
||||
ErrorService.logMessage("Firebase sign-out error", "error")
|
||||
ErrorService.logException(error)
|
||||
throw error
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Stores the authentication token using a provided token.
|
||||
* @param token - The authentication token to store.
|
||||
* @returns {Promise<User>} A promise that resolves with the authenticated user.
|
||||
* @throws {Error} Throws an error if the storage fails.
|
||||
*/
|
||||
private async _storeAuthCredential(context: ExtensionContext, credential: AuthCredential): Promise<void> {
|
||||
try {
|
||||
await storeSecret(context, "clineAccountId", JSON.stringify(credential.toJSON()))
|
||||
} catch (error) {
|
||||
ErrorService.logMessage("Firebase store token error", "error")
|
||||
ErrorService.logException(error)
|
||||
throw error
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -40,68 +111,45 @@ export class FirebaseAuthProvider {
|
||||
* @returns {Promise<User>} A promise that resolves with the authenticated user.
|
||||
* @throws {Error} Throws an error if the restoration fails.
|
||||
*/
|
||||
async retrieveClineAuthInfo(context: ExtensionContext): Promise<ClineAuthInfo | null> {
|
||||
const userRefreshToken = await getSecret(context, "clineAccountId")
|
||||
if (!userRefreshToken) {
|
||||
async restoreAuthCredential(context: ExtensionContext): Promise<User | null> {
|
||||
const credentialJSON = await getSecret(context, "clineAccountId")
|
||||
if (!credentialJSON) {
|
||||
console.error("No stored authentication credential found.")
|
||||
return null
|
||||
}
|
||||
try {
|
||||
// Exchange refresh token for new access token using Firebase's secure token endpoint
|
||||
// https://stackoverflow.com/questions/38233687/how-to-use-the-firebase-refreshtoken-to-reauthenticate/57119131#57119131
|
||||
const firebaseApiKey = this._config.apiKey
|
||||
const googleAccessTokenResponse = await axios.post(
|
||||
`https://securetoken.googleapis.com/v1/token?key=${firebaseApiKey}`,
|
||||
`grant_type=refresh_token&refresh_token=${encodeURIComponent(userRefreshToken)}`,
|
||||
{
|
||||
headers: {
|
||||
"Content-Type": "application/x-www-form-urlencoded",
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
// console.log("googleAccessTokenResponse", googleAccessTokenResponse)
|
||||
|
||||
// This returns an object with access_token, expires_in (3600), id_token (can be used as bearer token to authenticate requests, we'll use this in the future instead of firebase but need to be aware of how we use firebase sdk for e.g. user info like the profile image), project_id, refresh_token, token_type (always Bearer), and user_id
|
||||
const idToken = googleAccessTokenResponse.data.id_token
|
||||
// const idTokenExpirationDate = new Date(Date.now() + googleAccessTokenResponse.data.expires_in * 1000)
|
||||
|
||||
// Now retrieve the user info from the backend (this was an easy solution to keep providing user profile details like name and email, but we should move to using the fetchMe() function instead)
|
||||
// Fetch user info from Cline API
|
||||
// TODO: consolidate with fetchMe() instead of making the call directly here
|
||||
const userResponse = await axios.get("https://api.cline.bot/api/v1/users/me", {
|
||||
headers: {
|
||||
Authorization: `Bearer ${idToken}`,
|
||||
},
|
||||
})
|
||||
|
||||
// Store user data
|
||||
const userInfo: ClineAccountUserInfo = userResponse.data.data
|
||||
|
||||
return { idToken, userInfo }
|
||||
|
||||
// let userObject = JSON.parse(credentialJSON)
|
||||
// let user = User.
|
||||
// userObject = User.constructor._fromJSON(auth, user2);
|
||||
// const credentialData: AuthCredential = OAuthCredential.fromJSON(credentialJSON) as AuthCredential
|
||||
// const userCredential = await this._signInWithCredential(context, credentialData)
|
||||
// return userCredential.user
|
||||
const credentialData: AuthCredential = OAuthCredential.fromJSON(credentialJSON) as AuthCredential
|
||||
const userCredential = await this._signInWithCredential(credentialData)
|
||||
return userCredential.user
|
||||
} catch (error) {
|
||||
console.error("Firebase restore token error", error)
|
||||
ErrorService.logMessage("Firebase restore token error", "error")
|
||||
ErrorService.logException(error)
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
async _signInWithCredential(credential: AuthCredential): Promise<UserCredential> {
|
||||
const firebaseConfig = Object.assign({}, this._config)
|
||||
const app = initializeApp(firebaseConfig)
|
||||
const auth = getAuth(app)
|
||||
try {
|
||||
return await signInWithCredential(auth, credential)
|
||||
} catch (error) {
|
||||
ErrorService.logMessage("Firebase sign-in with credential error", "error")
|
||||
ErrorService.logException(error)
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Signs in the user using Firebase authentication with a custom token.
|
||||
* @returns {Promise<User>} A promise that resolves with the authenticated user.
|
||||
* @throws {Error} Throws an error if the sign-in fails.
|
||||
*/
|
||||
async signIn(context: ExtensionContext, token: string, provider: string): Promise<ClineAuthInfo | null> {
|
||||
async signIn(context: ExtensionContext, token: string, provider: string): Promise<User> {
|
||||
try {
|
||||
let credential
|
||||
let userCredential
|
||||
switch (provider) {
|
||||
case "google":
|
||||
credential = GoogleAuthProvider.credential(token)
|
||||
@@ -112,25 +160,9 @@ export class FirebaseAuthProvider {
|
||||
default:
|
||||
throw new Error(`Unsupported provider: ${provider}`)
|
||||
}
|
||||
// we've received the short-lived tokens from google/github, now we need to sign in to firebase with them
|
||||
const firebaseConfig = Object.assign({}, this._config)
|
||||
const app = initializeApp(firebaseConfig)
|
||||
const auth = getAuth(app)
|
||||
// this signs the user into firebase sdk internally
|
||||
const userCredential = (await signInWithCredential(auth, credential)).user
|
||||
// const userRefreshToken = await userCredential.getIdToken()
|
||||
|
||||
// store the long-lived refresh token in secret storage
|
||||
try {
|
||||
await storeSecret(context, "clineAccountId", userCredential.refreshToken)
|
||||
} catch (error) {
|
||||
ErrorService.logMessage("Firebase store token error", "error")
|
||||
ErrorService.logException(error)
|
||||
throw error
|
||||
}
|
||||
|
||||
// userCredential = await this._signInWithCredential(context, credential)
|
||||
return await this.retrieveClineAuthInfo(context)
|
||||
this._storeAuthCredential(context, credential)
|
||||
userCredential = await this._signInWithCredential(credential)
|
||||
return userCredential.user
|
||||
} catch (error) {
|
||||
ErrorService.logMessage("Firebase sign-in error", "error")
|
||||
ErrorService.logException(error)
|
||||
|
||||
@@ -0,0 +1,220 @@
|
||||
import * as fs from "fs"
|
||||
import * as path from "path"
|
||||
import * as os from "os"
|
||||
import { spawn, ChildProcess } from "child_process"
|
||||
import { Logger } from "@services/logging/Logger"
|
||||
import { AUDIO_PROGRAM_CONFIG } from "@/shared/audioProgramConstants"
|
||||
|
||||
function isExecutable(filePath: string): boolean {
|
||||
try {
|
||||
fs.accessSync(filePath, fs.constants.X_OK)
|
||||
return true
|
||||
} catch (e) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
export class AudioRecordingService {
|
||||
private recordingProcess: ChildProcess | null = null
|
||||
private isRecording: boolean = false
|
||||
private startTime: number = 0
|
||||
private outputFile: string = ""
|
||||
|
||||
constructor() {}
|
||||
|
||||
async startRecording(): Promise<{ success: boolean; error?: string }> {
|
||||
try {
|
||||
if (this.isRecording) {
|
||||
return { success: false, error: "Already recording" }
|
||||
}
|
||||
|
||||
// Check if recording software is available
|
||||
const checkResult = this.checkRecordingDependencies()
|
||||
if (!checkResult.available) {
|
||||
return { success: false, error: checkResult.error }
|
||||
}
|
||||
|
||||
// Create temporary file for audio output
|
||||
const tempDir = os.tmpdir()
|
||||
this.outputFile = path.join(tempDir, `cline_recording_${Date.now()}.webm`)
|
||||
|
||||
Logger.info("Starting audio recording...")
|
||||
|
||||
// Get the recording program path
|
||||
const recordProgram = this.getRecordProgram()
|
||||
if (!recordProgram) {
|
||||
return { success: false, error: "Recording program not found" }
|
||||
}
|
||||
Logger.info(`Using recording program: ${recordProgram.path}`)
|
||||
|
||||
// Set up recording arguments
|
||||
const args = recordProgram.getArgs(this.outputFile)
|
||||
|
||||
// Spawn the recording process
|
||||
this.recordingProcess = spawn(recordProgram.path, args)
|
||||
this.isRecording = true
|
||||
this.startTime = Date.now()
|
||||
|
||||
// Handle process errors
|
||||
this.recordingProcess.on("error", (error) => {
|
||||
Logger.error(`Recording process error: ${error.message}`)
|
||||
this.isRecording = false
|
||||
})
|
||||
|
||||
// Handle process exit
|
||||
this.recordingProcess.on("exit", (code) => {
|
||||
if (code !== 0 && code !== null) {
|
||||
Logger.warn(`Recording process exited with code: ${code}`)
|
||||
}
|
||||
})
|
||||
|
||||
this.recordingProcess.stderr?.on("data", (data) => {
|
||||
const message = data.toString().trim()
|
||||
if (message && !message.includes("In:") && !message.includes("Out:")) {
|
||||
Logger.info(`Recording stderr: ${message}`)
|
||||
}
|
||||
})
|
||||
|
||||
Logger.info("Audio recording started successfully")
|
||||
return { success: true }
|
||||
} catch (error) {
|
||||
const errorMessage = error instanceof Error ? error.message : String(error)
|
||||
Logger.error("Failed to start audio recording: " + errorMessage)
|
||||
return { success: false, error: `Failed to start recording: ${errorMessage}` }
|
||||
}
|
||||
}
|
||||
|
||||
async stopRecording(): Promise<{ success: boolean; audioBase64?: string; error?: string }> {
|
||||
try {
|
||||
if (!this.isRecording || !this.recordingProcess) {
|
||||
return { success: false, error: "Not currently recording" }
|
||||
}
|
||||
|
||||
Logger.info("Stopping audio recording...")
|
||||
|
||||
// Send SIGINT to stop recording gracefully (like Ctrl+C)
|
||||
this.recordingProcess.kill("SIGINT")
|
||||
|
||||
// Wait for the process to finish
|
||||
await new Promise<void>((resolve) => {
|
||||
if (this.recordingProcess) {
|
||||
// Timeout after 5 seconds
|
||||
const timeoutId = setTimeout(() => {
|
||||
resolve()
|
||||
}, 5000)
|
||||
|
||||
this.recordingProcess.on("exit", (code) => {
|
||||
clearTimeout(timeoutId) // Clear the timeout since process exited
|
||||
resolve()
|
||||
})
|
||||
} else {
|
||||
resolve()
|
||||
}
|
||||
})
|
||||
|
||||
this.recordingProcess = null
|
||||
this.isRecording = false
|
||||
|
||||
// Wait a moment for file to be fully written
|
||||
await new Promise((resolve) => setTimeout(resolve, 500))
|
||||
|
||||
// Read the audio file and convert to base64
|
||||
if (!fs.existsSync(this.outputFile)) {
|
||||
return { success: false, error: "Recording file not found" }
|
||||
}
|
||||
|
||||
const audioBuffer = fs.readFileSync(this.outputFile)
|
||||
const audioBase64 = audioBuffer.toString("base64")
|
||||
|
||||
// Clean up temporary file
|
||||
try {
|
||||
fs.unlinkSync(this.outputFile)
|
||||
} catch (cleanupError) {
|
||||
Logger.warn(
|
||||
"Failed to cleanup temporary audio file: " +
|
||||
(cleanupError instanceof Error ? cleanupError.message : String(cleanupError)),
|
||||
)
|
||||
}
|
||||
|
||||
Logger.info("Audio recording stopped and converted to base64")
|
||||
return { success: true, audioBase64 }
|
||||
} catch (error) {
|
||||
const errorMessage = error instanceof Error ? error.message : String(error)
|
||||
Logger.error("Failed to stop audio recording: " + errorMessage)
|
||||
return { success: false, error: `Failed to stop recording: ${errorMessage}` }
|
||||
}
|
||||
}
|
||||
|
||||
getRecordingStatus(): { isRecording: boolean; durationSeconds: number; error?: string } {
|
||||
const durationSeconds = this.isRecording ? (Date.now() - this.startTime) / 1000 : 0
|
||||
return {
|
||||
isRecording: this.isRecording,
|
||||
durationSeconds,
|
||||
}
|
||||
}
|
||||
|
||||
private checkRecordingDependencies(): { available: boolean; error?: string } {
|
||||
const program = this.getRecordProgram()
|
||||
if (!program) {
|
||||
const platform = os.platform() as keyof typeof AUDIO_PROGRAM_CONFIG
|
||||
const config = AUDIO_PROGRAM_CONFIG[platform]
|
||||
const error = config ? config.error : `Audio recording is not supported on platform: ${platform}`
|
||||
return { available: false, error }
|
||||
}
|
||||
return { available: true }
|
||||
}
|
||||
|
||||
private getRecordProgram(): { path: string; getArgs: (outputFile: string) => string[] } | undefined {
|
||||
const platform = os.platform() as keyof typeof AUDIO_PROGRAM_CONFIG
|
||||
const config = AUDIO_PROGRAM_CONFIG[platform]
|
||||
|
||||
if (!config) {
|
||||
return undefined
|
||||
}
|
||||
|
||||
// 1. Check if the command is in the system's PATH
|
||||
const pathDirs = (process.env.PATH || "").split(path.delimiter)
|
||||
for (const dir of pathDirs) {
|
||||
const fullPath = path.join(dir, config.command)
|
||||
if (fs.existsSync(fullPath) && isExecutable(fullPath)) {
|
||||
return { path: fullPath, getArgs: config.getArgs }
|
||||
}
|
||||
}
|
||||
|
||||
// 2. Check fallback paths if not in PATH
|
||||
for (const p of config.fallbackPaths) {
|
||||
if (fs.existsSync(p) && isExecutable(p)) {
|
||||
return { path: p, getArgs: config.getArgs }
|
||||
}
|
||||
}
|
||||
|
||||
return undefined
|
||||
}
|
||||
|
||||
// Cleanup method
|
||||
cleanup(): void {
|
||||
if (this.isRecording && this.recordingProcess) {
|
||||
try {
|
||||
this.recordingProcess.kill("SIGINT")
|
||||
this.recordingProcess = null
|
||||
this.isRecording = false
|
||||
} catch (error) {
|
||||
Logger.error("Error during cleanup: " + (error instanceof Error ? error.message : String(error)))
|
||||
}
|
||||
}
|
||||
|
||||
// Clean up any leftover temp files
|
||||
if (this.outputFile && fs.existsSync(this.outputFile)) {
|
||||
try {
|
||||
fs.unlinkSync(this.outputFile)
|
||||
} catch (error) {
|
||||
Logger.warn(
|
||||
"Failed to cleanup temp file during service cleanup: " +
|
||||
(error instanceof Error ? error.message : String(error)),
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export const audioRecordingService = new AudioRecordingService()
|
||||
@@ -0,0 +1,54 @@
|
||||
import { Logger } from "@services/logging/Logger"
|
||||
import { ClineAccountService } from "@/services/account/ClineAccountService"
|
||||
import axios from "axios"
|
||||
|
||||
export class VoiceTranscriptionService {
|
||||
private clineAccountService: ClineAccountService
|
||||
|
||||
constructor() {
|
||||
this.clineAccountService = ClineAccountService.getInstance()
|
||||
}
|
||||
|
||||
async transcribeAudio(audioBase64: string, language?: string): Promise<{ text?: string; error?: string }> {
|
||||
try {
|
||||
Logger.info("Transcribing audio with Cline transcription service...")
|
||||
|
||||
const result = await this.clineAccountService.transcribeAudio(audioBase64, language)
|
||||
|
||||
Logger.info("Transcription successful")
|
||||
|
||||
return { text: result.text }
|
||||
} catch (error) {
|
||||
Logger.error("Voice transcription error:", error)
|
||||
|
||||
// Handle axios errors with proper status code mapping
|
||||
if (axios.isAxiosError(error)) {
|
||||
const status = error.response?.status
|
||||
const message = error.response?.data?.message || error.message
|
||||
|
||||
switch (status) {
|
||||
case 401:
|
||||
return { error: "Authentication failed. Please reauthenticate your Cline account" }
|
||||
case 402:
|
||||
return { error: "Insufficient credits for transcription service." }
|
||||
case 400:
|
||||
return { error: "Invalid audio format or request data." }
|
||||
case 500:
|
||||
return { error: "Transcription server error. Please try again later." }
|
||||
default:
|
||||
return { error: `Transcription failed: ${message}` }
|
||||
}
|
||||
}
|
||||
|
||||
// Handle network errors
|
||||
const errorMessage = error instanceof Error ? error.message : String(error)
|
||||
if (errorMessage.includes("ECONNREFUSED") || errorMessage.includes("Network Error")) {
|
||||
return { error: "Cannot connect to transcription service." }
|
||||
}
|
||||
|
||||
return { error: `Network error: ${errorMessage}` }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export const voiceTranscriptionService = new VoiceTranscriptionService()
|
||||
@@ -27,13 +27,14 @@ interface Collection {
|
||||
* When adding a new category, add it both here and to the initial values in telemetryCategoryEnabled
|
||||
* Ensure `if (!this.isCategoryEnabled('<category_name>')` is added to the capture method
|
||||
*/
|
||||
type TelemetryCategory = "checkpoints" | "browser"
|
||||
type TelemetryCategory = "checkpoints" | "browser" | "voice"
|
||||
|
||||
class TelemetryService {
|
||||
// Map to control specific telemetry categories (event types)
|
||||
private telemetryCategoryEnabled: Map<TelemetryCategory, boolean> = new Map([
|
||||
["checkpoints", false], // Checkpoints telemetry disabled
|
||||
["browser", true], // Browser telemetry enabled
|
||||
["voice", true], // Voice telemetry enabled
|
||||
])
|
||||
|
||||
// Stores events when collect=true
|
||||
@@ -95,6 +96,20 @@ class TelemetryService {
|
||||
// Tracks when a button is clicked
|
||||
BUTTON_CLICKED: "ui.button_clicked",
|
||||
},
|
||||
// Voice-related events for tracking voice recording and transcription usage
|
||||
VOICE: {
|
||||
// Tracks when voice recording is started
|
||||
RECORDING_STARTED: "voice.recording_started",
|
||||
// Tracks when voice recording is stopped
|
||||
RECORDING_STOPPED: "voice.recording_stopped",
|
||||
// Tracks when voice transcription is started
|
||||
TRANSCRIPTION_STARTED: "voice.transcription_started",
|
||||
// Tracks when voice transcription is completed successfully
|
||||
TRANSCRIPTION_COMPLETED: "voice.transcription_completed",
|
||||
// Tracks when voice transcription fails
|
||||
TRANSCRIPTION_ERROR: "voice.transcription_error",
|
||||
// Tracks when voice feature is enabled or disabled in settings
|
||||
},
|
||||
}
|
||||
|
||||
/** Singleton instance of the TelemetryService */
|
||||
@@ -140,16 +155,9 @@ class TelemetryService {
|
||||
} else {
|
||||
// Only show warning if user has opted in to Cline telemetry but VS Code telemetry is disabled
|
||||
if (didUserOptIn) {
|
||||
void vscode.window
|
||||
.showWarningMessage(
|
||||
"Anonymous Cline error and usage reporting is enabled, but VSCode telemetry is disabled. To enable error and usage reporting for this extension, enable VSCode telemetry in settings.",
|
||||
"Open Settings",
|
||||
)
|
||||
.then((selection) => {
|
||||
if (selection === "Open Settings") {
|
||||
void vscode.commands.executeCommand("workbench.action.openSettings", "telemetry.telemetryLevel")
|
||||
}
|
||||
})
|
||||
void vscode.window.showWarningMessage(
|
||||
"Anonymous Cline error and usage reporting is enabled, but VSCode telemetry is disabled. To enable error and usage reporting for this extension, enable VSCode telemetry in settings.",
|
||||
)
|
||||
}
|
||||
this.telemetryEnabled = false
|
||||
}
|
||||
@@ -720,6 +728,164 @@ class TelemetryService {
|
||||
)
|
||||
}
|
||||
|
||||
// Voice events
|
||||
/**
|
||||
* Records when voice recording is started
|
||||
* @param taskId Optional task identifier if recording was started during a task
|
||||
* @param platform The platform where recording is happening (macOS, Windows, Linux)
|
||||
* @param collect If true, collect event instead of sending
|
||||
*/
|
||||
public captureVoiceRecordingStarted(taskId?: string, platform?: string, collect: boolean = false) {
|
||||
if (!this.isCategoryEnabled("voice")) {
|
||||
return
|
||||
}
|
||||
|
||||
this.capture(
|
||||
{
|
||||
event: TelemetryService.EVENTS.VOICE.RECORDING_STARTED,
|
||||
properties: {
|
||||
taskId,
|
||||
platform: platform || process.platform,
|
||||
timestamp: new Date().toISOString(),
|
||||
},
|
||||
},
|
||||
collect,
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Records when voice recording is stopped
|
||||
* @param taskId Optional task identifier if recording was stopped during a task
|
||||
* @param durationMs Duration of the recording in milliseconds
|
||||
* @param success Whether the recording was successful
|
||||
* @param platform The platform where recording happened
|
||||
* @param collect If true, collect event instead of sending
|
||||
*/
|
||||
public captureVoiceRecordingStopped(
|
||||
taskId?: string,
|
||||
durationMs?: number,
|
||||
success?: boolean,
|
||||
platform?: string,
|
||||
collect?: boolean,
|
||||
) {
|
||||
if (!this.isCategoryEnabled("voice")) {
|
||||
return
|
||||
}
|
||||
|
||||
this.capture(
|
||||
{
|
||||
event: TelemetryService.EVENTS.VOICE.RECORDING_STOPPED,
|
||||
properties: {
|
||||
taskId,
|
||||
durationMs,
|
||||
success,
|
||||
platform: platform || process.platform,
|
||||
timestamp: new Date().toISOString(),
|
||||
},
|
||||
},
|
||||
collect || false,
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Records when voice transcription is started
|
||||
* @param taskId Optional task identifier if transcription was started during a task
|
||||
* @param audioSizeBytes Size of the audio data being transcribed
|
||||
* @param language Language hint provided for transcription
|
||||
* @param collect If true, collect event instead of sending
|
||||
*/
|
||||
public captureVoiceTranscriptionStarted(
|
||||
taskId?: string,
|
||||
audioSizeBytes?: number,
|
||||
language?: string,
|
||||
collect: boolean = false,
|
||||
) {
|
||||
if (!this.isCategoryEnabled("voice")) {
|
||||
return
|
||||
}
|
||||
|
||||
this.capture(
|
||||
{
|
||||
event: TelemetryService.EVENTS.VOICE.TRANSCRIPTION_STARTED,
|
||||
properties: {
|
||||
taskId,
|
||||
audioSizeBytes,
|
||||
language,
|
||||
timestamp: new Date().toISOString(),
|
||||
},
|
||||
},
|
||||
collect,
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Records when voice transcription is completed successfully
|
||||
* @param taskId Optional task identifier if transcription was completed during a task
|
||||
* @param transcriptionLength Length of the transcribed text
|
||||
* @param durationMs Time taken for transcription in milliseconds
|
||||
* @param language Language used for transcription
|
||||
* @param collect If true, collect event instead of sending
|
||||
*/
|
||||
public captureVoiceTranscriptionCompleted(
|
||||
taskId?: string,
|
||||
transcriptionLength?: number,
|
||||
durationMs?: number,
|
||||
language?: string,
|
||||
collect: boolean = false,
|
||||
) {
|
||||
if (!this.isCategoryEnabled("voice")) {
|
||||
return
|
||||
}
|
||||
|
||||
this.capture(
|
||||
{
|
||||
event: TelemetryService.EVENTS.VOICE.TRANSCRIPTION_COMPLETED,
|
||||
properties: {
|
||||
taskId,
|
||||
transcriptionLength,
|
||||
durationMs,
|
||||
language,
|
||||
timestamp: new Date().toISOString(),
|
||||
},
|
||||
},
|
||||
collect,
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Records when voice transcription fails
|
||||
* @param taskId Optional task identifier if transcription failed during a task
|
||||
* @param errorType Type of error that occurred (e.g., "no_openai_key", "api_error", "network_error")
|
||||
* @param errorMessage The error message
|
||||
* @param durationMs Time taken before failure in milliseconds
|
||||
* @param collect If true, collect event instead of sending
|
||||
*/
|
||||
public captureVoiceTranscriptionError(
|
||||
taskId?: string,
|
||||
errorType?: string,
|
||||
errorMessage?: string,
|
||||
durationMs?: number,
|
||||
collect: boolean = false,
|
||||
) {
|
||||
if (!this.isCategoryEnabled("voice")) {
|
||||
return
|
||||
}
|
||||
|
||||
this.capture(
|
||||
{
|
||||
event: TelemetryService.EVENTS.VOICE.TRANSCRIPTION_ERROR,
|
||||
properties: {
|
||||
taskId,
|
||||
errorType,
|
||||
errorMessage,
|
||||
durationMs,
|
||||
timestamp: new Date().toISOString(),
|
||||
},
|
||||
},
|
||||
collect,
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks if telemetry is enabled
|
||||
* @returns Boolean indicating whether telemetry is enabled
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
export interface DictationSettings {
|
||||
voiceRecordingEnabled: boolean
|
||||
dictationLanguage: string
|
||||
}
|
||||
|
||||
export const DEFAULT_DICTATION_SETTINGS: DictationSettings = {
|
||||
voiceRecordingEnabled: true,
|
||||
dictationLanguage: "en",
|
||||
}
|
||||
|
||||
export interface LanguageItem {
|
||||
name: string
|
||||
code: string
|
||||
}
|
||||
|
||||
export const SUPPORTED_DICTATION_LANGUAGES: LanguageItem[] = [
|
||||
{ name: "English", code: "en" },
|
||||
{ name: "Spanish (Español)", code: "es" },
|
||||
{ name: "Chinese (中文)", code: "zh" },
|
||||
{ name: "Japanese (日本語)", code: "ja" },
|
||||
{ name: "Afrikaans", code: "af" },
|
||||
{ name: "Arabic (العربية)", code: "ar" },
|
||||
{ name: "Armenian (Հայերեն)", code: "hy" },
|
||||
{ name: "Azerbaijani (Azərbaycan)", code: "az" },
|
||||
{ name: "Belarusian (Беларуская)", code: "be" },
|
||||
{ name: "Bosnian (Bosanski)", code: "bs" },
|
||||
{ name: "Bulgarian (Български)", code: "bg" },
|
||||
{ name: "Catalan (Català)", code: "ca" },
|
||||
{ name: "Croatian (Hrvatski)", code: "hr" },
|
||||
{ name: "Czech (Čeština)", code: "cs" },
|
||||
{ name: "Danish (Dansk)", code: "da" },
|
||||
{ name: "Dutch (Nederlands)", code: "nl" },
|
||||
{ name: "Estonian (Eesti)", code: "et" },
|
||||
{ name: "Finnish (Suomi)", code: "fi" },
|
||||
{ name: "French (Français)", code: "fr" },
|
||||
{ name: "Galician (Galego)", code: "gl" },
|
||||
{ name: "German (Deutsch)", code: "de" },
|
||||
{ name: "Greek (Ελληνικά)", code: "el" },
|
||||
{ name: "Hebrew (עברית)", code: "he" },
|
||||
{ name: "Hindi (हिन्दी)", code: "hi" },
|
||||
{ name: "Hungarian (Magyar)", code: "hu" },
|
||||
{ name: "Icelandic (Íslenska)", code: "is" },
|
||||
{ name: "Indonesian (Bahasa Indonesia)", code: "id" },
|
||||
{ name: "Italian (Italiano)", code: "it" },
|
||||
{ name: "Kannada (ಕನ್ನಡ)", code: "kn" },
|
||||
{ name: "Kazakh (Қазақша)", code: "kk" },
|
||||
{ name: "Korean (한국어)", code: "ko" },
|
||||
{ name: "Latvian (Latviešu)", code: "lv" },
|
||||
{ name: "Lithuanian (Lietuvių)", code: "lt" },
|
||||
{ name: "Macedonian (Македонски)", code: "mk" },
|
||||
{ name: "Malay (Bahasa Melayu)", code: "ms" },
|
||||
{ name: "Marathi (मराठी)", code: "mr" },
|
||||
{ name: "Maori (Te Reo Māori)", code: "mi" },
|
||||
{ name: "Nepali (नेपाली)", code: "ne" },
|
||||
{ name: "Norwegian (Norsk)", code: "no" },
|
||||
{ name: "Persian (فارسی)", code: "fa" },
|
||||
{ name: "Polish (Polski)", code: "pl" },
|
||||
{ name: "Portuguese (Português)", code: "pt" },
|
||||
{ name: "Romanian (Română)", code: "ro" },
|
||||
{ name: "Russian (Русский)", code: "ru" },
|
||||
{ name: "Serbian (Српски)", code: "sr" },
|
||||
{ name: "Slovak (Slovenčina)", code: "sk" },
|
||||
{ name: "Slovenian (Slovenščina)", code: "sl" },
|
||||
{ name: "Swahili (Kiswahili)", code: "sw" },
|
||||
{ name: "Swedish (Svenska)", code: "sv" },
|
||||
{ name: "Tagalog", code: "tl" },
|
||||
{ name: "Tamil (தமிழ்)", code: "ta" },
|
||||
{ name: "Thai (ไทย)", code: "th" },
|
||||
{ name: "Turkish (Türkçe)", code: "tr" },
|
||||
{ name: "Ukrainian (Українська)", code: "uk" },
|
||||
{ name: "Urdu (اردو)", code: "ur" },
|
||||
{ name: "Vietnamese (Tiếng Việt)", code: "vi" },
|
||||
{ name: "Welsh (Cymraeg)", code: "cy" },
|
||||
]
|
||||
@@ -3,6 +3,7 @@ import { ApiConfiguration } from "./api"
|
||||
import { AutoApprovalSettings } from "./AutoApprovalSettings"
|
||||
import { BrowserSettings } from "./BrowserSettings"
|
||||
import { ChatSettings } from "./ChatSettings"
|
||||
import { DictationSettings } from "./DictationSettings"
|
||||
import { HistoryItem } from "./HistoryItem"
|
||||
import { TelemetrySetting } from "./TelemetrySetting"
|
||||
import { ClineRulesToggles } from "./cline-rules"
|
||||
@@ -31,6 +32,7 @@ export interface ExtensionState {
|
||||
apiConfiguration?: ApiConfiguration
|
||||
autoApprovalSettings: AutoApprovalSettings
|
||||
browserSettings: BrowserSettings
|
||||
dictationSettings: DictationSettings
|
||||
remoteBrowserHost?: string
|
||||
chatSettings: ChatSettings
|
||||
checkpointTrackerErrorMessage?: string
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
export const AUDIO_PROGRAM_CONFIG = {
|
||||
darwin: {
|
||||
command: "ffmpeg",
|
||||
fallbackPaths: ["/usr/local/bin/ffmpeg", "/opt/homebrew/bin/ffmpeg"],
|
||||
getArgs: (outputFile: string) => [
|
||||
"-f",
|
||||
"avfoundation",
|
||||
"-i",
|
||||
":default",
|
||||
"-c:a",
|
||||
"libopus",
|
||||
"-b:a",
|
||||
"32k",
|
||||
"-application",
|
||||
"voip",
|
||||
"-ar",
|
||||
"16000",
|
||||
"-ac",
|
||||
"1",
|
||||
outputFile,
|
||||
],
|
||||
error: "It looks like your system is missing the 'FFmpeg' utility, which is required for voice recording. To install it, please ensure you are in **Act Mode** and then send this message. I will handle the installation for you.\n\n**Installation command:** `brew install ffmpeg`",
|
||||
},
|
||||
linux: {
|
||||
command: "ffmpeg",
|
||||
fallbackPaths: ["/usr/bin/ffmpeg", "/usr/local/bin/ffmpeg"],
|
||||
getArgs: (outputFile: string) => [
|
||||
"-f",
|
||||
"pulse",
|
||||
"-i",
|
||||
"default",
|
||||
"-c:a",
|
||||
"libopus",
|
||||
"-b:a",
|
||||
"32k",
|
||||
"-application",
|
||||
"voip",
|
||||
"-ar",
|
||||
"16000",
|
||||
"-ac",
|
||||
"1",
|
||||
outputFile,
|
||||
],
|
||||
error: "It looks like your system is missing the 'FFmpeg' utility, which is required for voice recording. To install it, please ensure you are in **Act Mode** and then send this message. I will handle the installation for you.\n\n**Installation command:** `sudo apt-get install ffmpeg`",
|
||||
},
|
||||
win32: {
|
||||
command: "ffmpeg",
|
||||
fallbackPaths: ["C:\\ffmpeg\\bin\\ffmpeg.exe", "C:\\Program Files\\ffmpeg\\bin\\ffmpeg.exe"],
|
||||
getArgs: (outputFile: string) => [
|
||||
"-f",
|
||||
"dshow",
|
||||
"-i",
|
||||
"audio=default",
|
||||
"-c:a",
|
||||
"libopus",
|
||||
"-b:a",
|
||||
"32k",
|
||||
"-application",
|
||||
"voip",
|
||||
"-ar",
|
||||
"16000",
|
||||
"-ac",
|
||||
"1",
|
||||
outputFile,
|
||||
],
|
||||
error: "It looks like your system is missing the 'FFmpeg' utility, which is required for voice recording. To install it, please ensure you are in **Act Mode** and then send this message. I will handle the installation for you.\n\n**Installation command:** `winget install Gyan.FFmpeg`",
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
import { DictationSettings as ProtoDictationSettings } from "../../proto/state"
|
||||
import { DictationSettings } from "../../DictationSettings"
|
||||
|
||||
/**
|
||||
* Converts proto DictationSettings to TypeScript DictationSettings
|
||||
*/
|
||||
export function convertProtoDictationSettingsToDictationSettings(
|
||||
protoDictationSettings: ProtoDictationSettings,
|
||||
): DictationSettings {
|
||||
return {
|
||||
voiceRecordingEnabled: protoDictationSettings.voiceRecordingEnabled,
|
||||
dictationLanguage: protoDictationSettings.dictationLanguage,
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Converts TypeScript DictationSettings to proto DictationSettings
|
||||
*/
|
||||
export function convertDictationSettingsToProtoDictationSettings(dictationSettings: DictationSettings): ProtoDictationSettings {
|
||||
return ProtoDictationSettings.create({
|
||||
voiceRecordingEnabled: dictationSettings.voiceRecordingEnabled,
|
||||
dictationLanguage: dictationSettings.dictationLanguage,
|
||||
})
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
import { DiffViewProvider } from "@/integrations/editor/DiffViewProvider"
|
||||
|
||||
export class ExternalDiffViewProvider extends DiffViewProvider {
|
||||
override async openDiffEditor(): Promise<void> {
|
||||
// The host bridge proto changes are not submitted yet.
|
||||
//getHostBridgeProvider().diffClient.openDiff(this.absolutePath)
|
||||
}
|
||||
}
|
||||
@@ -4,14 +4,12 @@ import {
|
||||
WorkspaceServiceClientImpl,
|
||||
EnvServiceClientImpl,
|
||||
WindowServiceClientImpl,
|
||||
DiffServiceClientImpl,
|
||||
} from "@generated/standalone/host-bridge-clients"
|
||||
import {
|
||||
WatchServiceClientInterface,
|
||||
WorkspaceServiceClientInterface,
|
||||
EnvServiceClientInterface,
|
||||
WindowServiceClientInterface,
|
||||
DiffServiceClientInterface,
|
||||
} from "@generated/hosts/host-bridge-client-types"
|
||||
import { HostBridgeClientProvider } from "@/hosts/host-provider-types"
|
||||
|
||||
@@ -25,7 +23,6 @@ export class ExternalHostBridgeClientManager implements HostBridgeClientProvider
|
||||
workspaceClient: WorkspaceServiceClientInterface
|
||||
envClient: EnvServiceClientInterface
|
||||
windowClient: WindowServiceClientInterface
|
||||
diffClient: DiffServiceClientInterface
|
||||
|
||||
constructor() {
|
||||
const address = process.env.HOST_BRIDGE_ADDRESS || "localhost:50052"
|
||||
@@ -35,7 +32,6 @@ export class ExternalHostBridgeClientManager implements HostBridgeClientProvider
|
||||
this.workspaceClient = new WorkspaceServiceClientImpl(this.channel)
|
||||
this.envClient = new EnvServiceClientImpl(this.channel)
|
||||
this.windowClient = new WindowServiceClientImpl(this.channel)
|
||||
this.diffClient = new DiffServiceClientImpl(this.channel)
|
||||
}
|
||||
|
||||
public close(): void {
|
||||
|
||||
@@ -13,12 +13,11 @@ import { ExternalHostBridgeClientManager } from "./host-bridge-client-manager"
|
||||
import { ExternalWebviewProvider } from "./ExternalWebviewProvider"
|
||||
import { WebviewProviderType } from "@/shared/webview/types"
|
||||
import { v4 as uuidv4 } from "uuid"
|
||||
import { ExternalDiffViewProvider } from "./ExternalDiffviewProvider"
|
||||
|
||||
async function main() {
|
||||
log("Starting standalone service...")
|
||||
|
||||
hostProviders.initializeHostProviders(createWebview, createDiffView, new ExternalHostBridgeClientManager())
|
||||
hostProviders.initializeHostProviders(createWebview, new ExternalHostBridgeClientManager())
|
||||
activate(extensionContext)
|
||||
const controller = new Controller(extensionContext, outputChannel, postMessage, uuidv4())
|
||||
startProtobusService(controller)
|
||||
@@ -61,12 +60,9 @@ function getProtobusServiceNames(packageDefinition: { [x: string]: any }): strin
|
||||
return protobusServiceNames
|
||||
}
|
||||
|
||||
function createWebview() {
|
||||
const createWebview = () => {
|
||||
return new ExternalWebviewProvider(extensionContext, outputChannel, WebviewProviderType.SIDEBAR)
|
||||
}
|
||||
function createDiffView() {
|
||||
return new ExternalDiffViewProvider()
|
||||
}
|
||||
|
||||
/**
|
||||
* Wraps a Promise-based handler function to make it compatible with gRPC's callback-based API.
|
||||
|
||||
Generated
+19
-10
@@ -9,7 +9,6 @@
|
||||
"version": "0.3.0",
|
||||
"dependencies": {
|
||||
"@floating-ui/react": "^0.27.4",
|
||||
"@fontsource/azeret-mono": "^5.2.9",
|
||||
"@heroui/react": "^2.8.0-beta.2",
|
||||
"@vscode/webview-ui-toolkit": "^1.4.0",
|
||||
"debounce": "^2.1.1",
|
||||
@@ -24,6 +23,7 @@
|
||||
"posthog-js": "^1.224.0",
|
||||
"pretty-bytes": "^6.1.1",
|
||||
"react": "^18.3.1",
|
||||
"react-countup": "^6.5.3",
|
||||
"react-dom": "^18.3.1",
|
||||
"react-remark": "^2.1.0",
|
||||
"react-textarea-autosize": "^8.5.7",
|
||||
@@ -1261,15 +1261,6 @@
|
||||
"integrity": "sha512-MDWhGtE+eHw5JW7lq4qhc5yRLS11ERl1c7Z6Xd0a58DozHES6EnNNwUWbMiG4J9Cgj053Bhk8zvlhFYKVhULwg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@fontsource/azeret-mono": {
|
||||
"version": "5.2.9",
|
||||
"resolved": "https://registry.npmjs.org/@fontsource/azeret-mono/-/azeret-mono-5.2.9.tgz",
|
||||
"integrity": "sha512-1qnbVspQPI38qhSTSidWU4bjG5ynWCfkMwfPxahqxejJO/u4yT1FbPqG73s4fDmQSuDQYoA8jfTpoQiod7+fuA==",
|
||||
"license": "OFL-1.1",
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ayuhito"
|
||||
}
|
||||
},
|
||||
"node_modules/@formatjs/ecma402-abstract": {
|
||||
"version": "2.3.4",
|
||||
"resolved": "https://registry.npmjs.org/@formatjs/ecma402-abstract/-/ecma402-abstract-2.3.4.tgz",
|
||||
@@ -8336,6 +8327,12 @@
|
||||
"layout-base": "^1.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/countup.js": {
|
||||
"version": "2.8.0",
|
||||
"resolved": "https://registry.npmjs.org/countup.js/-/countup.js-2.8.0.tgz",
|
||||
"integrity": "sha512-f7xEhX0awl4NOElHulrl4XRfKoNH3rB+qfNSZZyjSZhaAoUk6elvhH+MNxMmlmuUJ2/QNTWPSA7U4mNtIAKljQ==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/create-error-class": {
|
||||
"version": "3.0.2",
|
||||
"resolved": "https://registry.npmjs.org/create-error-class/-/create-error-class-3.0.2.tgz",
|
||||
@@ -13714,6 +13711,18 @@
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/react-countup": {
|
||||
"version": "6.5.3",
|
||||
"resolved": "https://registry.npmjs.org/react-countup/-/react-countup-6.5.3.tgz",
|
||||
"integrity": "sha512-udnqVQitxC7QWADSPDOxVWULkLvKUWrDapn5i53HE4DPRVgs+Y5rr4bo25qEl8jSh+0l2cToJgGMx+clxPM3+w==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"countup.js": "^2.8.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"react": ">= 16.3.0"
|
||||
}
|
||||
},
|
||||
"node_modules/react-devtools": {
|
||||
"version": "6.1.2",
|
||||
"resolved": "https://registry.npmjs.org/react-devtools/-/react-devtools-6.1.2.tgz",
|
||||
|
||||
@@ -16,7 +16,6 @@
|
||||
},
|
||||
"dependencies": {
|
||||
"@floating-ui/react": "^0.27.4",
|
||||
"@fontsource/azeret-mono": "^5.2.9",
|
||||
"@heroui/react": "^2.8.0-beta.2",
|
||||
"@vscode/webview-ui-toolkit": "^1.4.0",
|
||||
"debounce": "^2.1.1",
|
||||
@@ -31,6 +30,7 @@
|
||||
"posthog-js": "^1.224.0",
|
||||
"pretty-bytes": "^6.1.1",
|
||||
"react": "^18.3.1",
|
||||
"react-countup": "^6.5.3",
|
||||
"react-dom": "^18.3.1",
|
||||
"react-remark": "^2.1.0",
|
||||
"react-textarea-autosize": "^8.5.7",
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
import { VSCodeButton, VSCodeDivider, VSCodeLink, VSCodeDropdown, VSCodeOption } from "@vscode/webview-ui-toolkit/react"
|
||||
import { memo, useCallback, useEffect, useState, useRef } from "react"
|
||||
import { memo, useCallback, useEffect, useState } from "react"
|
||||
import { BadgeCent } from "lucide-react"
|
||||
import { useClineAuth } from "@/context/ClineAuthContext"
|
||||
import VSCodeButtonLink from "../common/VSCodeButtonLink"
|
||||
import ClineLogoWhite from "../../assets/ClineLogoWhite"
|
||||
import CountUp from "react-countup"
|
||||
import CreditsHistoryTable from "./CreditsHistoryTable"
|
||||
import { UsageTransaction, PaymentTransaction } from "@shared/ClineAccount"
|
||||
import { useExtensionState } from "@/context/ExtensionStateContext"
|
||||
@@ -11,64 +13,6 @@ import { EmptyRequest } from "@shared/proto/common"
|
||||
import { UserOrganization, UserOrganizationUpdateRequest } from "@shared/proto/account"
|
||||
import { formatCreditsBalance } from "@/utils/format"
|
||||
|
||||
// Custom hook for animated credit display with styled decimals
|
||||
const useAnimatedCredits = (targetValue: number, duration: number = 660) => {
|
||||
const [currentValue, setCurrentValue] = useState(0)
|
||||
const animationRef = useRef<number>()
|
||||
const startTimeRef = useRef<number>()
|
||||
|
||||
useEffect(() => {
|
||||
const animate = (timestamp: number) => {
|
||||
if (!startTimeRef.current) {
|
||||
startTimeRef.current = timestamp
|
||||
}
|
||||
|
||||
const elapsed = timestamp - startTimeRef.current
|
||||
const progress = Math.min(elapsed / duration, 1)
|
||||
|
||||
// Easing function (ease-out)
|
||||
const easedProgress = 1 - Math.pow(1 - progress, 3)
|
||||
const newValue = easedProgress * targetValue
|
||||
|
||||
setCurrentValue(newValue)
|
||||
|
||||
if (progress < 1) {
|
||||
animationRef.current = requestAnimationFrame(animate)
|
||||
}
|
||||
}
|
||||
|
||||
// Reset and start animation
|
||||
startTimeRef.current = undefined
|
||||
animationRef.current = requestAnimationFrame(animate)
|
||||
|
||||
return () => {
|
||||
if (animationRef.current) {
|
||||
cancelAnimationFrame(animationRef.current)
|
||||
}
|
||||
}
|
||||
}, [targetValue, duration])
|
||||
|
||||
return currentValue
|
||||
}
|
||||
|
||||
// Custom component to handle styled credit display
|
||||
const StyledCreditDisplay = ({ balance }: { balance: number }) => {
|
||||
const animatedValue = useAnimatedCredits(formatCreditsBalance(balance))
|
||||
const formatted = animatedValue.toFixed(4)
|
||||
const parts = formatted.split(".")
|
||||
const wholePart = parts[0]
|
||||
const decimalPart = parts[1] || "0000"
|
||||
const firstTwoDecimals = decimalPart.slice(0, 2)
|
||||
const lastTwoDecimals = decimalPart.slice(2)
|
||||
|
||||
return (
|
||||
<span className="font-azeret-mono font-light tabular-nums">
|
||||
{wholePart}.{firstTwoDecimals}
|
||||
<span className="text-[var(--vscode-descriptionForeground)]">{lastTwoDecimals}</span>
|
||||
</span>
|
||||
)
|
||||
}
|
||||
|
||||
type VSCodeDropdownChangeEvent = Event & {
|
||||
target: {
|
||||
value: string
|
||||
@@ -202,13 +146,13 @@ export const ClineAccountView = () => {
|
||||
<div className="flex flex-col pr-3 h-full">
|
||||
<div className="flex flex-col w-full">
|
||||
<div className="flex items-center mb-6 flex-wrap gap-y-4">
|
||||
{/* {user.photoUrl ? (
|
||||
{user.photoUrl ? (
|
||||
<img src={user.photoUrl} alt="Profile" className="size-16 rounded-full mr-4" />
|
||||
) : ( */}
|
||||
<div className="size-16 rounded-full bg-[var(--vscode-button-background)] flex items-center justify-center text-2xl text-[var(--vscode-button-foreground)] mr-4">
|
||||
{user.displayName?.[0] || user.email?.[0] || "?"}
|
||||
</div>
|
||||
{/* )} */}
|
||||
) : (
|
||||
<div className="size-16 rounded-full bg-[var(--vscode-button-background)] flex items-center justify-center text-2xl text-[var(--vscode-button-foreground)] mr-4">
|
||||
{user.displayName?.[0] || user.email?.[0] || "?"}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="flex flex-col">
|
||||
{user.displayName && (
|
||||
@@ -256,9 +200,7 @@ export const ClineAccountView = () => {
|
||||
|
||||
{activeOrganization === null && (
|
||||
<div className="w-full flex flex-col items-center">
|
||||
<div className="text-sm text-[var(--vscode-descriptionForeground)] mb-3 font-azeret-mono font-light">
|
||||
CURRENT BALANCE
|
||||
</div>
|
||||
<div className="text-sm text-[var(--vscode-descriptionForeground)] mb-3">CURRENT BALANCE</div>
|
||||
|
||||
<div className="text-4xl font-bold text-[var(--vscode-foreground)] mb-6 flex items-center gap-2">
|
||||
{isLoading ? (
|
||||
@@ -269,7 +211,8 @@ export const ClineAccountView = () => {
|
||||
<span>----</span>
|
||||
) : (
|
||||
<>
|
||||
<StyledCreditDisplay balance={balance} />
|
||||
<BadgeCent className="size-6 text-[var(--vscode-foreground)]" />
|
||||
<CountUp end={formatCreditsBalance(balance)} duration={0.66} decimals={4} />
|
||||
</>
|
||||
)}
|
||||
<VSCodeButton appearance="icon" className="mt-1" onClick={getUserCredits}>
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { VSCodeBadge, VSCodeButton, VSCodeProgressRing } from "@vscode/webview-ui-toolkit/react"
|
||||
import { VSCodeBadge, VSCodeProgressRing } from "@vscode/webview-ui-toolkit/react"
|
||||
import deepEqual from "fast-deep-equal"
|
||||
import React, { memo, MouseEvent, useCallback, useEffect, useMemo, useRef, useState } from "react"
|
||||
import styled from "styled-components"
|
||||
@@ -36,7 +36,6 @@ import NewTaskPreview from "./NewTaskPreview"
|
||||
import ReportBugPreview from "./ReportBugPreview"
|
||||
import UserMessage from "./UserMessage"
|
||||
import QuoteButton from "./QuoteButton"
|
||||
import { useClineAuth } from "@/context/ClineAuthContext"
|
||||
|
||||
const normalColor = "var(--vscode-foreground)"
|
||||
const errorColor = "var(--vscode-errorForeground)"
|
||||
@@ -185,7 +184,6 @@ export const ChatRowContent = memo(
|
||||
sendMessageFromChatRow,
|
||||
onSetQuote,
|
||||
}: ChatRowContentProps) => {
|
||||
const { handleSignIn, clineUser } = useClineAuth()
|
||||
const { mcpServers, mcpMarketplaceCatalog, onRelinquishControl, apiConfiguration } = useExtensionState()
|
||||
const [seeNewChangesDisabled, setSeeNewChangesDisabled] = useState(false)
|
||||
const [quoteButtonState, setQuoteButtonState] = useState<QuoteButtonState>({
|
||||
@@ -1008,23 +1006,6 @@ export const ChatRowContent = memo(
|
||||
.
|
||||
</>
|
||||
)}
|
||||
{apiRequestFailedMessage?.includes(
|
||||
"Unauthorized: Please sign in to Cline before trying again.", // match with cline.ts (TODO: remove after some time)
|
||||
) && (
|
||||
<>
|
||||
<br />
|
||||
<br />
|
||||
{clineUser ? (
|
||||
<span style={{ color: "var(--vscode-descriptionForeground)" }}>
|
||||
(Click "Retry" below)
|
||||
</span>
|
||||
) : (
|
||||
<VSCodeButton onClick={handleSignIn} className="w-full mb-4">
|
||||
Sign in to Cline
|
||||
</VSCodeButton>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
</p>
|
||||
)
|
||||
})()}
|
||||
|
||||
@@ -45,6 +45,7 @@ import { useClickAway, useEvent, useWindowSize } from "react-use"
|
||||
import styled from "styled-components"
|
||||
import ClineRulesToggleModal from "../cline-rules/ClineRulesToggleModal"
|
||||
import ServersToggleModal from "./ServersToggleModal"
|
||||
import VoiceRecorder from "./VoiceRecorder"
|
||||
|
||||
const getImageDimensions = (dataUrl: string): Promise<{ width: number; height: number }> => {
|
||||
return new Promise((resolve, reject) => {
|
||||
@@ -280,6 +281,7 @@ const ChatTextArea = forwardRef<HTMLTextAreaElement, ChatTextAreaProps>(
|
||||
platform,
|
||||
localWorkflowToggles,
|
||||
globalWorkflowToggles,
|
||||
dictationSettings,
|
||||
} = useExtensionState()
|
||||
const [isTextAreaFocused, setIsTextAreaFocused] = useState(false)
|
||||
const [isDraggingOver, setIsDraggingOver] = useState(false)
|
||||
@@ -1564,7 +1566,7 @@ const ChatTextArea = forwardRef<HTMLTextAreaElement, ChatTextAreaProps>(
|
||||
// borderLeft: "9px solid transparent", // NOTE: react-textarea-autosize doesn't calculate correct height when using borderLeft/borderRight so we need to use horizontal padding instead
|
||||
// Instead of using boxShadow, we use a div with a border to better replicate the behavior when the textarea is focused
|
||||
// boxShadow: "0px 0px 0px 1px var(--vscode-input-border)",
|
||||
padding: "9px 28px 9px 9px",
|
||||
padding: `9px ${dictationSettings?.voiceRecordingEnabled ? "48" : "28"}px 9px 9px`,
|
||||
cursor: "text",
|
||||
flex: 1,
|
||||
zIndex: 1,
|
||||
@@ -1605,10 +1607,9 @@ const ChatTextArea = forwardRef<HTMLTextAreaElement, ChatTextAreaProps>(
|
||||
position: "absolute",
|
||||
right: 23,
|
||||
display: "flex",
|
||||
alignItems: "flex-end",
|
||||
alignItems: "center",
|
||||
height: textAreaBaseHeight || 31,
|
||||
bottom: 9.5, // should be 10 but doesn't look good on mac
|
||||
paddingBottom: "8px",
|
||||
zIndex: 2,
|
||||
}}>
|
||||
<div
|
||||
@@ -1617,6 +1618,35 @@ const ChatTextArea = forwardRef<HTMLTextAreaElement, ChatTextAreaProps>(
|
||||
flexDirection: "row",
|
||||
alignItems: "center",
|
||||
}}>
|
||||
{dictationSettings?.voiceRecordingEnabled === true && (
|
||||
<VoiceRecorder
|
||||
onTranscription={(text) => {
|
||||
// Remove any processing text first
|
||||
const cleanedValue = inputValue.replace(/\s*\[Transcribing\.\.\.\]$/, "")
|
||||
// Append the transcribed text to the cleaned input
|
||||
const newValue = cleanedValue + (cleanedValue ? " " : "") + text
|
||||
setInputValue(newValue)
|
||||
// Focus the textarea and move cursor to end
|
||||
setTimeout(() => {
|
||||
if (textAreaRef.current) {
|
||||
textAreaRef.current.focus()
|
||||
const length = newValue.length
|
||||
textAreaRef.current.setSelectionRange(length, length)
|
||||
}
|
||||
}, 0)
|
||||
}}
|
||||
onProcessingStateChange={(isProcessing, message) => {
|
||||
if (isProcessing && message) {
|
||||
// Show processing message in input
|
||||
const processingText = inputValue + (inputValue ? " " : "") + `[${message}]`
|
||||
setInputValue(processingText)
|
||||
}
|
||||
// When processing is done, the onTranscription callback will handle the final text
|
||||
}}
|
||||
disabled={sendingDisabled}
|
||||
language={dictationSettings?.dictationLanguage || "en"}
|
||||
/>
|
||||
)}
|
||||
{/* <div
|
||||
className={`input-icon-button ${shouldDisableImages ? "disabled" : ""} codicon codicon-device-camera`}
|
||||
onClick={() => {
|
||||
|
||||
@@ -0,0 +1,223 @@
|
||||
import React, { useState, useCallback, useEffect, useRef } from "react"
|
||||
import { VoiceServiceClient } from "@/services/grpc-client"
|
||||
import {
|
||||
StartRecordingRequest,
|
||||
StopRecordingRequest,
|
||||
TranscribeAudioRequest,
|
||||
GetRecordingStatusRequest,
|
||||
} from "@shared/proto/voice"
|
||||
|
||||
import HeroTooltip from "../common/HeroTooltip"
|
||||
import { formatSeconds } from "@/utils/format"
|
||||
|
||||
interface VoiceRecorderProps {
|
||||
onTranscription: (text: string) => void
|
||||
onProcessingStateChange?: (isProcessing: boolean, message?: string) => void
|
||||
disabled?: boolean
|
||||
language?: string
|
||||
}
|
||||
|
||||
const MAX_DURATION = 5 * 60 // 5 minutes in seconds
|
||||
|
||||
const VoiceRecorder: React.FC<VoiceRecorderProps> = ({
|
||||
onTranscription,
|
||||
onProcessingStateChange,
|
||||
disabled = false,
|
||||
language = "en",
|
||||
}) => {
|
||||
const [isRecording, setIsRecording] = useState(false)
|
||||
const [isProcessing, setIsProcessing] = useState(false)
|
||||
const [recordingDuration, setRecordingDuration] = useState(0)
|
||||
const [error, setError] = useState<string | null>(null)
|
||||
const pollingIntervalRef = useRef<NodeJS.Timeout | null>(null)
|
||||
|
||||
const startRecording = useCallback(async () => {
|
||||
try {
|
||||
setIsRecording(true)
|
||||
setError(null) // Clear any previous errors
|
||||
onProcessingStateChange?.(false) // Clear any previous processing state
|
||||
setRecordingDuration(0) // Reset recording duration
|
||||
|
||||
// Call Extension Host to start recording
|
||||
const response = await VoiceServiceClient.startRecording(StartRecordingRequest.create({}))
|
||||
|
||||
if (!response.success) {
|
||||
console.error("Failed to start recording:", response.error)
|
||||
setIsRecording(false)
|
||||
setError(response.error || "Failed to start recording")
|
||||
onProcessingStateChange?.(true, response.error || "Failed to start recording")
|
||||
return
|
||||
}
|
||||
|
||||
console.log("Recording started successfully")
|
||||
} catch (error) {
|
||||
console.error("Error starting recording:", error)
|
||||
setIsRecording(false)
|
||||
const errorMessage = error instanceof Error ? error.message : "Failed to start recording"
|
||||
setError(errorMessage)
|
||||
onProcessingStateChange?.(true, errorMessage)
|
||||
}
|
||||
}, [onProcessingStateChange])
|
||||
|
||||
const stopRecording = useCallback(async () => {
|
||||
try {
|
||||
setIsRecording(false)
|
||||
setIsProcessing(true)
|
||||
onProcessingStateChange?.(true, "Processing...")
|
||||
|
||||
// Call Extension Host to stop recording and get audio
|
||||
const response = await VoiceServiceClient.stopRecording(StopRecordingRequest.create({}))
|
||||
|
||||
if (!response.success) {
|
||||
console.error("Failed to stop recording:", response.error)
|
||||
setIsProcessing(false)
|
||||
const errorMessage = response.error || "Failed to stop recording"
|
||||
setError(errorMessage)
|
||||
onProcessingStateChange?.(true, errorMessage)
|
||||
return
|
||||
}
|
||||
|
||||
if (!response.audioBase64) {
|
||||
console.error("No audio data received")
|
||||
setIsProcessing(false)
|
||||
const errorMessage = "No audio data received"
|
||||
setError(errorMessage)
|
||||
onProcessingStateChange?.(true, errorMessage)
|
||||
return
|
||||
}
|
||||
|
||||
// Update processing state for transcription
|
||||
onProcessingStateChange?.(true, "Transcribing...")
|
||||
|
||||
// Transcribe the audio using OpenAI Whisper
|
||||
const transcriptionResponse = await VoiceServiceClient.transcribeAudio(
|
||||
TranscribeAudioRequest.create({
|
||||
audioBase64: response.audioBase64,
|
||||
language: language,
|
||||
}),
|
||||
)
|
||||
|
||||
if (transcriptionResponse.error) {
|
||||
console.error("Transcription error:", transcriptionResponse.error)
|
||||
setError(transcriptionResponse.error)
|
||||
// Show the error message in the UI
|
||||
onProcessingStateChange?.(true, transcriptionResponse.error)
|
||||
// Clear the error after a delay
|
||||
setTimeout(() => {
|
||||
setError(null)
|
||||
onProcessingStateChange?.(false)
|
||||
}, 5000)
|
||||
} else if (transcriptionResponse.text) {
|
||||
setError(null)
|
||||
onTranscription(transcriptionResponse.text)
|
||||
onProcessingStateChange?.(false)
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Error stopping recording:", error)
|
||||
const errorMessage = error instanceof Error ? error.message : "An error occurred"
|
||||
setError(errorMessage)
|
||||
onProcessingStateChange?.(true, errorMessage)
|
||||
} finally {
|
||||
setIsProcessing(false)
|
||||
}
|
||||
}, [onTranscription, onProcessingStateChange])
|
||||
|
||||
// Poll recording status while recording to update duration
|
||||
useEffect(() => {
|
||||
const pollRecordingStatus = async () => {
|
||||
try {
|
||||
const statusResponse = await VoiceServiceClient.getRecordingStatus(GetRecordingStatusRequest.create({}))
|
||||
if (statusResponse.isRecording) {
|
||||
setRecordingDuration(Math.floor(statusResponse.durationSeconds))
|
||||
|
||||
// Auto-stop if max duration reached
|
||||
if (statusResponse.durationSeconds >= MAX_DURATION) {
|
||||
stopRecording()
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Error polling recording status:", error)
|
||||
}
|
||||
}
|
||||
|
||||
if (isRecording && !isProcessing) {
|
||||
// Start polling immediately, then every second
|
||||
pollRecordingStatus()
|
||||
pollingIntervalRef.current = setInterval(pollRecordingStatus, 1000)
|
||||
} else {
|
||||
// Clear polling when not recording
|
||||
if (pollingIntervalRef.current) {
|
||||
clearInterval(pollingIntervalRef.current)
|
||||
pollingIntervalRef.current = null
|
||||
}
|
||||
}
|
||||
|
||||
// Cleanup on unmount
|
||||
return () => {
|
||||
if (pollingIntervalRef.current) {
|
||||
clearInterval(pollingIntervalRef.current)
|
||||
pollingIntervalRef.current = null
|
||||
}
|
||||
}
|
||||
}, [isRecording, isProcessing, stopRecording])
|
||||
|
||||
const handleClick = useCallback(() => {
|
||||
if (disabled || isProcessing) return
|
||||
|
||||
if (error) return setError(null)
|
||||
|
||||
if (isRecording) {
|
||||
stopRecording()
|
||||
} else {
|
||||
startRecording()
|
||||
}
|
||||
}, [isRecording, startRecording, stopRecording, disabled, isProcessing, error])
|
||||
|
||||
const getIconClass = () => {
|
||||
if (isProcessing) return "codicon-loading"
|
||||
if (isRecording) return "codicon-stop-circle"
|
||||
if (error) return "codicon-error"
|
||||
return "codicon-mic"
|
||||
}
|
||||
|
||||
const getIconColor = () => {
|
||||
if (isRecording) return "var(--vscode-errorForeground)"
|
||||
if (error) return "var(--vscode-errorForeground)"
|
||||
return ""
|
||||
}
|
||||
|
||||
const getIconAnimation = () => {
|
||||
if (isProcessing) return "animate-spin"
|
||||
if (isRecording) return "animate-pulse"
|
||||
return ""
|
||||
}
|
||||
|
||||
const getIconAdjustment = () => {
|
||||
if (isProcessing) return "mt-0"
|
||||
if (isRecording) return "mt-1"
|
||||
if (error) return "mt-1"
|
||||
return "mt-0.5"
|
||||
}
|
||||
|
||||
const getRecTooltipContent = () => {
|
||||
if (isProcessing) return "Transcribing..."
|
||||
if (isRecording) return `Stop Recording (${formatSeconds(recordingDuration)}/${formatSeconds(MAX_DURATION)})`
|
||||
if (error) return `Error: ${error}`
|
||||
return null
|
||||
}
|
||||
|
||||
return (
|
||||
<HeroTooltip content={getRecTooltipContent()} placement="top">
|
||||
<div
|
||||
className={`input-icon-button mr-1.5 text-base ${getIconAdjustment()} ${getIconAnimation()} ${disabled || isProcessing ? "disabled" : ""}`}
|
||||
onClick={handleClick}
|
||||
style={{
|
||||
color: getIconColor(),
|
||||
}}>
|
||||
<span className={`codicon ${getIconClass()}`} />
|
||||
</div>
|
||||
</HeroTooltip>
|
||||
)
|
||||
}
|
||||
|
||||
export default VoiceRecorder
|
||||
@@ -0,0 +1,16 @@
|
||||
import styled from "styled-components"
|
||||
|
||||
const CollapsibleContent = styled.div<{ isOpen: boolean }>`
|
||||
overflow: hidden;
|
||||
transition:
|
||||
max-height 0.3s ease-in-out,
|
||||
opacity 0.3s ease-in-out,
|
||||
margin-top 0.3s ease-in-out,
|
||||
visibility 0.3s ease-in-out;
|
||||
max-height: ${({ isOpen }) => (isOpen ? "1000px" : "0")};
|
||||
opacity: ${({ isOpen }) => (isOpen ? 1 : 0)};
|
||||
margin-top: ${({ isOpen }) => (isOpen ? "15px" : "0")};
|
||||
visibility: ${({ isOpen }) => (isOpen ? "visible" : "hidden")};
|
||||
`
|
||||
|
||||
export default CollapsibleContent
|
||||
@@ -5,6 +5,7 @@ import { BROWSER_VIEWPORT_PRESETS } from "../../../../../src/shared/BrowserSetti
|
||||
import { useExtensionState } from "../../../context/ExtensionStateContext"
|
||||
import { BrowserServiceClient } from "../../../services/grpc-client"
|
||||
import { EmptyRequest, StringRequest } from "@shared/proto/common"
|
||||
import CollapsibleContent from "../CollapsibleContent"
|
||||
import { updateBrowserSetting } from "../utils/settingsHandlers"
|
||||
import { DebouncedTextField } from "../common/DebouncedTextField"
|
||||
import Section from "../Section"
|
||||
@@ -43,19 +44,6 @@ const ConnectionStatusIndicator = ({
|
||||
)
|
||||
}
|
||||
|
||||
const CollapsibleContent = styled.div<{ isOpen: boolean }>`
|
||||
overflow: hidden;
|
||||
transition:
|
||||
max-height 0.3s ease-in-out,
|
||||
opacity 0.3s ease-in-out,
|
||||
margin-top 0.3s ease-in-out,
|
||||
visibility 0.3s ease-in-out;
|
||||
max-height: ${({ isOpen }) => (isOpen ? "1000px" : "0")}; // Sufficiently large height
|
||||
opacity: ${({ isOpen }) => (isOpen ? 1 : 0)};
|
||||
margin-top: ${({ isOpen }) => (isOpen ? "15px" : "0")};
|
||||
visibility: ${({ isOpen }) => (isOpen ? "visible" : "hidden")};
|
||||
`
|
||||
|
||||
export const BrowserSettingsSection: React.FC<BrowserSettingsSectionProps> = ({ renderSectionHeader }) => {
|
||||
const { browserSettings } = useExtensionState()
|
||||
const [isCheckingConnection, setIsCheckingConnection] = useState(false)
|
||||
|
||||
@@ -2,8 +2,11 @@ import { VSCodeCheckbox, VSCodeDropdown, VSCodeOption } from "@vscode/webview-ui
|
||||
import { useExtensionState } from "@/context/ExtensionStateContext"
|
||||
import { memo } from "react"
|
||||
import { OpenAIReasoningEffort } from "@shared/ChatSettings"
|
||||
import { SUPPORTED_DICTATION_LANGUAGES } from "@shared/DictationSettings"
|
||||
// import CollapsibleContent from "../CollapsibleContent"
|
||||
import { updateSetting } from "../utils/settingsHandlers"
|
||||
import { convertChatSettingsToProtoChatSettings } from "@shared/proto-conversions/state/chat-settings-conversion"
|
||||
import { convertDictationSettingsToProtoDictationSettings } from "@shared/proto-conversions/state/dictation-settings-conversion"
|
||||
import Section from "../Section"
|
||||
|
||||
interface FeatureSettingsSectionProps {
|
||||
@@ -11,8 +14,14 @@ interface FeatureSettingsSectionProps {
|
||||
}
|
||||
|
||||
const FeatureSettingsSection = ({ renderSectionHeader }: FeatureSettingsSectionProps) => {
|
||||
const { enableCheckpointsSetting, mcpMarketplaceEnabled, mcpRichDisplayEnabled, mcpResponsesCollapsed, chatSettings } =
|
||||
useExtensionState()
|
||||
const {
|
||||
enableCheckpointsSetting,
|
||||
mcpMarketplaceEnabled,
|
||||
mcpRichDisplayEnabled,
|
||||
mcpResponsesCollapsed,
|
||||
chatSettings,
|
||||
dictationSettings,
|
||||
} = useExtensionState()
|
||||
|
||||
const handleReasoningEffortChange = (newValue: OpenAIReasoningEffort) => {
|
||||
if (!chatSettings) return
|
||||
@@ -103,9 +112,65 @@ const FeatureSettingsSection = ({ renderSectionHeader }: FeatureSettingsSectionP
|
||||
<VSCodeOption value="high">High</VSCodeOption>
|
||||
</VSCodeDropdown>
|
||||
<p className="text-xs mt-[5px] text-[var(--vscode-descriptionForeground)]">
|
||||
Reasoning effort for the OpenAI family of models(applies to all OpenAI model providers)
|
||||
Reasoning effort for the OpenAI family of models (applies to all OpenAI model providers)
|
||||
</p>
|
||||
</div>
|
||||
<div style={{ marginTop: 10 }}>
|
||||
<div>
|
||||
<VSCodeCheckbox
|
||||
checked={dictationSettings?.voiceRecordingEnabled}
|
||||
onChange={(e: any) => {
|
||||
const checked = e.target.checked === true
|
||||
const updatedDictationSettings = {
|
||||
...dictationSettings,
|
||||
voiceRecordingEnabled: checked,
|
||||
}
|
||||
const protoDictationSettings =
|
||||
convertDictationSettingsToProtoDictationSettings(updatedDictationSettings)
|
||||
updateSetting("dictationSettings", protoDictationSettings)
|
||||
}}>
|
||||
Enable Dictation
|
||||
</VSCodeCheckbox>
|
||||
<p className="text-xs text-[var(--vscode-descriptionForeground)] mt-1">
|
||||
Enables voice recording with automatic transcription. Requires a Cline Account for the
|
||||
transcription model.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
{/* TODO: Fix and use CollapsibleContent, the animation is good but it breaks the dropdown
|
||||
<CollapsibleContent isOpen={dictationSettings?.voiceRecordingEnabled}> */}
|
||||
<div className={dictationSettings?.voiceRecordingEnabled ? "mt-4" : "hidden"}>
|
||||
<label
|
||||
htmlFor="dictation-language-dropdown"
|
||||
className="block text-sm font-medium text-[var(--vscode-foreground)] mb-1">
|
||||
Dictation Language
|
||||
</label>
|
||||
<VSCodeDropdown
|
||||
id="dictation-language-dropdown"
|
||||
currentValue={dictationSettings?.dictationLanguage || "en"}
|
||||
onChange={(e: any) => {
|
||||
const newValue = e.target.value
|
||||
const updatedDictationSettings = {
|
||||
...dictationSettings,
|
||||
dictationLanguage: newValue,
|
||||
}
|
||||
const protoDictationSettings =
|
||||
convertDictationSettingsToProtoDictationSettings(updatedDictationSettings)
|
||||
updateSetting("dictationSettings", protoDictationSettings)
|
||||
}}
|
||||
className="w-full">
|
||||
{SUPPORTED_DICTATION_LANGUAGES.map((language) => (
|
||||
<VSCodeOption key={language.code} value={language.code} className="py-0.5">
|
||||
{language.name}
|
||||
</VSCodeOption>
|
||||
))}
|
||||
</VSCodeDropdown>
|
||||
<p className="text-xs mt-1 text-[var(--vscode-descriptionForeground)]">
|
||||
The language you want to speak to the Dictation service in. Separate from preferred UI language.
|
||||
</p>
|
||||
</div>
|
||||
{/* </CollapsibleContent> */}
|
||||
</div>
|
||||
</div>
|
||||
</Section>
|
||||
</div>
|
||||
|
||||
@@ -16,6 +16,7 @@ import { convertProtoMcpServersToMcpServers } from "@shared/proto-conversions/mc
|
||||
import { DEFAULT_AUTO_APPROVAL_SETTINGS } from "@shared/AutoApprovalSettings"
|
||||
import { DEFAULT_BROWSER_SETTINGS, BrowserSettings } from "@shared/BrowserSettings"
|
||||
import { ChatSettings, DEFAULT_CHAT_SETTINGS } from "@shared/ChatSettings"
|
||||
import { DictationSettings, DEFAULT_DICTATION_SETTINGS } from "@shared/DictationSettings"
|
||||
import { DEFAULT_PLATFORM, ExtensionMessage, ExtensionState } from "@shared/ExtensionMessage"
|
||||
import { TelemetrySetting } from "@shared/TelemetrySetting"
|
||||
import { findLastIndex } from "@shared/array"
|
||||
@@ -68,6 +69,7 @@ interface ExtensionStateContextType extends ExtensionState {
|
||||
setTerminalOutputLineLimit: (value: number) => void
|
||||
setDefaultTerminalProfile: (value: string) => void
|
||||
setChatSettings: (value: ChatSettings) => void
|
||||
setDictationSettings: (value: DictationSettings) => void
|
||||
setMcpServers: (value: McpServer[]) => void
|
||||
setRequestyModels: (value: Record<string, ModelInfo>) => void
|
||||
setGlobalClineRulesToggles: (toggles: Record<string, boolean>) => void
|
||||
@@ -184,6 +186,7 @@ export const ExtensionStateContextProvider: React.FC<{
|
||||
shouldShowAnnouncement: false,
|
||||
autoApprovalSettings: DEFAULT_AUTO_APPROVAL_SETTINGS,
|
||||
browserSettings: DEFAULT_BROWSER_SETTINGS,
|
||||
dictationSettings: DEFAULT_DICTATION_SETTINGS,
|
||||
chatSettings: DEFAULT_CHAT_SETTINGS,
|
||||
platform: DEFAULT_PLATFORM,
|
||||
telemetrySetting: "unset",
|
||||
@@ -270,8 +273,13 @@ export const ExtensionStateContextProvider: React.FC<{
|
||||
const currentVersion = prevState.autoApprovalSettings?.version ?? 1
|
||||
const shouldUpdateAutoApproval = incomingVersion > currentVersion
|
||||
|
||||
// Always preserve chat settings from the current state
|
||||
// This prevents the backend from overwriting user changes
|
||||
const chatSettings = prevState.chatSettings || stateData.chatSettings
|
||||
|
||||
const newState = {
|
||||
...stateData,
|
||||
chatSettings, // Use preserved or incoming chat settings
|
||||
autoApprovalSettings: shouldUpdateAutoApproval
|
||||
? stateData.autoApprovalSettings
|
||||
: prevState.autoApprovalSettings,
|
||||
@@ -280,9 +288,7 @@ export const ExtensionStateContextProvider: React.FC<{
|
||||
// Update welcome screen state based on API configuration
|
||||
setShowWelcome(!newState.welcomeViewCompleted)
|
||||
setDidHydrateState(true)
|
||||
|
||||
console.log("[DEBUG] returning new state in ESC")
|
||||
|
||||
return newState
|
||||
})
|
||||
} catch (error) {
|
||||
@@ -816,6 +822,11 @@ export const ExtensionStateContextProvider: React.FC<{
|
||||
...prevState,
|
||||
browserSettings: value,
|
||||
})),
|
||||
setDictationSettings: (value: DictationSettings) =>
|
||||
setState((prevState) => ({
|
||||
...prevState,
|
||||
dictationSettings: value,
|
||||
})),
|
||||
}
|
||||
|
||||
return <ExtensionStateContext.Provider value={contextValue}>{children}</ExtensionStateContext.Provider>
|
||||
|
||||
@@ -6,11 +6,6 @@
|
||||
|
||||
@config "../tailwind.config.js";
|
||||
|
||||
/* Import Azeret Mono font from local package */
|
||||
@import "@fontsource/azeret-mono/300.css";
|
||||
@import "@fontsource/azeret-mono/400.css";
|
||||
@import "@fontsource/azeret-mono/700.css";
|
||||
|
||||
textarea:focus {
|
||||
outline: 1.5px solid var(--vscode-focusBorder, #007fd4);
|
||||
}
|
||||
|
||||
@@ -12,6 +12,8 @@ import { WebServiceDefinition } from "@shared/proto/web"
|
||||
import { ModelsServiceDefinition } from "@shared/proto/models"
|
||||
import { SlashServiceDefinition } from "@shared/proto/slash"
|
||||
import { UiServiceDefinition } from "@shared/proto/ui"
|
||||
import { VoiceServiceDefinition } from "@shared/proto/voice"
|
||||
|
||||
const AccountServiceClient = createGrpcClient(AccountServiceDefinition)
|
||||
const BrowserServiceClient = createGrpcClient(BrowserServiceDefinition)
|
||||
const CheckpointsServiceClient = createGrpcClient(CheckpointsServiceDefinition)
|
||||
@@ -23,6 +25,8 @@ const WebServiceClient = createGrpcClient(WebServiceDefinition)
|
||||
const ModelsServiceClient = createGrpcClient(ModelsServiceDefinition)
|
||||
const SlashServiceClient = createGrpcClient(SlashServiceDefinition)
|
||||
const UiServiceClient = createGrpcClient(UiServiceDefinition)
|
||||
const VoiceServiceClient = createGrpcClient(VoiceServiceDefinition)
|
||||
|
||||
export {
|
||||
AccountServiceClient,
|
||||
BrowserServiceClient,
|
||||
@@ -35,5 +39,6 @@ export {
|
||||
ModelsServiceClient,
|
||||
SlashServiceClient,
|
||||
UiServiceClient,
|
||||
VoiceServiceClient,
|
||||
}
|
||||
//# sourceMappingURL=grpc-client.js.map
|
||||
|
||||
@@ -62,3 +62,16 @@ export function formatSize(bytes?: number) {
|
||||
|
||||
return prettyBytes(bytes)
|
||||
}
|
||||
|
||||
export function formatSeconds(seconds?: number): string {
|
||||
if (seconds === undefined) {
|
||||
return "--:--"
|
||||
}
|
||||
|
||||
const mins = Math.floor(seconds / 60)
|
||||
const secs = Math.floor(seconds % 60)
|
||||
.toString()
|
||||
.padStart(2, "0")
|
||||
|
||||
return `${mins}:${secs}`
|
||||
}
|
||||
|
||||
@@ -4,11 +4,7 @@ const { heroui } = require("@heroui/react")
|
||||
module.exports = {
|
||||
content: ["./src/**/*.{js,ts,jsx,tsx,mdx}", "./node_modules/@heroui/theme/dist/**/*.{js,ts,jsx,tsx}"],
|
||||
theme: {
|
||||
extend: {
|
||||
fontFamily: {
|
||||
"azeret-mono": ['"Azeret Mono"', "monospace"],
|
||||
},
|
||||
},
|
||||
extend: {},
|
||||
},
|
||||
darkMode: "class",
|
||||
plugins: [
|
||||
|
||||
Reference in New Issue
Block a user