Files
cline/.env.example
T
cdffc002eb feat(telemetry): add OpenTelemetry integration (#6605)
* feat: Modular telemetry architecture with Jitsu provider support

- Add dual-provider telemetry architecture supporting both Jitsu and PostHog
- Implement JitsuTelemetryProvider with full API compatibility
- Add required telemetry bypass for critical system health events
- Create modular event handler base class for future extensibility
- Add Jitsu configuration with environment variable controls
- Update TelemetryService to support multiple providers with error isolation
- Add .env.example template for development setup
- Maintain backward compatibility with existing PostHog integration
- Enable easy PostHog removal via POSTHOG_TELEMETRY_ENABLED=false
- Install dotenv for local development environment support

Key benefits:
- Dual tracking during transition period
- Error isolation between providers
- Memory efficient static method architecture
- Easy provider enable/disable via environment variables
- Wednesday deployment ready for Jitsu migration

* fix(build): Load environment variables from .env file during development builds

- Add dotenv.config() to esbuild.mjs to load .env variables
- Include all telemetry-related environment variables in build injection:
  - TELEMETRY_SERVICE_API_KEY (PostHog)
  - ERROR_SERVICE_API_KEY (PostHog error tracking)
  - JITSU_WRITE_KEY (Jitsu telemetry)
  - JITSU_HOST (Jitsu host URL)
  - JITSU_ENABLED (Jitsu provider control)
  - POSTHOG_TELEMETRY_ENABLED (PostHog provider control)

This ensures telemetry services work correctly in development builds
by properly injecting API keys and configuration from .env file.

Also updates TelemetryService tests to support multi-provider architecture.

* fix(telemetry): Replace Record<string, unknown> with proper JSON-serializable types

- Add TelemetryPrimitive, TelemetryValue, TelemetryObject, and TelemetryProperties types to ITelemetryProvider
- Update JitsuTelemetryProvider to use TelemetryProperties instead of Record<string, unknown>
- Update PostHogTelemetryProvider to use TelemetryProperties instead of Record<string, unknown>
- Update TelemetryService to use TelemetryProperties for type-safe telemetry data
- Ensures all telemetry properties are JSON-serializable, preventing runtime errors
- Fixes TypeScript compatibility issue between Jitsu's JSONObject type and Record<string, unknown>

* moved and organized the telemetry files and updated the example env file to be more descriptive

* refactor: remove Jitsu telemetry provider

- Remove Jitsu provider implementation and config files
- Remove Jitsu environment variables from .env.example
- Remove Jitsu build configuration from esbuild.mjs
- Update TelemetryProviderFactory to only support PostHog
- Uninstall @jitsu/js dependency
- Add .env to .gitignore to prevent committing local env files

* chore: add changeset for Jitsu removal

* removed jitsu

* fix: update import paths after PostHogClientProvider relocation

* fix: remove race condition in captureToProviders and reorganize PostHog providers

- Changed captureToProviders from async to synchronous method
- Removed unnecessary Promise.allSettled overhead since provider.log() and provider.logRequired() are synchronous
- Changed from .map() to .forEach() for better clarity
- Moved PostHog provider files into posthog/ subdirectory for better organization
- Updated all import paths to reflect new folder structure

* refactor(telemetry): remove unnecessary addProperties method and improve type safety

- Remove addProperties helper method that used 'any' types
- Replace with inline typed spread operations in capture(), captureRequired(), and identifyAccount()
- Fix type errors in captureConversationTurnEvent and captureBrowserError
- All telemetry properties now properly typed as TelemetryProperties
- Ensures OpenTelemetry compatibility through type system enforcement

* refactor: remove dotenv dependency and use launch.json envFile

- Remove dotenv import and config() call from esbuild.mjs
- Add envFile parameter to all launch.json configurations to load .env
- Remove dotenv from package.json devDependencies

Environment variables are now loaded via VSCode's envFile feature for local
development, while CI/production continues to inject via GitHub Actions.
This provides cleaner separation between build-time and runtime environment
handling.

* feat(telemetry): add browser telemetry properties and improve typing

- Add remoteBrowserHost and endpoint fields to browser telemetry events
- Replace generic Record<string, unknown> with TelemetryObject type in EventHandlerBase for better type safety
- Import TelemetryObject type from ITelemetryProvider

These changes enhance browser telemetry tracking capabilities and improve type consistency across the telemetry service.

* feat(telemetry): add OpenTelemetry integration

Add comprehensive OpenTelemetry support alongside existing PostHog telemetry:

- Add OpenTelemetry provider with metrics and logs/events support
- Support multiple exporters: console, OTLP (gRPC/HTTP/Protobuf), and Prometheus
- Implement flexible configuration via environment variables
- Add detailed .env.example documentation with usage examples
- Integrate with existing telemetry infrastructure via TelemetryClient
- Support independent or parallel operation with PostHog
- Add proper attribute flattening for OpenTelemetry primitives
- Include configurable export intervals and protocols

This enables users to export telemetry data to any OpenTelemetry-compatible
backend (Grafana, Jaeger, etc.) while maintaining backward compatibility
with PostHog integration.

* add changeset

* Update packages

* .vscodeignore

* fixed type error

* fix(telemetry): Fix OpenTelemetry gRPC exporter endpoint format

- Strip http:// prefix from gRPC endpoints (gRPC requires 'localhost:4317' not 'http://localhost:4317')
- Clean up debug logging from OpenTelemetry provider classes
- Add helpful comment to .env.example about gRPC endpoint format

This fixes the issue where metrics were being recorded in-memory but silently
failing to export to the OpenTelemetry collector. Metrics now flow end-to-end
from the extension through the collector to Prometheus.

Verified working with test infrastructure at ~/code/@cline/cline-otel-testing

* merged from main and handled conflcits

* fix: ensure exportTimeoutMillis is less than exportIntervalMillis in OpenTelemetry metrics

Changed the timeout calculation to dynamically compute as 80% of the export interval,
capped at 30 seconds. This fixes the error: 'exportIntervalMillis must be greater than
or equal to exportTimeoutMillis' that occurred when the configured interval was less
than 30 seconds.

* feat(otel): add insecure gRPC connection support for development

- Add OTEL_EXPORTER_OTLP_INSECURE config option
- Support insecure (non-TLS) gRPC connections for local testing
- Update OpenTelemetryClientProvider to use grpcCredentials.createInsecure()
- Add comprehensive debug logging for troubleshooting
- Tested and validated with local OTel collector

This enables testing of OTLP gRPC protocol without TLS certificates,
useful for local development and testing environments.

* feat(otel): add comprehensive debug logging for troubleshooting

- Add configuration summary logging at initialization
- Log all exporter creation steps with success/failure status
- Log connection details (protocol, endpoint, insecure mode)
- Log header presence (keys only, not values for security)
- Add try-catch blocks around exporter creation with error logging
- Log reader/processor counts for validation
- Improve visibility for TLS handshake and authentication issues

* test: validate HTTP/Protobuf protocol with path appending fix

- Tested HTTP/Protobuf exporter with binary encoding
- Confirmed path appending fix works for /v1/metrics and /v1/logs
- Validated bearer token authentication over HTTP/Protobuf
- All exports successful with complete data fidelity
- Documented test results in scenario-5-http-protobuf.md

Test Status:  PASSED - HTTP/Protobuf production ready

* pre-cleanup

* refactor(telemetry): clean up OpenTelemetry provider architecture

Major refactoring to improve code quality, maintainability, and align with domain-driven design principles:

**Architecture Improvements:**
- Created OpenTelemetryExporterFactory with pure functions for exporter creation
- Extracted exporter logic from OpenTelemetryClientProvider into factory
- Removed Prometheus support (not a requirement)
- Simplified diagnostic logging with minimal wrapper gated by TEL_DEBUG_DIAGNOSTICS flag

**Interface & Provider Updates:**
- Extended ITelemetryProvider with optional incrementCounter() and recordHistogram() methods
- No OpenTelemetry types leak into provider interface (provider-agnostic)
- Implemented no-op metric stubs in PostHogTelemetryProvider
- Removed eventCounter from OpenTelemetryTelemetryProvider (was incorrectly tracking events as metrics)
- Added lazy counter/histogram creation with Map caches in OpenTelemetry provider
- Logs are now the primary telemetry path, metrics are optional/future-ready

**Code Quality:**
- ~50% reduction in complexity through factory pattern
- Clear separation of concerns between interface, implementation, client management, and exporter creation
- Improved testability with pure functions and lazy instrument creation
- Better maintainability with cleaner code structure

**Configuration:**
- Updated .env.example with comprehensive OpenTelemetry documentation
- Added TEL_DEBUG_DIAGNOSTICS flag for enabling diagnostic logging
- Clarified all configuration options with detailed comments
- Removed Prometheus references

**Verified Working:**
- All protocols tested and working: gRPC, HTTP/JSON, HTTP/Protobuf
- Bearer token authentication validated
- Console exporter functional
- Maintains full compatibility with TelemetryService interface

* OTel: make flattenProperties circular-safe with depth guard and array truncation

Use WeakSet to detect circular references; add MAX_DEPTH=10; limit arrays to 100 items with _truncated and _original_length flags; handle Date via toISOString and Error via message; skip __proto__, constructor, prototype keys; wrap JSON.stringify in try/catch.

* security: restrict sensitive OTel logging to debug mode only

Only log OTLP endpoints and header information when TEL_DEBUG_DIAGNOSTICS=true or IS_DEV=true. In production mode, only show whether these values are configured without exposing actual values. This prevents sensitive infrastructure details and authentication information from appearing in production logs.

* removed debug logging from non debug mode

* feat: add batch configuration for OpenTelemetry log processor

Add configurable batch settings for BatchLogRecordProcessor to allow tuning for different use cases:

- OTEL_LOG_BATCH_SIZE: Maximum logs per batch (default: 512)
- OTEL_LOG_BATCH_TIMEOUT: Maximum wait time in ms (default: 5000)
- OTEL_LOG_MAX_QUEUE_SIZE: Maximum queue size (default: 2048)

Benefits:
- High-volume scenarios can increase queue size to prevent dropped events
- Real-time monitoring can reduce timeout for faster exports
- Low-volume scenarios can reduce batch size to minimize delays

All settings are optional with sensible defaults matching OpenTelemetry SDK standards. Configuration is validated to ensure positive values.

* feat(telemetry): add build-time OpenTelemetry environment variable injection

Add support for injecting OpenTelemetry configuration at build time from
GitHub Actions secrets, following the same pattern as PostHog telemetry.
This enables production builds to have default OpenTelemetry collector
configuration while still allowing runtime overrides.

Changes:

1. esbuild.mjs:
   - Added build-time injection for 7 OpenTelemetry environment variables:
     * OTEL_TELEMETRY_ENABLED - Enable/disable OpenTelemetry
     * OTEL_LOGS_EXPORTER - Logs exporter type (console/otlp)
     * OTEL_METRICS_EXPORTER - Metrics exporter type (console/otlp)
     * OTEL_EXPORTER_OTLP_PROTOCOL - OTLP protocol (grpc/http/json/http/protobuf)
     * OTEL_EXPORTER_OTLP_ENDPOINT - Collector endpoint URL
     * OTEL_EXPORTER_OTLP_HEADERS - Authentication headers (e.g., bearer tokens)
     * OTEL_METRIC_EXPORT_INTERVAL - Metric export interval in milliseconds
   - Variables are read from process.env at build time and injected into
     the bundle via esbuild's define option
   - Follows exact same pattern as existing PostHog API key injection

2. .github/workflows/publish.yml:
   - Added OpenTelemetry environment variables to 'Package and Publish Extension' step
   - Variables are populated from GitHub Actions secrets
   - Applied to both release and pre-release builds

3. .github/workflows/publish-nightly.yml:
   - Added same OpenTelemetry environment variables to nightly builds
   - Ensures consistent configuration across all build types

How it works:

- Build Time (Production):
  * GitHub Actions reads secrets and sets environment variables
  * esbuild.mjs injects these values into the bundled code
  * Production builds ship with default OpenTelemetry configuration

- Runtime (Development):
  * Developers use .env file with their own configuration
  * No changes needed to existing development workflow

- Runtime (Production):
  * Users can override build-time defaults by setting environment variables
  * Runtime values take complete precedence over build-time defaults
  * Enterprise users can point to their own collectors

Next steps:
- Add GitHub secrets to repository (Settings → Secrets and variables → Actions)
- Required secrets: OTEL_TELEMETRY_ENABLED, OTEL_LOGS_EXPORTER,
  OTEL_EXPORTER_OTLP_PROTOCOL, OTEL_EXPORTER_OTLP_ENDPOINT,
  OTEL_EXPORTER_OTLP_HEADERS
- Optional secrets: OTEL_METRICS_EXPORTER, OTEL_METRIC_EXPORT_INTERVAL

Benefits:
- Consistent with existing PostHog telemetry pattern
- Secure: production secrets stay in GitHub, not in code
- Flexible: users can override defaults at runtime
- Development-friendly: .env file continues to work as before
- Production-ready: default collector configuration for all users

* removed ai slop

* updated lock file

* fix: use ExtensionRegistryInfo.version for cross-platform compatibility

Replace process.env.npm_package_version with ExtensionRegistryInfo.version
in OpenTelemetry service version to ensure compatibility across VSCode,
JetBrains, and CLI environments.

Addresses PR #6605 inline comment from Sarah Fortune (sjf)

* fix: restore package-lock.json with proper biome dependencies

Fixes CI test failures caused by corrupted biome package entries.
Restores package-lock.json from main and reinstalls to properly
update OpenTelemetry dependencies while preserving biome integrity.

Addresses PR #6605 comment from Sarah Fortune (sjf) about test failures

---------

Co-authored-by: NightTrek <Daniels@dual4t.com>
Co-authored-by: Daniel Steigman <35793213+NightTrek@users.noreply.github.com>
2025-10-13 15:49:48 -07:00

117 lines
4.5 KiB
Bash

# Cline Development Environment Variables
# Copy this file to .env and fill in your actual values
# Values should be obtained from 1Password shared vault for development
# ============================================================================
# DEVELOPMENT FLAGS
# Recomend not changing these unless you know what you're doing they are set by the launch.json normally
# ============================================================================
# IS_DEV=true
# CLINE_ENVIRONMENT=local
# ============================================================================
# POSTHOG TELEMETRY (Existing)
# ============================================================================
# Get these values from 1Password shared vault
TELEMETRY_SERVICE_API_KEY=your-posthog-telemetry-api-key
ERROR_SERVICE_API_KEY=your-posthog-error-tracking-api-key
# ============================================================================
# TELEMETRY PROVIDER CONTROL
# ============================================================================
# Control which telemetry providers are active
POSTHOG_TELEMETRY_ENABLED=true # Enable PostHog telemetry (default: true)
# Set to false to disable Telemetry completely
# ============================================================================
# OPENTELEMETRY (Optional - for advanced telemetry)
# ============================================================================
# OpenTelemetry provides flexible telemetry collection with multiple export options
# Can run alongside PostHog or independently
# Primary focus: Logs (events), with optional metrics support
# Enable OpenTelemetry (set to 1 to enable)
# OTEL_TELEMETRY_ENABLED=1
# Exporters: "console" for local debugging, "otlp" for remote collector
# Logs are the primary signal (recommended)
# OTEL_LOGS_EXPORTER=console
# OTEL_METRICS_EXPORTER=otlp
# OTLP Protocol: "grpc", "http/json", or "http/protobuf"
# OTEL_EXPORTER_OTLP_PROTOCOL=grpc
# OTLP Endpoint (without /v1/logs or /v1/metrics path - auto-appended)
# For gRPC: use "localhost:4317" (no http:// prefix)
# For HTTP: use "http://localhost:4318"
# OTEL_EXPORTER_OTLP_ENDPOINT=localhost:4317
# OTLP Headers (for authentication, e.g., bearer tokens)
# OTEL_EXPORTER_OTLP_HEADERS=authorization=Bearer your-token-here
# Use insecure gRPC connections (for local testing only, NOT for production)
# OTEL_EXPORTER_OTLP_INSECURE=true
# Metric export interval in milliseconds (default: 60000)
# OTEL_METRIC_EXPORT_INTERVAL=10000
# Batch configuration for logs (optional)
# OTEL_LOG_BATCH_SIZE=512 # Max logs per batch (default: 512)
# OTEL_LOG_BATCH_TIMEOUT=5000 # Max wait time in ms (default: 5000)
# OTEL_LOG_MAX_QUEUE_SIZE=2048 # Max queue size (default: 2048)
# Enable detailed export diagnostics (for debugging)
# TEL_DEBUG_DIAGNOSTICS=true
# Advanced: Separate endpoints for metrics and logs (optional)
# OTEL_EXPORTER_OTLP_METRICS_PROTOCOL=http/protobuf
# OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=http://metrics.example.com:4318
# OTEL_EXPORTER_OTLP_LOGS_PROTOCOL=grpc
# OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=logs.example.com:4317
# Example configurations:
#
# Console debugging (logs only):
# OTEL_TELEMETRY_ENABLED=1
# OTEL_LOGS_EXPORTER=console
# TEL_DEBUG_DIAGNOSTICS=true
#
# OTLP with gRPC (insecure, for local testing):
# OTEL_TELEMETRY_ENABLED=1
# OTEL_LOGS_EXPORTER=otlp
# OTEL_EXPORTER_OTLP_PROTOCOL=grpc
# OTEL_EXPORTER_OTLP_ENDPOINT=localhost:4317
# OTEL_EXPORTER_OTLP_INSECURE=true
# OTEL_EXPORTER_OTLP_HEADERS=authorization=Bearer your-token
#
# OTLP with HTTP/JSON (production):
# OTEL_TELEMETRY_ENABLED=1
# OTEL_LOGS_EXPORTER=otlp
# OTEL_EXPORTER_OTLP_PROTOCOL=http/json
# OTEL_EXPORTER_OTLP_ENDPOINT=https://otel.example.com
# OTEL_EXPORTER_OTLP_HEADERS=authorization=Bearer your-token
# ============================================================================
# OPTIONAL DEVELOPMENT SETTINGS
# ============================================================================
# Uncomment and modify as needed for development
# Multi-root workspace debugging
# MULTI_ROOT_TRACE=true
# gRPC recorder for testing
# GRPC_RECORDER_ENABLED=true
# GRPC_RECORDER_FILE_NAME=test-recording
# Test mode
# E2E_TEST=true
# IS_TEST=true
# ============================================================================
# USAGE INSTRUCTIONS
# ============================================================================
# 1. Copy this file: cp .env.example .env
# 2. Get PostHog keys from 1Password shared vault
# 3. Update the values in .env
# 4. The .env file is gitignored for security