mirror of
https://github.com/Kilo-Org/kilocode.git
synced 2026-10-31 03:46:24 +08:00
* feat(session-export): scaffold module with config constants * feat(session-export): add zstd compression wrapper * feat(session-export): event and envelope type definitions * feat(session-export): eligibility check with kill-switch * feat(session-export): org signal collector with auth resolver * feat(session-export): worker SQLite schema and storage helpers * feat(session-export): content-addressed chunker with zstd dedup * feat(session-export): client-side light scrubber * feat(session-export): IPC contract and inbox with back-pressure * feat(session-export): persist scrubbed events with chunked payloads * feat(session-export): worker entry point with inbox drain loop * feat(session-export): main-thread capture module * feat(session-export): workspace baseline and delta fibers * feat(session-export): sync subscriber and tool io chunking * feat(session-export): bootstrap wiring and compaction hook * feat(session-export): wire capture hooks into sessions * feat(session-export): uploader and buffer cap * chore(session-export): annotate shared session hooks * changeset(session-export): add release note * fix(session-export): keep bootstrap non-blocking without instance context * feat(session-export): respawn worker after failures * test(session-export): add performance budget assertions * test(session-export): add worker end-to-end smoke test * fix(session-export): strip identity and high-risk baseline paths * test(session-export): gate perf assertions for stable sweeps * fix(session-export): bundle worker in single-file builds * fix(session-export): make capture envelopes cloneable * fix(session-export): drain worker on CLI shutdown * feat(session-export): capture workspace baseline and deltas * fix(session-export): preserve request metadata * test(session-export): cover request metadata capture * feat(cli): send indexed session export batches * feat(cli): authorize session export uploads * fix(cli): flush session export shutdown uploads * feat(cli): optimize session export replay payloads * fix(cli): include session export surface metadata * fix(session-export): decrement chunk refs after upload decRefChunks was never called, so chunk ref_count stayed at 1 (or higher with dedup) and DELETE FROM chunk WHERE ref_count <= 0 never matched. Chunks accumulated in the local SQLite buffer until the 50 GB cap. Call decRefChunks alongside markUploaded so deleteUploaded can reclaim the rows. * fix(session-export): add periodic uploader flush timer flushIntervalMs and retryBackoffMaxMs were both defined in config and neither was referenced anywhere; scheduleFlush only ran on inbound events or reconnect. After a 5xx or network failure the row was backed off for 1 s, but if no further event arrived the retry never fired and the events stranded until the CLI restarted. Drive a periodic flush from a setInterval, unref the handle so it doesn't pin the process, and clear it from a new dispose() hook the worker calls on shutdown. * fix(session-export): stop emitting absolute workspace root in baseline CaptureMetadata.root was the literal absolute filesystem path (/Users/<name>/Projects/<repo>), shipped in every workspace_baseline_completed event and not stripped by handlers' identity filter. The field was set but never read anywhere downstream — file paths in the baseline are already relative, so the root added no replay signal. Remove the field outright. * fix(session-export): cap pendingEvents result set The SELECT had no LIMIT clause, so under a backlog (network outage, crashed receiver) it would materialize the full pending table into a JavaScript array before the byte-limit truncation applied. With a 50 GB buffer cap that is hundreds of MB of heap inside the worker per drain. Add LIMIT 500 — the drain loop already re-queries until empty, so no events are missed. * fix(session-export): exponential retry backoff up to retryBackoffMaxMs Both the 5xx and network-error branches always retried after the floor delay regardless of how many attempts had already failed, and retryBackoffMaxMs was unreferenced. During a sustained outage every session re-tried at roughly 1 Hz against the dead receiver. Surface upload_attempts on EventRow and compute the next delay as min * 2^attempts capped at max. * fix(session-export): evict superseded workspace snapshots on remember createWorkspaceProvider retained every captured snapshot — in-memory and inside the persisted state file — even after the session moved on to a newer one. For a 1k-file repo over a 100-turn session that is ~2 GB of unreachable heap plus a state file that grows monotonically. Drop the previous snapshot for the session on remember() when no other session still references it. * fix(session-export): anchor aws_secret_key scrubber to key name The bare 40-char base64 pattern matched every 40-character hex string, including all git commit SHAs. Tool outputs, diffs, and conversation messages were silently rewritten as <<REDACTED:aws_secret_key>>, destroying lineage information in training data. Require the key name context — naked secrets in unstructured text are rare and the .env / .aws/credentials high-risk path strip already covers the common case. * fix(session-export): preserve root linkage in SyncSubscriber events SyncSubscriber hardcoded rootSessionId = sessionId on every tool, permission, and feedback event, so sub-agent sessions lost their root linkage and a future training pipeline could not reconstruct the agent topology from these side-channel events. Expose the rootSessionId mapping from Capture and plumb it through the same pattern as getTurnId. * fix(session-export): atomic chunk GC after upload markUploaded + decRefChunks + deleteUploaded were three separate SQL statements; a crash between markUploaded and decRefChunks would leave events flagged uploaded (never retried) and chunks with stale ref_count (never reclaimed by deleteUploaded). Bundle the three calls into a single transactional commitUploaded helper so either all three land or none of them do. * fix(session-export): wait on transient sqlite locks * fix(session-export): snapshot current workspace directory * fix(cli): preserve Kilo model export metadata * fix(cli): send anon id for session export * fix(cli): fallback to telemetry anon id * chore(cli): annotate session export config test * chore: remove session export docs * fix(cli): harden session export uploads * fix(cli): preserve stream lifecycle for session export * fix(kilo-docs): exclude session export ingest link * fix(cli): restrict session export workspace sync to git repos * chore: remove session export changeset * fix(cli): tighten session export payload types * fix(cli): type session export model payloads * fix(cli): simplify session export cleanup * fix(cli): avoid session export shutdown race * refactor(cli): use drizzle for session export storage * fix(cli): finalize session export sqlite statements * fix(cli): link compaction exports to root sessions * fix(cli): stop exporting raw stream parts * fix(cli): prune stale workspace snapshots * refactor(cli): clarify chunk ref counting * test(cli): clarify dropped upload assertions * ci: avoid visual path filter action failure * fix(cli): preserve in-flight workspace snapshots * fix(cli): stop uploading baseline start events * fix(cli): trim redundant export metadata * fix(cli): trim workspace export bookkeeping * fix(cli): fold terminal outcome into tool exports * fix(cli): dedupe request context in export batches * fix(cli): normalize compaction export payloads * fix(cli): avoid duplicate tool result exports * test(cli): align session export expectations * fix(cli): run secretlint during session export scrubbing * fix(cli): keep retried export batches contiguous * fix(cli): include agent info in session exports * fix(cli): flush pending session exports on serve startup * fix(cli): drop exports when scrubbing fails * fix(cli): narrow secretlint value extraction * fix(cli): limit exported agent info * test(cli): remove brittle agent export source assertion * fix(cli): ignore corrupt session export workspace state * fix(cli): keep session export close best effort * fix(cli): avoid following session export symlinks * fix(cli): preserve session export chunk ref counts * fix(cli): retry transient session export uploads * fix(cli): validate session export ingest endpoint * fix(cli): tolerate missing export token details * fix(cli): fail closed on session export org lookup * fix(cli): decode session export permission replies * fix(cli): finalize session export on stream close * fix(cli): bound session export baseline timeout * fix(cli): avoid persisting workspace file contents * fix(cli): bound workspace snapshot capture * fix(cli): validate session export worker messages * fix(cli): revoke stale session export eligibility * fix(cli): scope session export workspaces * fix(cli): extend session export shutdown flush * fix(cli): infer free Kilo models for export * fix(cli): avoid duplicate chunk refs * fix(cli): throttle session export uploads * fix(cli): harden session export capture * feat: disclose free model data collection (#10767) * feat: disclose free model data collection * chore(cli): document free model footer sorting * fix(vscode): add free model data translations * fix: simplify free model data label * fix: simplify data collection badges * fix: remove duplicate model info disclosure * fix: restore composer data tooltip * fix: align jetbrains data collection indicator * fix: align free model data indicators * fix(vscode): show data collection in model preview * fix: limit data indicators to kilo gateway * test(cli): relax prompt cancel timeout * test(cli): annotate prompt cancel timeout * test(cli): classify prompt queue runtime test * fix(cli): defer session export startup
81 lines
3.1 KiB
TypeScript
81 lines
3.1 KiB
TypeScript
// IMPORTANT: Set env vars BEFORE any imports from src/ directory
|
|
// xdg-basedir reads env vars at import time, so we must set these first
|
|
import os from "os"
|
|
import path from "path"
|
|
import fs from "fs/promises"
|
|
import { afterAll } from "bun:test"
|
|
import { remove as cleanup } from "./kilocode/cleanup" // kilocode_change
|
|
|
|
// Set XDG env vars FIRST, before any src/ imports
|
|
const dir = path.join(os.tmpdir(), "opencode-test-data-" + process.pid)
|
|
await fs.mkdir(dir, { recursive: true })
|
|
afterAll(async () => {
|
|
const { SessionExport } = await import("../src/kilocode/session-export") // kilocode_change
|
|
const { Database } = await import("../src/storage/db")
|
|
await SessionExport.shutdown() // kilocode_change
|
|
Database.close()
|
|
await cleanup(dir) // kilocode_change
|
|
})
|
|
|
|
process.env["XDG_DATA_HOME"] = path.join(dir, "share")
|
|
process.env["XDG_CACHE_HOME"] = path.join(dir, "cache")
|
|
process.env["XDG_CONFIG_HOME"] = path.join(dir, "config")
|
|
process.env["XDG_STATE_HOME"] = path.join(dir, "state")
|
|
process.env["KILO_MODELS_PATH"] = path.join(import.meta.dir, "tool", "fixtures", "models-api.json")
|
|
process.env["KILO_EXPERIMENTAL_EVENT_SYSTEM"] = "true"
|
|
|
|
// Set test home directory to isolate tests from user's actual home directory
|
|
// This prevents tests from picking up real user configs/skills from ~/.claude/skills
|
|
const testHome = path.join(dir, "home")
|
|
await fs.mkdir(testHome, { recursive: true })
|
|
process.env["KILO_TEST_HOME"] = testHome
|
|
|
|
// Set test managed config directory to isolate tests from system managed settings
|
|
const testManagedConfigDir = path.join(dir, "managed")
|
|
process.env["KILO_TEST_MANAGED_CONFIG_DIR"] = testManagedConfigDir
|
|
process.env["KILO_DISABLE_DEFAULT_PLUGINS"] = "true"
|
|
|
|
// Write the cache version file to prevent global/index.ts from clearing the cache
|
|
const cacheDir = path.join(dir, "cache", "kilo")
|
|
await fs.mkdir(cacheDir, { recursive: true })
|
|
await fs.writeFile(path.join(cacheDir, "version"), "21")
|
|
|
|
// Clear provider and server auth env vars to ensure clean test state
|
|
delete process.env["ANTHROPIC_API_KEY"]
|
|
delete process.env["OPENAI_API_KEY"]
|
|
delete process.env["GOOGLE_API_KEY"]
|
|
delete process.env["GOOGLE_GENERATIVE_AI_API_KEY"]
|
|
delete process.env["AZURE_OPENAI_API_KEY"]
|
|
delete process.env["AWS_ACCESS_KEY_ID"]
|
|
delete process.env["AWS_PROFILE"]
|
|
delete process.env["AWS_REGION"]
|
|
delete process.env["AWS_BEARER_TOKEN_BEDROCK"]
|
|
delete process.env["OPENROUTER_API_KEY"]
|
|
delete process.env["LLM_GATEWAY_API_KEY"]
|
|
delete process.env["GROQ_API_KEY"]
|
|
delete process.env["MISTRAL_API_KEY"]
|
|
delete process.env["PERPLEXITY_API_KEY"]
|
|
delete process.env["TOGETHER_API_KEY"]
|
|
delete process.env["XAI_API_KEY"]
|
|
delete process.env["DEEPSEEK_API_KEY"]
|
|
delete process.env["FIREWORKS_API_KEY"]
|
|
delete process.env["CEREBRAS_API_KEY"]
|
|
delete process.env["SAMBANOVA_API_KEY"]
|
|
delete process.env["KILO_SERVER_PASSWORD"]
|
|
delete process.env["KILO_SERVER_USERNAME"]
|
|
|
|
// Use in-memory sqlite
|
|
process.env["KILO_DB"] = ":memory:"
|
|
|
|
// Now safe to import from src/
|
|
const { Log } = await import("@opencode-ai/core/util/log")
|
|
const { initProjectors } = await import("../src/server/projectors")
|
|
|
|
void Log.init({
|
|
print: false,
|
|
dev: true,
|
|
level: "DEBUG",
|
|
})
|
|
|
|
initProjectors()
|