opensource/sim
3
0
Fork 0
mirror of https://github.com/simstudioai/sim.git synced 2026-10-13 15:48:31 +08:00
Code Issues Packages Projects Releases Wiki Activity

v0.9.15: mship improvements, memory improvements, nextjs bump, plane and ramp integrations, outbox hardening

Browse Source
This commit is contained in:
Waleed
2026-10-07 23:35:16 -07:00
committed by GitHub
parent 546d4e7e5d e3800f6f01
commit 670def1a73
1188 changed files with 310145 additions and 13074 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/skills/add-settings-page/SKILL.md
+2 -2
View File
@@ -66,8 +66,8 @@ Each grep lists candidates; review every match against the expected ones named b
- Editable pages: confirm Save/Discard go through `saveDiscardActions()` and
dirty is wired via `useSettingsUnsavedGuard` (called before early-return
gates) — flag any hand-rolled Save button, `beforeunload`, or unsaved modal.
`git grep -n "beforeunload" -- 'apps/sim/**/settings/**' 'apps/sim/ee/'`
should only hit the centralized `use-settings-before-unload.ts`.
`git grep -n "beforeunload" -- 'apps/sim/**/settings/**' 'apps/sim/ee/' 'apps/sim/components/settings/' ':(exclude,glob)**/*.test.*'`
should only hit the centralized `use-settings-browser-navigation.ts`.
5. Fix each finding with the smallest structural change that satisfies the checklist;
do not touch handlers, state, queries, or gate returns. A pixel-size fix swaps
only the size class for its exact-pixel token (`text-[12px]` → `text-caption`).
.agents/skills/ship/SKILL.md
+5 -2
View File
@@ -47,7 +47,7 @@ When the user runs `/ship`:
- Run `/db-migrate` to review the migration for zero-downtime safety (expand/contract phasing, backward-compatibility with the deployed app version).
- `(cd packages/db && bunx drizzle-kit generate && git status --porcelain ./migrations)` must print nothing (CI's schema/migration sync step).
- `bun run check:migrations origin/staging` must pass (staging is the PR base). Do not silence a flagged statement with a `-- migration-safe:` annotation unless `/db-migrate` confirmed the old code no longer depends on it; otherwise split the destructive change into a later deploy.
6. **Run pre-ship checks** from the repo root before staging. This has two phases: first **regenerate** every committed artifact so generated files never drift into a CI failure (this is what catches things like `agent-stream-docs` going stale after a `models.ts` edit), then run the **full audit suite** CI's `Lint and Test` job enforces. Both phases parallelize — but only across commands that write **disjoint** outputs — and a bare `wait` swallows child exit codes, so both phases below explicitly collect each job's status and abort ship if any failed.
6. **Run pre-ship checks** from the repo root before staging. This has two phases: first **regenerate** every committed artifact so generated files never drift into a CI failure (this is what catches things like `agent-stream-docs` going stale after a `models.ts` edit), then run the **full audit suite** CI's `lint` job enforces. Both phases parallelize — but only across commands that write **disjoint** outputs — and a bare `wait` swallows child exit codes, so both phases below explicitly collect each job's status and abort ship if any failed.
**Phase A — regenerate the always-in-repo committed artifacts (parallel), then let step 7 stage whatever changed.** Regenerate only the generators whose inputs live entirely in this repo and that any ordinary code change can drift — `agent-stream-docs:generate` (derives from the provider model registry), `docs-manifest:generate` (derives from docs page paths), and `skills:sync` (derives from `.agents/skills/**`). They write disjoint outputs (`apps/docs/…/agent.mdx`, `apps/sim/lib/mothership/generated/docs-manifest.ts`, and `.claude/skills` links), so they parallelize safely, and each is idempotent (a no-op when already in sync):
```bash
@@ -66,7 +66,7 @@ When the user runs `/ship`:
**Do NOT blanket-run the domain generators here.** `mship:generate` (`generate-mship-contracts.ts`) is an **umbrella** that drives all nine mothership contract generators (`mship-contracts`, `billing-protocol-contract`, `mship-tools`, the four `trace-*`, `metrics-contract`, `vfs-snapshot-contract`) and biome-formats `apps/sim/lib/mothership/generated/` — never run it *and* its constituents (they write the same files and corrupt each other in parallel), and never run it on an ordinary ship: it reads an **external** copilot-contract source that isn't checked out in most worktrees, so it hard-fails with `ENOENT` and would abort ship for an unrelated reason. `generate:pi-model-catalog` (under `apps/sim`) likewise regenerates from the installed Pi package, not repo source. `scripts/generate-docs.ts` rewrites the integration docs and client-safe catalog; run it when this PR changes their block/icon/landing-content inputs or when `integration-catalog:check` reports drift, then review its broad generated diff. Only when **this PR's diff actually touches** a domain generator's input do you regenerate it deliberately and run its matching `:check` (`bun run mship:check` / the individual `*:check`) — with the external source present.
**Phase B — run lint + every audit CI enforces, in parallel, and abort ship if any fails.** Before running the commands, compare this list with `.github/workflows/test-build.yml`; when CI adds an audit, run it and update this skill instead of trusting a stale snapshot. The env-flag audit is currently an inline workflow block rather than a package script: when `apps/sim/lib/core/config/env-flags.ts` changed, run that current workflow block verbatim instead of copying a second version into this skill. Run `bun run lint` first (it autofixes formatting and mutates files, so don't parallelize it with the read-only audits), then run the base-sensitive block-registry check, then fan the independent audits out and collect exit codes:
**Phase B — run lint + every audit CI enforces, in parallel, and abort ship if any fails.** Before running the commands, compare this list with `.github/workflows/checks.yml`; when CI adds an audit, run it and update this skill instead of trusting a stale snapshot. The env-flag audit is currently an inline workflow block rather than a package script: when `apps/sim/lib/core/config/env-flags.ts` changed, run that current workflow block verbatim instead of copying a second version into this skill. Run `bun run lint` first (it autofixes formatting and mutates files, so don't parallelize it with the read-only audits), then run the base-sensitive block-registry check, then fan the independent audits out and collect exit codes:
```bash
# autofix formatting first (mutating; not parallel-safe with the audits). Gate its exit too —
# a non-zero lint (unfixable errors) must abort before the audits run, not be ignored.
@@ -77,6 +77,9 @@ When the user runs `/ship`:
}
# Runs every audit CI runs, concurrently, and replays the output of any that fail.
# The audit list is derived in scripts/run-audits.ts — do not hand-list audits here.
# Install CI's pinned actionlint version for the host OS/architecture and verify its
# artifact against the official release checksums in a local mktemp directory.
# Preserve CI's -shellcheck= -pyflakes= flags; lint all workflows and abort ship if it fails.
bun run check:audits || { echo "❌ audit(s) failed — do not ship"; exit 1; }
bun run type-check || { echo "❌ type-check failed — do not ship"; exit 1; }
# CI's "Verify docs manifest is in sync" step is not a `check:*` script, so the runner above
.claude/rules/landing-seo-geo.md
+1
View File
@@ -17,6 +17,7 @@ paths:
- All copy is server-rendered text: no text baked into images, no content that exists only after a client effect runs.
- Navbar is a Server Component (no `'use client'`) for immediate crawlability. Logo `<Image>` has `priority` (LCP element). The navbar `<nav>` carries `SiteNavigationElement` schema.org markup.
- Structured data: emit JSON-LD (`Organization`, `WebSite`, `WebApplication` with `featureList`, `FAQPage` if an FAQ exists) from a server component rendered before visible content. Keep `featureList` in sync with the features the page shows (`apps/sim/app/(landing)/components/home-structured-data/`).
- `/library` and `/blog` pagination stays server-rendered with real next/previous links. Each unfiltered page is indexable and uses its own canonical URL (`?page=2`, etc.); page 1 canonicalizes to the bare index. Tag-filtered variants remain `noindex,follow`. See [Google pagination guidance](https://developers.google.com/search/docs/specialty/ecommerce/pagination-and-incremental-page-loading).
- After adding routes or anchors, verify `app/sitemap.ts` and `app/robots.ts` still reflect reality.
## GEO (Generative Engine Optimisation)
.claude/rules/sim-settings-pages.md
+27 -28
View File
@@ -339,34 +339,33 @@ rendered through `<SettingsActionChips actions={…} />` from
`@/components/settings/settings-header` — that is the shared chip path, and it
is what keeps tone/icon/variant/tooltip handling from drifting between the two
shells. Reach for it before hand-rolling a `Chip`.
- **`useSettingsUnsavedGuard({ isDirty })`** (`…/settings/hooks/use-settings-unsaved-guard`)
— syncs the page's local `isDirty` into the shared `useSettingsDirtyStore` (so
the sidebar's **section-switch** confirm + the centralized `beforeunload` both
apply for free) and returns `{ showUnsavedModal, setShowUnsavedModal, guardBack,
confirmDiscard }` for a detail view's **in-view back** chip.
- **Top-level pages** (whitelabeling, sso): call it **unassigned** —
`useSettingsUnsavedGuard({ isDirty: hasChanges })` — they only need the
store-sync; the sidebar/`beforeunload` do the rest.
- **Detail sub-views** (data-retention, access-control group-detail): route the
back chip through `onClick={() => guard.guardBack(closeFn)}` and render the
shared `<UnsavedChangesModal open={guard.showUnsavedModal}
onOpenChange={guard.setShowUnsavedModal} onDiscard={guard.confirmDiscard} />`
(from `@/app/workspace/[workspaceId]/components/credential-detail`). The
in-view header **Discard** chip (via `saveDiscardActions({ onDiscard })`) is a
*reset to original* — distinct from the back-confirm's discard, which leaves.
- **`useSettingsBeforeUnload`** is mounted by the settings shells
(`settings/layout.tsx` and `components/settings/standalone-settings-shell.tsx`) —
never add a per-page `beforeunload`.
- **Dirty *computation* stays local** (shapes differ: field-compare vs
normalize+stringify) — only how dirty is *consumed* is shared. Derive it (a
`const`/`useMemo`), never store it in `useState`.
- **CRITICAL — rules of hooks:** call `useSettingsUnsavedGuard(...)`
**unconditionally, before every early-return gate** (entitlement / loading /
not-entitled `return <SettingsEmptyState>`). A hook placed after a gate is
skipped on gated renders and crashes.
- The route-based credential detail keeps its own `useUnsavedChangesGuard` (it
guards real `router.push` navigation + browser Back via a history sentinel);
it already shares `UnsavedChangesModal`, so copy stays unified.
- **`useSettingsUnsavedGuard({ isDirty, navigationBlocked, onDiscard })`** from
`@/components/settings/use-settings-unsaved-guard` registers one editor with the
shared settings store. Register every full-page draft, including inline creation
forms. Dirty computation stays local and derived; fetched defaults stay clean.
- **`onDiscard` resets the registered draft.** The root `SettingsNavigationGuard`
owns the discard dialog, internal link interception, browser Back/Forward, and
refresh protection for registered editors. Do not add a per-page dialog or
history listener for an editor using this hook. Skill create/detail pages retain
their existing `useUnsavedChangesGuard` and local dialog until they migrate to
the shared registration.
- **Native history needs entry indexes.** The Navigation API supplies them for
legacy entries. Without it, native traversal to an unindexed entry cannot be
cancelled reliably; never guess a direction or rewrite the history stack.
- **History confirmation authorizes traversal before discarding.** Drafts are
discarded only when the browser reports the confirmed traversal, so a no-op
Back or Forward retains edits. Known cross-document traversal uses native unload
protection. An unknown cross-document target on a classic History browser can
require a second native confirmation after the shared dialog.
- **Detail back controls** call `guard.guardBack(closeFn)`. Other destructive view
transitions, such as switching an editor's direction or environment, also use
the shared `requestLeave` action before resetting their draft.
- **`navigationBlocked` covers pending saves and uploads.** The shared guard blocks
navigation while requests are pending. Attempts are not queued; retry after they
settle. Disable or preserve edits made during requests;
failed saves retain drafts, and successful saves only clear committed values.
- **Call the hook unconditionally, before every early-return gate** (entitlement,
loading, or empty-state return), so gated renders preserve hook order.
## Detail sub-views
.claude/rules/sim-testing.md
+3 -3
View File
@@ -32,11 +32,11 @@ contracts, and demonstrated regressions.
| Suffix | Needs | Run with | In CI |
|--------|-------|----------|-------|
| `*.test.ts(x)` | nothing; global mocks from `vitest.setup.ts` | `vitest run` | Lint and Test job |
| `*.integration.ts` | real PostgreSQL (`TEST_DATABASE_URL`), optionally Redis (`TEST_REDIS_URL`) | `vitest run --mode integration` | `PostgreSQL integration` job, by glob |
| `*.test.ts(x)` | nothing; global mocks from `vitest.setup.ts` | `vitest run` | `test` jobs (sharded) |
| `*.integration.ts` | real PostgreSQL (`TEST_DATABASE_URL`), optionally Redis (`TEST_REDIS_URL`) | `vitest run --mode integration` | `integration` jobs, by glob (sharded per provisioning path) |
| `*.live.test.ts` | provider APIs, hosted sandboxes, local runtimes, or sibling checkouts | `vitest run --mode live <file>` (apps/sim) | never |
| `apps/desktop/e2e/*.spec.ts` | the packaged Electron app | Playwright | desktop E2E workflow |
| `apps/sim/scripts/test-*-e2e.ts` | a running app over HTTP | its `package.json` script when one exists (`bun run test:scim:e2e`; `test:workflow-version-compare:e2e` adds `--no-env-file`), else `bun scripts/test-<suite>-e2e.ts` from apps/sim | End-to-end over real HTTP job |
| `apps/sim/scripts/test-*-e2e.ts` | a running app over HTTP | its `package.json` script when one exists (`bun run test:scim:e2e`; `test:workflow-version-compare:e2e` adds `--no-env-file`), else `bun scripts/test-<suite>-e2e.ts` from apps/sim | `e2e` jobs (`.github/scripts/http-e2e.sh`) |
- A unit test lives next to its source: `feature.ts` → `feature.test.ts`. No network, no database,
no real timers.
.cursor/rules/landing-seo-geo.mdc
+1
View File
@@ -17,6 +17,7 @@ globs: ["apps/sim/app/(landing)/**/*.tsx","apps/sim/content/**/*.mdx"]
- All copy is server-rendered text: no text baked into images, no content that exists only after a client effect runs.
- Navbar is a Server Component (no `'use client'`) for immediate crawlability. Logo `<Image>` has `priority` (LCP element). The navbar `<nav>` carries `SiteNavigationElement` schema.org markup.
- Structured data: emit JSON-LD (`Organization`, `WebSite`, `WebApplication` with `featureList`, `FAQPage` if an FAQ exists) from a server component rendered before visible content. Keep `featureList` in sync with the features the page shows (`apps/sim/app/(landing)/components/home-structured-data/`).
- `/library` and `/blog` pagination stays server-rendered with real next/previous links. Each unfiltered page is indexable and uses its own canonical URL (`?page=2`, etc.); page 1 canonicalizes to the bare index. Tag-filtered variants remain `noindex,follow`. See [Google pagination guidance](https://developers.google.com/search/docs/specialty/ecommerce/pagination-and-incremental-page-loading).
- After adding routes or anchors, verify `app/sitemap.ts` and `app/robots.ts` still reflect reality.
## GEO (Generative Engine Optimisation)
.cursor/rules/sim-settings-pages.mdc
+27 -28
View File
@@ -336,34 +336,33 @@ rendered through `<SettingsActionChips actions={…} />` from
`@/components/settings/settings-header` — that is the shared chip path, and it
is what keeps tone/icon/variant/tooltip handling from drifting between the two
shells. Reach for it before hand-rolling a `Chip`.
- **`useSettingsUnsavedGuard({ isDirty })`** (`…/settings/hooks/use-settings-unsaved-guard`)
— syncs the page's local `isDirty` into the shared `useSettingsDirtyStore` (so
the sidebar's **section-switch** confirm + the centralized `beforeunload` both
apply for free) and returns `{ showUnsavedModal, setShowUnsavedModal, guardBack,
confirmDiscard }` for a detail view's **in-view back** chip.
- **Top-level pages** (whitelabeling, sso): call it **unassigned** —
`useSettingsUnsavedGuard({ isDirty: hasChanges })` — they only need the
store-sync; the sidebar/`beforeunload` do the rest.
- **Detail sub-views** (data-retention, access-control group-detail): route the
back chip through `onClick={() => guard.guardBack(closeFn)}` and render the
shared `<UnsavedChangesModal open={guard.showUnsavedModal}
onOpenChange={guard.setShowUnsavedModal} onDiscard={guard.confirmDiscard} />`
(from `@/app/workspace/[workspaceId]/components/credential-detail`). The
in-view header **Discard** chip (via `saveDiscardActions({ onDiscard })`) is a
*reset to original* — distinct from the back-confirm's discard, which leaves.
- **`useSettingsBeforeUnload`** is mounted by the settings shells
(`settings/layout.tsx` and `components/settings/standalone-settings-shell.tsx`) —
never add a per-page `beforeunload`.
- **Dirty *computation* stays local** (shapes differ: field-compare vs
normalize+stringify) — only how dirty is *consumed* is shared. Derive it (a
`const`/`useMemo`), never store it in `useState`.
- **CRITICAL — rules of hooks:** call `useSettingsUnsavedGuard(...)`
**unconditionally, before every early-return gate** (entitlement / loading /
not-entitled `return <SettingsEmptyState>`). A hook placed after a gate is
skipped on gated renders and crashes.
- The route-based credential detail keeps its own `useUnsavedChangesGuard` (it
guards real `router.push` navigation + browser Back via a history sentinel);
it already shares `UnsavedChangesModal`, so copy stays unified.
- **`useSettingsUnsavedGuard({ isDirty, navigationBlocked, onDiscard })`** from
`@/components/settings/use-settings-unsaved-guard` registers one editor with the
shared settings store. Register every full-page draft, including inline creation
forms. Dirty computation stays local and derived; fetched defaults stay clean.
- **`onDiscard` resets the registered draft.** The root `SettingsNavigationGuard`
owns the discard dialog, internal link interception, browser Back/Forward, and
refresh protection for registered editors. Do not add a per-page dialog or
history listener for an editor using this hook. Skill create/detail pages retain
their existing `useUnsavedChangesGuard` and local dialog until they migrate to
the shared registration.
- **Native history needs entry indexes.** The Navigation API supplies them for
legacy entries. Without it, native traversal to an unindexed entry cannot be
cancelled reliably; never guess a direction or rewrite the history stack.
- **History confirmation authorizes traversal before discarding.** Drafts are
discarded only when the browser reports the confirmed traversal, so a no-op
Back or Forward retains edits. Known cross-document traversal uses native unload
protection. An unknown cross-document target on a classic History browser can
require a second native confirmation after the shared dialog.
- **Detail back controls** call `guard.guardBack(closeFn)`. Other destructive view
transitions, such as switching an editor's direction or environment, also use
the shared `requestLeave` action before resetting their draft.
- **`navigationBlocked` covers pending saves and uploads.** The shared guard blocks
navigation while requests are pending. Attempts are not queued; retry after they
settle. Disable or preserve edits made during requests;
failed saves retain drafts, and successful saves only clear committed values.
- **Call the hook unconditionally, before every early-return gate** (entitlement,
loading, or empty-state return), so gated renders preserve hook order.
## Detail sub-views
.cursor/rules/sim-testing.mdc
+3 -3
View File
@@ -30,11 +30,11 @@ contracts, and demonstrated regressions.
| Suffix | Needs | Run with | In CI |
|--------|-------|----------|-------|
| `*.test.ts(x)` | nothing; global mocks from `vitest.setup.ts` | `vitest run` | Lint and Test job |
| `*.integration.ts` | real PostgreSQL (`TEST_DATABASE_URL`), optionally Redis (`TEST_REDIS_URL`) | `vitest run --mode integration` | `PostgreSQL integration` job, by glob |
| `*.test.ts(x)` | nothing; global mocks from `vitest.setup.ts` | `vitest run` | `test` jobs (sharded) |
| `*.integration.ts` | real PostgreSQL (`TEST_DATABASE_URL`), optionally Redis (`TEST_REDIS_URL`) | `vitest run --mode integration` | `integration` jobs, by glob (sharded per provisioning path) |
| `*.live.test.ts` | provider APIs, hosted sandboxes, local runtimes, or sibling checkouts | `vitest run --mode live <file>` (apps/sim) | never |
| `apps/desktop/e2e/*.spec.ts` | the packaged Electron app | Playwright | desktop E2E workflow |
| `apps/sim/scripts/test-*-e2e.ts` | a running app over HTTP | its `package.json` script when one exists (`bun run test:scim:e2e`; `test:workflow-version-compare:e2e` adds `--no-env-file`), else `bun scripts/test-<suite>-e2e.ts` from apps/sim | End-to-end over real HTTP job |
| `apps/sim/scripts/test-*-e2e.ts` | a running app over HTTP | its `package.json` script when one exists (`bun run test:scim:e2e`; `test:workflow-version-compare:e2e` adds `--no-env-file`), else `bun scripts/test-<suite>-e2e.ts` from apps/sim | `e2e` jobs (`.github/scripts/http-e2e.sh`) |
- A unit test lives next to its source: `feature.ts` → `feature.test.ts`. No network, no database,
no real timers.
.github/actions/cache-mount/action.yml → .github/actions/cache/action.yml
+1 -1
View File
@@ -1,4 +1,4 @@
name: Cache Mount
name: cache
description: Mount a build cache directory using Blacksmith sticky disks, or the GitHub Actions cache when running on GitHub-hosted runners.
inputs:
.github/actions/docker-build/action.yml → .github/actions/image/action.yml
+2 -2
View File
@@ -1,4 +1,4 @@
name: Docker Build and Push
name: image
description: Set up a buildx builder and build/push an image, using Blacksmith's builder or the upstream Docker actions on GitHub-hosted runners.
inputs:
@@ -35,7 +35,7 @@ inputs:
required: false
# Registry logins must precede this action. provenance/sbom stay off: attestation
# manifests break `imagetools create` retagging in promote-images.
# manifests break `imagetools create` retagging in the `promote` job of ci.yml.
runs:
using: composite
steps:
.github/actions/setup-workspace/action.yml → .github/actions/setup/action.yml
+16 -6
View File
@@ -1,15 +1,23 @@
name: Setup Workspace
name: setup
description: Install the pinned Bun and Node toolchain, mount the dependency (and optionally Turbo) caches, and install workspace dependencies.
inputs:
provider:
description: The CI_PROVIDER repo variable, forwarded to cache-mount.
description: The CI_PROVIDER repo variable, forwarded to the cache action.
required: false
default: ''
turbo-cache-key:
description: Suffix for a Turbo cache mounted at ./.turbo. Empty skips the mount. Jobs that write Turbo entries need distinct suffixes, or last-writer-wins commits evict each other's entries.
required: false
default: ''
node-version:
description: Node version to install. Empty keeps the runner's preinstalled Node, for jobs that only ever ran Bun.
required: false
default: '24'
registry-url:
description: npm registry to write an .npmrc for, so `npm publish` reads NODE_AUTH_TOKEN. Empty writes none.
required: false
default: ''
# Cache keys are scoped by event name, and fork PRs get their own namespace on
# top: untrusted fork runs must never share a cache with push runs (whose caches
@@ -30,19 +38,21 @@ runs:
bun-version: 1.4.2
- name: Setup Node
if: inputs.node-version != ''
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: 24
node-version: ${{ inputs.node-version }}
registry-url: ${{ inputs.registry-url }}
- name: Mount Bun cache
uses: ./.github/actions/cache-mount
uses: ./.github/actions/cache
with:
provider: ${{ inputs.provider }}
key: ${{ github.repository }}-bun-cache-${{ github.event_name }}${{ github.event.pull_request.head.repo.fork && '-fork' || '' }}
path: ~/.bun/install/cache
- name: Mount node_modules
uses: ./.github/actions/cache-mount
uses: ./.github/actions/cache
with:
provider: ${{ inputs.provider }}
key: ${{ github.repository }}-node-modules-${{ github.event_name }}${{ github.event.pull_request.head.repo.fork && '-fork' || '' }}-${{ hashFiles('bun.lock') }}
@@ -50,7 +60,7 @@ runs:
- name: Mount Turbo cache
if: inputs.turbo-cache-key != ''
uses: ./.github/actions/cache-mount
uses: ./.github/actions/cache
with:
provider: ${{ inputs.provider }}
key: ${{ github.repository }}-${{ inputs.turbo-cache-key }}-${{ github.event_name }}${{ github.event.pull_request.head.repo.fork && '-fork' || '' }}
.github/dependabot.yml
+19
View File
@@ -0,0 +1,19 @@
version: 2
# GitHub Actions only: every `uses:` is pinned to a commit SHA, and without this the pins drift
# apart (checkout had three different SHAs across workflows). One grouped PR a week keeps every
# workflow and composite action on the same version of each action.
updates:
- package-ecosystem: github-actions
directories:
- /
- /.github/actions/*
schedule:
interval: weekly
target-branch: staging
groups:
actions:
patterns:
- '*'
commit-message:
prefix: ci
.github/scripts/desktop-live-changes.sh
Executable
+24
View File
@@ -0,0 +1,24 @@
#!/usr/bin/env bash
# Prints `changed=false` only when every file a pull request changes is clearly unrelated to the
# live desktop suite, and `changed=true` otherwise, including when the diff cannot be worked out.
#
# Usage: desktop-live-changes.sh <base-sha>
set -u
unrelated='^(apps/docs/|apps/pii/|apps/sim/content/|packages/(python-sdk|ts-sdk)/)|\.mdx?$|(^|/)LICENSE$'
base=${1:-}
run() {
echo "changed=true"
echo "Running the live desktop suite: $1" >&2
exit 0
}
[ -n "$base" ] || run 'no base commit'
git fetch --quiet --depth=1 origin "$base" || run "could not fetch $base"
names=$(git diff --no-renames --name-only "$base" HEAD) || run "could not diff against $base"
[ -n "$names" ] || run 'no changed files listed'
if printf '%s\n' "$names" | grep -qvE "$unrelated"; then
run 'a change may affect it'
fi
echo "changed=false"
echo 'Skipping the live desktop suite: every change is unrelated to it' >&2
.github/scripts/http-e2e.sh
Executable
+158
View File
@@ -0,0 +1,158 @@
#!/usr/bin/env bash
# Runs one end-to-end suite group over real HTTP, each against its own `next dev` app.
#
# Usage: http-e2e.sh <scim|cli|stop-after|desktop-inbox> (run from apps/sim)
#
# The job provides DATABASE_URL, BETTER_AUTH_SECRET and ENCRYPTION_KEY; each group sets the rest of
# its app's environment here. Reports and server logs land in $RUNNER_TEMP/e2e.
#
# The first request cold-compiles the app under Turbopack, which takes 42-150s on CI runners, so
# the readiness deadline only has to catch a hung boot: an exited server fails immediately, and
# either way the server log tail lands in the job log.
#
# Each app starts from an empty Turbopack dev cache: a cache written under other NEXT_PUBLIC_*
# values, by a server that `next dev` SIGKILLs 100ms after SIGTERM, can panic Turbopack or wedge a
# route compile on restore. It runs in its own session under an E2E_APP tag, and stop-session.sh
# returns only once every process in that session or carrying that tag has exited (Next's
# telemetry flush runs detached and still writes .next/dev).
set -euo pipefail
group=${1:?usage: http-e2e.sh <scim|cli|stop-after|desktop-inbox>}
report_dir="$RUNNER_TEMP/e2e"
ready_timeout_seconds=300
mkdir -p "$report_dir"
server_pid=''
app_tag=''
server_log=''
status_log=''
finish() {
local status=$?
if [ -n "$server_pid" ]; then
bash "$GITHUB_WORKSPACE/.github/scripts/stop-session.sh" "$server_pid" "$app_tag" || status=1
wait "$server_pid" 2>/dev/null || true
if [ -n "$status_log" ]; then
awk '/^ (GET|POST|PUT|PATCH|DELETE|HEAD) \/api\// { print }' "$server_log" > "$status_log"
fi
if [ "$status" -ne 0 ]; then
tail -n 200 "$server_log"
fi
fi
exit "$status"
}
trap finish EXIT
# start_app <name> <port> <label> [record-http-status]
start_app() {
local name=$1 port=$2 label=$3
server_log="$report_dir/$name-next.log"
if [ "${4:-}" = record-http-status ]; then
status_log="$report_dir/$name-http-status.log"
fi
app_tag="$name-$GITHUB_RUN_ID-$GITHUB_RUN_ATTEMPT-$$"
export NEXT_PUBLIC_APP_URL="http://127.0.0.1:$port"
export BETTER_AUTH_URL="$NEXT_PUBLIC_APP_URL"
export DISABLE_TELEMETRY=true NEXT_TELEMETRY_DISABLED=1
rm -rf .next/dev
E2E_APP="$app_tag" setsid node ../../node_modules/next/dist/bin/next dev --hostname 127.0.0.1 \
--port "$port" > "$server_log" 2>&1 &
server_pid=$!
local started=$SECONDS
until curl --fail --silent --max-time 10 "$NEXT_PUBLIC_APP_URL/api/health" > /dev/null; do
if ! kill -0 "$server_pid" 2>/dev/null; then
echo "::error::Local $label app exited during startup."
exit 1
fi
if [ $((SECONDS - started)) -ge "$ready_timeout_seconds" ]; then
echo "::error::Local $label app did not become ready within $ready_timeout_seconds seconds."
exit 1
fi
sleep 2
done
echo "Local $label app ready after $((SECONDS - started))s"
}
case "$group" in
scim)
export NEXT_PUBLIC_FORCE_HOSTED=true
export BILLING_ENABLED=true NEXT_PUBLIC_BILLING_ENABLED=true
export ENTERPRISE_ENABLED=true NEXT_PUBLIC_ENTERPRISE_ENABLED=true
export SCIM_ENABLED=true NEXT_PUBLIC_SCIM_ENABLED=true
export SSO_ENABLED=true NEXT_PUBLIC_SSO_ENABLED=true
export ORGANIZATIONS_ENABLED=true NEXT_PUBLIC_ORGANIZATIONS_ENABLED=true
export INTERNAL_API_SECRET=scim-http-ci-local-secret-at-least-32-characters
export DB_TX_TRIPWIRE=throw
export NEXT_PUBLIC_CHAT_DISABLED=true
start_app scim 3017 SCIM record-http-status
SCIM_E2E_BASE_URL="$NEXT_PUBLIC_APP_URL" \
SCIM_E2E_DATABASE_URL="$DATABASE_URL" \
SCIM_E2E_AUTH_SECRET="$BETTER_AUTH_SECRET" \
SCIM_E2E_REPORT_PATH="$report_dir/scim-e2e-report.json" \
bun run test:scim:e2e
VERSION_COMPARE_E2E_BASE_URL="$NEXT_PUBLIC_APP_URL" \
VERSION_COMPARE_E2E_DATABASE_URL="$DATABASE_URL" \
VERSION_COMPARE_E2E_AUTH_SECRET="$BETTER_AUTH_SECRET" \
VERSION_COMPARE_E2E_REPORT_PATH="$report_dir/version-compare-http-report.json" \
bun run test:workflow-version-compare:e2e
;;
# The search suites serve their own fixtures in-process and need no app. They share this group
# because they finish in seconds. Self-hosted without billing: hosted billing admits runs through
# Redis, which the CLI app is not given.
cli)
for search in google-content lucid zoom google-meet; do
report_var="SEARCH_$(echo "$search" | tr 'a-z-' 'A-Z_')_REPORT_PATH"
env NEXT_PUBLIC_APP_URL=http://127.0.0.1:3040 NEXT_PUBLIC_FORCE_HOSTED=false \
"$report_var=$report_dir/search-$search.json" \
bun "scripts/test-search-$search-e2e.ts"
done
export NEXT_PUBLIC_FORCE_HOSTED=false
export INTERNAL_API_SECRET=cli-http-ci-local-secret-at-least-32-characters
start_app cli 3018 CLI
CLI_LATENCY_E2E_BASE_URL="$NEXT_PUBLIC_APP_URL" \
CLI_LATENCY_E2E_DATABASE_URL="$DATABASE_URL" \
CLI_LATENCY_E2E_RUNS=3 \
CLI_LATENCY_E2E_WARMUP=1 \
CLI_LATENCY_E2E_REPORT_PATH="$report_dir/cli-run-latency-report.json" \
bun run test:cli-run-latency:e2e
;;
# Self-hosted: hosted billing admits a run only through a Redis usage reservation.
stop-after)
export NEXT_PUBLIC_FORCE_HOSTED=false
export CRON_SECRET=stop-after-e2e-local-cron-secret
export INTERNAL_API_SECRET=stop-after-http-ci-local-secret-at-least-32-characters
export DB_TX_TRIPWIRE=throw
export NEXT_PUBLIC_CHAT_DISABLED=true
start_app stop-after 3018 workflow record-http-status
STOP_AFTER_E2E_BASE_URL="$NEXT_PUBLIC_APP_URL" \
STOP_AFTER_E2E_DATABASE_URL="$DATABASE_URL" \
STOP_AFTER_E2E_REPORT_PATH="$report_dir/stop-after-http-report.json" \
STOP_AFTER_E2E_CRON_SECRET="$CRON_SECRET" \
bun run test:workflow-stop-after:e2e
;;
# The desktop background executor's protocol: device registration, the SSE doorbell over Redis
# pub/sub, presence, leased claims, Stop and isolation. The only group whose app gets Redis.
desktop-inbox)
export REDIS_URL=redis://127.0.0.1:6379
export NEXT_PUBLIC_FORCE_HOSTED=false
export COPILOT_TOOL_PERMISSIONS_ENABLED=true
export INTERNAL_API_SECRET=desktop-inbox-http-ci-local-secret-at-least-32-characters
export DB_TX_TRIPWIRE=throw
start_app desktop-inbox 3019 'desktop executor' record-http-status
DESKTOP_INBOX_E2E_BASE_URL="$NEXT_PUBLIC_APP_URL" \
DESKTOP_INBOX_E2E_DATABASE_URL="$DATABASE_URL" \
DESKTOP_INBOX_E2E_REDIS_URL="$REDIS_URL" \
DESKTOP_INBOX_E2E_AUTH_SECRET="$BETTER_AUTH_SECRET" \
DESKTOP_INBOX_E2E_REPORT_PATH="$report_dir/desktop-inbox-http-report.json" \
bun run test:desktop-inbox:e2e
;;
*)
echo "::error::Unknown end-to-end group: $group" >&2
exit 2
;;
esac
.github/scripts/stop-session.sh
Executable
+153
View File
@@ -0,0 +1,153 @@
#!/usr/bin/env bash
# Stops every process an app started, and returns only once none is left.
#
# Usage: stop-session.sh <session-leader-pid> <app-tag>
#
# Start the app as `E2E_APP=<app-tag> setsid <command> &`, with a tag no other process of this
# user carries (CI uses `<app>-<run id>-<attempt>-<shell pid>`). Its processes are the members of
# the leader's session, plus any process whose environment holds exactly `E2E_APP=<app-tag>` but
# left the session. This script, its ancestors and its own children are never signalled, even
# when they carry the tag.
# `next dev` needs both: its server and workers stay in the session, while its telemetry flush is
# spawned detached, in a session of its own, and writes .next/dev/trace after `next dev` exits.
# The next app in the job wipes that directory, so it must not start while any of them runs.
#
# Every member is signalled, including one that moved to its own process group, and one that
# started after the first signal.
#
# Zombies don't count as running: they have exited and only await reaping. The leader is one
# until the shell that started it waits on it, which that shell does after this returns.
set -u
leader=${1:-}
tag=${2:-}
grace_seconds=10
if ! [[ "$leader" =~ ^[0-9]+$ ]] || [ -z "$tag" ]; then
echo "::error::Usage: stop-session.sh <session-leader-pid> <app-tag>, with a non-empty tag." >&2
exit 2
fi
# Sets `state` and `parent` from /proc/<pid>/stat without forking, so a scan stays cheap, and
# fails once the process is gone. The command name can hold spaces and parentheses, so the
# fields are read after its last ")".
read_stat() {
local stat rest
read -r stat 2>/dev/null < "/proc/$1/stat" || return 1
rest=${stat##*") "}
state=${rest%% *}
rest=${rest#* }
parent=${rest%% *}
}
ancestors=" "
pid=$$
while read_stat "$pid" && [ "$parent" -gt 0 ]; do
ancestors+="$parent "
pid=$parent
[ "$pid" -eq 1 ] && break
done
# This script, an ancestor of it, or one of its own subshells and commands, which inherit the tag
# when the caller exported it. When a process on the walk exits mid-walk, the candidate has been
# reparented, so the walk restarts from it; ancestry that still won't resolve counts as not ours,
# so an app process is never spared by accident.
is_own() {
local candidate=$1 pid attempt
[[ "$ancestors" == *" $candidate "* ]] && return 0
for attempt in 1 2 3; do
pid=$candidate
while [ "$pid" -gt 1 ]; do
[ "$pid" -eq $$ ] && return 0
read_stat "$pid" || continue 2
pid=$parent
done
return 1
done
return 1
}
app_pids() {
local pid
{
ps -s "$leader" -o pid= 2>/dev/null | tr -d ' '
grep -Flzx "E2E_APP=$tag" /proc/[0-9]*/environ 2>/dev/null | cut -d/ -f3
} | sort -nu | while read -r pid; do
read_stat "$pid" || continue
[ "$state" = Z ] && continue
is_own "$pid" || echo "$pid"
done
}
signal_app() {
local pids
pids=$(app_pids)
[ -z "$pids" ] || kill "-$1" $pids 2>/dev/null || true
}
running() {
[ -n "$(app_pids)" ]
}
leader_running() {
ps -p "$leader" -o stat= 2>/dev/null | grep -qv '^Z'
}
# Prints one scan of the app's processes as a table, the expected telemetry flush labelled in a
# leading column, and fails when that scan finds none, so no caller prints an empty heading.
describe_app() {
local pids table
pids=$(app_pids | paste -sd, -)
[ -n "$pids" ] || return 1
table=$(ps -p "$pids" -o pid,sid,stat,etimes,args 2>/dev/null)
[ "$(wc -l <<< "$table")" -gt 1 ] || return 1
awk '{
note = ""
if (NR > 1 && $0 ~ /next\/dist\/telemetry\/detached-flush\.js/) note = "(expected telemetry flush)"
printf "%-26s %s\n", note, $0
}' <<< "$table" | cut -c1-240
}
# Centiseconds since boot: monotonic, and independent of the locale's decimal separator.
now_cs() {
local uptime
read -r uptime _ < /proc/uptime
echo "${uptime/./}"
}
# Re-sends the signal every 0.1s while the check holds, until the shared deadline. Succeeds
# once the check has stopped holding on two scans 0.1s apart, so a process missed by one scan
# (spawned, or mid-reparenting, while it ran) is still caught.
signal_while() {
local signal=$1 clear=0
shift
while ((10#$(now_cs) < deadline)); do
if "$@"; then
clear=0
signal_app "$signal"
else
clear=$((clear + 1))
((clear >= 2)) && return 0
fi
sleep 0.1
done
"$@" && return 1
sleep 0.1
! "$@"
}
deadline=$((10#$(now_cs) + grace_seconds * 100))
signal_while TERM leader_running
if ! leader_running && table=$(describe_app); then
echo "Processes from app $tag outlived its leader:"
echo "$table"
fi
signal_while TERM running && exit 0
echo "::warning::Processes from app $tag were still running ${grace_seconds}s after SIGTERM:"
describe_app || true
deadline=$((10#$(now_cs) + grace_seconds * 100))
signal_while KILL running && exit 0
echo "::error::Processes from app $tag survived SIGKILL for ${grace_seconds}s."
exit 1
.github/workflows/checks.yml
+737
View File
@@ -0,0 +1,737 @@
name: checks
on:
workflow_call:
workflow_dispatch:
permissions:
contents: read
jobs:
integration:
# Runs the real-infrastructure test layer: every `*.integration.ts` in packages/db and
# apps/sim, discovered by glob (`vitest run --mode integration`), against the database each
# provisioning path produces. A new integration suite needs no workflow change.
#
# The two paths build different schemas (migrations add triggers, checks and NOT VALID
# constraints that `db:push` does not), so each runs the whole suite. Vitest splits each
# suite's files across four shards by measured duration (DurationBalancedSequencer in vitest.shared.ts); a
# shard runs its files one at a time against its own database. Files run serially, so a shard
# barely uses more than one core: 4 vCPU is enough.
name: integration (${{ matrix.provision }}, ${{ matrix.shard }}/4)
runs-on: &runner-4vcpu ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
provision: [push, migrate]
shard: [1, 2, 3, 4]
services:
redis:
image: redis:8.2-alpine
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 5s
--health-timeout 5s
--health-retries 10
postgres:
image: pgvector/pgvector:pg17
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: sim_test
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U postgres -d sim_test"
--health-interval 5s
--health-timeout 5s
--health-retries 10
postgres-legacy:
image: postgres:16-alpine
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: sim_test
ports:
- 5433:5432
options: >-
--health-cmd "pg_isready -U postgres -d sim_test"
--health-interval 5s
--health-timeout 5s
--health-retries 10
env:
TEST_DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5432/sim_test
TEST_REDIS_URL: redis://127.0.0.1:6379
DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5432/sim_test
BETTER_AUTH_SECRET: oauth-postgres-ci-secret-at-least-32-characters
NEXT_PUBLIC_APP_URL: https://test.sim.ai
ENCRYPTION_KEY: '0000000000000000000000000000000000000000000000000000000000000000'
steps:
# A same-repository pull request checks out through a git mirror on a sticky disk, so a slow
# clone from GitHub (1 in 10 took a minute or more, up to 4) no longer sets the run's pace.
# The mirror is one disk per repository that job steps can write to, so fork pull requests
# and pushes, whose checks gate a deploy, keep the plain checkout: same trust split as the
# setup action's caches.
- &checkout-mirror
name: Checkout code (mirror)
if: github.event_name == 'pull_request' && !github.event.pull_request.head.repo.fork
uses: useblacksmith/checkout@25227e61ff9dafe400e22fa487b673eac4e4409a # v1.8.1
- &checkout-plain
name: Checkout code
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
- name: Setup workspace
uses: ./.github/actions/setup
with:
provider: ${{ vars.CI_PROVIDER }}
- name: Provision a fresh database through the supported command
working-directory: packages/db
run: |
bun -e 'import postgres from "postgres"; const sql = postgres(process.env.DATABASE_URL); for (const extension of ["vector", "btree_gin", "pg_trgm"]) await sql`CREATE EXTENSION IF NOT EXISTS ${sql(extension)}`; await sql.end()'
bun run db:${{ matrix.provision }}
- name: Verify migration replay is a no-op
if: matrix.provision == 'migrate' && matrix.shard == 1
working-directory: packages/db
run: bun run db:migrate
- name: Run packages/db integration tests
working-directory: packages/db
run: bun run test --mode integration --shard=${{ matrix.shard }}/4
- name: Run apps/sim integration tests
working-directory: apps/sim
# A non-UTC process zone keeps timestamp-without-time-zone handling honest.
env:
TZ: America/Los_Angeles
run: bun run test --mode integration --shard=${{ matrix.shard }}/4
- name: Verify cumulative billing timeout recovery on PostgreSQL 16
if: matrix.provision == 'push' && matrix.shard == 1
working-directory: apps/sim
env:
TEST_DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5433/sim_test
run: >-
bun run test --mode integration lib/billing/core/usage-log.integration.ts
--outputFile.json=test-results/integration-pg16.json
- name: Upload integration test reports
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: integration-reports-${{ matrix.provision }}-${{ matrix.shard }}
path: |
packages/db/test-results/*.json
apps/sim/test-results/*.json
if-no-files-found: warn
retention-days: 14
# Acceptance suites that cross a real HTTP boundary, one job per app, each on its own database.
# Off the integration jobs' path: the SCIM app boots hosted, which starts background usage replay
# against DATABASE_URL, so it must never share a database with suites asserting on billing rows.
# The suites exercise HTTP behavior rather than a provisioning path, so they run once, against
# the production (migrate) path. Each group's app environment lives in http-e2e.sh.
#
# SCIM runs two suites against a hosted app and is the longest group, so it keeps the 8 vCPU
# runner; the others boot a smaller self-hosted app and fit on 4.
e2e:
name: e2e (${{ matrix.group }})
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && matrix.runner || 'ubuntu-latest' }}
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
include:
- group: scim
runner: blacksmith-8vcpu-ubuntu-2404
- group: cli
runner: blacksmith-4vcpu-ubuntu-2404
- group: stop-after
runner: blacksmith-4vcpu-ubuntu-2404
- group: desktop-inbox
runner: blacksmith-4vcpu-ubuntu-2404
services:
postgres:
image: pgvector/pgvector:pg17
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: sim_test
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U postgres -d sim_test"
--health-interval 5s
--health-timeout 5s
--health-retries 10
# Only the desktop executor's app is given REDIS_URL: its doorbell and presence live there.
redis:
image: redis:8.2-alpine
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 5s
--health-timeout 5s
--health-retries 10
env:
DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5432/sim_test
BETTER_AUTH_SECRET: http-e2e-ci-secret-at-least-32-characters
ENCRYPTION_KEY: '0000000000000000000000000000000000000000000000000000000000000000'
steps:
- *checkout-mirror
- *checkout-plain
- name: Setup workspace
uses: ./.github/actions/setup
with:
provider: ${{ vars.CI_PROVIDER }}
# Migrations create their own extensions, as on a fresh self-hosted install.
- name: Provision the database through migrations
working-directory: packages/db
run: bun run db:migrate
# No step timeout: the job's bound covers a hang without cutting a slow but healthy suite
# short of writing its report.
- name: Run end-to-end suites
working-directory: apps/sim
run: bash ../../.github/scripts/http-e2e.sh "${{ matrix.group }}"
- name: Upload end-to-end reports and server logs
if: failure()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: http-e2e-${{ matrix.group }}
path: ${{ runner.temp }}/e2e/
if-no-files-found: ignore
retention-days: 7
# Pull requests skip the live desktop suite only when every change is clearly unrelated to the
# app it drives (docs, other apps, published content). Anything else, and any failure to work
# out the diff, runs it: a pull request that skipped it wrongly would first fail on staging.
desktop-changes:
name: desktop-changes
runs-on: &runner-2vcpu ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 5
outputs:
changed: ${{ github.event_name != 'pull_request' || steps.diff.outputs.changed != 'false' }}
steps:
- name: Checkout code (mirror)
if: github.event_name == 'pull_request' && !github.event.pull_request.head.repo.fork
uses: useblacksmith/checkout@25227e61ff9dafe400e22fa487b673eac4e4409a # v1.8.1
with:
fetch-depth: 2
- name: Checkout code
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
with:
fetch-depth: 2
- name: Diff against the pull request's base
id: diff
if: github.event_name == 'pull_request'
env:
BASE: ${{ github.event.pull_request.base.sha }}
run: bash .github/scripts/desktop-live-changes.sh "$BASE" >> "$GITHUB_OUTPUT"
# Desktop tools in the real Electron app against a local app, on its own runner: the
# Electron app, the dev app and its realtime server together outgrow the e2e runner.
desktop-live:
name: desktop-live
needs: desktop-changes
if: needs.desktop-changes.outputs.changed == 'true'
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-16vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 30
services:
postgres:
image: pgvector/pgvector:pg17
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: sim_test
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U postgres -d sim_test"
--health-interval 5s
--health-timeout 5s
--health-retries 10
redis:
image: redis:8.2-alpine
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 5s
--health-timeout 5s
--health-retries 10
env:
DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5432/sim_test
BETTER_AUTH_SECRET: desktop-live-e2e-ci-secret-at-least-32-characters
ENCRYPTION_KEY: '0000000000000000000000000000000000000000000000000000000000000000'
steps:
- *checkout-mirror
- *checkout-plain
- name: Setup workspace
uses: ./.github/actions/setup
with:
provider: ${{ vars.CI_PROVIDER }}
- name: Provision the database through migrations
working-directory: packages/db
run: bun run db:migrate
# Turbopack's dev cache turns the spec's route warm-up from a cold compile (~4 min) into a
# restore. It is content-addressed, so a pull request's changed modules still recompile; the
# key carries the installed Next version so an upgrade starts from an empty cache, and the
# event and fork segments keep untrusted runs off the cache trusted runs read.
- name: Resolve Turbopack dev cache key
id: next-cache
run: echo "key=${GITHUB_REPOSITORY}-next-dev-desktop-live-${GITHUB_EVENT_NAME}${FORK_SUFFIX}-$(jq -r .version node_modules/next/package.json)" >> "$GITHUB_OUTPUT"
env:
FORK_SUFFIX: ${{ github.event.pull_request.head.repo.fork && '-fork' || '' }}
- name: Mount Turbopack dev cache
uses: ./.github/actions/cache
with:
provider: ${{ vars.CI_PROVIDER }}
key: ${{ steps.next-cache.outputs.key }}
path: ./apps/sim/.next/dev
# Chat switches, Stop, sign-out, approval, the dormant-executor round trip, and background
# runs across a chat switch and a network cut. The spec runs the recording proxy (the app's
# public origin) and the stand-in worker.
- name: Verify desktop tools in the Electron app against a local app
env:
NEXT_PUBLIC_APP_URL: http://127.0.0.1:3020
BETTER_AUTH_URL: http://127.0.0.1:3020
REDIS_URL: redis://127.0.0.1:6379
SIM_AGENT_API_URL: http://127.0.0.1:3022
NEXT_PUBLIC_SOCKET_URL: http://127.0.0.1:3023
SOCKET_SERVER_URL: http://127.0.0.1:3023
NEXT_PUBLIC_FORCE_HOSTED: 'false'
COPILOT_API_KEY: desktop-tools-e2e-ci-local-copilot-key
COPILOT_TOOL_PERMISSIONS_ENABLED: 'true'
MOTHERSHIP_SIM_TRANSPORT: direct
INTERNAL_API_SECRET: desktop-tools-e2e-ci-local-secret-at-least-32-characters
DISABLE_TELEMETRY: 'true'
NEXT_TELEMETRY_DISABLED: '1'
READY_TIMEOUT_SECONDS: 300
run: |
report_dir="$RUNNER_TEMP/e2e"
next_log="$report_dir/desktop-tools-next.log"
mkdir -p "$report_dir"
# Each app runs in its own session under an E2E_APP tag, and stop-session.sh returns once
# every process it started has exited.
realtime_tag="desktop-realtime-$GITHUB_RUN_ID-$GITHUB_RUN_ATTEMPT-$$"
server_tag="desktop-tools-$GITHUB_RUN_ID-$GITHUB_RUN_ATTEMPT-$$"
# Keeps the restored cache under the cap the dev scripts apply locally.
(cd apps/sim && bun run dev:cache:cap)
start_sim() {
(cd apps/sim && E2E_APP="$server_tag" exec setsid node ../../node_modules/next/dist/bin/next dev --hostname 127.0.0.1 \
--port 3021 >> "$next_log" 2>&1) &
server_pid=$!
}
# A corrupted cache aborts Turbopack instead of falling back. Only then is the shared cache
# dropped: a failing test must not cost every later run its warm cache.
cache_broken() {
grep -qiE 'cache corruption|turbopack.*panic|panicked' "$next_log" 2>/dev/null
}
# The cache directory is a mount point: empty it rather than remove it. Absolute, because
# the EXIT trap runs after the step has moved into apps/desktop.
clear_cache() {
find "$GITHUB_WORKSPACE/apps/sim/.next/dev" -mindepth 1 -maxdepth 1 -exec rm -rf {} +
}
# SIGINT first: `next dev` SIGKILLs its server 100ms after SIGTERM, which discards a cache
# write in flight. stop-session.sh then removes anything still running.
# SIGINT is best-effort; the cleanup always runs, since workers and the detached telemetry
# flush can outlive a server that has already exited.
stop_sim() {
if kill -INT "$server_pid" 2>/dev/null; then
for _ in $(seq 1 30); do kill -0 "$server_pid" 2>/dev/null || break; sleep 1; done
fi
bash "$GITHUB_WORKSPACE/.github/scripts/stop-session.sh" "$server_pid" "$server_tag"
}
(cd apps/realtime && PORT=3023 SIM_DB_ROLE=realtime ALLOWED_ORIGINS="$NEXT_PUBLIC_APP_URL" \
E2E_APP="$realtime_tag" exec setsid bun src/index.ts > "$report_dir/desktop-tools-realtime.log" 2>&1) &
realtime_pid=$!
start_sim
finish() {
status=$?
stop_sim || status=1
bash "$GITHUB_WORKSPACE/.github/scripts/stop-session.sh" "$realtime_pid" "$realtime_tag" || status=1
wait "$server_pid" "$realtime_pid" 2>/dev/null || true
if cache_broken; then
echo "::warning::Turbopack reported a broken dev cache; clearing it for the next run."
clear_cache
fi
exit "$status"
}
trap finish EXIT
# The apps boot while the runner installs Electron's libraries and bundles the shell.
sudo apt-get update -q
sudo apt-get install -yq xvfb libgtk-3-0t64 libnss3 libasound2t64 libgbm1 libxss1 \
libxtst6 libatk-bridge2.0-0t64 libxkbcommon0 > /dev/null
# Bundle only: `bun run build` also fetches the macOS node-pty prebuilds for packaging,
# which a Linux run does not use.
(cd apps/desktop && bun run scripts/build.ts)
started=$SECONDS
retried=0
until curl --fail --silent --max-time 10 http://127.0.0.1:3021/api/health > /dev/null &&
curl --fail --silent --max-time 10 http://127.0.0.1:3023/health > /dev/null; do
if ! kill -0 "$server_pid" 2>/dev/null; then
tail -n 200 "$next_log"
if [ "$retried" = 0 ] && cache_broken; then
echo "::warning::Turbopack rejected the restored dev cache; restarting from an empty cache."
bash "$GITHUB_WORKSPACE/.github/scripts/stop-session.sh" "$server_pid" "$server_tag"
clear_cache
: > "$next_log"
retried=1
start_sim
continue
fi
exit 1
fi
kill -0 "$realtime_pid" 2>/dev/null || { tail -n 200 "$report_dir/desktop-tools-realtime.log"; exit 1; }
[ $((SECONDS - started)) -lt "$READY_TIMEOUT_SECONDS" ] || { echo '::error::Local app did not become ready'; exit 1; }
sleep 2
done
echo "Local apps ready $((SECONDS - started))s after the Electron bundle"
cd apps/desktop
SIM_DESKTOP_E2E_SIM_URL=http://127.0.0.1:3021 \
SIM_DESKTOP_E2E_PROXY_PORT=3020 \
SIM_DESKTOP_E2E_AGENT_PORT=3022 \
SIM_DESKTOP_E2E_DATABASE_URL="$DATABASE_URL" \
SIM_DESKTOP_E2E_REDIS_URL="$REDIS_URL" \
SIM_DESKTOP_E2E_AUTH_SECRET="$BETTER_AUTH_SECRET" \
xvfb-run -a -s '-screen 0 1920x1200x24' bunx playwright test e2e/desktop-tools-live-sim.spec.ts \
--output "$report_dir/desktop-tools-results" --retries=0
- name: Upload Electron E2E results and server logs
if: failure()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: desktop-live-e2e-results
path: ${{ runner.temp }}/e2e/
if-no-files-found: ignore
retention-days: 7
# Lint, audits, type-check and the schema sync check: about two minutes of mostly cached work,
# kept off the test shards so neither waits on the other.
lint:
name: lint
runs-on: *runner-4vcpu
timeout-minutes: 15
steps:
# The diff-based audits below need a base commit to read, and the default
# depth of 1 clones a single commit with no parent. They normally fetch
# their base by SHA (see "Resolve base ref"), so this depth only covers the
# `HEAD~1` fallback — but without it that fallback resolves to nothing.
#
# Worth stating because the failure was invisible for so long: the migration
# audit read the resulting `git diff` failure as "no migrations changed" and
# exited 0, so it had never actually run on a push build.
# Same mirror/plain split as the integration job's checkout.
- &checkout-mirror-depth2
name: Checkout code (mirror)
if: github.event_name == 'pull_request' && !github.event.pull_request.head.repo.fork
uses: useblacksmith/checkout@25227e61ff9dafe400e22fa487b673eac4e4409a # v1.8.1
with:
fetch-depth: 2
- &checkout-plain-depth2
name: Checkout code
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
with:
fetch-depth: 2
- name: Setup workspace
uses: ./.github/actions/setup
with:
provider: ${{ vars.CI_PROVIDER }}
turbo-cache-key: turbo-cache
# Surfaces known CVEs in the dependency tree. Non-blocking until the
# existing advisory backlog is triaged, then flip to a required gate by
# removing continue-on-error.
- name: Security audit
run: bun audit
continue-on-error: true
- name: Validate env flags
run: |
FILE="apps/sim/lib/core/config/env-flags.ts"
ERRORS=""
echo "Checking for hardcoded boolean env flags..."
# Use perl for multiline matching to catch both:
# export const isHosted = true
# export const isHosted =
# true
HARDCODED=$(perl -0777 -ne 'while (/export const (is[A-Za-z]+)\s*=\s*\n?\s*(true|false)\b/g) { print " $1 = $2\n" }' "$FILE")
if [ -n "$HARDCODED" ]; then
ERRORS="${ERRORS}\n❌ Env flags must not be hardcoded to boolean literals!\n\nFound hardcoded flags:\n${HARDCODED}\n\nEnv flags should derive their values from environment variables.\n"
fi
echo "Checking env flag naming conventions..."
# Check that all export const (except functions) start with 'is'
# This finds exports like "export const someFlag" that don't start with "is" or "get"
BAD_NAMES=$(grep -E "^export const [a-z]" "$FILE" | grep -vE "^export const (is|get)" | sed 's/export const \([a-zA-Z]*\).*/ \1/')
if [ -n "$BAD_NAMES" ]; then
ERRORS="${ERRORS}\n❌ Env flags must use 'is' prefix for boolean flags!\n\nFound incorrectly named flags:\n${BAD_NAMES}\n\nExample: 'hostedMode' should be 'isHostedMode'\n"
fi
if [ -n "$ERRORS" ]; then
echo ""
echo -e "$ERRORS"
exit 1
fi
echo "✅ All env flags are properly configured"
# One fetch for both base-ref audits, and no `|| true`: a swallowed fetch leaves
# the base ref absent, which neither audit can tell apart from a branch that
# changed nothing. The block-registry check at least degrades to a visible
# `⚠ … skipping` line; the migration audit printed `✓ No new migrations to
# check` and exited 0, clearing the only guard on production DDL.
#
# Depth stays at 1 — without a merge-base the migration audit diffs the two
# tips, which under `--diff-filter=AM` is exactly the migrations new here.
#
# On push the base is `github.event.before`, the tip the branch had before
# this push — not `HEAD~1`, which names only the last commit and would let a
# multi-commit push slip every earlier commit's migrations past the audit.
# It is fetched by SHA at depth 1; the audits diff two tips and need no
# common ancestry. An all-zero `before` means the branch is new and has no
# predecessor to diff, so `HEAD~1` remains the fallback there.
- name: Resolve base ref for diff-based audits
id: audit_base
run: |
if [ "${{ github.event_name }}" = "pull_request" ]; then
git fetch --depth=1 origin "${{ github.base_ref }}"
echo "ref=origin/${{ github.base_ref }}" >> "$GITHUB_OUTPUT"
elif [ -n "${{ github.event.before }}" ] &&
[ "${{ github.event.before }}" != "0000000000000000000000000000000000000000" ]; then
git fetch --depth=1 origin "${{ github.event.before }}"
echo "ref=${{ github.event.before }}" >> "$GITHUB_OUTPUT"
else
echo "ref=HEAD~1" >> "$GITHUB_OUTPUT"
fi
- name: Check block registry invariants
run: bun run apps/sim/scripts/check-block-registry.ts "${{ steps.audit_base.outputs.ref }}"
- name: Lint code
run: bun run lint:check
# Workflow syntax, expressions, `needs` references and runner labels. ShellCheck stays off
# here: the existing run blocks carry info-level findings that are their own cleanup.
- name: Lint workflows
env:
ACTIONLINT_VERSION: 1.7.12
ACTIONLINT_SHA256: 8aca8db96f1b94770f1b0d72b6dddcb1ebb8123cb3712530b08cc387b349a3d8
run: |
archive="$RUNNER_TEMP/actionlint.tar.gz"
curl -fsSL -o "$archive" \
"https://github.com/rhysd/actionlint/releases/download/v${ACTIONLINT_VERSION}/actionlint_${ACTIONLINT_VERSION}_linux_amd64.tar.gz"
echo "${ACTIONLINT_SHA256} ${archive}" | sha256sum -c -
tar -xzf "$archive" -C "$RUNNER_TEMP" actionlint
"$RUNNER_TEMP/actionlint" -color -shellcheck= -pyflakes=
# Every zero-argument `check:*` script, run concurrently. The list is derived in
# scripts/run-audits.ts, which also writes the per-audit timing table to the job
# summary and annotates failures. Audits needing a base ref stay separate below.
- name: Repo audits
run: bun run check:audits
- name: Verify docs manifest is in sync
run: bun run docs-manifest:check
- name: Migration safety (zero-downtime) audit
run: bun run check:migrations "${{ steps.audit_base.outputs.ref }}"
# Every workspace, not just realtime. packages/emcn, packages/utils,
# apps/desktop and apps/docs had no type check in CI at all; apps/sim's
# source was covered only as a side effect of `next build` in the separate
# `build` job. Note this does NOT cover apps/sim's tests — its tsconfig
# excludes *.test.ts(x), and including them today surfaces ~2.2k errors,
# so that is its own cleanup rather than a gate to switch on here.
- name: Type-check all workspaces
run: bunx turbo run type-check
- name: Check schema and migrations are in sync
working-directory: packages/db
run: |
bunx drizzle-kit generate --config=./drizzle.config.ts
if [ -n "$(git status --porcelain ./migrations)" ]; then
echo "❌ Schema and migrations are out of sync!"
echo "Run 'cd packages/db && bunx drizzle-kit generate' and commit the new migrations."
git status --porcelain ./migrations
git diff ./migrations
exit 1
fi
echo "✅ Schema and migrations are in sync"
# The root scripts and every workspace's Vitest suite (`bun run test`), split in two. apps/sim is
# about 98% of the time, so only its files are sharded; shard 1 also runs the root scripts and
# the other workspaces. Each shard keeps its own Turbo cache: pass-through args are part of the
# task hash, so a shard only ever replays its own result.
test:
name: test (${{ matrix.shard }}/2)
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-8vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
shard: [1, 2]
steps:
- *checkout-mirror-depth2
- *checkout-plain-depth2
- name: Setup workspace
uses: ./.github/actions/setup
with:
provider: ${{ vars.CI_PROVIDER }}
turbo-cache-key: turbo-cache-test-${{ matrix.shard }}
# cloud-review-tools.test.ts runs the real helper on the runner, which shells
# out to rg. Blacksmith's image ships it, GitHub's doesn't.
- name: Install ripgrep
run: command -v rg || (sudo apt-get update && sudo apt-get install -y ripgrep)
- name: Verify shell placeholder compilation in Bash
if: matrix.shard == 1
working-directory: apps/sim
env:
SHELL_PLACEHOLDERS_REPORT_PATH: ${{ runner.temp }}/shell-placeholders.json
run: bun scripts/test-shell-placeholders-e2e.ts
- name: Upload shell placeholder execution report
if: failure() && matrix.shard == 1
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: shell-placeholders
path: ${{ runner.temp }}/shell-placeholders.json
if-no-files-found: warn
retention-days: 7
- name: Run tests
env:
NODE_OPTIONS: '--no-warnings --max-old-space-size=8192'
NEXT_PUBLIC_APP_URL: 'https://www.sim.ai'
DATABASE_URL: 'postgresql://postgres:postgres@localhost:5432/simstudio'
ENCRYPTION_KEY: '0000000000000000000000000000000000000000000000000000000000000000' # dummy key for CI only
TURBO_CACHE_DIR: .turbo
SHARD: ${{ matrix.shard }}
run: |
if [ "$SHARD" = 1 ]; then
bun run test:scripts
bunx turbo run test --filter='!@sim/app'
fi
bunx turbo run test --filter=@sim/app -- --shard="$SHARD/2"
# Next.js production build, in parallel with lint + tests. Sticky disks are
# cloned from the last committed snapshot per job and committed last-writer-
# wins, so concurrent mounts are safe. The bun/node_modules disks are shared
# with the test jobs (the lockfile-hashed key means they only ever share when the
# dependency tree really is identical, so LWW loss is harmless), but the Turbo
# cache gets its own key: with a shared key, only the last committer's new
# entries survive each run, so the test and build Turbo entries would evict
# each other nondeterministically.
#
# Runner is sized for the COLD-cache build, which is what OOM-killed the 8vcpu
# tier (23 kills / 1074 runs at 98% of its 30.4 GB): warm peaks ~12 GB, cold
# peaked 51 GB. NODE_OPTIONS' --max-old-space-size caps only Node's JS heap,
# not the native Turbopack workers that dominate, so it cannot prevent this.
build:
name: build
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-16vcpu-ubuntu-2404' || 'linux-x64-8-core' }}
# Build durations crossed 15 minutes as the app grew (10m02 on Jul 29 AM,
# 14m44 after the folders/desktop/library merges, then two straight
# timeouts) — GitHub reports a job timeout as "cancelled". 25 keeps
# headroom without masking a genuine hang.
timeout-minutes: 25
steps:
- *checkout-mirror
- *checkout-plain
- name: Setup workspace
uses: ./.github/actions/setup
with:
provider: ${{ vars.CI_PROVIDER }}
turbo-cache-key: turbo-cache-build
# No `.next/cache` mount: the Turbopack persistent build cache is off. A
# controlled A/B on one branch (PR #6078) with a byte-identical module graph
# measured compile at 113s with the cache off, 162s cold with it on, and
# 360s warm — the cache made the same build 3.2x slower, and it grew
# 5.1 GB -> 12 GB across two runs of an unchanged tree, so a disk degrades
# the more it is used. Mounting a disk nothing reads would only cost storage.
# Running out of RAM kills the whole VM and surfaces only as "the runner
# has received a shutdown signal" — no mention of memory, ~12 min in. Warn
# with the real numbers so that failure is a one-line diagnosis instead of
# a mystery. Warn, never fail: a warm build peaks ~12 GB and a partial one
# ~28 GB, so a 32 GB runner still completes plenty of builds, and the
# GitHub fallback is the break-glass path — degrading it to a guaranteed
# failure would be worse than the risk this flags.
- name: Check runner memory headroom
run: |
TOTAL_GB=$(awk '/MemTotal/ {printf "%d", $2/1048576}' /proc/meminfo)
echo "Runner memory: ${TOTAL_GB} GB"
if [ "$TOTAL_GB" -lt 40 ]; then
echo "::warning::Runner has ${TOTAL_GB} GB. A cold-cache build peaks ~51 GB, so this run may be OOM-killed (reported only as 'the runner has received a shutdown signal'). Warm/partial builds should still fit."
fi
- name: Build application
env:
NODE_OPTIONS: '--no-warnings --max-old-space-size=8192'
NEXT_PUBLIC_APP_URL: 'https://www.sim.ai'
DATABASE_URL: 'postgresql://postgres:postgres@localhost:5432/simstudio'
STRIPE_SECRET_KEY: 'dummy_key_for_ci_only'
STRIPE_WEBHOOK_SECRET: 'dummy_secret_for_ci_only'
RESEND_API_KEY: 'dummy_key_for_ci_only'
AWS_REGION: 'us-west-2'
ENCRYPTION_KEY: '0000000000000000000000000000000000000000000000000000000000000000' # dummy key for CI only
TURBO_CACHE_DIR: .turbo
run: bunx turbo run build --filter=@sim/app
# One status for every check above: the single check to require on a branch ruleset, so adding,
# sharding or renaming a job never means editing the ruleset. Skipped is a pass (the desktop live
# suite skips on pull requests that cannot affect it); a failure or a cancellation is not.
# `always()`, not `!cancelled()`: a skipped job satisfies a required check, so a gate that skips
# on a cancelled run would report a cancelled head commit as passing.
ci:
name: ci
needs: [integration, e2e, desktop-changes, desktop-live, lint, test, build]
if: ${{ always() }}
runs-on: *runner-2vcpu
timeout-minutes: 5
steps:
- name: Require every check to pass
env:
RESULTS: ${{ toJSON(needs) }}
run: |
failed="$(jq -r 'to_entries[] | select(.value.result != "success" and .value.result != "skipped") | "\(.key): \(.value.result)"' <<< "$RESULTS")"
if [ -n "$failed" ]; then
echo "::error::Checks did not pass:"
echo "$failed"
exit 1
fi
echo "All checks passed."
.github/workflows/ci.yml
+226 -217
View File
@@ -1,4 +1,4 @@
name: CI
name: ci
# Runner provider toggle, read from the CI_PROVIDER repo variable:
#
@@ -18,13 +18,11 @@ name: CI
on:
push:
branches: [main, staging, dev]
# No path filter: `checks / ci` is the required status check, and a pull request whose run
# never starts never reports it, so it could not merge. Docs-only pull requests are rare enough
# that running CI on them costs nothing measurable.
pull_request:
branches: [main, staging, dev]
# Docs content and markdown don't affect the app build or images; push
# runs stay unfiltered because they feed the deploy pipeline.
paths-ignore:
- 'apps/docs/content/**'
- '**/*.md'
concurrency:
group: ci-${{ github.ref }}
@@ -34,18 +32,17 @@ permissions:
contents: read
jobs:
test-build:
name: Test and Build
checks:
name: checks
if: github.ref != 'refs/heads/dev' || github.event_name == 'pull_request'
uses: ./.github/workflows/test-build.yml
secrets: inherit
uses: ./.github/workflows/checks.yml
# Detect if this is a version release commit (e.g., "v0.5.24: ...")
# Smallest runner on purpose: a few seconds of pure shell over the commit
# message, no checkout and no install.
detect-version:
name: Detect Version
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
version:
name: version
runs-on: &runner-2vcpu ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 5
if: github.event_name == 'push' && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/staging' || github.ref == 'refs/heads/dev')
outputs:
@@ -74,9 +71,9 @@ jobs:
# the Electron app or the bridge packages trigger a per-env prerelease build
# (dev → dev stream, staging → staging) that the env's update feed
# (/api/desktop/update) starts offering automatically.
detect-desktop-changes:
name: Detect Desktop Changes
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
desktop-changes:
name: desktop-changes
runs-on: *runner-2vcpu
timeout-minutes: 5
if: github.event_name == 'push' && (github.ref == 'refs/heads/dev' || github.ref == 'refs/heads/staging')
outputs:
@@ -116,36 +113,41 @@ jobs:
# in place before the new app version deploys (replaces the removed ECS
# migration sidecar)
migrate:
name: Migrate DB
needs: [test-build]
name: migrate
needs: [checks]
# Explicit need results instead of the implicit success(): a skipped job
# anywhere in the transitive needs chain silently fails implicit success()
# and cascade-skips the deploy chain (migrate -> promote-images ->
# and cascade-skips the deploy chain (migrate -> promote ->
# CodeDeploy) — this bit us on 2026-07-23. State requirements explicitly.
if: >-
!cancelled() &&
needs.test-build.result == 'success' &&
needs.checks.result == 'success' &&
github.event_name == 'push' &&
(github.ref == 'refs/heads/main' || github.ref == 'refs/heads/staging')
uses: ./.github/workflows/migrations.yml
uses: ./.github/workflows/migrate.yml
with:
environment: ${{ github.ref == 'refs/heads/main' && 'production' || 'staging' }}
secrets: inherit
secrets:
DATABASE_URL: ${{ secrets.DATABASE_URL }}
STAGING_DATABASE_URL: ${{ secrets.STAGING_DATABASE_URL }}
MIGRATION_DATABASE_URL: ${{ secrets.MIGRATION_DATABASE_URL }}
STAGING_MIGRATION_DATABASE_URL: ${{ secrets.STAGING_MIGRATION_DATABASE_URL }}
# Same ordering for dev (schema push before the dev image lands in ECR)
migrate-dev:
name: Migrate Dev DB
name: migrate-dev
if: github.event_name == 'push' && github.ref == 'refs/heads/dev'
uses: ./.github/workflows/migrations.yml
uses: ./.github/workflows/migrate.yml
with:
environment: dev
secrets: inherit
secrets:
DEV_DATABASE_URL: ${{ secrets.DEV_DATABASE_URL }}
# Dev: build immutable images alongside the schema push and Trigger upload.
build-dev:
name: Build Dev ECR
image-dev:
name: image-dev (${{ matrix.id }})
if: github.event_name == 'push' && github.ref == 'refs/heads/dev'
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && matrix.bs_runner || matrix.gh_runner }}
runs-on: &runner-matrix ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && matrix.bs_runner || matrix.gh_runner }}
timeout-minutes: 30
permissions:
contents: read
@@ -162,29 +164,33 @@ jobs:
# deploy — nothing ships until the image is pushed — and its two
# dominant steps both scale with cores (`bun install` ~300-400s, `next
# build` ~260s). The same `next build` runs on 16 vCPU in the separate
# Build App verification job, which does not gate anything; this one
# checks `build` job, which does not gate anything; this one
# was doing comparable work on half the cores.
#
# cache_mb is the layer cache the post-job prune retains, and it is the
# only reason the sticky disks stay bounded — see docker-build's
# only reason the sticky disks stay bounded — see the image action's
# action.yml. Rows that omit it take the small-image default there. The
# app image overrides because it carries ~34 layers plus apt and bun
# cache mounts for the whole monorepo; 100 GB is several builds' worth
# of headroom over that working set.
- dockerfile: ./docker/app.Dockerfile
- id: app
dockerfile: ./docker/app.Dockerfile
cache_mb: '102400'
ecr_repo_secret: ECR_APP
gh_runner: linux-x64-8-core
bs_runner: blacksmith-16vcpu-ubuntu-2404
- dockerfile: ./docker/db.Dockerfile
- id: db
dockerfile: ./docker/db.Dockerfile
ecr_repo_secret: ECR_MIGRATIONS
gh_runner: ubuntu-latest
bs_runner: blacksmith-2vcpu-ubuntu-2404
- dockerfile: ./docker/realtime.Dockerfile
- id: realtime
dockerfile: ./docker/realtime.Dockerfile
ecr_repo_secret: ECR_REALTIME
gh_runner: ubuntu-latest
bs_runner: blacksmith-2vcpu-ubuntu-2404
- dockerfile: ./docker/pii.Dockerfile
- id: pii
dockerfile: ./docker/pii.Dockerfile
ecr_repo_secret: ECR_PII
gh_runner: ubuntu-latest
bs_runner: blacksmith-2vcpu-ubuntu-2404
@@ -215,7 +221,7 @@ jobs:
ECR_REPO: ${{ matrix.ecr_repo_secret == 'ECR_APP' && secrets.ECR_APP || matrix.ecr_repo_secret == 'ECR_MIGRATIONS' && secrets.ECR_MIGRATIONS || matrix.ecr_repo_secret == 'ECR_REALTIME' && secrets.ECR_REALTIME || matrix.ecr_repo_secret == 'ECR_PII' && secrets.ECR_PII || '' }}
- name: Build and push
uses: ./.github/actions/docker-build
uses: ./.github/actions/image
with:
provider: ${{ vars.CI_PROVIDER }}
file: ${{ matrix.dockerfile }}
@@ -226,13 +232,13 @@ jobs:
max-cache-size-mb: ${{ matrix.cache_mb }}
# Staging/production coordinate task releases with app traffic cutover.
# Dev tasks deploy independently in deploy-trigger-dev.yml.
prepare-trigger:
name: Prepare Trigger.dev
# Dev tasks deploy independently in trigger-dev.yml.
trigger-upload:
name: trigger-upload
if: >-
github.event_name == 'push' &&
(github.ref == 'refs/heads/main' || github.ref == 'refs/heads/staging')
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
runs-on: &runner-4vcpu ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 30
outputs:
version: ${{ steps.deploy.outputs.deploymentVersion }}
@@ -242,24 +248,11 @@ jobs:
- name: Checkout code
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
- name: Setup workspace
uses: ./.github/actions/setup
with:
bun-version: 1.4.2
- name: Cache Bun dependencies
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
with:
path: |
~/.bun/install/cache
node_modules
**/node_modules
key: ${{ runner.os }}-bun-${{ hashFiles('**/bun.lock') }}
restore-keys: |
${{ runner.os }}-bun-
- name: Install dependencies
run: bun install --frozen-lockfile --ignore-scripts
provider: ${{ vars.CI_PROVIDER }}
node-version: ''
- name: Select Trigger environment
id: target
@@ -305,13 +298,13 @@ jobs:
# Runs in parallel with tests — only immutable sha tags are pushed here, and
# the CodePipeline EventBridge triggers filter on exactly the
# latest/staging/dev ECR tags, so nothing deploys and no mutable tag moves
# until promote-images / create-ghcr-manifests retag after the gate.
build-amd64:
name: Build AMD64
# until promote / ghcr retag after the gate.
image-amd64:
name: image-amd64 (${{ matrix.id }})
if: >-
github.event_name == 'push' &&
(github.ref == 'refs/heads/main' || github.ref == 'refs/heads/staging')
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && matrix.bs_runner || matrix.gh_runner }}
runs-on: *runner-matrix
timeout-minutes: 30
permissions:
contents: read
@@ -321,30 +314,35 @@ jobs:
fail-fast: false
matrix:
include:
- dockerfile: ./docker/app.Dockerfile
- id: app
dockerfile: ./docker/app.Dockerfile
cache_mb: '102400'
ghcr_image: ghcr.io/simstudioai/simstudio
ecr_repo_secret: ECR_APP
gh_runner: linux-x64-8-core
bs_runner: blacksmith-16vcpu-ubuntu-2404
- dockerfile: ./docker/db.Dockerfile
- id: db
dockerfile: ./docker/db.Dockerfile
ghcr_image: ghcr.io/simstudioai/migrations
ecr_repo_secret: ECR_MIGRATIONS
gh_runner: ubuntu-latest
bs_runner: blacksmith-2vcpu-ubuntu-2404
- dockerfile: ./docker/realtime.Dockerfile
- id: realtime
dockerfile: ./docker/realtime.Dockerfile
ghcr_image: ghcr.io/simstudioai/realtime
ecr_repo_secret: ECR_REALTIME
gh_runner: ubuntu-latest
bs_runner: blacksmith-2vcpu-ubuntu-2404
- dockerfile: ./docker/pii.Dockerfile
- id: pii
dockerfile: ./docker/pii.Dockerfile
ghcr_image: ghcr.io/simstudioai/pii
ecr_repo_secret: ECR_PII
gh_runner: ubuntu-latest
bs_runner: blacksmith-2vcpu-ubuntu-2404
# No ECR repo is provisioned for cron, so it publishes to GHCR only.
# The tag step below omits the ECR tag when the repo name is empty.
- dockerfile: ./docker/cron.Dockerfile
- id: cron
dockerfile: ./docker/cron.Dockerfile
ghcr_image: ghcr.io/simstudioai/cron
gh_runner: ubuntu-latest
bs_runner: blacksmith-2vcpu-ubuntu-2404
@@ -383,8 +381,8 @@ jobs:
ECR_REPO: ${{ matrix.ecr_repo_secret == 'ECR_APP' && secrets.ECR_APP || matrix.ecr_repo_secret == 'ECR_MIGRATIONS' && secrets.ECR_MIGRATIONS || matrix.ecr_repo_secret == 'ECR_REALTIME' && secrets.ECR_REALTIME || matrix.ecr_repo_secret == 'ECR_PII' && secrets.ECR_PII || '' }}
# Only sha tags here — the ECR deploy tags (latest/staging) are applied
# by promote-images and the GHCR latest-amd64/version tags by
# create-ghcr-manifests, both after tests and migrations pass.
# by promote and the GHCR latest-amd64/version tags by
# ghcr, both after tests and migrations pass.
- name: Generate tags
id: meta
run: |
@@ -419,7 +417,7 @@ jobs:
- name: Build and push images
if: steps.meta.outputs.skip != 'true'
uses: ./.github/actions/docker-build
uses: ./.github/actions/image
with:
provider: ${{ vars.CI_PROVIDER }}
file: ${{ matrix.dockerfile }}
@@ -434,33 +432,33 @@ jobs:
# the image builds themselves run in parallel with the tests. A single job
# (not a matrix) so all four sha manifests are verified before any tag
# moves; a missing image can't produce a partial mixed-version deploy.
promote-images:
name: Promote Images
needs: [migrate, build-amd64, prepare-trigger, migrate-dev, build-dev]
promote:
name: promote
needs: [migrate, image-amd64, trigger-upload, migrate-dev, image-dev]
# Explicit results: see migrate's comment.
if: >-
!cancelled() && github.event_name == 'push' &&
(
((github.ref == 'refs/heads/main' || github.ref == 'refs/heads/staging') &&
needs.prepare-trigger.result == 'success' &&
needs.trigger-upload.result == 'success' &&
needs.migrate.result == 'success' &&
needs.build-amd64.result == 'success') ||
needs.image-amd64.result == 'success') ||
(github.ref == 'refs/heads/dev' &&
needs.migrate-dev.result == 'success' &&
needs.build-dev.result == 'success')
needs.image-dev.result == 'success')
)
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
runs-on: *runner-2vcpu
timeout-minutes: 10
permissions:
contents: read
id-token: write
outputs:
# Whether the deploy tag was actually moved (false on a stale-run guard
# skip). promote-trigger keys off this so tasks are never promoted when
# skip). trigger-promote keys off this so tasks are never promoted when
# the app itself wasn't.
promoted: ${{ steps.guard.outputs.fresh }}
# Epoch when the deploy tag was retagged (this push's ECS pipeline trigger).
# promote-trigger passes it to the poll script so a stale pipeline execution
# trigger-promote passes it to the poll script so a stale pipeline execution
# reusing the same image digest can't satisfy the cutover gate.
retag_epoch: ${{ steps.promote.outputs.retag_epoch }}
# Unchanged tags verify the latest execution's cutover without an epoch bound.
@@ -552,21 +550,21 @@ jobs:
# traffic cutover (CodeDeploy AllowTraffic on every target). The image retag
# triggers the ECS pipeline; this job correlates it via the digest + retag epoch
# (rejecting a stale execution reusing the digest) and promotes at cutover.
# Skipped when promote-images skipped the tag move (stale run) — tasks then
# Skipped when promote skipped the tag move (stale run) — tasks then
# correctly stay on the old version. If the app deploy fails or never cuts over,
# promote never fires and this job fails visibly.
promote-trigger:
name: Promote Trigger.dev
needs: [promote-images, prepare-trigger]
trigger-promote:
name: trigger-promote
needs: [promote, trigger-upload]
# Explicit results also suppress skip propagation from optional ancestors.
if: >-
!cancelled() &&
github.event_name == 'push' &&
(github.ref == 'refs/heads/main' || github.ref == 'refs/heads/staging') &&
needs.promote-images.result == 'success' &&
needs.prepare-trigger.result == 'success' &&
needs.promote-images.outputs.promoted == 'true'
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
needs.promote.result == 'success' &&
needs.trigger-upload.result == 'success' &&
needs.promote.outputs.promoted == 'true'
runs-on: *runner-4vcpu
# Leave setup/promotion headroom above the 70-minute cutover poll.
timeout-minutes: 90
permissions:
@@ -576,24 +574,11 @@ jobs:
- name: Checkout code
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
- name: Setup workspace
uses: ./.github/actions/setup
with:
bun-version: 1.4.2
- name: Cache Bun dependencies
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
with:
path: |
~/.bun/install/cache
node_modules
**/node_modules
key: ${{ runner.os }}-bun-${{ hashFiles('**/bun.lock') }}
restore-keys: |
${{ runner.os }}-bun-
- name: Install dependencies
run: bun install --frozen-lockfile --ignore-scripts
provider: ${{ vars.CI_PROVIDER }}
node-version: ''
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@e7f100cf4c008499ea8adda475de1042d6975c7b # v6
@@ -607,10 +592,10 @@ jobs:
- name: Wait for ECS traffic cutover
env:
OVERALL_TIMEOUT: 4200
APP_IMAGE_CHANGED: ${{ needs.promote-images.outputs.app_image_changed }}
DIGEST: ${{ needs.promote-images.outputs.app_image_digest }}
APP_IMAGE_CHANGED: ${{ needs.promote.outputs.app_image_changed }}
DIGEST: ${{ needs.promote.outputs.app_image_digest }}
PIPELINE: sim-${{ github.ref == 'refs/heads/main' && 'production' || github.ref == 'refs/heads/dev' && 'dev' || 'staging' }}-us-east-1-app-deployment
RETAG_EPOCH: ${{ needs.promote-images.outputs.retag_epoch }}
RETAG_EPOCH: ${{ needs.promote.outputs.retag_epoch }}
run: |
set -eo pipefail
case "$APP_IMAGE_CHANGED" in
@@ -625,9 +610,9 @@ jobs:
env:
TRIGGER_ACCESS_TOKEN: ${{ secrets.TRIGGER_ACCESS_TOKEN }}
TRIGGER_PROJECT_ID: ${{ secrets.TRIGGER_PROJECT_ID }}
TRIGGER_ENV: ${{ needs.prepare-trigger.outputs.environment }}
TRIGGER_BRANCH: ${{ needs.prepare-trigger.outputs.preview_branch }}
VERSION: ${{ needs.prepare-trigger.outputs.version }}
TRIGGER_ENV: ${{ needs.trigger-upload.outputs.environment }}
TRIGGER_BRANCH: ${{ needs.trigger-upload.outputs.preview_branch }}
VERSION: ${{ needs.trigger-upload.outputs.version }}
run: |
set -eo pipefail
if [ -z "$TRIGGER_ACCESS_TOKEN" ] || [ -z "$TRIGGER_PROJECT_ID" ]; then
@@ -635,7 +620,7 @@ jobs:
exit 1
fi
if [ -z "$VERSION" ]; then
echo "ERROR: no deployed version passed from prepare-trigger" >&2
echo "ERROR: no deployed version passed from trigger-upload" >&2
exit 1
fi
echo "Promoting Trigger.dev version $VERSION ($TRIGGER_ENV)"
@@ -647,11 +632,11 @@ jobs:
# Build ARM64 images for GHCR (main branch only, runs in parallel with
# tests). Pushes only the immutable sha tag — latest-arm64/version-arm64
# are applied by create-ghcr-manifests after the gate, so a failing run
# are applied by ghcr after the gate, so a failing run
# never moves a documented tag.
build-ghcr-arm64:
name: Build ARM64 (GHCR Only)
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && matrix.bs_runner || matrix.gh_runner }}
image-arm64:
name: image-arm64 (${{ matrix.id }})
runs-on: *runner-matrix
timeout-minutes: 30
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
permissions:
@@ -665,24 +650,29 @@ jobs:
# and this job only runs on push to main — an unprovisioned label would
# hang a release in `queued` rather than fail a PR.
include:
- dockerfile: ./docker/app.Dockerfile
- id: app
dockerfile: ./docker/app.Dockerfile
cache_mb: '102400'
image: ghcr.io/simstudioai/simstudio
gh_runner: linux-arm64-8-core
bs_runner: blacksmith-16vcpu-ubuntu-2404-arm
- dockerfile: ./docker/db.Dockerfile
- id: db
dockerfile: ./docker/db.Dockerfile
image: ghcr.io/simstudioai/migrations
gh_runner: ubuntu-24.04-arm
bs_runner: blacksmith-4vcpu-ubuntu-2404-arm
- dockerfile: ./docker/realtime.Dockerfile
- id: realtime
dockerfile: ./docker/realtime.Dockerfile
image: ghcr.io/simstudioai/realtime
gh_runner: ubuntu-24.04-arm
bs_runner: blacksmith-4vcpu-ubuntu-2404-arm
- dockerfile: ./docker/pii.Dockerfile
- id: pii
dockerfile: ./docker/pii.Dockerfile
image: ghcr.io/simstudioai/pii
gh_runner: ubuntu-24.04-arm
bs_runner: blacksmith-4vcpu-ubuntu-2404-arm
- dockerfile: ./docker/cron.Dockerfile
- id: cron
dockerfile: ./docker/cron.Dockerfile
image: ghcr.io/simstudioai/cron
gh_runner: ubuntu-24.04-arm
bs_runner: blacksmith-4vcpu-ubuntu-2404-arm
@@ -699,7 +689,7 @@ jobs:
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push ARM64 to GHCR
uses: ./.github/actions/docker-build
uses: ./.github/actions/image
with:
provider: ${{ vars.CI_PROVIDER }}
file: ${{ matrix.dockerfile }}
@@ -709,18 +699,18 @@ jobs:
# Publish all mutable GHCR tags (latest, latest-amd64/arm64, version tags)
# and the multi-arch manifests from the immutable sha tags — only on main,
# after the deploy gate (promote-images) and the ARM64 build both pass.
create-ghcr-manifests:
name: Create GHCR Manifests
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
# after the deploy gate (promote) and the ARM64 build both pass.
ghcr:
name: ghcr (${{ matrix.id }})
runs-on: *runner-2vcpu
timeout-minutes: 10
needs: [promote-images, build-ghcr-arm64, detect-version]
needs: [promote, image-arm64, version]
# Explicit results: see migrate's comment.
if: >-
!cancelled() &&
needs.promote-images.result == 'success' &&
needs.build-ghcr-arm64.result == 'success' &&
needs.detect-version.result == 'success' &&
needs.promote.result == 'success' &&
needs.image-arm64.result == 'success' &&
needs.version.result == 'success' &&
github.event_name == 'push' && github.ref == 'refs/heads/main'
permissions:
contents: read
@@ -733,11 +723,16 @@ jobs:
strategy:
matrix:
include:
- image: ghcr.io/simstudioai/simstudio
- image: ghcr.io/simstudioai/migrations
- image: ghcr.io/simstudioai/realtime
- image: ghcr.io/simstudioai/pii
- image: ghcr.io/simstudioai/cron
- id: simstudio
image: ghcr.io/simstudioai/simstudio
- id: migrations
image: ghcr.io/simstudioai/migrations
- id: realtime
image: ghcr.io/simstudioai/realtime
- id: pii
image: ghcr.io/simstudioai/pii
- id: cron
image: ghcr.io/simstudioai/cron
steps:
- name: Login to GHCR
@@ -747,7 +742,7 @@ jobs:
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
# Same monotonic guard as promote-images, applied to the public latest
# Same monotonic guard as promote, applied to the public latest
# tags only — immutable sha and version tags are always published.
- name: Guard against stale latest tags
id: guard
@@ -771,8 +766,8 @@ jobs:
docker buildx imagetools create -t "${IMAGE}:${SHA}" \
"${IMAGE}:${SHA}-amd64" "${IMAGE}:${SHA}-arm64"
if [ "${{ needs.detect-version.outputs.is_release }}" = "true" ]; then
VERSION="${{ needs.detect-version.outputs.version }}"
if [ "${{ needs.version.outputs.is_release }}" = "true" ]; then
VERSION="${{ needs.version.outputs.version }}"
echo "📦 Publishing version tags: ${VERSION}"
docker buildx imagetools create -t "${IMAGE}:${VERSION}-amd64" "${IMAGE}:${SHA}-amd64"
docker buildx imagetools create -t "${IMAGE}:${VERSION}-arm64" "${IMAGE}:${SHA}-arm64"
@@ -789,10 +784,10 @@ jobs:
# Sign the published images and attach SLSA provenance and an SBOM to each.
#
# This runs after create-ghcr-manifests rather than inside the build because
# This runs after ghcr rather than inside the build because
# buildx's own provenance/sbom attestations stay off (see the note in
# .github/actions/docker-build): the extra manifests they add to an index
# break the `imagetools create` retagging that promote-images depends on.
# .github/actions/image): the extra manifests they add to an index
# break the `imagetools create` retagging that promote depends on.
# Attaching attestations here instead leaves the index itself untouched — they
# are stored as separate referrer manifests that point at it.
#
@@ -804,18 +799,18 @@ jobs:
# `:<sha>-amd64` manifest it wraps — attesting the manifest leaves the tag
# people actually pin unverifiable. Which tags exist also varies per run:
# version tags only on a release, and the latest tags only when the monotonic
# guard in create-ghcr-manifests passed. Resolving tag -> digest here and
# guard in ghcr passed. Resolving tag -> digest here and
# de-duplicating is what keeps the two in step without hardcoding that logic
# twice.
attest-subjects:
name: Resolve Attestation Subjects
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
name: attest-subjects
runs-on: *runner-2vcpu
timeout-minutes: 10
needs: [create-ghcr-manifests, detect-version]
needs: [ghcr, version]
if: >-
!cancelled() &&
needs.create-ghcr-manifests.result == 'success' &&
needs.detect-version.result == 'success' &&
needs.ghcr.result == 'success' &&
needs.version.result == 'success' &&
github.event_name == 'push' && github.ref == 'refs/heads/main'
permissions:
contents: read
@@ -834,10 +829,10 @@ jobs:
- name: Resolve published tags to digests
id: resolve
env:
IS_RELEASE: ${{ needs.detect-version.outputs.is_release }}
VERSION: ${{ needs.detect-version.outputs.version }}
IS_RELEASE: ${{ needs.version.outputs.is_release }}
VERSION: ${{ needs.version.outputs.version }}
SHA: ${{ github.sha }}
LATEST_FRESH: ${{ needs.create-ghcr-manifests.outputs.latest_fresh }}
LATEST_FRESH: ${{ needs.ghcr.outputs.latest_fresh }}
run: |
set -euo pipefail
@@ -911,7 +906,7 @@ jobs:
# A moving alias is only taken when it resolves to the same index
# digest this run published — content identity, which is what a
# digest can prove. create-ghcr-manifests holds the latest tags back when
# digest can prove. ghcr holds the latest tags back when
# its monotonic guard sees a newer commit, and they then still point
# at an older build — attesting those would put this run's signature
# and provenance on an image it did not produce. The per-arch
@@ -983,9 +978,9 @@ jobs:
# One leg per distinct published digest. Attesting each subject separately is
# also what makes the SBOMs truthful: the amd64 and arm64 images contain
# different packages, and one SBOM attached to the index cannot describe both.
attest-images:
name: Attest Images
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
attest:
name: attest
runs-on: *runner-2vcpu
timeout-minutes: 15
needs: [attest-subjects]
if: >-
@@ -1064,9 +1059,9 @@ jobs:
# Check if docs changed
# Smallest runner on purpose: a depth-2 checkout plus a path filter, no
# install and no build.
check-docs-changes:
name: Check Docs Changes
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
docs-changes:
name: docs-changes
runs-on: *runner-2vcpu
timeout-minutes: 5
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
outputs:
@@ -1085,41 +1080,43 @@ jobs:
- 'apps/sim/lib/chunkers/**'
# Process docs embeddings (only when docs change, after images are promoted)
process-docs:
name: Process Docs
needs: [promote-images, check-docs-changes]
docs:
name: docs
needs: [promote, docs-changes]
# Explicit results: see migrate's comment.
if: >-
!cancelled() &&
needs.promote-images.result == 'success' &&
needs.check-docs-changes.result == 'success' &&
needs.check-docs-changes.outputs.docs_changed == 'true'
uses: ./.github/workflows/docs-embeddings.yml
secrets: inherit
needs.promote.result == 'success' &&
needs.docs-changes.result == 'success' &&
needs.docs-changes.outputs.docs_changed == 'true'
uses: ./.github/workflows/docs.yml
secrets:
DATABASE_URL: ${{ secrets.DATABASE_URL }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
# Create GitHub Release (only for version commits on main, after all builds complete)
create-release:
name: Create GitHub Release
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
release:
name: release
runs-on: *runner-4vcpu
timeout-minutes: 10
needs: [create-ghcr-manifests, attest-subjects, attest-images, detect-version]
# Explicit results: see migrate's comment. attest-images is a gate, not just
needs: [ghcr, attest-subjects, attest, version]
# Explicit results: see migrate's comment. attest is a gate, not just
# an ordering edge — a release must not advertise images whose signature or
# attestation failed to publish. The count check is belt and braces: today
# attest-subjects already fails on an empty subject set, so this only bites
# if that guard is ever removed.
#
# Note this gates the GitHub release, not the production deploy: CodePipeline
# fires from the ECR tags moved by promote-images, upstream of this job, and
# fires from the ECR tags moved by promote, upstream of this job, and
# only the GHCR mirrors are attested.
if: >-
!cancelled() &&
needs.create-ghcr-manifests.result == 'success' &&
needs.ghcr.result == 'success' &&
needs.attest-subjects.result == 'success' &&
needs.attest-subjects.outputs.count != '0' &&
needs.attest-images.result == 'success' &&
needs.detect-version.result == 'success' &&
needs.detect-version.outputs.is_release == 'true'
needs.attest.result == 'success' &&
needs.version.result == 'success' &&
needs.version.outputs.is_release == 'true'
permissions:
contents: write
steps:
@@ -1128,18 +1125,16 @@ jobs:
with:
fetch-depth: 0
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
- name: Setup workspace
uses: ./.github/actions/setup
with:
bun-version: 1.4.2
- name: Install dependencies
run: bun install --frozen-lockfile --ignore-scripts
provider: ${{ vars.CI_PROVIDER }}
node-version: ''
- name: Create release
env:
GH_PAT: ${{ secrets.GITHUB_TOKEN }}
run: bun run scripts/create-single-release.ts ${{ needs.detect-version.outputs.version }}
run: bun run scripts/create-single-release.ts ${{ needs.version.outputs.version }}
# Desktop release: builds, signs, notarizes, and attaches the macOS app to
# the GitHub release created above. Gated on the Apple signing secrets so a
@@ -1147,14 +1142,14 @@ jobs:
# Developer account is provisioned — the moment the six secrets exist, the
# next vX.Y.Z release ships desktop artifacts with no further changes.
# Job-level `if:` cannot read the secrets context, hence the probe job.
check-desktop-signing:
name: Check Desktop Signing Secrets
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
desktop-signing:
name: desktop-signing
runs-on: *runner-2vcpu
timeout-minutes: 2
needs: [detect-version, detect-desktop-changes]
# !cancelled(): detect-desktop-changes is skipped on main (and
# detect-version tags only on main); either path may need the probe.
if: ${{ !cancelled() && (needs.detect-version.outputs.is_release == 'true' || needs.detect-desktop-changes.outputs.changed == 'true') }}
needs: [version, desktop-changes]
# !cancelled(): desktop-changes is skipped on main (and
# version tags only on main); either path may need the probe.
if: ${{ !cancelled() && (needs.version.outputs.is_release == 'true' || needs.desktop-changes.outputs.changed == 'true') }}
outputs:
configured: ${{ steps.check.outputs.configured }}
steps:
@@ -1169,24 +1164,31 @@ jobs:
fi
desktop-release:
name: Desktop Release
needs: [create-release, check-desktop-signing, detect-version]
# Suppress the implicit success() check: check-desktop-signing has an
name: desktop-release
needs: [release, desktop-signing, version]
# Suppress the implicit success() check: desktop-signing has an
# intentionally skipped transitive dependency on main, which would
# otherwise cascade-skip this job even when every direct need succeeded.
if: >-
!cancelled() &&
needs.create-release.result == 'success' &&
needs.check-desktop-signing.result == 'success' &&
needs.detect-version.result == 'success' &&
needs.check-desktop-signing.outputs.configured == 'true'
needs.release.result == 'success' &&
needs.desktop-signing.result == 'success' &&
needs.version.result == 'success' &&
needs.desktop-signing.outputs.configured == 'true'
permissions:
contents: write
uses: ./.github/workflows/desktop-release.yml
with:
version: ${{ needs.detect-version.outputs.version }}
version: ${{ needs.version.outputs.version }}
publish: true
secrets: inherit
secrets:
DESKTOP_RELEASE_TOKEN: ${{ secrets.DESKTOP_RELEASE_TOKEN }}
CSC_LINK: ${{ secrets.CSC_LINK }}
CSC_KEY_PASSWORD: ${{ secrets.CSC_KEY_PASSWORD }}
APPLE_API_KEY_P8: ${{ secrets.APPLE_API_KEY_P8 }}
APPLE_API_KEY_ID: ${{ secrets.APPLE_API_KEY_ID }}
APPLE_API_ISSUER: ${{ secrets.APPLE_API_ISSUER }}
APPLE_TEAM_ID: ${{ secrets.APPLE_TEAM_ID }}
# Per-env desktop prereleases: a dev/staging push that touches shell code
# publishes an environment-tagged GitHub prerelease to the public,
@@ -1194,19 +1196,19 @@ jobs:
# builds out of this source repository prevents its followers from receiving
# every internal shell release. Each environment's /api/desktop/update feed
# still offers only its own signed stream.
create-desktop-prerelease:
name: Create Desktop Prerelease
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
prerelease-create:
name: prerelease-create
runs-on: *runner-2vcpu
timeout-minutes: 5
needs: [detect-desktop-changes, check-desktop-signing]
needs: [desktop-changes, desktop-signing]
# Requires the signing probe to have actually succeeded (not just "not
# cancelled") so a probe failure can't produce a release with no build.
if: >-
${{
!cancelled() &&
needs.detect-desktop-changes.outputs.changed == 'true' &&
needs.check-desktop-signing.result == 'success' &&
needs.check-desktop-signing.outputs.configured == 'true'
needs.desktop-changes.outputs.changed == 'true' &&
needs.desktop-signing.result == 'success' &&
needs.desktop-signing.outputs.configured == 'true'
}}
permissions:
contents: read
@@ -1265,9 +1267,9 @@ jobs:
echo "version=$TAG" >> "$GITHUB_OUTPUT"
echo "✅ Created draft prerelease $TAG"
desktop-prerelease:
name: Desktop Prerelease Build
needs: [create-desktop-prerelease, check-desktop-signing]
prerelease-build:
name: prerelease-build
needs: [prerelease-create, desktop-signing]
# The reusable workflow declares contents: write for its stable-release
# path. GitHub cannot elevate a caller's token, even though this prerelease
# path uses the dedicated cross-repository token for its actual upload.
@@ -1275,25 +1277,32 @@ jobs:
contents: write
uses: ./.github/workflows/desktop-release.yml
with:
version: ${{ needs.create-desktop-prerelease.outputs.version }}
version: ${{ needs.prerelease-create.outputs.version }}
publish: true
sign: true
secrets: inherit
secrets:
DESKTOP_RELEASE_TOKEN: ${{ secrets.DESKTOP_RELEASE_TOKEN }}
CSC_LINK: ${{ secrets.CSC_LINK }}
CSC_KEY_PASSWORD: ${{ secrets.CSC_KEY_PASSWORD }}
APPLE_API_KEY_P8: ${{ secrets.APPLE_API_KEY_P8 }}
APPLE_API_KEY_ID: ${{ secrets.APPLE_API_KEY_ID }}
APPLE_API_ISSUER: ${{ secrets.APPLE_API_ISSUER }}
APPLE_TEAM_ID: ${{ secrets.APPLE_TEAM_ID }}
# The draft only becomes visible to the update feed once its artifacts are
# attached — this is what makes a dev/staging push atomic from the shell's
# point of view.
publish-desktop-prerelease:
name: Publish Desktop Prerelease
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
prerelease-publish:
name: prerelease-publish
runs-on: *runner-2vcpu
timeout-minutes: 5
needs: [create-desktop-prerelease, desktop-prerelease]
needs: [prerelease-create, prerelease-build]
permissions:
contents: read
env:
GH_TOKEN: ${{ secrets.DESKTOP_RELEASE_TOKEN }}
GH_REPO: simstudioai/sim-desktop-releases
TAG: ${{ needs.create-desktop-prerelease.outputs.version }}
TAG: ${{ needs.prerelease-create.outputs.version }}
steps:
- name: Publish the draft release
run: |
@@ -1307,17 +1316,17 @@ jobs:
# and delete the rest (with their tags, so dev force-resets don't strand
# commits behind stale tags). Leftover drafts (failed or superseded builds)
# are always garbage by this point — the current run's release is published.
prune-desktop-prereleases:
name: Prune Desktop Prereleases
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
prerelease-prune:
name: prerelease-prune
runs-on: *runner-2vcpu
timeout-minutes: 5
needs: [create-desktop-prerelease, publish-desktop-prerelease]
needs: [prerelease-create, prerelease-publish]
permissions:
contents: read
env:
GH_TOKEN: ${{ secrets.DESKTOP_RELEASE_TOKEN }}
GH_REPO: simstudioai/sim-desktop-releases
CURRENT_TAG: ${{ needs.create-desktop-prerelease.outputs.version }}
CURRENT_TAG: ${{ needs.prerelease-create.outputs.version }}
steps:
- name: Delete stale prereleases
run: |
.github/workflows/codeql.yml
+3 -3
View File
@@ -1,4 +1,4 @@
name: CodeQL
name: codeql
# Advanced setup, replacing the repo-settings "default setup".
#
@@ -77,7 +77,7 @@ permissions:
jobs:
analyze:
name: Analyze ${{ matrix.language }}
name: analyze (${{ matrix.language }})
# Sized per language, not per workflow. The two analyses are nothing alike:
# javascript-typescript peaks at 19.5 GB (p95 over 3090 runs), so it needs
# the 8 vCPU tier's 30.4 GB and would OOM on the 4 vCPU tier's 15.2 GB; the
@@ -108,7 +108,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09 # v5
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
with:
persist-credentials: false
.github/workflows/companion-pr-check.yml → .github/workflows/companion.yml
+2 -1
View File
@@ -1,4 +1,4 @@
name: companion-pr-check
name: companion
# Soft, NON-BLOCKING warning: when a PR targeting staging/main declares a
# cross-repo "Companion:" PR, surface whether that companion is merged yet, so
@@ -40,6 +40,7 @@ permissions:
jobs:
companion:
name: companion
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 5
steps:
.github/workflows/desktop-e2e.yml
+66 -7
View File
@@ -1,4 +1,4 @@
name: Desktop E2E
name: desktop-e2e
# Smoke coverage of the real Electron shell on desktop changes, plus a weekly
# advisory canary against electron@latest so Chromium-cadence breakage surfaces
@@ -57,19 +57,34 @@ concurrency:
cancel-in-progress: true
jobs:
e2e-pinned:
name: E2E (pinned)
e2e:
name: e2e
if: github.event_name != 'schedule'
runs-on: macos-26
timeout-minutes: 30
steps:
- name: Checkout code
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
with:
persist-credentials: false
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.4.2
# Electron, electron-builder and Playwright download their binaries on every cold install. All
# three caches are keyed by version inside, so an older entry restored by prefix is harmless.
- name: Cache Electron and Playwright downloads
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
with:
path: |
~/Library/Caches/electron
~/Library/Caches/electron-builder
~/Library/Caches/ms-playwright
key: desktop-downloads-${{ runner.os }}-${{ hashFiles('bun.lock') }}
restore-keys: desktop-downloads-${{ runner.os }}-
- name: Install dependencies
run: bun install --frozen-lockfile
@@ -81,9 +96,16 @@ jobs:
working-directory: apps/desktop
run: bunx playwright install chromium
# The terminal-cancel suite drives a real tmux server; without tmux its tmux scenarios skip.
- name: Install tmux
run: brew install tmux
- name: Run Playwright _electron smoke suite
working-directory: apps/desktop
run: bunx playwright test
env:
BACKGROUND_EXECUTOR_REPORT_PATH: test-results/background-executor-report.json
TERMINAL_CANCEL_REPORT_PATH: test-results/terminal-cancel-report.json
- name: Upload test results
if: failure()
@@ -93,20 +115,35 @@ jobs:
path: apps/desktop/test-results
retention-days: 7
e2e-latest-canary:
name: E2E (electron@latest canary)
e2e-canary:
name: e2e-canary
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: macos-26
timeout-minutes: 30
continue-on-error: true
steps:
- name: Checkout code
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
with:
persist-credentials: false
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.4.2
# Electron, electron-builder and Playwright download their binaries on every cold install. All
# three caches are keyed by version inside, so an older entry restored by prefix is harmless.
- name: Cache Electron and Playwright downloads
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
with:
path: |
~/Library/Caches/electron
~/Library/Caches/electron-builder
~/Library/Caches/ms-playwright
key: desktop-downloads-${{ runner.os }}-${{ hashFiles('bun.lock') }}
restore-keys: desktop-downloads-${{ runner.os }}-
- name: Install dependencies
run: bun install --frozen-lockfile
@@ -122,9 +159,16 @@ jobs:
working-directory: apps/desktop
run: bunx playwright install chromium
# The terminal-cancel suite drives a real tmux server; without tmux its tmux scenarios skip.
- name: Install tmux
run: brew install tmux
- name: Run Playwright _electron smoke suite
working-directory: apps/desktop
run: bunx playwright test
env:
BACKGROUND_EXECUTOR_REPORT_PATH: test-results/background-executor-report.json
TERMINAL_CANCEL_REPORT_PATH: test-results/terminal-cancel-report.json
- name: Upload test results
if: failure()
@@ -135,12 +179,15 @@ jobs:
retention-days: 7
package-smoke:
name: Unsigned package smoke
name: package-smoke
if: github.event_name != 'schedule'
runs-on: macos-26
timeout-minutes: 20
steps:
- name: Checkout code
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
with:
persist-credentials: false
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
@@ -148,10 +195,22 @@ jobs:
bun-version: 1.4.2
- name: Setup Node
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: 22
# Electron, electron-builder and Playwright download their binaries on every cold install. All
# three caches are keyed by version inside, so an older entry restored by prefix is harmless.
- name: Cache Electron and Playwright downloads
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
with:
path: |
~/Library/Caches/electron
~/Library/Caches/electron-builder
~/Library/Caches/ms-playwright
key: desktop-downloads-${{ runner.os }}-${{ hashFiles('bun.lock') }}
restore-keys: desktop-downloads-${{ runner.os }}-
- name: Install dependencies
run: bun install --frozen-lockfile
.github/workflows/desktop-release.yml
+20 -5
View File
@@ -1,4 +1,4 @@
name: Desktop Release (macOS)
name: desktop-release
# Builds, signs, notarizes, and uploads the desktop app to an existing GitHub
# release. Stable releases live in this source repository; dev and staging
@@ -24,6 +24,21 @@ on:
required: false
type: boolean
default: true
secrets:
DESKTOP_RELEASE_TOKEN:
required: false
CSC_LINK:
required: false
CSC_KEY_PASSWORD:
required: false
APPLE_API_KEY_P8:
required: false
APPLE_API_KEY_ID:
required: false
APPLE_API_ISSUER:
required: false
APPLE_TEAM_ID:
required: false
workflow_dispatch:
inputs:
version:
@@ -45,8 +60,8 @@ permissions:
contents: write
jobs:
build-sign-notarize:
name: Build, Sign, Notarize
build:
name: build
runs-on: macos-26
timeout-minutes: 60
steps:
@@ -131,12 +146,12 @@ jobs:
bun-version: 1.4.2
- name: Setup Node
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: 22
- name: Cache Electron binaries
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
with:
path: |
~/Library/Caches/electron
.github/workflows/stickydisk-gc.yml → .github/workflows/disk-gc.yml
+4 -4
View File
@@ -1,7 +1,7 @@
name: Sticky Disk GC
name: disk-gc
# node_modules sticky disks are keyed on hashFiles('bun.lock') by design — see the
# "Mount node_modules" comment in test-build.yml. A sticky disk is a mutable volume
# cache-key comment in .github/actions/setup/action.yml. A sticky disk is a mutable volume
# and `bun install --frozen-lockfile` adds what the lockfile needs without pruning
# what it dropped, so branches on different lockfiles must not share one. That
# design is correct and is preserved here.
@@ -42,7 +42,7 @@ concurrency:
jobs:
gc:
name: Reclaim idle node_modules sticky disks
name: gc
# GitHub-hosted on purpose, not a Blacksmith runner: the CLI is a pure API
# client, and collection has to keep working during a Blacksmith outage or a
# CI_PROVIDER break-glass switch — exactly when disks sit idle and still bill.
@@ -120,7 +120,7 @@ jobs:
# because the key interpolates ${{ github.event_name }} and a
# workflow_dispatch disk already exists that an enumerated list would have
# skipped forever. `-fork` is a separate optional segment rather than part
# of that class: test-build.yml appends it after the event name, so a fork
# of that class: the setup action appends it after the event name, so a fork
# key reads `pull_request-fork` and a character class cannot span the
# hyphen. Without it, fork disks would never be collected.
#
.github/workflows/docs-embeddings.yml
-51
View File
@@ -1,51 +0,0 @@
name: Process Docs Embeddings
on:
workflow_call:
workflow_dispatch: # Allow manual triggering
permissions:
contents: read
jobs:
process-docs-embeddings:
name: Process Documentation Embeddings
# Network-bound on the embeddings API: ~9% CPU and ~3% peak memory on 8 vCPU.
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 30
if: github.ref == 'refs/heads/main'
steps:
- name: Checkout code
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.4.2
- name: Setup Node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
with:
node-version: 22
- name: Cache Bun dependencies
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
with:
path: |
~/.bun/install/cache
node_modules
**/node_modules
key: ${{ runner.os }}-bun-${{ hashFiles('**/bun.lock') }}
restore-keys: |
${{ runner.os }}-bun-
- name: Install dependencies
run: bun install --frozen-lockfile --ignore-scripts
- name: Process docs embeddings
working-directory: ./apps/sim
env:
DATABASE_URL: ${{ secrets.DATABASE_URL }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: bun run scripts/process-docs.ts --clear
.github/workflows/docs.yml
+40
View File
@@ -0,0 +1,40 @@
name: docs
on:
workflow_call:
secrets:
DATABASE_URL:
required: false
OPENAI_API_KEY:
required: false
workflow_dispatch: # Allow manual triggering
permissions:
contents: read
jobs:
embeddings:
name: embeddings
# Network-bound on the embeddings API: ~9% CPU and ~3% peak memory on 8 vCPU.
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 30
if: github.ref == 'refs/heads/main'
steps:
- name: Checkout code
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
with:
persist-credentials: false
- name: Setup workspace
uses: ./.github/actions/setup
with:
provider: ${{ vars.CI_PROVIDER }}
node-version: '22'
- name: Process docs embeddings
working-directory: ./apps/sim
env:
DATABASE_URL: ${{ secrets.DATABASE_URL }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: bun run scripts/process-docs.ts --clear
.github/workflows/helm.yml
+15 -15
View File
@@ -1,4 +1,4 @@
name: Helm Chart
name: helm
on:
push:
@@ -33,11 +33,11 @@ permissions:
jobs:
chart:
name: Lint, test, and validate chart
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
name: chart
runs-on: &runner-2vcpu ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 15
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
with:
# ct diffs the chart against the PR base to decide whether the version
# was bumped, so a shallow clone would leave it nothing to compare.
@@ -154,12 +154,12 @@ jobs:
done
install:
name: Install on kind and run helm test
name: install
needs: chart
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 25
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
with:
persist-credentials: false
@@ -198,9 +198,9 @@ jobs:
# and a release becoming public between two independent API calls would be
# enough to make the OCI artifact and the HTTP repo install different Sims.
release-version:
name: Resolve the app release
name: release-version
if: github.event_name == 'push' && github.ref == 'refs/heads/main' && github.repository == 'simstudioai/sim'
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
runs-on: *runner-2vcpu
timeout-minutes: 5
permissions:
contents: read # Read the release list.
@@ -211,7 +211,7 @@ jobs:
# it -- so the subject is the only source available, and the latest release
# is the source of truth for every other push.
#
# The subject pattern MUST stay identical to detect-version in ci.yml, which
# The subject pattern MUST stay identical to the `version` job in ci.yml, which
# is what actually creates the tag and builds the images. If this one
# matched a subject that one rejects, the chart would publish naming a
# release that was never cut.
@@ -246,10 +246,10 @@ jobs:
# rendered clean under kubeconform, and actually installed on a kind cluster.
# A separate workflow would race those instead of waiting for them.
publish:
name: Publish chart to GHCR
name: publish
needs: [chart, install, release-version]
if: github.event_name == 'push' && github.ref == 'refs/heads/main' && github.repository == 'simstudioai/sim'
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
runs-on: *runner-2vcpu
timeout-minutes: 15
permissions:
contents: read # Read the chart source.
@@ -257,7 +257,7 @@ jobs:
id-token: write # Sigstore signs against the runner's OIDC identity; no key material is stored.
attestations: write # Let actions/attest-build-provenance record the SLSA provenance.
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
with:
persist-credentials: false
@@ -439,15 +439,15 @@ jobs:
# to cut a release and push the index, and there is no reason to hand that to
# the job holding the signing identity.
publish-http:
name: Publish chart to the Helm repo
name: publish-http
needs: [chart, install, release-version]
if: github.event_name == 'push' && github.ref == 'refs/heads/main' && github.repository == 'simstudioai/sim'
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-2vcpu-ubuntu-2404' || 'ubuntu-latest' }}
runs-on: *runner-2vcpu
timeout-minutes: 15
permissions:
contents: write # Cut the chart release and push index.yaml to the pages branch.
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
with:
# chart-releaser diffs against the previous tag to decide which charts
# changed, so it needs the full history rather than a shallow clone.
.github/workflows/migrations.yml → .github/workflows/migrate.yml
+18 -18
View File
@@ -1,4 +1,4 @@
name: Database Migrations
name: migrate
on:
workflow_call:
@@ -7,6 +7,17 @@ on:
description: Target environment (production, staging, or dev)
required: true
type: string
secrets:
DATABASE_URL:
required: false
STAGING_DATABASE_URL:
required: false
DEV_DATABASE_URL:
required: false
MIGRATION_DATABASE_URL:
required: false
STAGING_MIGRATION_DATABASE_URL:
required: false
workflow_dispatch:
inputs:
environment:
@@ -23,7 +34,7 @@ permissions:
jobs:
migrate:
name: Apply Database Migrations
name: migrate
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
# Bulk projection loads and concurrent index builds can outlast ordinary schema changes.
timeout-minutes: 300
@@ -31,25 +42,14 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.4.2
persist-credentials: false
- name: Cache Bun dependencies
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
- name: Setup workspace
uses: ./.github/actions/setup
with:
path: |
~/.bun/install/cache
node_modules
**/node_modules
key: ${{ runner.os }}-bun-${{ hashFiles('**/bun.lock') }}
restore-keys: |
${{ runner.os }}-bun-
- name: Install dependencies
run: bun install --frozen-lockfile --ignore-scripts
provider: ${{ vars.CI_PROVIDER }}
node-version: ''
# The expression maps the explicit environment input to exactly one repo
# secret, so the job never holds another environment's database URL. An
.github/workflows/publish-cli.yml
+19 -26
View File
@@ -1,4 +1,4 @@
name: Publish CLI Package
name: publish-cli
on:
push:
@@ -9,39 +9,28 @@ on:
permissions:
contents: read
# A newer push waits for the running publish rather than racing its version check.
concurrency:
group: publish-cli-${{ github.ref }}
cancel-in-progress: false
jobs:
publish-npm:
publish:
name: publish
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 15
steps:
- name: Checkout repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.4.2
persist-credentials: false
- name: Setup Node.js for npm publishing
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
- name: Setup workspace
uses: ./.github/actions/setup
with:
provider: ${{ vars.CI_PROVIDER }}
node-version: '22'
registry-url: 'https://registry.npmjs.org/'
- name: Cache Bun dependencies
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
with:
path: |
~/.bun/install/cache
node_modules
**/node_modules
key: ${{ runner.os }}-bun-${{ hashFiles('**/bun.lock') }}
restore-keys: |
${{ runner.os }}-bun-
- name: Install dependencies
working-directory: packages/cli
run: bun install --frozen-lockfile --ignore-scripts
registry-url: https://registry.npmjs.org/
- name: Build package
working-directory: packages/cli
@@ -54,8 +43,10 @@ jobs:
- name: Check if version already exists
id: version_check
env:
VERSION: ${{ steps.package_version.outputs.version }}
run: |
if npm view simstudio@${{ steps.package_version.outputs.version }} version &> /dev/null; then
if npm view "simstudio@$VERSION" version &> /dev/null; then
echo "exists=true" >> $GITHUB_OUTPUT
else
echo "exists=false" >> $GITHUB_OUTPUT
@@ -70,4 +61,6 @@ jobs:
- name: Log skipped publish
if: steps.version_check.outputs.exists == 'true'
run: echo "Skipped publishing because version ${{ steps.package_version.outputs.version }} already exists on npm"
env:
VERSION: ${{ steps.package_version.outputs.version }}
run: echo "Skipped publishing because version $VERSION already exists on npm"
.github/workflows/publish-python-sdk.yml
+24 -9
View File
@@ -1,4 +1,4 @@
name: Publish Python SDK
name: publish-python-sdk
on:
push:
@@ -9,13 +9,21 @@ on:
permissions:
contents: write
# A newer push waits for the running publish rather than racing its version check.
concurrency:
group: publish-python-sdk-${{ github.ref }}
cancel-in-progress: false
jobs:
publish-pypi:
publish:
name: publish
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 15
steps:
- name: Checkout repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
with:
persist-credentials: false
- name: Setup Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5
@@ -40,12 +48,17 @@ jobs:
- name: Check if version already exists
id: version_check
# Exact lookup: PyPI answers 404 only for a version it has never published. Anything other
# than 200 or 404 stops the job rather than guessing.
env:
VERSION: ${{ steps.package_version.outputs.version }}
run: |
if pip index versions simstudio-sdk | grep -q "${{ steps.package_version.outputs.version }}"; then
echo "exists=true" >> $GITHUB_OUTPUT
else
echo "exists=false" >> $GITHUB_OUTPUT
fi
status="$(curl -sS -o /dev/null -w '%{http_code}' "https://pypi.org/pypi/simstudio-sdk/$VERSION/json")"
case "$status" in
200) echo "exists=true" >> "$GITHUB_OUTPUT" ;;
404) echo "exists=false" >> "$GITHUB_OUTPUT" ;;
*) echo "::error::PyPI returned HTTP $status for simstudio-sdk $VERSION" >&2; exit 1 ;;
esac
- name: Build package
if: steps.version_check.outputs.exists == 'false'
@@ -67,11 +80,13 @@ jobs:
- name: Log skipped publish
if: steps.version_check.outputs.exists == 'true'
run: echo "Skipped publishing because version ${{ steps.package_version.outputs.version }} already exists on PyPI"
env:
VERSION: ${{ steps.package_version.outputs.version }}
run: echo "Skipped publishing because version $VERSION already exists on PyPI"
- name: Create GitHub Release
if: steps.version_check.outputs.exists == 'false'
uses: softprops/action-gh-release@de2c0eb89ae2a093876385947365aca7b0e5f844 # v1
uses: softprops/action-gh-release@efb35369e0ad2afab669f228072c1b0d510eae64 # v3.0.3
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
.github/workflows/publish-sim-cli.yml
+7 -22
View File
@@ -1,4 +1,4 @@
name: Publish Sim API CLI Package
name: publish-sim-cli
on:
push:
@@ -19,39 +19,24 @@ concurrency:
cancel-in-progress: false
jobs:
publish-npm:
publish:
name: publish
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 15
steps:
- name: Checkout repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.4.2
persist-credentials: false
# Vitest 5 requires Node 22.12+; the published bundle is still
# smoke-tested on Node 20 below, its minimum supported runtime.
- name: Setup Node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
- name: Setup workspace
uses: ./.github/actions/setup
with:
provider: ${{ vars.CI_PROVIDER }}
node-version: '22'
- name: Cache Bun dependencies
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
with:
path: |
~/.bun/install/cache
node_modules
**/node_modules
key: ${{ runner.os }}-bun-${{ hashFiles('**/bun.lock') }}
restore-keys: |
${{ runner.os }}-bun-
- name: Install dependencies
run: bun install --frozen-lockfile --ignore-scripts
- name: Verify npm authentication
env:
NPM_CONFIG_TOKEN: ${{ secrets.NPM_TOKEN }}
.github/workflows/publish-sim-setup.yml
+7 -22
View File
@@ -1,4 +1,4 @@
name: Publish Sim Setup Package
name: publish-sim-setup
on:
push:
@@ -19,39 +19,24 @@ concurrency:
cancel-in-progress: false
jobs:
publish-npm:
publish:
name: publish
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 15
steps:
- name: Checkout repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.4.2
persist-credentials: false
# Vitest 5 requires Node 22.12+; the published bundle is still
# smoke-tested on Node 20 below, its minimum supported runtime.
- name: Setup Node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
- name: Setup workspace
uses: ./.github/actions/setup
with:
provider: ${{ vars.CI_PROVIDER }}
node-version: '22'
- name: Cache Bun dependencies
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
with:
path: |
~/.bun/install/cache
node_modules
**/node_modules
key: ${{ runner.os }}-bun-${{ hashFiles('**/bun.lock') }}
restore-keys: |
${{ runner.os }}-bun-
- name: Install dependencies
run: bun install --frozen-lockfile --ignore-scripts
- name: Verify npm authentication
env:
NPM_CONFIG_TOKEN: ${{ secrets.NPM_TOKEN }}
.github/workflows/publish-ts-sdk.yml
+20 -26
View File
@@ -1,4 +1,4 @@
name: Publish TypeScript SDK
name: publish-ts-sdk
on:
push:
@@ -9,38 +9,28 @@ on:
permissions:
contents: write
# A newer push waits for the running publish rather than racing its version check.
concurrency:
group: publish-ts-sdk-${{ github.ref }}
cancel-in-progress: false
jobs:
publish-npm:
publish:
name: publish
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 15
steps:
- name: Checkout repository
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.4.2
persist-credentials: false
- name: Setup Node.js for npm publishing
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
- name: Setup workspace
uses: ./.github/actions/setup
with:
provider: ${{ vars.CI_PROVIDER }}
node-version: '22'
registry-url: 'https://registry.npmjs.org/'
- name: Cache Bun dependencies
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
with:
path: |
~/.bun/install/cache
node_modules
**/node_modules
key: ${{ runner.os }}-bun-${{ hashFiles('**/bun.lock') }}
restore-keys: |
${{ runner.os }}-bun-
- name: Install dependencies
run: bun install --frozen-lockfile --ignore-scripts
registry-url: https://registry.npmjs.org/
- name: Run tests
working-directory: packages/ts-sdk
@@ -57,8 +47,10 @@ jobs:
- name: Check if version already exists
id: version_check
env:
VERSION: ${{ steps.package_version.outputs.version }}
run: |
if npm view simstudio-ts-sdk@${{ steps.package_version.outputs.version }} version &> /dev/null; then
if npm view "simstudio-ts-sdk@$VERSION" version &> /dev/null; then
echo "exists=true" >> $GITHUB_OUTPUT
else
echo "exists=false" >> $GITHUB_OUTPUT
@@ -73,11 +65,13 @@ jobs:
- name: Log skipped publish
if: steps.version_check.outputs.exists == 'true'
run: echo "Skipped publishing because version ${{ steps.package_version.outputs.version }} already exists on npm"
env:
VERSION: ${{ steps.package_version.outputs.version }}
run: echo "Skipped publishing because version $VERSION already exists on npm"
- name: Create GitHub Release
if: steps.version_check.outputs.exists == 'false'
uses: softprops/action-gh-release@de2c0eb89ae2a093876385947365aca7b0e5f844 # v1
uses: softprops/action-gh-release@efb35369e0ad2afab669f228072c1b0d510eae64 # v3.0.3
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
.github/workflows/test-build.yml
-482
View File
@@ -1,482 +0,0 @@
name: Test and Build
on:
workflow_call:
workflow_dispatch:
permissions:
contents: read
jobs:
postgres-integration:
# Runs the real-infrastructure test layer: every `*.integration.ts` in packages/db and
# apps/sim, discovered by glob (`vitest run --mode integration`), against the database each
# provisioning path produces. A new integration suite needs no workflow change.
name: PostgreSQL integration (${{ matrix.provision }})
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-8vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 25
strategy:
fail-fast: false
matrix:
provision: [push, migrate]
services:
redis:
image: redis:8.2-alpine
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 5s
--health-timeout 5s
--health-retries 10
postgres:
image: pgvector/pgvector:pg17
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: sim_test
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U postgres -d sim_test"
--health-interval 5s
--health-timeout 5s
--health-retries 10
postgres-legacy:
image: postgres:16-alpine
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: sim_test
ports:
- 5433:5432
options: >-
--health-cmd "pg_isready -U postgres -d sim_test"
--health-interval 5s
--health-timeout 5s
--health-retries 10
env:
TEST_DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5432/sim_test
TEST_REDIS_URL: redis://127.0.0.1:6379
DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5432/sim_test
BETTER_AUTH_SECRET: oauth-postgres-ci-secret-at-least-32-characters
NEXT_PUBLIC_APP_URL: https://test.sim.ai
ENCRYPTION_KEY: '0000000000000000000000000000000000000000000000000000000000000000'
steps:
- name: Checkout code
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
- name: Setup workspace
uses: ./.github/actions/setup-workspace
with:
provider: ${{ vars.CI_PROVIDER }}
- name: Provision a fresh database through the supported command
working-directory: packages/db
run: |
bun -e 'import postgres from "postgres"; const sql = postgres(process.env.DATABASE_URL); for (const extension of ["vector", "btree_gin", "pg_trgm"]) await sql`CREATE EXTENSION IF NOT EXISTS ${sql(extension)}`; await sql.end()'
bun run db:${{ matrix.provision }}
- name: Verify migration replay is a no-op
if: matrix.provision == 'migrate'
working-directory: packages/db
run: bun run db:migrate
- name: Run packages/db integration tests
working-directory: packages/db
run: bun run test --mode integration
- name: Run apps/sim integration tests
working-directory: apps/sim
# A non-UTC process zone keeps timestamp-without-time-zone handling honest.
env:
TZ: America/Los_Angeles
run: bun run test --mode integration
- name: Verify cumulative billing timeout recovery on PostgreSQL 16
if: matrix.provision == 'push'
working-directory: apps/sim
env:
TEST_DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5433/sim_test
run: >-
bun run test --mode integration lib/billing/core/usage-log.integration.ts
--outputFile.json=test-results/integration-pg16.json
- name: Upload integration test reports
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: integration-reports-${{ matrix.provision }}
path: |
packages/db/test-results/*.json
apps/sim/test-results/*.json
if-no-files-found: warn
retention-days: 14
# Acceptance suites that cross a real HTTP boundary. Its own job, off the
# integration legs' critical path and on its own database: the SCIM app boots
# hosted, which starts background usage replay against DATABASE_URL, so it must
# never share a database with suites asserting on billing rows. The suites
# exercise HTTP behavior rather than a provisioning path, so they run once,
# against the production (migrate) path.
http-e2e:
name: End-to-end over real HTTP
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-8vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 20
services:
postgres:
image: pgvector/pgvector:pg17
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: sim_test
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U postgres -d sim_test"
--health-interval 5s
--health-timeout 5s
--health-retries 10
env:
DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5432/sim_test
BETTER_AUTH_SECRET: http-e2e-ci-secret-at-least-32-characters
ENCRYPTION_KEY: '0000000000000000000000000000000000000000000000000000000000000000'
steps:
- name: Checkout code
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
- name: Setup workspace
uses: ./.github/actions/setup-workspace
with:
provider: ${{ vars.CI_PROVIDER }}
# Migrations create their own extensions, as on a fresh self-hosted install.
- name: Provision the database through migrations
working-directory: packages/db
run: bun run db:migrate
- name: Verify Google document reads over real HTTP
working-directory: apps/sim
env:
NEXT_PUBLIC_APP_URL: http://127.0.0.1:3040
NEXT_PUBLIC_FORCE_HOSTED: 'false'
SEARCH_GOOGLE_CONTENT_REPORT_PATH: ${{ runner.temp }}/e2e/search-google-content.json
run: bun scripts/test-search-google-content-e2e.ts
- name: Verify Lucid MCP search and complete diagram reads over real HTTP
working-directory: apps/sim
env:
NEXT_PUBLIC_APP_URL: http://127.0.0.1:3040
NEXT_PUBLIC_FORCE_HOSTED: 'false'
SEARCH_LUCID_REPORT_PATH: ${{ runner.temp }}/e2e/search-lucid.json
run: bun scripts/test-search-lucid-e2e.ts
- name: Verify Zoom search over real HTTP
working-directory: apps/sim
env:
NEXT_PUBLIC_APP_URL: http://127.0.0.1:3040
NEXT_PUBLIC_FORCE_HOSTED: 'false'
SEARCH_ZOOM_REPORT_PATH: ${{ runner.temp }}/e2e/search-zoom.json
run: bun scripts/test-search-zoom-e2e.ts
- name: Verify Google Meet search over real HTTP
working-directory: apps/sim
env:
NEXT_PUBLIC_APP_URL: http://127.0.0.1:3040
NEXT_PUBLIC_FORCE_HOSTED: 'false'
SEARCH_GOOGLE_MEET_REPORT_PATH: ${{ runner.temp }}/e2e/search-google-meet.json
run: bun scripts/test-search-google-meet-e2e.ts
# The first request cold-compiles the app under Turbopack, which took 42-150s
# on this runner class: a fixed 120s readiness deadline failed on the slow
# tail. The deadline only has to catch a hung boot; an exited server fails
# immediately, and either way the server log tail lands in the job log. No
# step timeout: the job's bound covers a hang without cutting a slow but
# healthy suite short of writing its report.
- name: Verify SCIM, administration and workflow comparisons over real HTTP
working-directory: apps/sim
env:
NEXT_PUBLIC_APP_URL: http://127.0.0.1:3017
BETTER_AUTH_URL: http://127.0.0.1:3017
NEXT_PUBLIC_FORCE_HOSTED: 'true'
BILLING_ENABLED: 'true'
NEXT_PUBLIC_BILLING_ENABLED: 'true'
ENTERPRISE_ENABLED: 'true'
NEXT_PUBLIC_ENTERPRISE_ENABLED: 'true'
SCIM_ENABLED: 'true'
NEXT_PUBLIC_SCIM_ENABLED: 'true'
SSO_ENABLED: 'true'
NEXT_PUBLIC_SSO_ENABLED: 'true'
ORGANIZATIONS_ENABLED: 'true'
NEXT_PUBLIC_ORGANIZATIONS_ENABLED: 'true'
INTERNAL_API_SECRET: scim-http-ci-local-secret-at-least-32-characters
DB_TX_TRIPWIRE: throw
DISABLE_TELEMETRY: 'true'
NEXT_TELEMETRY_DISABLED: '1'
NEXT_PUBLIC_CHAT_DISABLED: 'true'
READY_TIMEOUT_SECONDS: 300
run: |
report_dir="$RUNNER_TEMP/e2e"
server_log="$report_dir/scim-next.log"
mkdir -p "$report_dir"
node ../../node_modules/next/dist/bin/next dev --hostname 127.0.0.1 --port 3017 > "$server_log" 2>&1 &
server_pid=$!
finish() {
kill "$server_pid" 2>/dev/null || true
wait "$server_pid" 2>/dev/null || true
awk '/^ (GET|POST|PUT|PATCH|DELETE|HEAD) \/api\// { print }' "$server_log" > "$report_dir/scim-http-status.log"
}
trap finish EXIT
fail_startup() {
echo "::error::$1"
tail -n 200 "$server_log"
exit 1
}
started=$SECONDS
until curl --fail --silent --max-time 10 http://127.0.0.1:3017/api/health > /dev/null; do
kill -0 "$server_pid" 2>/dev/null || fail_startup 'Local SCIM app exited during startup.'
[ $((SECONDS - started)) -lt "$READY_TIMEOUT_SECONDS" ] ||
fail_startup "Local SCIM app did not become ready within $READY_TIMEOUT_SECONDS seconds."
sleep 2
done
echo "Local SCIM app ready after $((SECONDS - started))s"
SCIM_E2E_BASE_URL="$NEXT_PUBLIC_APP_URL" \
SCIM_E2E_DATABASE_URL="$DATABASE_URL" \
SCIM_E2E_AUTH_SECRET="$BETTER_AUTH_SECRET" \
SCIM_E2E_REPORT_PATH="$report_dir/scim-e2e-report.json" \
bun run test:scim:e2e
VERSION_COMPARE_E2E_BASE_URL="$NEXT_PUBLIC_APP_URL" \
VERSION_COMPARE_E2E_DATABASE_URL="$DATABASE_URL" \
VERSION_COMPARE_E2E_AUTH_SECRET="$BETTER_AUTH_SECRET" \
VERSION_COMPARE_E2E_REPORT_PATH="$report_dir/version-compare-http-report.json" \
bun run test:workflow-version-compare:e2e
- name: Upload end-to-end reports and server logs
if: failure()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: http-e2e-reports
path: ${{ runner.temp }}/e2e/
if-no-files-found: ignore
retention-days: 7
test-build:
name: Lint and Test
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-8vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 15
steps:
# The diff-based audits below need a base commit to read, and the default
# depth of 1 clones a single commit with no parent. They normally fetch
# their base by SHA (see "Resolve base ref"), so this depth only covers the
# `HEAD~1` fallback — but without it that fallback resolves to nothing.
#
# Worth stating because the failure was invisible for so long: the migration
# audit read the resulting `git diff` failure as "no migrations changed" and
# exited 0, so it had never actually run on a push build.
- name: Checkout code
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
with:
fetch-depth: 2
- name: Setup workspace
uses: ./.github/actions/setup-workspace
with:
provider: ${{ vars.CI_PROVIDER }}
turbo-cache-key: turbo-cache
# Surfaces known CVEs in the dependency tree. Non-blocking until the
# existing advisory backlog is triaged, then flip to a required gate by
# removing continue-on-error.
- name: Security audit
run: bun audit
continue-on-error: true
- name: Validate env flags
run: |
FILE="apps/sim/lib/core/config/env-flags.ts"
ERRORS=""
echo "Checking for hardcoded boolean env flags..."
# Use perl for multiline matching to catch both:
# export const isHosted = true
# export const isHosted =
# true
HARDCODED=$(perl -0777 -ne 'while (/export const (is[A-Za-z]+)\s*=\s*\n?\s*(true|false)\b/g) { print " $1 = $2\n" }' "$FILE")
if [ -n "$HARDCODED" ]; then
ERRORS="${ERRORS}\n❌ Env flags must not be hardcoded to boolean literals!\n\nFound hardcoded flags:\n${HARDCODED}\n\nEnv flags should derive their values from environment variables.\n"
fi
echo "Checking env flag naming conventions..."
# Check that all export const (except functions) start with 'is'
# This finds exports like "export const someFlag" that don't start with "is" or "get"
BAD_NAMES=$(grep -E "^export const [a-z]" "$FILE" | grep -vE "^export const (is|get)" | sed 's/export const \([a-zA-Z]*\).*/ \1/')
if [ -n "$BAD_NAMES" ]; then
ERRORS="${ERRORS}\n❌ Env flags must use 'is' prefix for boolean flags!\n\nFound incorrectly named flags:\n${BAD_NAMES}\n\nExample: 'hostedMode' should be 'isHostedMode'\n"
fi
if [ -n "$ERRORS" ]; then
echo ""
echo -e "$ERRORS"
exit 1
fi
echo "✅ All env flags are properly configured"
# One fetch for both base-ref audits, and no `|| true`: a swallowed fetch leaves
# the base ref absent, which neither audit can tell apart from a branch that
# changed nothing. The block-registry check at least degrades to a visible
# `⚠ … skipping` line; the migration audit printed `✓ No new migrations to
# check` and exited 0, clearing the only guard on production DDL.
#
# Depth stays at 1 — without a merge-base the migration audit diffs the two
# tips, which under `--diff-filter=AM` is exactly the migrations new here.
#
# On push the base is `github.event.before`, the tip the branch had before
# this push — not `HEAD~1`, which names only the last commit and would let a
# multi-commit push slip every earlier commit's migrations past the audit.
# It is fetched by SHA at depth 1; the audits diff two tips and need no
# common ancestry. An all-zero `before` means the branch is new and has no
# predecessor to diff, so `HEAD~1` remains the fallback there.
- name: Resolve base ref for diff-based audits
id: audit_base
run: |
if [ "${{ github.event_name }}" = "pull_request" ]; then
git fetch --depth=1 origin "${{ github.base_ref }}"
echo "ref=origin/${{ github.base_ref }}" >> "$GITHUB_OUTPUT"
elif [ -n "${{ github.event.before }}" ] &&
[ "${{ github.event.before }}" != "0000000000000000000000000000000000000000" ]; then
git fetch --depth=1 origin "${{ github.event.before }}"
echo "ref=${{ github.event.before }}" >> "$GITHUB_OUTPUT"
else
echo "ref=HEAD~1" >> "$GITHUB_OUTPUT"
fi
- name: Check block registry invariants
run: bun run apps/sim/scripts/check-block-registry.ts "${{ steps.audit_base.outputs.ref }}"
- name: Lint code
run: bun run lint:check
# Every zero-argument `check:*` script, run concurrently. The list is derived in
# scripts/run-audits.ts, which also writes the per-audit timing table to the job
# summary and annotates failures. Audits needing a base ref stay separate below.
- name: Repo audits
run: bun run check:audits
- name: Verify docs manifest is in sync
run: bun run docs-manifest:check
- name: Migration safety (zero-downtime) audit
run: bun run check:migrations "${{ steps.audit_base.outputs.ref }}"
# Every workspace, not just realtime. packages/emcn, packages/utils,
# apps/desktop and apps/docs had no type check in CI at all; apps/sim's
# source was covered only as a side effect of `next build` in the separate
# Build App job. Note this does NOT cover apps/sim's tests — its tsconfig
# excludes *.test.ts(x), and including them today surfaces ~2.2k errors,
# so that is its own cleanup rather than a gate to switch on here.
- name: Type-check all workspaces
run: bunx turbo run type-check
# cloud-review-tools.test.ts runs the real helper on the runner, which shells
# out to rg. Blacksmith's image ships it, GitHub's doesn't.
- name: Install ripgrep
run: command -v rg || (sudo apt-get update && sudo apt-get install -y ripgrep)
# Runs the root scripts and each workspace's Vitest suite,
# without `--coverage`.
- name: Run tests
env:
NODE_OPTIONS: '--no-warnings --max-old-space-size=8192'
NEXT_PUBLIC_APP_URL: 'https://www.sim.ai'
DATABASE_URL: 'postgresql://postgres:postgres@localhost:5432/simstudio'
ENCRYPTION_KEY: '0000000000000000000000000000000000000000000000000000000000000000' # dummy key for CI only
TURBO_CACHE_DIR: .turbo
run: bun run test
- name: Check schema and migrations are in sync
working-directory: packages/db
run: |
bunx drizzle-kit generate --config=./drizzle.config.ts
if [ -n "$(git status --porcelain ./migrations)" ]; then
echo "❌ Schema and migrations are out of sync!"
echo "Run 'cd packages/db && bunx drizzle-kit generate' and commit the new migrations."
git status --porcelain ./migrations
git diff ./migrations
exit 1
fi
echo "✅ Schema and migrations are in sync"
# Next.js production build, in parallel with lint + tests. Sticky disks are
# cloned from the last committed snapshot per job and committed last-writer-
# wins, so concurrent mounts are safe. The bun/node_modules disks are shared
# with test-build (the lockfile-hashed key means they only ever share when the
# dependency tree really is identical, so LWW loss is harmless), but the Turbo
# cache gets its own key: with a shared key, only the last committer's new
# entries survive each run, so the test and build Turbo entries would evict
# each other nondeterministically.
#
# Runner is sized for the COLD-cache build, which is what OOM-killed the 8vcpu
# tier (23 kills / 1074 runs at 98% of its 30.4 GB): warm peaks ~12 GB, cold
# peaked 51 GB. NODE_OPTIONS' --max-old-space-size caps only Node's JS heap,
# not the native Turbopack workers that dominate, so it cannot prevent this.
build:
name: Build App
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-16vcpu-ubuntu-2404' || 'linux-x64-8-core' }}
# Build durations crossed 15 minutes as the app grew (10m02 on Jul 29 AM,
# 14m44 after the folders/desktop/library merges, then two straight
# timeouts) — GitHub reports a job timeout as "cancelled". 25 keeps
# headroom without masking a genuine hang.
timeout-minutes: 25
steps:
- name: Checkout code
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
- name: Setup workspace
uses: ./.github/actions/setup-workspace
with:
provider: ${{ vars.CI_PROVIDER }}
turbo-cache-key: turbo-cache-build
# No `.next/cache` mount: the Turbopack persistent build cache is off. A
# controlled A/B on one branch (PR #6078) with a byte-identical module graph
# measured compile at 113s with the cache off, 162s cold with it on, and
# 360s warm — the cache made the same build 3.2x slower, and it grew
# 5.1 GB -> 12 GB across two runs of an unchanged tree, so a disk degrades
# the more it is used. Mounting a disk nothing reads would only cost storage.
# Running out of RAM kills the whole VM and surfaces only as "the runner
# has received a shutdown signal" — no mention of memory, ~12 min in. Warn
# with the real numbers so that failure is a one-line diagnosis instead of
# a mystery. Warn, never fail: a warm build peaks ~12 GB and a partial one
# ~28 GB, so a 32 GB runner still completes plenty of builds, and the
# GitHub fallback is the break-glass path — degrading it to a guaranteed
# failure would be worse than the risk this flags.
- name: Check runner memory headroom
run: |
TOTAL_GB=$(awk '/MemTotal/ {printf "%d", $2/1048576}' /proc/meminfo)
echo "Runner memory: ${TOTAL_GB} GB"
if [ "$TOTAL_GB" -lt 40 ]; then
echo "::warning::Runner has ${TOTAL_GB} GB. A cold-cache build peaks ~51 GB, so this run may be OOM-killed (reported only as 'the runner has received a shutdown signal'). Warm/partial builds should still fit."
fi
- name: Build application
env:
NODE_OPTIONS: '--no-warnings --max-old-space-size=8192'
NEXT_PUBLIC_APP_URL: 'https://www.sim.ai'
DATABASE_URL: 'postgresql://postgres:postgres@localhost:5432/simstudio'
STRIPE_SECRET_KEY: 'dummy_key_for_ci_only'
STRIPE_WEBHOOK_SECRET: 'dummy_secret_for_ci_only'
RESEND_API_KEY: 'dummy_key_for_ci_only'
AWS_REGION: 'us-west-2'
ENCRYPTION_KEY: '0000000000000000000000000000000000000000000000000000000000000000' # dummy key for CI only
TURBO_CACHE_DIR: .turbo
run: bunx turbo run build --filter=@sim/app
.github/workflows/deploy-trigger-dev.yml → .github/workflows/trigger-dev.yml
+7 -18
View File
@@ -1,4 +1,4 @@
name: Deploy Dev Tasks
name: trigger-dev
# Independent of app CI: task packaging and deployment must never delay dev images.
on:
@@ -16,31 +16,20 @@ concurrency:
jobs:
deploy:
name: Deploy Trigger.dev preview
name: deploy
runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
timeout-minutes: 30
steps:
- name: Checkout code
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: 1.4.2
persist-credentials: false
- name: Cache Bun dependencies
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5
- name: Setup workspace
uses: ./.github/actions/setup
with:
path: |
~/.bun/install/cache
node_modules
**/node_modules
key: ${{ runner.os }}-bun-${{ hashFiles('**/bun.lock') }}
restore-keys: |
${{ runner.os }}-bun-
- name: Install dependencies
run: bun install --frozen-lockfile --ignore-scripts
provider: ${{ vars.CI_PROVIDER }}
node-version: ''
- name: Upload preview version
id: deploy
apps/desktop/README.md
+2 -2
View File
@@ -107,8 +107,8 @@ Local unsigned pre-release share: `SIM_DESKTOP_DEFAULT_ORIGIN=https://www.dev.si
The build also derives the app icon from `SIM_DESKTOP_DEFAULT_ORIGIN`. Every channel uses the exact production icon with its white background and black `sim` mark. Non-production channels add a thin outline using existing platform colors: dev uses orange, staging uses Loop blue, and localhost uses Workflow violet. The macOS menu-bar icon also carries a compact `D`, `S`, or `L` subscript for those environments; production remains unmarked. Native Icon Composer assets live in `build/`; `scripts/build.ts` copies the selected variant to the ignored `build/generated-icon.icon` path consumed by electron-builder. Electron-builder compiles it to `Assets.car` and derives the legacy `.icns` fallback from the same source. Matching 512px PNGs in `static/` provide the Dock icon for unpackaged runs.
CI (`.github/workflows/desktop-release.yml`, wired into `ci.yml`):
- Stable builds run only after `create-release` on a `vX.Y.Z:` commit to main — **never before**: `scripts/create-single-release.ts` skips creation if the tag exists, so a desktop job publishing first would eat the changelog. Stable assets remain on `simstudioai/sim`; dev/staging assets publish to the public `simstudioai/sim-desktop-releases` repository so source-repository followers are not notified for internal shell builds. The job builds `--publish never`; reruns verify the size and SHA-256 digest of existing release assets instead of overwriting them.
- **Secrets gate**: `check-desktop-signing` in `ci.yml` probes the six Apple secrets and skips the desktop job with a warning until they exist — releases never fail on a missing Apple account, and the first release after the secrets land ships desktop artifacts automatically. Manual/one-off builds: Actions → "Desktop Release (macOS)" → Run workflow with a `vX.Y.Z` version (`publish: false` uploads artifacts to the run instead of the release).
- Stable builds run only after `release` on a `vX.Y.Z:` commit to main — **never before**: `scripts/create-single-release.ts` skips creation if the tag exists, so a desktop job publishing first would eat the changelog. Stable assets remain on `simstudioai/sim`; dev/staging assets publish to the public `simstudioai/sim-desktop-releases` repository so source-repository followers are not notified for internal shell builds. The job builds `--publish never`; reruns verify the size and SHA-256 digest of existing release assets instead of overwriting them.
- **Secrets gate**: `desktop-signing` in `ci.yml` probes the six Apple secrets and skips the desktop job with a warning until they exist — releases never fail on a missing Apple account, and the first release after the secrets land ships desktop artifacts automatically. Manual/one-off builds: Actions → "desktop-release" → Run workflow with a `vX.Y.Z` version (`publish: false` uploads artifacts to the run instead of the release).
- The product semver is **injected** from the release tag into `apps/desktop/package.json` at build time (repo package versions are placeholders). A mismatch guard fails the build.
- Fuses are flipped at package time (`electronFuses` in `electron-builder.yml`): runAsNode off, NODE_OPTIONS off, inspect args off, ASAR-only + integrity validation, and cookie encryption on. The packaged smoke test asserts every fuse byte so Electron upgrades fail until new fuses receive an explicit policy.
- **Cookie-encryption go/no-go**: on every Electron bump, verify a packaged build keeps its session across relaunch (there are historical cookie-persistence bugs with the `EnableCookieEncryption` fuse). If it reproduces, set `enableCookieEncryption: false` and record it here.
apps/desktop/e2e/background-executor.spec.ts
+575
View File
@@ -0,0 +1,575 @@
import { mkdirSync, mkdtempSync, readFileSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { type ElectronApplication, expect, test } from '@playwright/test'
import type { SimDesktopApi } from '@sim/desktop-bridge'
import { getErrorMessage } from '@sim/utils/errors'
import { sleep } from '@sim/utils/helpers'
import { recordCheck } from './check-report'
import {
type FixtureCall,
FixtureSim,
launch,
processRunning,
RECONCILE_MS,
readFileSafe,
refFor,
registeredDevice,
settled,
sha256,
WORKSPACE,
} from './executor-sim'
/**
* The Sim desktop app's background executor against a fixture Sim that speaks the executor's
* device protocol (register, inbox, doorbell, claim, lease, complete, import) the way Sim's
* routes do.
* The window navigates, reloads and leaves the chats while their calls run: nothing in this
* suite depends on a chat view, which is the point. Each check lands in a JSON report at
* BACKGROUND_EXECUTOR_REPORT_PATH as it finishes.
*/
const CHAT_A = 'chat-browser-a'
const CHAT_B = 'chat-terminal-b'
const CHAT_C = 'chat-idle-c'
const sim = new FixtureSim()
/** Runs one check and adds its outcome to the report as soon as it is known. */
async function check(name: string, body: () => Promise<void>): Promise<void> {
const startedAt = Date.now()
const record = (status: 'passed' | 'failed', error?: unknown) =>
recordCheck(process.env.BACKGROUND_EXECUTOR_REPORT_PATH, 'background-executor', {
name,
status,
durationMs: Date.now() - startedAt,
retry: test.info().retry,
...(error === undefined ? {} : { error: getErrorMessage(error) }),
})
try {
await body()
record('passed')
} catch (error) {
record('failed', error)
throw error
}
}
test.describe('background executor', () => {
let app: ElectronApplication | null = null
test.beforeAll(async () => {
await sim.start()
})
test.afterEach(async () => {
await app?.close().catch(() => {})
app = null
sim.reset()
})
test.afterAll(async () => {
await sim.stop()
})
test('A: two chats run browser and terminal work while the user is elsewhere and reloads', async () => {
const userData = mkdtempSync(join(tmpdir(), 'sim-executor-a-'))
const launched = await launch(sim, userData)
app = launched.app
const window = launched.window
const deviceId = await registeredDevice(sim)
await window.goto(`${sim.origin}/workspace/${WORKSPACE}/chat/${CHAT_C}`)
const marker = join(userData, 'terminal-marker.txt')
const readable = join(userData, 'notes.txt')
writeFileSync(readable, 'background read fixture')
await check('A: chat A opens its page in the background', async () => {
const opened = sim.issue(deviceId, CHAT_A, 'browser_open_url', {
url: `${sim.origin}/counter?chat=A`,
})
const completion = await settled(sim, opened)
expect(completion.status, completion.message).toBe('success')
})
const opened = [...sim.calls.values()][0] as FixtureCall
const outline = (opened.completions[0]?.data?.snapshot as { outline: string }).outline
const button = refFor(outline, 'Count visit')
const clicks = Array.from({ length: 10 }, () =>
sim.issue(deviceId, CHAT_A, 'browser_click', { elementId: button })
)
const terminalRuns = [1, 2, 3].map((n) =>
sim.issue(deviceId, CHAT_B, 'terminal', {
operation: 'run',
args: { command: `sleep 1; echo B-${n} >> '${marker}'`, waitSeconds: 30 },
})
)
const localRead = sim.issue(deviceId, CHAT_B, 'read_local_file', { path: readable })
await window.goto(`${sim.origin}/workspace/ws-other/home`)
await window.reload()
await window.goto(`${sim.origin}/workspace/${WORKSPACE}/chat/${CHAT_C}`)
await check('A: every call completes exactly once with its own token', async () => {
for (const id of [...clicks, ...terminalRuns, localRead]) {
const completion = await settled(sim, id, 90_000)
const call = sim.requireCall(id)
expect(completion.status, `${call.toolName}: ${completion.message}`).toBe('success')
expect(completion.outcome).toBe('recorded')
expect(call.claims).toBe(1)
expect(call.completions).toHaveLength(1)
}
})
await check('A: the page saw each click exactly once', async () => {
await expect.poll(() => sim.hits.get('A')).toBe(10)
})
await check('A: chat B ran each command once, in order, in its own terminal', async () => {
expect(readFileSync(marker, 'utf8').trim().split('\n')).toEqual(['B-1', 'B-2', 'B-3'])
const read = sim.requireCall(localRead).completions[0]
expect(JSON.stringify(read?.data)).toContain('background read fixture')
})
await check("A: chat B cannot see chat A's tabs", async () => {
const listed = sim.issue(deviceId, CHAT_B, 'browser_list_tabs', {})
const completion = await settled(sim, listed)
expect(JSON.stringify(completion.data)).not.toContain('/counter')
})
await check('A: no chat view executed or reported a call', async () => {
expect(sim.requests.filter((request) => request.includes('/api/copilot/confirm'))).toEqual([])
expect(
sim.requests.filter((request) => request.includes('/api/desktop/tool/authorize'))
).toEqual([])
})
await check('A: calls are picked up within 1.5 s at p95', async () => {
const latencies = [...sim.calls.values()]
.map((call) => (call.claimedAt ?? Number.POSITIVE_INFINITY) - call.issuedAt)
.sort((a, b) => a - b)
const p95 = latencies[Math.ceil(latencies.length * 0.95) - 1] ?? Number.POSITIVE_INFINITY
expect(p95).toBeLessThan(1_500)
})
await check('A: the lease is renewed while calls wait and run', async () => {
expect(Math.max(...terminalRuns.map((id) => sim.requireCall(id).renewals))).toBeGreaterThan(0)
})
})
test('B: a result produced while the network is cut is delivered once after reconnecting', async () => {
const userData = mkdtempSync(join(tmpdir(), 'sim-executor-b-'))
app = (await launch(sim, userData)).app
const deviceId = await registeredDevice(sim)
const run = sim.issue(deviceId, CHAT_B, 'terminal', {
operation: 'run',
args: { command: 'sleep 2; echo offline-done', waitSeconds: 30 },
})
await expect.poll(() => sim.requireCall(run).claims).toBe(1)
sim.disconnect()
await check(
'B: the cut drops the doorbell and nothing reaches Sim while it lasts',
async () => {
await expect.poll(() => sim.streams.get(deviceId)?.size ?? 0).toBe(0)
await expect.poll(() => sim.droppedWhileOffline, { timeout: 15_000 }).toBeGreaterThan(0)
await sleep(4_000)
expect(sim.requireCall(run).completions).toHaveLength(0)
}
)
sim.reconnect()
await check('B: the result arrives once after reconnecting', async () => {
const completion = await settled(sim, run, 60_000)
expect(completion.status, completion.message).toBe('success')
expect(JSON.stringify(completion.data)).toContain('offline-done')
await sleep(3_000)
expect(sim.requireCall(run).completions).toHaveLength(1)
expect(sim.requireCall(run).claims).toBe(1)
})
await check('B: the device reopens its doorbell and picks up new work', async () => {
await expect.poll(() => sim.streams.get(deviceId)?.size ?? 0, { timeout: 45_000 }).toBe(1)
const after = sim.issue(deviceId, CHAT_B, 'browser_list_tabs', {})
const completion = await settled(sim, after)
expect(completion.status, completion.message).toBe('success')
})
})
test('C: a crash mid-command reports the outcome as unknown and never reruns it', async () => {
const userData = mkdtempSync(join(tmpdir(), 'sim-executor-c-'))
const marker = join(userData, 'crash-marker.txt')
const first = await launch(sim, userData)
const deviceId = await registeredDevice(sim)
const run = sim.issue(deviceId, CHAT_B, 'terminal', {
operation: 'run',
args: { command: `echo started >> '${marker}'; sleep 40`, waitSeconds: 60 },
})
await expect.poll(() => readFileSafe(marker), { timeout: 30_000 }).toContain('started')
first.app.process().kill('SIGKILL')
sim.streams.get(deviceId)?.clear()
app = (await launch(sim, userData)).app
await check('C: the restarted app reports the lost result as outcome unknown', async () => {
const completion = await settled(sim, run, 60_000)
expect(completion.data).toMatchObject({ outcomeUnknown: true, doNotRetry: true })
})
await check('C: the command ran exactly once', async () => {
expect(sim.requireCall(run).claims).toBe(1)
expect(readFileSafe(marker).trim().split('\n')).toEqual(['started'])
})
})
test('D: a folder import lands in Sim while the user is in another chat', async () => {
const userData = mkdtempSync(join(tmpdir(), 'sim-executor-d-'))
const launched = await launch(sim, userData)
app = launched.app
const deviceId = await registeredDevice(sim)
const source = join(userData, 'Reports')
mkdirSync(join(source, 'q3'), { recursive: true })
writeFileSync(join(source, 'notes.txt'), 'remember the numbers')
// Larger than one 8 MB read, so the file crosses Electron in several chunks.
const large = Buffer.alloc(9 * 1024 * 1024, 7)
writeFileSync(join(source, 'q3', 'export.bin'), large)
await launched.window.goto(`${sim.origin}/workspace/${WORKSPACE}/chat/${CHAT_C}`)
const call = sim.issue(deviceId, CHAT_B, 'import_local_files', {
path: source,
targetWorkspaceId: WORKSPACE,
folderId: 'folder-e2e',
})
await check('D: the import completes with every entry it stored', async () => {
const completion = await settled(sim, call, 60_000)
expect(completion.status).toBe('success')
expect(completion.data).toMatchObject({
success: true,
workspaceId: WORKSPACE,
folders: [
{ id: 'entry-1', relativePath: '' },
{ id: 'entry-3', relativePath: 'q3' },
],
files: [
{ id: 'entry-2', relativePath: 'notes.txt' },
{ id: 'entry-4', relativePath: 'q3/export.bin' },
],
})
})
await check('D: Sim received the tree with the bytes on disk, once', async () => {
expect(sim.imported).toEqual([
{ toolCallId: call, kind: 'directory', sourceName: 'Reports', relativePath: '' },
{
toolCallId: call,
kind: 'file',
sourceName: 'Reports',
relativePath: 'notes.txt',
sha256: sha256(Buffer.from('remember the numbers')),
bytes: 20,
},
{ toolCallId: call, kind: 'directory', sourceName: 'Reports', relativePath: 'q3' },
{
toolCallId: call,
kind: 'file',
sourceName: 'Reports',
relativePath: 'q3/export.bin',
sha256: sha256(large),
bytes: large.length,
},
])
expect(sim.requireCall(call).claims).toBe(1)
})
})
test('E: Stop from another chat stops a running browser wait and terminal command', async () => {
const userData = mkdtempSync(join(tmpdir(), 'sim-executor-e-'))
app = (await launch(sim, userData)).app
const deviceId = await registeredDevice(sim)
const opened = sim.issue(deviceId, CHAT_A, 'browser_open_url', {
url: `${sim.origin}/counter?chat=E`,
})
await settled(sim, opened)
const wait = sim.issue(deviceId, CHAT_A, 'browser_wait_for', {
text: 'never appears',
timeoutMs: 30_000,
})
const command = 'sleep 47'
const run = sim.issue(deviceId, CHAT_B, 'terminal', {
operation: 'run',
args: { command, waitSeconds: 60 },
})
await expect.poll(() => sim.requireCall(wait).claims).toBe(1)
await expect.poll(() => processRunning(command), { timeout: 20_000 }).toBe(true)
const stoppedAt = Date.now()
sim.stopCall(wait)
sim.stopCall(run)
await check('E: both stopped calls are acknowledged within seconds', async () => {
await settled(sim, wait, 10_000)
await settled(sim, run, 15_000)
expect(sim.requireCall(wait).completions[0]?.outcome).toBe('superseded')
expect(sim.requireCall(run).completions[0]?.outcome).toBe('superseded')
expect((sim.requireCall(wait).completions[0]?.at ?? 0) - stoppedAt).toBeLessThan(5_000)
})
await check('E: the stopped command no longer runs on the machine', async () => {
await expect.poll(() => processRunning(command), { timeout: 10_000 }).toBe(false)
})
})
test('F: a call waiting for approval in a background chat notifies, then runs once approved', async () => {
const userData = mkdtempSync(join(tmpdir(), 'sim-executor-f-'))
app = (await launch(sim, userData)).app
await app.evaluate(({ Notification }) => {
const shown: Array<{ title: string; body: string }> = []
const target = globalThis as typeof globalThis & { __shownNotifications?: typeof shown }
target.__shownNotifications = shown
Notification.prototype.show = function show(this: { title: string; body: string }) {
shown.push({ title: this.title, body: this.body })
}
})
const deviceId = await registeredDevice(sim)
const gated = sim.issue(
deviceId,
CHAT_B,
'terminal',
{ operation: 'run', args: { command: 'echo approved-run', waitSeconds: 30 } },
'awaiting_approval'
)
await check('F: the user is notified, without the command in the notification', async () => {
await expect
.poll(
() =>
app?.evaluate(
() =>
(globalThis as { __shownNotifications?: Array<{ body: string }> })
.__shownNotifications ?? []
),
{ timeout: 15_000 }
)
.toHaveLength(1)
const shown = await app?.evaluate(
() =>
(globalThis as { __shownNotifications?: Array<{ body: string }> }).__shownNotifications
)
expect(JSON.stringify(shown)).not.toContain('approved-run')
})
await check('F: nothing runs before approval', async () => {
await sleep(RECONCILE_MS * 2)
expect(sim.requireCall(gated).claims).toBe(0)
})
sim.approve(gated)
const approvedAt = Date.now()
await check('F: the approved call is claimed promptly and runs once', async () => {
const completion = await settled(sim, gated)
expect((sim.requireCall(gated).claimedAt ?? 0) - approvedAt).toBeLessThan(1_500)
expect(JSON.stringify(completion.data)).toContain('approved-run')
})
})
test('F: a call the user declines in a background chat never runs', async () => {
const userData = mkdtempSync(join(tmpdir(), 'sim-executor-f-declined-'))
const marker = join(userData, 'declined-marker.txt')
app = (await launch(sim, userData)).app
const deviceId = await registeredDevice(sim)
const pulls = () =>
sim.requests.filter((request) => request.startsWith('GET /api/desktop/inbox')).length
const claimAttempts = () =>
sim.requests.filter((request) => request === 'POST /api/desktop/tool/claim').length
const pullsBefore = pulls()
const claimAttemptsBefore = claimAttempts()
const gated = sim.issue(
deviceId,
CHAT_B,
'terminal',
{ operation: 'run', args: { command: `echo ran >> '${marker}'`, waitSeconds: 30 } },
'awaiting_approval'
)
// The device has pulled its inbox and seen the call waiting for approval.
await expect.poll(pulls).toBeGreaterThan(pullsBefore)
sim.decline(gated)
await check('F: the declined call is never claimed, and its command never runs', async () => {
await sleep(RECONCILE_MS * 2)
// Attempts, not successful claims: the fixture refuses a declined call's claim itself.
expect(claimAttempts() - claimAttemptsBefore).toBe(0)
expect(sim.requireCall(gated).completions).toEqual([])
expect(readFileSafe(marker)).toBe('')
})
})
test('G: only the device a turn is bound to claims its calls', async () => {
const first = await launch(sim, mkdtempSync(join(tmpdir(), 'sim-executor-g1-')))
const firstDevice = await registeredDevice(sim)
const second = await launch(sim, mkdtempSync(join(tmpdir(), 'sim-executor-g2-')))
const secondDevice = await registeredDevice(sim, new Set([firstDevice]))
app = first.app
const call = sim.issue(firstDevice, CHAT_A, 'browser_list_tabs', {})
await check('G: the bound device runs it and the other never claims it', async () => {
await settled(sim, call)
expect(sim.requireCall(call).claims).toBe(1)
expect(
sim.requests.filter((request) => request === 'POST /api/desktop/tool/claim').length
).toBe(1)
expect(secondDevice).not.toBe(firstDevice)
})
await second.app.close()
})
test('H: a device Sim has not enabled offers no binding', async () => {
sim.enabled = false
const launched = await launch(sim, mkdtempSync(join(tmpdir(), 'sim-executor-h-')))
app = launched.app
await launched.window.goto(`${sim.origin}/workspace/${WORKSPACE}/chat/${CHAT_C}`)
await expect.poll(() => sim.requests.includes('POST /api/desktop/devices')).toBe(true)
await check(
'H: the composer gets no device, so its turns stay with the chat view',
async () => {
const device = await launched.window.evaluate(() =>
(
globalThis as typeof globalThis & { simDesktop: SimDesktopApi }
).simDesktop.desktopExecutor?.getDevice()
)
expect(device).toBeNull()
}
)
sim.enabled = true
await launched.app.close()
const enabled = await launch(sim, mkdtempSync(join(tmpdir(), 'sim-executor-h2-')))
app = enabled.app
const deviceId = await registeredDevice(sim)
await enabled.window.goto(`${sim.origin}/workspace/${WORKSPACE}/chat/${CHAT_C}`)
await check('H: an enabled device offers itself for binding', async () => {
const device = await enabled.window.evaluate(() =>
(
globalThis as typeof globalThis & { simDesktop: SimDesktopApi }
).simDesktop.desktopExecutor?.getDevice()
)
expect(device).toEqual({ deviceId, protocolVersion: 1 })
})
})
test('I: the agent yields its page while the user works in it, then takes it back', async () => {
app = (await launch(sim, mkdtempSync(join(tmpdir(), 'sim-executor-i-')))).app
const deviceId = await registeredDevice(sim)
const opened = sim.issue(deviceId, CHAT_A, 'browser_open_url', {
url: `${sim.origin}/counter?chat=I`,
})
const outline = ((await settled(sim, opened)).data?.snapshot as { outline: string }).outline
const button = refFor(outline, 'Count visit')
const typeInAgentPage = () =>
app?.evaluate(({ webContents }) => {
const page = webContents
.getAllWebContents()
.find((contents) => contents.getURL().includes('/counter?chat=I'))
page?.sendInputEvent({ type: 'keyDown', keyCode: 'Tab' })
page?.sendInputEvent({ type: 'keyUp', keyCode: 'Tab' })
})
await typeInAgentPage()
const click = sim.issue(deviceId, CHAT_A, 'browser_click', { elementId: button })
for (let i = 0; i < 4; i++) {
await sleep(500)
await typeInAgentPage()
}
const lastUserInputAt = Date.now()
await check('I: the click waits until the user stops, then runs once', async () => {
const completion = await settled(sim, click, 30_000)
expect(completion.status, completion.message).toBe('success')
expect(completion.at - lastUserInputAt).toBeGreaterThanOrEqual(3_000)
await expect.poll(() => sim.hits.get('I')).toBe(1)
})
})
test('I: an action the user never stops working long enough for does not run', async () => {
app = (await launch(sim, mkdtempSync(join(tmpdir(), 'sim-executor-i2-')))).app
const deviceId = await registeredDevice(sim)
const opened = sim.issue(deviceId, CHAT_A, 'browser_open_url', {
url: `${sim.origin}/counter?chat=I2`,
})
const outline = ((await settled(sim, opened)).data?.snapshot as { outline: string }).outline
const button = refFor(outline, 'Count visit')
const typeInAgentPage = () =>
app?.evaluate(({ webContents }) => {
const page = webContents
.getAllWebContents()
.find((contents) => contents.getURL().includes('/counter?chat=I2'))
page?.sendInputEvent({ type: 'keyDown', keyCode: 'Tab' })
page?.sendInputEvent({ type: 'keyUp', keyCode: 'Tab' })
})
await typeInAgentPage()
const click = sim.issue(deviceId, CHAT_A, 'browser_click', { elementId: button })
let typing = true
const keepTyping = (async () => {
while (typing) {
await typeInAgentPage()
await sleep(1_000)
}
})()
await check('I: the click reports it never ran, instead of timing out', async () => {
const completion = await settled(sim, click, 60_000)
typing = false
await keepTyping
expect(completion.status).toBe('error')
expect(completion.message).toContain('Not run: the user kept working in this page')
expect(completion.data).not.toMatchObject({ outcomeUnknown: true })
expect(sim.hits.get('I2') ?? 0).toBe(0)
})
})
test('J: the machine stays awake only while a chat has work running', async () => {
app = (await launch(sim, mkdtempSync(join(tmpdir(), 'sim-executor-j-')))).app
await app.evaluate(({ powerSaveBlocker }) => {
const log: string[] = []
const active = new Set<number>()
let next = 1
const target = globalThis as typeof globalThis & { __sleepBlocks?: string[] }
target.__sleepBlocks = log
powerSaveBlocker.start = (type) => {
log.push(`start:${type}`)
active.add(next)
return next++
}
powerSaveBlocker.stop = (id) => {
log.push('stop')
active.delete(id)
return true
}
powerSaveBlocker.isStarted = (id) => active.has(id)
})
const deviceId = await registeredDevice(sim)
const sleepLog = () =>
app?.evaluate(() => (globalThis as { __sleepBlocks?: string[] }).__sleepBlocks ?? [])
const run = sim.issue(deviceId, CHAT_B, 'terminal', {
operation: 'run',
args: { command: 'sleep 3; echo awake', waitSeconds: 30 },
})
await check('J: a blocker is held while the command runs', async () => {
await expect.poll(sleepLog, { timeout: 15_000 }).toEqual(['start:prevent-app-suspension'])
})
await check('J: it is released once the result is delivered', async () => {
await settled(sim, run)
await expect
.poll(sleepLog, { timeout: 10_000 })
.toEqual(['start:prevent-app-suspension', 'stop'])
})
})
})
apps/desktop/e2e/check-report.spec.ts
+49
View File
@@ -0,0 +1,49 @@
import { spawnSync } from 'node:child_process'
import { mkdtempSync, readFileSync, writeFileSync } from 'node:fs'
import { createRequire } from 'node:module'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { fileURLToPath } from 'node:url'
import { expect, test } from '@playwright/test'
import { omit } from '@sim/utils/object'
import type { ReportCheck } from './check-report'
/**
* The suites' JSON reports against a real worker restart: a nested Playwright run of a fixture
* whose first test fails, so the second runs in a fresh worker with fresh module state. The report
* must keep the failure, and must not carry checks from an earlier run.
*/
const CLI = createRequire(import.meta.url).resolve('@playwright/test/cli')
const CONFIG = fileURLToPath(new URL('./fixtures/worker-restart.config.ts', import.meta.url))
test('a check that failed before the worker restarted stays in the report', () => {
const reportPath = join(mkdtempSync(join(tmpdir(), 'sim-check-report-')), 'report.json')
writeFileSync(
reportPath,
JSON.stringify({
suite: 'worker-restart',
run: -1,
checks: [{ name: 'left by an earlier run', status: 'passed', durationMs: 0 }],
})
)
// The outer run's worker variables would make the nested runner think it is a worker.
const env = omit(
process.env,
Object.keys(process.env).filter((key) => /^(TEST_|PW_)/.test(key))
)
const run = spawnSync(process.execPath, [CLI, 'test', '--config', CONFIG], {
env: { ...env, WORKER_RESTART_REPORT_PATH: reportPath },
encoding: 'utf8',
timeout: 60_000,
})
expect(run.status, run.stdout + run.stderr).toBe(1)
const checks = (JSON.parse(readFileSync(reportPath, 'utf8')) as { checks: ReportCheck[] }).checks
expect(checks.map(({ name, status }) => ({ name, status }))).toEqual([
{ name: 'fails first', status: 'failed' },
{ name: 'passes after the worker restarts', status: 'passed' },
])
// Proof the second check ran in a replacement worker, not the one that failed.
expect(checks[0]?.error).not.toBe(checks[1]?.error)
})
apps/desktop/e2e/check-report.ts
+44
View File
@@ -0,0 +1,44 @@
import { readFileSync, writeFileSync } from 'node:fs'
/** One check's outcome in a suite's JSON report. */
export interface ReportCheck {
name: string
status: string
durationMs: number
error?: string
retry?: number
}
interface Report {
suite: string
/** The Playwright run that wrote it: its workers' parent process. */
run: number
checks: ReportCheck[]
}
/**
* Adds a check to the suite's report at `reportPath` as soon as its outcome is known, keeping the
* checks already there from the same run.
*
* Playwright replaces a worker after a failure, and the new one starts with empty module state, so
* a report kept in memory and written at the end would drop everything before the failure, the
* failure included, and could read green. The file holds it instead. Every worker of one run is a
* child of the same runner, so a report left by an earlier run is replaced rather than added to.
*/
export function recordCheck(
reportPath: string | undefined,
suite: string,
check: ReportCheck
): void {
if (!reportPath) return
const run = process.ppid
let checks: ReportCheck[] = []
try {
const existing = JSON.parse(readFileSync(reportPath, 'utf8')) as Report
if (existing.suite === suite && existing.run === run) checks = existing.checks
} catch {
// No report yet from this run.
}
const report: Report = { suite, run, checks: [...checks, check] }
writeFileSync(reportPath, JSON.stringify(report, null, 2))
}
apps/desktop/e2e/desktop-tools-live-sim.spec.ts
+871
View File
@@ -0,0 +1,871 @@
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { dirname, join } from 'node:path'
import { fileURLToPath } from 'node:url'
import {
type ElectronApplication,
_electron as electron,
expect,
type Locator,
type Page,
test,
} from '@playwright/test'
import type { SimDesktopApi } from '@sim/desktop-bridge'
import { sleep } from '@sim/utils/helpers'
import { generateId } from '@sim/utils/id'
import {
type LiveSimConfig,
liveSimConfig,
RedisMonitor,
ScriptedAgent,
type SeededUser,
SimDatabase,
SimProxy,
} from './fixtures/live-sim'
/**
* Desktop tools in the real Electron app against a real local Sim: the renderer is Sim's own chat
* view, every claim and result crosses Sim's routes into PostgreSQL and Redis, and only the
* model's decisions are scripted (a stand-in worker at `SIM_AGENT_API_URL`). Each test checks
* what the user or the model would observe: the result the model is resumed with, what landed in
* the workspace, what Sim persisted, and which requests reached Sim.
*
* Sim runs the background executor wherever it has Redis, as it does here. Most tests cover the
* chat view running desktop tools itself, as on an install without Redis: the proxy answers the
* app's registration as such an install does, so the app stays dormant and no turn binds to it.
* The tests marked as running in the background let Sim's own answer through.
*/
const DESKTOP_DIR = fileURLToPath(new URL('..', import.meta.url))
const config = liveSimConfig()
const PICKUP_GRACE_MS = 15_000
/** Longer than the default tool budget (60 s) plus the resume grace (30 s). */
const LONG_IMPORT_MS = 120_000
/** The execution lease a running import holds and renews (`SIM_TOOL_EXECUTION_LEASE_SECONDS`). */
const LEASE_MS = 60_000
/** How long a held request may take to arrive once the step that sends it ran. */
const ARRIVAL_MS = 60_000
/** First requests to a route compile it, which takes minutes on a cold dev app. */
const COMPILE_MS = 300_000
const REGISTRATION_PATH = '/api/desktop/devices'
const DESKTOP_COMPLETION_PATH = '/api/desktop/tool/complete'
/** Sim's answer to registration on an install that cannot run the background executor. */
const executorUnavailable = (answer: Record<string, unknown>) => {
answer.enabled = false
}
type DesktopWindow = typeof globalThis & { simDesktop: SimDesktopApi }
test.describe('desktop tools against a live Sim', () => {
test.skip(typeof config === 'string', typeof config === 'string' ? config : '')
test.describe.configure({ timeout: 360_000 })
let sim: LiveSimConfig
let proxy: SimProxy
let agent: ScriptedAgent
let db: SimDatabase
let app: ElectronApplication | undefined
let scratch: string
/** Errors the current window reported, shown when a click is blocked. */
let pageErrors: string[] = []
let testStartedAt = 0
test.beforeAll(async () => {
test.setTimeout(1_500_000)
if (typeof config === 'string') throw new Error(config)
sim = config
proxy = new SimProxy(sim)
agent = new ScriptedAgent(sim.agentPort)
db = new SimDatabase(sim)
await Promise.all([proxy.start(), agent.start()])
proxy.rewriteAnswer(REGISTRATION_PATH, executorUnavailable)
await test.step('warm up the routes the tests use', warmUp)
})
test.afterAll(async () => {
await Promise.all([proxy?.stop(), agent?.stop(), db?.close()])
})
test.beforeEach(() => {
testStartedAt = Date.now()
scratch = mkdtempSync(join(tmpdir(), 'sim-desktop-tools-live-'))
})
test.afterEach(async () => {
const testInfo = test.info()
if (testInfo.status !== testInfo.expectedStatus) {
const page = app?.windows()[0]
await page?.screenshot({ path: testInfo.outputPath('failure.png') }).catch(() => {})
writeFileSync(
testInfo.outputPath('diagnostics.json'),
JSON.stringify(
{
url: page?.url(),
composer: await page
?.getByRole('textbox')
.last()
.inputValue({ timeout: 2_000 })
.catch(() => null),
pageErrors,
requests: proxy
.seen(testStartedAt)
.filter((entry) => !entry.path.startsWith('/_next/')),
},
null,
2
)
)
}
proxy.clearHolds()
proxy.restoreNetwork()
await app?.close().catch(() => {})
app = undefined
proxy.rewriteAnswer(REGISTRATION_PATH, executorUnavailable)
proxy.rewriteAnswer(DESKTOP_COMPLETION_PATH, undefined)
rmSync(scratch, { recursive: true, force: true })
})
/**
* A dev app compiles each route on its first request, one at a time, so a route first reached
* mid-test stalls every other request (Electron gives a tool authorization 8 s). The warm-up runs
* the tests' own flows once: a read and an import, a chat switch during a live turn and back,
* Stop, and the login page. It then waits until every request the app made has been answered,
* so no route the tests reach is left to compile. It ends with a marker request
* (`/api/health?e2e=warm-up-done`) so the dev server's log shows anything compiled after it.
*/
async function warmUp(): Promise<void> {
scratch = mkdtempSync(join(tmpdir(), 'sim-desktop-tools-warm-'))
try {
const user = await db.seedUser(['Warm chat', 'Warm other chat'])
const headers = {
'Content-Type': 'application/json',
Cookie: `better-auth.session_token=${user.cookie}`,
Origin: proxy.origin,
}
const compile = (path: string, method: 'GET' | 'POST' | 'PUT' = 'GET') =>
fetch(new URL(path, sim.upstream), {
method,
headers,
body: method === 'GET' ? undefined : '{}',
redirect: 'manual',
signal: AbortSignal.timeout(COMPILE_MS),
}).then((response) => response.arrayBuffer())
// Routes the tests reach that a warm-up turn alone would not: Stop's, registration, and the
// ones a running app loads in the background.
for (const path of [
chatPath(user, 'Warm chat'),
'/login',
`/api/mothership/chats/${user.chats['Warm other chat']}`,
`/api/mothership/chat/stream?chatId=${user.chats['Warm chat']}`,
`/api/workspaces/${user.workspaceId}/files/folders`,
'/api/copilot/chats',
'/api/users/me/settings',
'/api/auth/oauth/connections',
'/api/desktop/inbox',
])
await compile(path)
for (const path of [
'/api/mothership/chat/stop',
'/api/mothership/chat/abort',
'/api/desktop/devices',
'/api/desktop/tool/authorize',
'/api/desktop/tool/claim',
'/api/desktop/tool/complete',
'/api/copilot/confirm',
'/api/files/uploads',
'/api/files/uploads/warm-up/parts',
'/api/files/uploads/warm-up/complete',
`/api/workspaces/${user.workspaceId}/files/folders`,
])
await compile(path, 'POST')
await compile('/api/v2/uploads/warm-up', 'PUT')
const file = writeFile(join(scratch, 'warm.txt'), 'warm')
const folder = join(scratch, 'Warm folder')
writeFile(join(folder, 'warm.txt'), 'warm')
let proceed!: () => void
const proceeding = new Promise<void>((resolve) => {
proceed = resolve
})
agent.script(
'[warm-up]',
async (turn) => {
turn.text('Warming up.')
await proceeding
turn.toolCall({ toolName: 'read_local_file', args: { path: file } })
turn.toolCall({
toolName: 'import_local_files',
args: { path: folder, targetWorkspaceId: user.workspaceId },
})
turn.pause()
},
(_resume, turn) => turn.complete('Warmed up.')
)
agent.script('[warm-stop]', async (turn) => {
turn.text('Stopping soon.')
// The leg stays open until Stop ends it.
await turn.closed
})
const page = await openApp(user, 'Warm chat', COMPILE_MS)
await send(page, '[warm-up] read and import', COMPILE_MS)
await expect(page.getByText('Warming up.')).toBeVisible({ timeout: COMPILE_MS })
// Leaving and reopening a chat while its turn runs re-attaches to the turn's stream.
await openChat(page, user, 'Warm other chat', COMPILE_MS)
await openChat(page, user, 'Warm chat', COMPILE_MS)
proceed()
await expect(page.getByText('Warmed up.')).toBeVisible({ timeout: 2 * COMPILE_MS })
await send(page, '[warm-stop] wait for Stop', COMPILE_MS)
await expect(page.getByText('Stopping soon.')).toBeVisible({ timeout: COMPILE_MS })
await click(page, page.getByRole('button', { name: 'Stop generation' }), COMPILE_MS)
await expect(page.getByRole('button', { name: 'Stop generation' })).toBeHidden({
timeout: COMPILE_MS,
})
await proxy.settled(COMPILE_MS)
await compile('/api/health?e2e=warm-up-done')
} finally {
await app?.close().catch(() => {})
app = undefined
rmSync(scratch, { recursive: true, force: true })
}
}
const chatPath = (user: SeededUser, title: string) =>
`/workspace/${user.workspaceId}/chat/${user.chats[title]}`
/** Launches the app signed in as `user`, showing the chat titled `title`. */
async function openApp(user: SeededUser, title: string, timeout = 120_000): Promise<Page> {
app = await electron.launch({
args: [process.env.SIM_DESKTOP_E2E_MAIN ?? '.'],
cwd: DESKTOP_DIR,
env: {
...process.env,
SIM_DESKTOP_ORIGIN: proxy.origin,
SIM_DESKTOP_USER_DATA: join(scratch, 'profile'),
},
})
const page = await app.firstWindow({ timeout })
pageErrors = []
page.on('pageerror', (error) => pageErrors.push(error.message))
page.on('console', (message) => {
if (message.type() === 'error') pageErrors.push(message.text())
})
const signIn = new URL('/__e2e/sign-in', proxy.origin)
signIn.searchParams.set('cookie', user.cookie)
signIn.searchParams.set('to', chatPath(user, title))
await page.goto(signIn.toString(), { waitUntil: 'commit', timeout })
await expect(composer(page)).toBeVisible({ timeout })
return page
}
const composer = (page: Page) => page.getByRole('textbox').last()
/** The dev app's error overlay, if it is up, with the errors the window reported. */
async function devOverlay(page: Page): Promise<{ text: string; consoleOnly: boolean } | null> {
const overlay = page.locator('nextjs-portal [data-nextjs-dialog]')
if ((await overlay.count()) === 0) return null
const text = await overlay
.first()
.innerText()
.catch(() => '')
return {
text: `${text}\n${pageErrors.join('\n')}`,
consoleOnly: /^\s*Console Error/.test(text),
}
}
/**
* Clicks `target`. When the dev app's error overlay intercepts the click, a console-error notice
* (dev-only chrome over a logged error) is dismissed and the click retried; a runtime error fails
* at once with its message instead of waiting out a blocked click.
*/
async function click(page: Page, target: Locator, timeout = 15_000): Promise<void> {
for (let attempt = 0; ; attempt++) {
try {
await target.click({ timeout: attempt === 0 ? Math.min(timeout, 5_000) : timeout })
return
} catch (error) {
const overlay = await devOverlay(page)
if (overlay && !overlay.consoleOnly)
throw new Error(`Next.js error overlay: ${overlay.text}`)
if (overlay) await page.keyboard.press('Escape')
else if (attempt > 0) throw error
}
}
}
/**
* Sends `message` and waits for its turn to reach Sim. Before the page hydrates, typing or the
* click can be lost: the Send button is missing or does nothing and the composer is not emptied,
* and only then is the message typed and sent again. Once the UI takes the message (the
* composer empties after the click), a turn that never reaches Sim is a lost send and fails.
*/
async function send(page: Page, message: string, timeout = 60_000): Promise<void> {
const since = Date.now()
const deadline = since + timeout
const sent = () =>
proxy
.seen(since)
.some((entry) => entry.method === 'POST' && entry.path === '/api/mothership/chat')
while (!sent()) {
if (Date.now() > deadline)
throw new Error(
`The UI never took the message: ${message} (errors: ${pageErrors.join(' | ')})`
)
if ((await composer(page).inputValue()) !== message) await composer(page).fill(message)
const clicked = await click(page, page.getByRole('button', { name: 'Send message' }), 5_000)
.then(() => true)
.catch((error: unknown) => {
if (String(error).includes('Next.js error overlay')) throw error
return false
})
if (!clicked) continue
const taken = await expect
.poll(() => composer(page).inputValue(), { timeout: 5_000 })
.toBe('')
.then(
() => true,
() => false
)
if (!taken) continue
await expect
.poll(sent, {
timeout: Math.max(deadline - Date.now(), 30_000),
message: `The UI took the message but its turn never reached Sim: ${message} (errors: ${pageErrors.join(' | ')})`,
})
.toBe(true)
}
}
/** Switches chats in-app, the way the sidebar does, without reloading the page. */
async function openChat(
page: Page,
user: SeededUser,
title: string,
timeout = 30_000
): Promise<void> {
await click(page, page.getByRole('link', { name: title }).first(), timeout)
await expect(page).toHaveURL(new RegExp(`${user.chats[title]}$`), { timeout })
}
/** The chat's first call as `status: error`, so a wrong terminal state says why. */
async function callState(chatId: string): Promise<string> {
const [call] = await db.toolCalls(chatId)
return call ? `${call.status}: ${call.error ?? ''}` : 'no call'
}
function writeFile(path: string, contents: string): string {
mkdirSync(dirname(path), { recursive: true })
writeFileSync(path, contents)
return path
}
/** A folder to import: a file, then a subfolder holding another file. */
function importSource(): string {
const root = join(scratch, 'Reports')
writeFile(join(root, 'a.txt'), 'first file')
writeFile(join(root, 'later', 'b.txt'), 'second file')
return root
}
/** The first request of a file's upload session. */
const isUploadStart = (method: string, path: string) =>
method === 'POST' && path === '/api/files/uploads'
const isFolderCreate = (method: string, path: string) =>
method === 'POST' && /^\/api\/workspaces\/[^/]+\/files\/folders$/.test(path)
const isDesktopClaim = (method: string, path: string) =>
method === 'POST' && path === '/api/desktop/tool/authorize'
/** The chat turn's response stream, as the chat view reads it. */
const isChatStream = (entry: { method: string; path: string }) =>
entry.method === 'POST' && entry.path === '/api/mothership/chat'
/** The background executor's report of a call's result. */
const isDesktopCompletion = (method: string, path: string) =>
method === 'POST' && path === DESKTOP_COMPLETION_PATH
/** A client tool's report of its own result. */
const isToolReport = (method: string, path: string) =>
method === 'POST' && path === '/api/copilot/confirm'
test('a browser call issued after the user switched chats fails as not started after the pickup grace', async () => {
const user = await db.seedUser(['Browser chat', 'Other chat'])
let issue!: () => void
const issued = new Promise<void>((resolve) => {
issue = resolve
})
let callId = ''
let issuedAt = 0
agent.script('[pickup-grace]', async (turn) => {
turn.text('Checking your browser tabs.')
await issued
callId = turn.toolCall({ toolName: 'browser_list_tabs', args: {} })
issuedAt = Date.now()
turn.pause()
})
const page = await openApp(user, 'Browser chat')
await send(page, '[pickup-grace] which tabs are open?')
await expect(page.getByText('Checking your browser tabs.')).toBeVisible({ timeout: 60_000 })
await openChat(page, user, 'Other chat')
issue()
await agent.waitForResume(() => Boolean(callId && agent.resultFor(callId)), 60_000)
const result = agent.resultFor(callId)
expect(result?.success).toBe(false)
expect(result?.data).toMatchObject({ notStarted: true })
const waited = (result?.at ?? 0) - issuedAt
expect(waited).toBeGreaterThanOrEqual(PICKUP_GRACE_MS - 1_000)
expect(waited).toBeLessThan(PICKUP_GRACE_MS + 20_000)
const [call] = await db.toolCalls(user.chats['Browser chat'])
expect(call).toMatchObject({ toolName: 'browser_list_tabs', status: 'failed', claimedBy: null })
})
test('leaving the chat does not end a local file read already under way', async () => {
const user = await db.seedUser(['Read chat', 'Other chat'])
const marker = generateId()
const file = writeFile(join(scratch, 'notes.txt'), `notes from disk ${marker}`)
let callId = ''
agent.script('[leave-read]', (turn) => {
callId = turn.toolCall({ toolName: 'read_local_file', args: { path: file } })
turn.pause()
})
const page = await openApp(user, 'Read chat')
// Visit the other chat once, so leaving for it later is a quick client-side switch.
await openChat(page, user, 'Other chat')
await openChat(page, user, 'Read chat')
const claim = proxy.hold(isDesktopClaim)
const since = Date.now()
await send(page, '[leave-read] read my notes')
await claim.arrival(ARRIVAL_MS, 'The read’s claim')
const turnStream = proxy.seen(since).find(isChatStream)
if (!turnStream) throw new Error('The turn’s stream never reached Sim')
await click(page, page.getByRole('link', { name: 'Other chat' }).first())
// The view let go of the turn's stream, the moment a read tied to that view would end.
// Electron gives the held claim 8 s, so this waits only a few.
await expect.poll(() => turnStream.clientClosedAt, { timeout: 6_000 }).toBeDefined()
claim.release()
await expect(page).toHaveURL(new RegExp(`${user.chats['Other chat']}$`), { timeout: 30_000 })
await agent.waitForResume(() => Boolean(callId && agent.resultFor(callId)), 60_000)
const result = agent.resultFor(callId)
expect(result?.success).toBe(true)
expect(JSON.stringify(result?.data)).toContain(marker)
const [call] = await db.toolCalls(user.chats['Read chat'])
expect(call).toMatchObject({ toolName: 'read_local_file', status: 'completed' })
})
test('an import keeps running across chat switches, and Stop from the reopened chat ends it', async () => {
const user = await db.seedUser(['Import chat', 'Other chat'])
const chatId = user.chats['Import chat']
const source = importSource()
agent.script('[stop-import]', (turn) => {
turn.toolCall({
toolName: 'import_local_files',
args: { path: source, targetWorkspaceId: user.workspaceId },
})
turn.pause()
})
const page = await openApp(user, 'Import chat')
const firstUpload = proxy.hold(isUploadStart)
await send(page, '[stop-import] import my reports')
await firstUpload.arrival(ARRIVAL_MS, 'The first upload')
await openChat(page, user, 'Other chat')
await openChat(page, user, 'Import chat')
const laterFolder = proxy.hold(isFolderCreate)
firstUpload.release()
// The import outlived both view changes: its first file landed.
await expect
.poll(
async () =>
`${(await db.workspaceFileNames(user.workspaceId)).join(',')} | ${await callState(chatId)}`,
{ timeout: 60_000 }
)
.toMatch(/^a\.txt /)
await laterFolder.arrival(ARRIVAL_MS, 'The next folder')
const stoppedAt = Date.now()
await click(page, page.getByRole('button', { name: 'Stop generation' }))
// Stop cancels the import's request in flight, which ends the import, and records the call as
// cancelled; the stopped import reports nothing that could contest that record.
await expect.poll(() => laterFolder.isAbandoned, { timeout: 15_000 }).toBe(true)
await expect.poll(() => callState(chatId), { timeout: 30_000 }).toMatch(/^cancelled/)
laterFolder.release()
expect(proxy.seen(stoppedAt).filter((entry) => isToolReport(entry.method, entry.path))).toEqual(
[]
)
const [call] = await db.toolCalls(chatId)
expect(call).toMatchObject({ toolName: 'import_local_files', status: 'cancelled' })
expect(await db.workspaceFolderNames(user.workspaceId)).not.toContain('later')
expect(await db.workspaceFileNames(user.workspaceId)).toEqual(['a.txt'])
})
/** An import whose first upload the proxy holds until released, as a large file's would take. */
async function slowImport(user: SeededUser, title: string, marker: string) {
const source = importSource()
let callId = ''
agent.script(marker, (turn) => {
callId = turn.toolCall({
toolName: 'import_local_files',
args: { path: source, targetWorkspaceId: user.workspaceId },
})
turn.pause()
})
const page = await openApp(user, title)
const firstUpload = proxy.hold(isUploadStart)
await send(page, `${marker} import my reports`)
await firstUpload.arrival(ARRIVAL_MS, 'The first upload')
return { page, firstUpload, callId: () => callId }
}
test('an import that runs longer than the default tool budget still completes', async () => {
test.setTimeout(420_000)
const user = await db.seedUser(['Long import chat', 'Other chat'])
const chatId = user.chats['Long import chat']
const { page, firstUpload, callId } = await slowImport(
user,
'Long import chat',
'[long-import]'
)
// The import is alive and working past the 60 s default budget and its 30 s grace, while the
// user is in another chat: its lease is renewed by the import, not by the chat view.
await openChat(page, user, 'Other chat')
await page.waitForTimeout(LONG_IMPORT_MS)
expect(agent.resultFor(callId())).toBeUndefined()
firstUpload.release()
await agent.waitForResume(() => Boolean(agent.resultFor(callId())), 120_000)
const result = agent.resultFor(callId())
expect(JSON.stringify(result?.data)).not.toContain('outcomeUnknown')
expect(result?.success).toBe(true)
await expect
.poll(() => db.workspaceFileNames(user.workspaceId), { timeout: 30_000 })
.toEqual(['a.txt', 'b.txt'])
expect(await callState(chatId)).toMatch(/^completed/)
})
test('a long import whose window crashed settles as outcome unknown about one lease later', async () => {
test.setTimeout(420_000)
const user = await db.seedUser(['Crashing import chat'])
const { callId } = await slowImport(user, 'Crashing import chat', '[crash-import]')
await sleep(LONG_IMPORT_MS)
expect(agent.resultFor(callId())).toBeUndefined()
// A crash reports nothing on its way out, unlike a closed window: only the lapse of the
// lease the page was renewing tells Sim the import is gone.
const crashedAt = Date.now()
await app?.evaluate(({ webContents }) => {
for (const contents of webContents.getAllWebContents())
if (contents.getURL().includes('/workspace/')) contents.forcefullyCrashRenderer()
})
await agent.waitForResume(() => Boolean(agent.resultFor(callId())), 150_000)
const result = agent.resultFor(callId())
expect(result?.data).toMatchObject({ outcomeUnknown: true })
expect((result?.at ?? 0) - crashedAt).toBeLessThan(LEASE_MS + 20_000)
})
test('signing out ends a desktop tool still running', async () => {
const user = await db.seedUser(['Import chat'])
const chatId = user.chats['Import chat']
const source = importSource()
agent.script('[sign-out-import]', (turn) => {
turn.toolCall({
toolName: 'import_local_files',
args: { path: source, targetWorkspaceId: user.workspaceId },
})
turn.pause()
})
const page = await openApp(user, 'Import chat')
const firstUpload = proxy.hold(isUploadStart)
await send(page, '[sign-out-import] import my reports')
await firstUpload.arrival(ARRIVAL_MS, 'The first upload')
// The desktop reloads into the login page once signing out completes, which would end any
// tool; holding the sign-out shows the tool ends at sign-out itself, not at that reload.
const signOut = proxy.hold((method, path) => method === 'POST' && path === '/api/auth/sign-out')
await click(page, page.getByRole('button', { name: user.name }))
await click(page, page.getByRole('menuitem', { name: 'Sign out' }))
await signOut.arrival(ARRIVAL_MS, 'The sign-out')
await expect.poll(() => firstUpload.isAbandoned, { timeout: 15_000 }).toBe(true)
const abandonedAt = Date.now()
signOut.release()
// The reload into the login page replaces the document, so nothing of the import can run after.
await expect(page).toHaveURL(/\/login/, { timeout: 30_000 })
firstUpload.release()
// Between sign-out and that reload the cancelled import sent nothing more, not even a report.
const toolRequests = proxy
.seen(abandonedAt)
.filter(
(entry) =>
isUploadStart(entry.method, entry.path) ||
isFolderCreate(entry.method, entry.path) ||
isToolReport(entry.method, entry.path)
)
expect(toolRequests).toEqual([])
expect(await callState(chatId)).toMatch(/^running/)
expect(await db.workspaceFileNames(user.workspaceId)).toEqual([])
expect(await db.workspaceFolderNames(user.workspaceId)).not.toContain('later')
})
/** Sends `message` and returns the `toolName` call Sim persisted for it, held for approval. */
async function gatedCall(page: Page, chatId: string, message: string, toolName: string) {
await send(page, message)
await expect
.poll(async () => (await db.toolCalls(chatId)).some((call) => call.toolName === toolName), {
timeout: 60_000,
})
.toBe(true)
const call = (await db.toolCalls(chatId)).find((entry) => entry.toolName === toolName)
if (!call) throw new Error(`Missing ${toolName} call`)
expect(call.status).toBe('pending')
expect(call.permissionRequestedAt).not.toBeNull()
return call
}
test("a local read awaiting the user's approval cannot be claimed", async () => {
const user = await db.seedUser(['Approval chat'])
const secret = generateId()
const file = writeFile(join(scratch, 'secret.txt'), `private ${secret}`)
agent.script('[unapproved-read]', (turn) => {
turn.toolCall({
toolName: 'read_local_file',
args: { path: file },
status: 'awaiting_approval',
})
turn.pause()
})
const page = await openApp(user, 'Approval chat')
const chatId = user.chats['Approval chat']
const read = await gatedCall(
page,
chatId,
'[unapproved-read] read my secret',
'read_local_file'
)
// A renderer acting on a call the user has not allowed (a replayed event) is refused.
const response = await page.evaluate(
(toolCallId) =>
(globalThis as DesktopWindow).simDesktop.localFiles?.({ operation: 'read', toolCallId }),
read.toolCallId
)
expect(response?.ok).toBe(false)
expect(JSON.stringify(response)).not.toContain(secret)
const [after] = await db.toolCalls(chatId)
expect(after).toMatchObject({ status: 'pending', claimedBy: null })
})
test("a terminal command awaiting the user's approval cannot be claimed", async () => {
const user = await db.seedUser(['Approval chat'])
const command = `touch '${join(scratch, 'terminal-ran')}'`
agent.script('[unapproved-run]', (turn) => {
turn.toolCall({ toolName: 'terminal', args: { operation: 'run', command } })
turn.pause()
})
const page = await openApp(user, 'Approval chat')
const chatId = user.chats['Approval chat']
const run = await gatedCall(page, chatId, '[unapproved-run] run a command', 'terminal')
// The guard is the claim: Sim must not hand an unapproved command to the desktop, so the call
// stays pending and unclaimed. Whether a handed-over command then runs depends on a terminal
// being open for the chat, which this test does not set up, so it checks the claim only.
const response = await page.evaluate(
({ toolCallId, command }) =>
(globalThis as DesktopWindow).simDesktop.terminal
.executeTool(toolCallId, 'run', { command }, 'unapproved-e2e')
.catch((error: unknown) => ({ ok: false, error: String(error) })),
{ toolCallId: run.toolCallId, command }
)
expect(response).toMatchObject({ ok: false })
const [after] = await db.toolCalls(chatId)
expect(after).toMatchObject({ status: 'pending', claimedBy: null })
})
test('a claim that reaches Sim after the user pressed Stop is refused', async () => {
const user = await db.seedUser(['Stop chat'])
const file = writeFile(join(scratch, 'notes.txt'), 'notes')
agent.script('[stopped-claim]', (turn) => {
turn.toolCall({ toolName: 'read_local_file', args: { path: file } })
turn.pause()
})
const page = await openApp(user, 'Stop chat')
const chatId = user.chats['Stop chat']
// Delivered to Sim on release even should Electron have given up on it meanwhile.
const lateClaim = proxy.hold(isDesktopClaim, { deliverIfAbandoned: true })
await send(page, '[stopped-claim] read my secret')
const claim = await lateClaim.arrival(ARRIVAL_MS, 'The read’s claim')
await click(page, page.getByRole('button', { name: 'Stop generation' }))
// Stop settles the call nobody has claimed yet as never started.
await expect.poll(() => callState(chatId), { timeout: 30_000 }).toMatch(/^cancelled/)
lateClaim.release()
// The claim held across Stop reaches Sim after it and is refused, and so is a replay of it.
await expect.poll(() => claim.status, { timeout: 15_000 }).toBe(410)
const [call] = await db.toolCalls(chatId)
const replay = await page.evaluate(
(toolCallId) =>
(globalThis as DesktopWindow).simDesktop.localFiles?.({ operation: 'read', toolCallId }),
call.toolCallId
)
expect(replay?.ok).toBe(false)
const answered = () =>
proxy.seen(claim.at, '/api/desktop/tool/authorize').filter((entry) => entry.status)
await expect.poll(() => answered().length, { timeout: 15_000 }).toBeGreaterThan(0)
for (const entry of answered()) expect(entry.status).toBe(410)
const [after] = await db.toolCalls(chatId)
expect(after).toMatchObject({ status: 'cancelled', claimedBy: null })
})
test('where Sim cannot run the background executor, the app stays dormant and a desktop round trip runs in the chat view', async () => {
const user = await db.seedUser(['Round trip'])
const marker = generateId()
const file = writeFile(join(scratch, 'plan.txt'), `plan ${marker}`)
const monitor = new RedisMonitor(sim.redisUrl)
await monitor.start()
try {
let callId = ''
agent.script(
'[round-trip]',
(turn) => {
turn.text('Reading the plan.')
callId = turn.toolCall({ toolName: 'read_local_file', args: { path: file } })
turn.pause()
},
(resume, turn) => {
const result = resume.results.find((entry) => entry.callId === callId)
turn.complete(
JSON.stringify(result?.data).includes(marker) ? 'The plan says go.' : 'Could not read.'
)
}
)
const since = Date.now()
const page = await openApp(user, 'Round trip')
await send(page, '[round-trip] what does my plan say?')
// Each layer is checked on its own (soft), so a regression shows every layer it reaches:
// the round trip, the turn's binding, the routes the app calls, and the doorbell.
await expect
.soft(page.getByText('The plan says go.'), 'foreground round trip')
.toBeVisible({ timeout: 60_000 })
expect.soft(agent.resultFor(callId)?.success, 'read result').toBe(true)
// The app registers once signed out (refused) and again on sign-in.
const registeredSignedIn = () =>
proxy
.seen(since, REGISTRATION_PATH)
.some((entry) => entry.method === 'POST' && entry.status === 200)
await expect.poll(registeredSignedIn, { timeout: 30_000 }).toBe(true)
const runs = await db.runs(user.chats['Round trip'])
expect(runs.length).toBeGreaterThan(0)
expect
.soft(
runs.map((run) => run.desktopDeviceId),
'turn binding'
)
.toEqual(runs.map(() => null))
const [call] = await db.toolCalls(user.chats['Round trip'])
expect
.soft(call, 'foreground call')
.toMatchObject({ toolName: 'read_local_file', status: 'completed' })
expect.soft(call?.persistSeq, 'persist order').not.toBeNull()
// Only registration and the foreground claim: no inbox, doorbell stream, executor claim,
// lease or completion. The workspace's activity poll is Sim's page, not the app: this Sim
// has Redis, so its page asks.
const desktopRoutes = new Set(proxy.seen(since, '/api/desktop/').map((entry) => entry.path))
desktopRoutes.delete(REGISTRATION_PATH)
desktopRoutes.delete('/api/desktop/activity')
expect
.soft(desktopRoutes, 'desktop routes the app called')
.toEqual(new Set(['/api/desktop/tool/authorize']))
expect(monitor.lines.length).toBeGreaterThan(0)
expect.soft(monitor.publishesTo('desktop:inbox'), 'doorbell').toEqual([])
} finally {
monitor.stop()
}
})
test('in the background, a call issued after the user switched chats runs on the desktop', async () => {
proxy.rewriteAnswer(REGISTRATION_PATH, undefined)
const user = await db.seedUser(['Background chat', 'Other chat'])
const marker = generateId()
const file = writeFile(join(scratch, 'notes.txt'), `notes from disk ${marker}`)
let issue!: () => void
const issued = new Promise<void>((resolve) => {
issue = resolve
})
let callId = ''
let issuedAt = 0
agent.script('[background-read]', async (turn) => {
turn.text('Reading your notes.')
await issued
callId = turn.toolCall({ toolName: 'read_local_file', args: { path: file } })
issuedAt = Date.now()
turn.pause()
})
const page = await openApp(user, 'Background chat')
await send(page, '[background-read] read my notes')
await expect(page.getByText('Reading your notes.')).toBeVisible({ timeout: 60_000 })
await openChat(page, user, 'Other chat')
issue()
await agent.waitForResume(() => Boolean(callId && agent.resultFor(callId)), 60_000)
const result = agent.resultFor(callId)
expect(result?.success).toBe(true)
expect(JSON.stringify(result?.data)).toContain(marker)
// No pickup grace: the desktop, not the chat view the user left, ran it.
expect((result?.at ?? 0) - issuedAt).toBeLessThan(PICKUP_GRACE_MS)
const chatId = user.chats['Background chat']
const runs = await db.runs(chatId)
expect(runs.some((run) => run.desktopDeviceId !== null)).toBe(true)
const [call] = await db.toolCalls(chatId)
expect(call).toMatchObject({ toolName: 'read_local_file', status: 'completed' })
expect(proxy.seen(issuedAt, '/api/desktop/tool/authorize')).toEqual([])
})
test('in the background, a result reported across a network cut reaches the agent exactly once', async () => {
proxy.rewriteAnswer(REGISTRATION_PATH, undefined)
const user = await db.seedUser(['Cut chat'])
const chatId = user.chats['Cut chat']
const marker = generateId()
const file = writeFile(join(scratch, 'notes.txt'), `notes from disk ${marker}`)
let callId = ''
agent.script('[network-cut]', (turn) => {
callId = turn.toolCall({ toolName: 'read_local_file', args: { path: file } })
turn.pause()
})
const page = await openApp(user, 'Cut chat')
// Reaches Sim on release although the cut made the app give up on it, as a report already on
// the wire would: the app cannot know it landed, so it reports again once back online.
const completion = proxy.hold(isDesktopCompletion, { deliverIfAbandoned: true })
const answers = proxy.recordAnswers(DESKTOP_COMPLETION_PATH)
await send(page, '[network-cut] read my notes')
await completion.arrival(ARRIVAL_MS, 'The result report')
proxy.cutNetwork()
await expect.poll(() => completion.isAbandoned, { timeout: 15_000 }).toBe(true)
completion.release()
await expect.poll(() => callState(chatId), { timeout: 30_000 }).toMatch(/^completed/)
// The late report alone resumed the agent while the app was still offline.
await agent.waitForResume(() => Boolean(agent.resultFor(callId)), 60_000)
const restoredAt = Date.now()
proxy.restoreNetwork()
// Back online, the app reopens its doorbell and reports the result again.
await expect
.poll(() => proxy.seen(restoredAt, '/api/desktop/inbox/stream').length, { timeout: 60_000 })
.toBeGreaterThan(0)
const retried = () =>
proxy
.seen(restoredAt)
.filter((entry) => isDesktopCompletion(entry.method, entry.path) && entry.status)
await expect.poll(() => retried().length, { timeout: 60_000 }).toBeGreaterThan(0)
for (const entry of retried()) expect(entry.status).toBeLessThan(300)
await proxy.settled(30_000)
// Sim had already recorded the late report, so every retry the app heard back from is a no-op.
expect(answers.length).toBeGreaterThan(0)
expect(answers).toEqual(answers.map(() => expect.objectContaining({ outcome: 'duplicate' })))
const delivered = agent.resumes.filter((resume) =>
resume.results.some((entry) => entry.callId === callId)
)
expect(delivered).toHaveLength(1)
expect(JSON.stringify(agent.resultFor(callId)?.data)).toContain(marker)
const [call] = await db.toolCalls(chatId)
expect(call).toMatchObject({ toolName: 'read_local_file', status: 'completed' })
})
})
apps/desktop/e2e/executor-sim.ts
+523
View File
@@ -0,0 +1,523 @@
import { execFileSync } from 'node:child_process'
import { createHash } from 'node:crypto'
import { mkdtempSync, readFileSync } from 'node:fs'
import { createServer, type IncomingMessage, type Server, type ServerResponse } from 'node:http'
import type { Socket } from 'node:net'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { fileURLToPath } from 'node:url'
import {
type ElectronApplication,
_electron as electron,
expect,
type Page,
} from '@playwright/test'
/**
* A fixture Sim that speaks the background executor's device protocol (register, inbox,
* doorbell, claim, lease, complete, import) the way Sim's routes do, and the helpers the
* executor's Electron suites share.
*/
const DESKTOP_DIR = fileURLToPath(new URL('..', import.meta.url))
export const WORKSPACE = 'ws-e2e'
const LEASE_RENEW_MS = 1_000
export const RECONCILE_MS = 2_000
type CallStatus = 'pending' | 'awaiting_approval' | 'running' | 'completed' | 'failed' | 'cancelled'
export interface Completion {
status: string
message: string
data?: Record<string, unknown>
outcome: 'recorded' | 'duplicate' | 'superseded'
at: number
}
export interface FixtureCall {
toolCallId: string
toolName: string
args: Record<string, unknown>
chatId: string
deviceId: string
status: CallStatus
issuedAt: number
claimedAt?: number
token?: string
claims: number
renewals: number
completions: Completion[]
acknowledged: boolean
}
export interface ImportedEntry {
toolCallId: string
kind: string
sourceName: string
relativePath: string
sha256?: string
bytes?: number
}
export function sha256(bytes: Buffer): string {
return createHash('sha256').update(bytes).digest('hex')
}
export class FixtureSim {
readonly calls = new Map<string, FixtureCall>()
readonly devices = new Map<string, { name: string; platform: string }>()
readonly requests: string[] = []
readonly hits = new Map<string, number>()
readonly streams = new Map<string, Set<ServerResponse>>()
readonly imported: ImportedEntry[] = []
enabled = true
/** Connections reset while the network was cut. */
droppedWhileOffline = 0
private offline = false
private readonly sockets = new Set<Socket>()
private server: Server | null = null
origin = ''
private nextToken = 0
async start(): Promise<void> {
this.server = createServer((request, response) => {
void this.handle(request, response)
})
this.server.on('connection', (socket) => {
if (this.offline) {
this.droppedWhileOffline += 1
socket.destroy()
return
}
this.sockets.add(socket)
socket.on('close', () => this.sockets.delete(socket))
})
await new Promise<void>((resolve) => this.server?.listen(0, '127.0.0.1', resolve))
const address = this.server.address()
if (!address || typeof address === 'string') throw new Error('Missing fixture address')
this.origin = `http://127.0.0.1:${address.port}`
}
async stop(): Promise<void> {
for (const streams of this.streams.values()) for (const stream of streams) stream.destroy()
this.server?.closeAllConnections()
await new Promise<void>((resolve) => this.server?.close(() => resolve()))
}
/**
* Cuts the network between the app and Sim: every open connection drops, the doorbell stream and
* kept-alive request sockets included, and every new one is reset until {@link reconnect}.
*/
disconnect(): void {
this.offline = true
for (const socket of this.sockets) socket.destroy()
}
reconnect(): void {
this.offline = false
}
reset(): void {
this.calls.clear()
this.devices.clear()
this.requests.length = 0
this.hits.clear()
this.imported.length = 0
this.enabled = true
this.offline = false
this.droppedWhileOffline = 0
}
/** Persists a call for a device and rings its doorbell, as Sim's pre-persist and offer do. */
issue(
deviceId: string,
chatId: string,
toolName: string,
args: Record<string, unknown>,
status: CallStatus = 'pending'
): string {
const toolCallId = `call-${this.calls.size + 1}-${toolName}`
this.calls.set(toolCallId, {
toolCallId,
toolName,
args,
chatId,
deviceId,
status,
issuedAt: Date.now(),
claims: 0,
renewals: 0,
completions: [],
acknowledged: false,
})
this.ring(deviceId, status === 'awaiting_approval' ? 'approval' : 'call')
return toolCallId
}
/** Stop from any surface: Sim settles the call and tells the device to cancel. */
stopCall(toolCallId: string): void {
const call = this.requireCall(toolCallId)
call.status = 'cancelled'
this.ring(call.deviceId, 'cancel')
}
approve(toolCallId: string): void {
const call = this.requireCall(toolCallId)
call.status = 'pending'
this.ring(call.deviceId, 'approval')
}
/** The user declines a held call: Sim settles it without the device and rings again. */
decline(toolCallId: string): void {
const call = this.requireCall(toolCallId)
call.status = 'cancelled'
this.ring(call.deviceId, 'approval')
}
requireCall(toolCallId: string): FixtureCall {
const call = this.calls.get(toolCallId)
if (!call) throw new Error(`No fixture call ${toolCallId}`)
return call
}
ring(deviceId: string, reason: string): void {
for (const stream of this.streams.get(deviceId) ?? []) {
stream.write(`event: inbox_changed\ndata: ${JSON.stringify({ reason })}\n\n`)
}
}
private async raw(request: IncomingMessage): Promise<Buffer> {
const chunks: Buffer[] = []
for await (const chunk of request) chunks.push(Buffer.from(chunk))
return Buffer.concat(chunks)
}
private async body(request: IncomingMessage): Promise<Record<string, unknown>> {
const text = (await this.raw(request)).toString()
return text ? (JSON.parse(text) as Record<string, unknown>) : {}
}
/** Stores one entry of a claimed, running import, as Sim's import route does. */
private async importEntry(
url: URL,
request: IncomingMessage,
response: ServerResponse
): Promise<void> {
const query = Object.fromEntries(url.searchParams)
const call = this.calls.get(query.toolCallId ?? '')
if (
request.method !== 'PUT' ||
!call ||
call.deviceId !== query.deviceId ||
call.toolName !== 'import_local_files' ||
call.token !== request.headers['x-sim-execution-token'] ||
call.status !== 'running'
) {
this.json(response, 404, { error: 'Desktop import not found' })
return
}
// As Sim's route does: a file must declare its length, and arrive whole.
if (query.kind === 'file' && request.headers['content-length'] === undefined) {
this.json(response, 411, { error: 'A file import must declare its length' })
return
}
const content = await this.raw(request)
if (query.kind === 'file' && content.length !== Number(request.headers['content-length'])) {
this.json(response, 400, { error: 'The file did not arrive whole' })
return
}
const entry: ImportedEntry = {
toolCallId: call.toolCallId,
kind: query.kind ?? '',
sourceName: query.sourceName ?? '',
relativePath: query.relativePath ?? '',
...(query.kind === 'file' ? { sha256: sha256(content), bytes: content.length } : {}),
}
this.imported.push(entry)
this.json(response, 200, {
id: `entry-${this.imported.length}`,
name: entry.relativePath.split('/').at(-1) || entry.sourceName,
})
}
private json(response: ServerResponse, status: number, body: unknown): void {
response.writeHead(status, { 'Content-Type': 'application/json' })
response.end(JSON.stringify(body))
}
private signedIn(request: IncomingMessage): boolean {
return request.headers.cookie?.includes('better-auth.session_token=fixture') ?? false
}
private async handle(request: IncomingMessage, response: ServerResponse): Promise<void> {
const url = new URL(request.url ?? '/', this.origin)
const path = url.pathname
this.requests.push(`${request.method} ${path}`)
if (path === '/api/auth/get-session') {
this.json(
response,
200,
this.signedIn(request) ? { user: { id: 'user-e2e' }, session: { id: 'session-e2e' } } : null
)
return
}
if (path.startsWith('/api/desktop/') && !this.signedIn(request)) {
this.json(response, 401, { error: 'Unauthorized' })
return
}
if (path === '/api/desktop/devices' && request.method === 'POST') {
const body = await this.body(request)
if (this.enabled) {
this.devices.set(String(body.deviceId), {
name: String(body.name),
platform: String(body.platform),
})
}
this.json(response, 200, {
enabled: this.enabled,
protocolVersion: 1,
leaseMs: 60_000,
leaseRenewMs: LEASE_RENEW_MS,
reconcileMs: RECONCILE_MS,
})
return
}
const deviceId = url.searchParams.get('deviceId')
if (path === '/api/desktop/inbox/stream') {
if (!deviceId || !this.devices.has(deviceId)) {
this.json(response, 401, { error: 'Unregistered' })
return
}
response.writeHead(200, { 'Content-Type': 'text/event-stream', 'Cache-Control': 'no-cache' })
response.write(': connected\n\n')
const streams = this.streams.get(deviceId) ?? new Set<ServerResponse>()
streams.add(response)
this.streams.set(deviceId, streams)
request.on('close', () => streams.delete(response))
return
}
if (path === '/api/desktop/inbox') {
if (!deviceId || !this.devices.has(deviceId)) {
this.json(response, 401, { error: 'Unregistered' })
return
}
const items = [...this.calls.values()]
.filter((call) => call.deviceId === deviceId)
.flatMap((call) => {
if (call.status === 'pending' && !call.token)
return [
{
kind: 'call',
toolCallId: call.toolCallId,
toolName: call.toolName,
chatId: call.chatId,
workspaceId: WORKSPACE,
createdAt: new Date(call.issuedAt).toISOString(),
},
]
if (call.status === 'awaiting_approval')
return [
{
kind: 'approval_needed',
toolCallId: call.toolCallId,
toolName: call.toolName,
chatId: call.chatId,
chatTitle: 'Fix CI',
workspaceId: WORKSPACE,
summary: String((call.args.args as { command?: string } | undefined)?.command),
},
]
if (call.token && call.status === 'cancelled' && !call.acknowledged)
return [{ kind: 'cancel', toolCallId: call.toolCallId }]
return []
})
this.json(response, 200, { items })
return
}
if (path === '/api/desktop/tool/import') {
await this.importEntry(url, request, response)
return
}
if (path.startsWith('/api/desktop/tool/')) {
const body = await this.body(request)
const call = this.calls.get(String(body.toolCallId))
if (!call || call.deviceId !== body.deviceId) {
this.json(response, 404, { error: 'Desktop tool call not found' })
return
}
if (path === '/api/desktop/tool/claim') {
if (call.status !== 'pending' || call.token) {
this.json(response, 404, { error: 'This call is no longer waiting for this device' })
return
}
call.claims += 1
call.status = 'running'
call.claimedAt = Date.now()
call.token = `token-${++this.nextToken}`
this.json(response, 200, {
toolName: call.toolName,
args: call.args,
chatId: call.chatId,
workspaceId: WORKSPACE,
executionToken: call.token,
})
return
}
if (body.executionToken !== call.token) {
this.json(response, 404, { error: 'Desktop tool call not found' })
return
}
if (path === '/api/desktop/tool/lease') {
call.renewals += 1
if (call.status !== 'running') {
this.json(response, 410, { error: 'This call was stopped or settled. Stop running it.' })
return
}
this.json(response, 200, { renewed: true })
return
}
if (path === '/api/desktop/tool/complete') {
const status = String(body.status)
const settled =
status === 'success' ? 'completed' : status === 'cancelled' ? 'cancelled' : 'failed'
const outcome =
call.status === 'running'
? 'recorded'
: call.completions.length > 0
? 'duplicate'
: 'superseded'
if (outcome === 'recorded') call.status = settled
if (call.status === 'cancelled') call.acknowledged = true
call.completions.push({
status,
message: String(body.message ?? ''),
...(body.data && typeof body.data === 'object'
? { data: body.data as Record<string, unknown> }
: {}),
outcome,
at: Date.now(),
})
this.json(response, 200, {
outcome,
status: call.status === 'running' ? settled : call.status,
})
return
}
}
if (path === '/hit') {
const chat = url.searchParams.get('chat') ?? ''
this.hits.set(chat, (this.hits.get(chat) ?? 0) + 1)
this.json(response, 200, { count: this.hits.get(chat) })
return
}
if (path === '/session-ended') {
// Another account signing in replaces the session: the old session cookie is cleared.
response.writeHead(200, {
'Content-Type': 'text/html',
'Set-Cookie': 'better-auth.session_token=; Max-Age=0; HttpOnly; SameSite=Lax; Path=/',
})
response.end('<!doctype html><title>Signed in elsewhere</title>')
return
}
response.writeHead(200, {
'Content-Type': 'text/html',
'Set-Cookie': 'better-auth.session_token=fixture; HttpOnly; SameSite=Lax; Path=/',
})
response.end(
path === '/counter'
? `<!doctype html><title>Counter ${url.searchParams.get('chat')}</title>
<button onclick="fetch('/hit?chat=${url.searchParams.get('chat')}').then(r => r.json()).then(b => { document.getElementById('count').textContent = String(b.count) })">Count visit</button>
<p id="count">0</p>`
: `<!doctype html><title>Sim fixture</title><h1>${path}</h1>`
)
}
}
/**
* Launches the app against the fixture Sim. Its shells are zsh with an empty config directory, so
* what they do is the app's and not this machine's: a developer's `.zshrc` can stop at a question
* (oh-my-zsh asks before updating) and hold the first prompt for as long as nobody answers it.
* The terminal's handling of slow and stalled startup files has its own tests.
*/
export async function launch(
sim: FixtureSim,
userData: string,
env: Record<string, string> = {}
): Promise<{ app: ElectronApplication; window: Page }> {
const app = await electron.launch({
args: [process.env.SIM_DESKTOP_E2E_MAIN ?? '.'],
cwd: DESKTOP_DIR,
env: {
...process.env,
SHELL: '/bin/zsh',
ZDOTDIR: mkdtempSync(join(tmpdir(), 'sim-e2e-zdotdir-')),
SIM_DESKTOP_ORIGIN: sim.origin,
SIM_DESKTOP_USER_DATA: userData,
...env,
},
})
// A dialog listener stops Playwright auto-dismissing page dialogs, so the desktop's own handling
// decides their outcome exactly as it does in production.
const leaveDialogsToDesktop = (page: Page) => page.on('dialog', () => {})
app.context().pages().forEach(leaveDialogsToDesktop)
app.context().on('page', leaveDialogsToDesktop)
const window = await app.firstWindow()
return { app, window }
}
/** The device this app registered, once it has opened its doorbell. */
export async function registeredDevice(
sim: FixtureSim,
knownDevices: ReadonlySet<string> = new Set()
): Promise<string> {
let deviceId: string | undefined
await expect
.poll(
() => {
deviceId = [...sim.devices.keys()].find(
(id) => !knownDevices.has(id) && (sim.streams.get(id)?.size ?? 0) > 0
)
return deviceId
},
{ timeout: 30_000 }
)
.toBeTruthy()
return deviceId as string
}
export async function settled(
sim: FixtureSim,
toolCallId: string,
timeout = 60_000
): Promise<Completion> {
await expect
.poll(() => sim.requireCall(toolCallId).completions.length, { timeout })
.toBeGreaterThan(0)
return sim.requireCall(toolCallId).completions[0] as Completion
}
export function refFor(outline: string, name: string): number {
const match = outline
.split('\n')
.find((line) => line.includes(`"${name}"`) && /\[ref=\d+\]/.test(line))
?.match(/\[ref=(\d+)\]/)
if (!match) throw new Error(`No reference for ${name}: ${outline}`)
return Number(match[1])
}
export function processRunning(pattern: string): boolean {
try {
execFileSync('pgrep', ['-f', pattern])
return true
} catch {
return false
}
}
export function readFileSafe(path: string): string {
try {
return readFileSync(path, 'utf8')
} catch {
return ''
}
}
apps/desktop/e2e/fixtures/live-sim.ts
+794
View File
@@ -0,0 +1,794 @@
import { createHmac } from 'node:crypto'
import {
createServer,
request as httpRequest,
type IncomingHttpHeaders,
type IncomingMessage,
type Server,
type ServerResponse,
} from 'node:http'
import { connect, type Socket } from 'node:net'
import type { Duplex } from 'node:stream'
import { sleep } from '@sim/utils/helpers'
import { generateId, generateShortId } from '@sim/utils/id'
import { toArray } from '@sim/utils/object'
import postgres from 'postgres'
/**
* A real local Sim app for the desktop E2E suite: the Electron app talks to it through a
* recording proxy, and Sim talks to a scripted agent standing in for the Mothership worker at
* `SIM_AGENT_API_URL`. Both sides of the desktop tools are the shipped code; only the model's
* decisions are scripted.
*
* Start Sim (for example `next dev`) with `NEXT_PUBLIC_APP_URL` and `BETTER_AUTH_URL` set to the
* proxy's origin, `SIM_AGENT_API_URL` set to the agent's, `REDIS_URL`, and
* `COPILOT_TOOL_PERMISSIONS_ENABLED=true`, then provide the environment `liveSimConfig` reads.
*/
export interface LiveSimConfig {
/** Where Sim itself listens. */
upstream: string
/** The proxy's port: the origin Sim and Electron both use. */
proxyPort: number
/** The scripted agent's port. */
agentPort: number
databaseUrl: string
redisUrl: string
/** Sim's `BETTER_AUTH_SECRET`, to sign the session cookie of a seeded session. */
authSecret: string
}
/** The live Sim this run was given, or why the suite is skipped. */
export function liveSimConfig(): LiveSimConfig | string {
const env = process.env
const missing = [
'SIM_DESKTOP_E2E_SIM_URL',
'SIM_DESKTOP_E2E_PROXY_PORT',
'SIM_DESKTOP_E2E_AGENT_PORT',
'SIM_DESKTOP_E2E_DATABASE_URL',
'SIM_DESKTOP_E2E_REDIS_URL',
'SIM_DESKTOP_E2E_AUTH_SECRET',
].filter((name) => !env[name])
if (missing.length > 0) return `Needs a local Sim app: set ${missing.join(', ')}`
const databaseUrl = env.SIM_DESKTOP_E2E_DATABASE_URL ?? ''
if (!/(?:^|[_-])test$/i.test(new URL(databaseUrl).pathname.slice(1)))
return 'SIM_DESKTOP_E2E_DATABASE_URL must name a dedicated test database'
return {
upstream: env.SIM_DESKTOP_E2E_SIM_URL ?? '',
proxyPort: Number(env.SIM_DESKTOP_E2E_PROXY_PORT),
agentPort: Number(env.SIM_DESKTOP_E2E_AGENT_PORT),
databaseUrl,
redisUrl: env.SIM_DESKTOP_E2E_REDIS_URL ?? '',
authSecret: env.SIM_DESKTOP_E2E_AUTH_SECRET ?? '',
}
}
async function readBody(request: IncomingMessage): Promise<Buffer> {
const chunks: Buffer[] = []
for await (const chunk of request)
chunks.push(Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk))
return Buffer.concat(chunks)
}
function listen(server: Server, port: number): Promise<void> {
return new Promise((resolve, reject) => {
server.once('error', reject)
server.listen(port, '127.0.0.1', () => {
server.off('error', reject)
resolve()
})
})
}
function close(server: Server): Promise<void> {
server.closeAllConnections()
return new Promise((resolve) => server.close(() => resolve()))
}
/** One request the proxy saw, from the renderer or Electron's main process. */
interface ProxiedRequest {
method: string
path: string
at: number
status?: number
/** When the client gave up on the response before it finished (a reader that let go). */
clientClosedAt?: number
}
/**
* A request the proxy holds before it reaches Sim, so a test can act while a tool's request is
* in flight. A request whose client gave up while held is dropped, never forwarded, as a
* cancelled request on a real network would be.
*/
class HeldRequest {
readonly reached: Promise<ProxiedRequest>
/** Settles once the client gives up on the held request (it cancelled it). */
readonly abandoned: Promise<void>
isAbandoned = false
private reach!: (request: ProxiedRequest) => void
private abandon!: () => void
private releaseHeld: (() => void) | undefined
private released = false
constructor(
readonly matches: (method: string, path: string) => boolean,
/** Deliver the request to Sim on release even if its client gave up, as a late request would arrive. */
readonly deliverIfAbandoned = false
) {
this.reached = new Promise((resolve) => {
this.reach = resolve
})
this.abandoned = new Promise((resolve) => {
this.abandon = resolve
})
}
/** Called by the proxy when the matching request arrives; resolves when the test releases it. */
hold(request: ProxiedRequest, response: ServerResponse): Promise<boolean> {
response.once('close', () => {
if (response.writableFinished || this.released) return
this.isAbandoned = true
this.abandon()
})
this.reach(request)
return new Promise((resolve) => {
this.releaseHeld = () => resolve(!this.isAbandoned)
if (this.released) this.releaseHeld()
})
}
/** The held request, once it arrives; fails if it has not within `timeoutMs`. */
async arrival(timeoutMs: number, label: string): Promise<ProxiedRequest> {
let timer: ReturnType<typeof setTimeout> | undefined
const timeout = new Promise<never>((_, reject) => {
timer = setTimeout(
() => reject(new Error(`${label} did not arrive in ${timeoutMs} ms`)),
timeoutMs
)
})
try {
return await Promise.race([this.reached, timeout])
} finally {
clearTimeout(timer)
}
}
release(): void {
this.released = true
this.releaseHeld?.()
}
}
/**
* The origin Electron and Sim share. It forwards everything to Sim, records each request, can
* hold one until the test releases it, rewrite an answer or cut the network, and serves one
* test-only route that installs a seeded session's cookie the way Sim's own sign-in response
* would.
*/
export class SimProxy {
readonly requests: ProxiedRequest[] = []
readonly origin: string
private readonly server: Server
private holds: HeldRequest[] = []
/** Requests a hold is keeping from Sim right now. */
private readonly heldEntries = new Set<ProxiedRequest>()
private readonly answerRewrites = new Map<string, (body: Record<string, unknown>) => void>()
private readonly sockets = new Set<Duplex>()
/** Every client connection open now, HTTP and upgraded alike. */
private readonly connections = new Set<Socket>()
private networkCut = false
constructor(private readonly config: LiveSimConfig) {
this.origin = `http://127.0.0.1:${config.proxyPort}`
this.server = createServer((request, response) => {
void this.handle(request, response).catch((error: unknown) => {
if (!response.headersSent) response.writeHead(502)
response.end(String(error))
})
})
// While the network is cut, a new connection is reset as an unreachable host's would be.
this.server.on('connection', (socket: Socket) => {
if (this.networkCut) {
socket.destroy()
return
}
this.connections.add(socket)
socket.once('close', () => this.connections.delete(socket))
})
// Next's dev server pushes over a websocket; pass upgrades straight through.
this.server.on('upgrade', (request, socket, head) => {
const target = new URL(config.upstream)
const upstream = connect(Number(target.port), target.hostname, () => {
const lines = [`${request.method} ${request.url} HTTP/1.1`]
for (let i = 0; i < request.rawHeaders.length; i += 2)
lines.push(`${request.rawHeaders[i]}: ${request.rawHeaders[i + 1]}`)
upstream.write(`${lines.join('\r\n')}\r\n\r\n`)
upstream.write(head)
upstream.pipe(socket)
socket.pipe(upstream)
})
const client = socket
this.sockets.add(upstream)
this.sockets.add(client)
const end = () => {
upstream.destroy()
client.destroy()
this.sockets.delete(upstream)
this.sockets.delete(client)
}
for (const side of [upstream, client]) {
side.on('error', end)
side.on('close', end)
}
})
}
start(): Promise<void> {
return listen(this.server, this.config.proxyPort)
}
async stop(): Promise<void> {
for (const socket of this.sockets) socket.destroy()
await close(this.server)
}
/** Holds the next request that matches until the returned handle releases it. */
hold(
matches: (method: string, path: string) => boolean,
options: { deliverIfAbandoned?: boolean } = {}
): HeldRequest {
const held = new HeldRequest(matches, options.deliverIfAbandoned)
this.holds.push(held)
return held
}
/** Drops holds that never matched, so none outlives the test that set it. */
clearHolds(): void {
for (const held of this.holds) held.release()
this.holds = []
}
/**
* Rewrites Sim's JSON answer to `path`, as a Sim configured otherwise would answer it; `undefined`
* stops rewriting it.
*/
rewriteAnswer(
path: string,
rewrite: ((body: Record<string, unknown>) => void) | undefined
): void {
if (rewrite) this.answerRewrites.set(path, rewrite)
else this.answerRewrites.delete(path)
}
/**
* Sim's JSON answers to `path` that reach the app from now on, recorded as they pass and left
* unchanged. An answer to a request whose client gave up reaches no one, so it is not recorded.
*/
recordAnswers(path: string): Record<string, unknown>[] {
const answers: Record<string, unknown>[] = []
this.rewriteAnswer(path, (body) => {
answers.push(structuredClone(body))
})
return answers
}
/**
* Cuts the network between the app and Sim: every open connection drops mid-flight, the
* doorbell stream included, and new ones are reset until `restoreNetwork`.
*/
cutNetwork(): void {
this.networkCut = true
for (const socket of this.connections) socket.destroy()
}
restoreNetwork(): void {
this.networkCut = false
}
/**
* Resolves once every request the app made has been answered (held ones aside), staying so for
* a moment, which on a dev app means none is waiting on a route to compile.
*/
async settled(timeoutMs: number): Promise<void> {
const deadline = Date.now() + timeoutMs
let quietSince = Date.now()
for (;;) {
const waiting = this.requests.filter(
(entry) =>
entry.status === undefined &&
entry.clientClosedAt === undefined &&
!this.heldEntries.has(entry)
)
if (waiting.length > 0) quietSince = Date.now()
else if (Date.now() - quietSince >= 3_000) return
if (Date.now() > deadline)
throw new Error(
`Requests still unanswered: ${waiting.map((entry) => entry.path).join(', ')}`
)
await sleep(250)
}
}
/** The requests seen since `since`, optionally only those under a path prefix. */
seen(since = 0, prefix = '/'): ProxiedRequest[] {
return this.requests.filter((entry) => entry.at >= since && entry.path.startsWith(prefix))
}
private async handle(request: IncomingMessage, response: ServerResponse): Promise<void> {
const url = new URL(request.url ?? '/', this.origin)
const method = request.method ?? 'GET'
const entry: ProxiedRequest = { method, path: url.pathname, at: Date.now() }
this.requests.push(entry)
response.once('close', () => {
if (!response.writableFinished) entry.clientClosedAt = Date.now()
})
if (url.pathname === '/__e2e/sign-in') {
const cookie = url.searchParams.get('cookie') ?? ''
response.writeHead(302, {
'Set-Cookie': `better-auth.session_token=${cookie}; Path=/; HttpOnly; SameSite=Lax`,
Location: url.searchParams.get('to') ?? '/',
})
response.end()
entry.status = 302
return
}
const body = await readBody(request)
const held = this.holds.find((candidate) => candidate.matches(method, url.pathname))
if (held) {
this.holds = this.holds.filter((candidate) => candidate !== held)
this.heldEntries.add(entry)
const deliver = await held.hold(entry, response)
this.heldEntries.delete(entry)
if (!deliver && !held.deliverIfAbandoned) return
}
const target = new URL(url.pathname + url.search, this.config.upstream)
const rewriteAnswer = this.answerRewrites.get(url.pathname)
// The body is forwarded whole, so it is sent with a length rather than chunked. An answer to
// rewrite is asked for uncompressed.
const {
'transfer-encoding': _chunked,
'accept-encoding': acceptEncoding,
...forwarded
} = request.headers
const headers: IncomingHttpHeaders = {
...forwarded,
...(rewriteAnswer ? {} : { 'accept-encoding': acceptEncoding }),
'content-length': String(body.length),
}
await new Promise<void>((resolve, reject) => {
const clientGone = response.destroyed
const upstream = httpRequest(target, { method, headers }, (upstreamResponse) => {
entry.status = upstreamResponse.statusCode
upstreamResponse.on('end', resolve)
upstreamResponse.on('error', reject)
// A request delivered after its client gave up is answered to no one.
if (clientGone) {
upstreamResponse.resume()
return
}
if (rewriteAnswer && upstreamResponse.statusCode === 200) {
void readBody(upstreamResponse).then((raw) => {
const answer: Record<string, unknown> = JSON.parse(raw.toString('utf8'))
rewriteAnswer(answer)
const rewritten = Buffer.from(JSON.stringify(answer))
const { 'transfer-encoding': _chunked, ...answerHeaders } = upstreamResponse.headers
response.writeHead(200, {
...answerHeaders,
'content-length': String(rewritten.length),
})
response.end(rewritten)
}, reject)
return
}
response.writeHead(upstreamResponse.statusCode ?? 502, upstreamResponse.headers)
upstreamResponse.pipe(response)
})
if (!clientGone) response.on('close', () => upstream.destroy())
upstream.on('error', reject)
upstream.end(body)
})
}
}
/** One tool call a scripted turn issues. */
interface ScriptedToolCall {
toolName: string
args: Record<string, unknown>
/** `awaiting_approval` asks Sim to hold the call for the user's decision. */
status?: 'awaiting_approval'
}
/** One result Sim sends back when it resumes the agent after a checkpoint. */
interface ResumedResult {
callId: string
name?: string
success?: boolean
data?: unknown
}
/** A resume request Sim sent, with when it arrived. */
interface Resume {
at: number
streamId: string
results: ResumedResult[]
}
/** A chat turn Sim opened against the agent, written to as the scripted model acts. */
class AgentTurn {
readonly toolCallIds: string[] = []
/** Settles once this leg's stream has ended, from either side. */
readonly closed: Promise<void>
private markClosed!: () => void
private seq = 0
private readonly keepAlive: ReturnType<typeof setInterval>
private ended = false
constructor(
readonly body: Record<string, unknown>,
readonly streamId: string,
private readonly response: ServerResponse
) {
this.closed = new Promise((resolve) => {
this.markClosed = resolve
})
response.writeHead(200, {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
Connection: 'keep-alive',
})
// The worker keeps an idle stream alive with SSE comments; so does this one.
this.keepAlive = setInterval(() => {
if (!this.ended) response.write(': keepalive\n\n')
}, 2_000)
response.on('close', () => this.finish())
this.emit('session', { kind: 'start' })
}
get message(): string {
return typeof this.body.message === 'string' ? this.body.message : ''
}
emit(type: string, payload: Record<string, unknown>): void {
if (this.ended) return
this.seq += 1
const chatId = typeof this.body.chatId === 'string' ? this.body.chatId : undefined
const envelope = {
v: 1,
type,
seq: this.seq,
ts: new Date().toISOString(),
stream: { streamId: this.streamId, ...(chatId ? { chatId } : {}), cursor: String(this.seq) },
trace: { requestId: `agent-${this.streamId}` },
payload,
}
this.response.write(`data: ${JSON.stringify(envelope)}\n\n`)
}
text(text: string): void {
this.emit('text', { channel: 'assistant', text })
}
/** Issues a client-executed tool call and returns its id. */
toolCall(call: ScriptedToolCall): string {
const toolCallId = `toolu_${generateShortId()}`
this.toolCallIds.push(toolCallId)
this.emit('tool', {
phase: 'call',
toolCallId,
toolName: call.toolName,
executor: 'client',
mode: 'async',
arguments: call.args,
ui: { clientExecutable: true },
...(call.status ? { status: call.status } : {}),
})
return toolCallId
}
/** Ends this leg waiting on the turn's tool calls, as the worker does at a checkpoint. */
pause(): void {
this.emit('run', {
kind: 'checkpoint_pause',
checkpointId: generateId(),
executionId: generateId(),
runId: generateId(),
pendingToolCallIds: [...this.toolCallIds],
})
this.finish()
}
complete(text?: string): void {
if (text) this.text(text)
this.emit('complete', { status: 'complete' })
this.finish()
}
private finish(): void {
if (this.ended) return
this.ended = true
clearInterval(this.keepAlive)
this.response.end()
this.markClosed()
}
}
type TurnScript = (turn: AgentTurn) => void | Promise<void>
type ResumeScript = (resume: Resume, turn: AgentTurn) => void | Promise<void>
/**
* Stands in for the Mothership worker at Sim's `SIM_AGENT_API_URL`: each chat turn runs the
* script registered for its message, and each resume after a checkpoint records the tool results
* Sim delivers, which is what the real model would read.
*/
export class ScriptedAgent {
readonly turns: AgentTurn[] = []
readonly resumes: Resume[] = []
readonly unexpected: string[] = []
private readonly server: Server
private readonly scripts = new Map<string, TurnScript>()
private readonly resumeScripts = new Map<string, ResumeScript>()
private readonly waiters: (() => void)[] = []
constructor(private readonly port: number) {
this.server = createServer((request, response) => {
void this.handle(request, response).catch((error: unknown) => {
this.unexpected.push(String(error))
if (!response.headersSent) response.writeHead(500)
response.end()
})
})
}
get url(): string {
return `http://127.0.0.1:${this.port}`
}
start(): Promise<void> {
return listen(this.server, this.port)
}
stop(): Promise<void> {
return close(this.server)
}
/**
* Runs `script` for the turn whose message contains `marker`; `onResume` answers each resume of
* that turn (by default the turn completes with a short reply).
*/
script(marker: string, script: TurnScript, onResume?: ResumeScript): void {
this.scripts.set(marker, script)
if (onResume) this.resumeScripts.set(marker, onResume)
}
/** Resolves with the first resume matching `predicate`, failing after `timeoutMs`. */
async waitForResume(predicate: (resume: Resume) => boolean, timeoutMs: number): Promise<Resume> {
const deadline = Date.now() + timeoutMs
for (;;) {
const found = this.resumes.find(predicate)
if (found) return found
const remaining = deadline - Date.now()
if (remaining <= 0) throw new Error(`No matching resume within ${timeoutMs} ms`)
await new Promise<void>((resolve) => {
const timer = setTimeout(resolve, remaining)
this.waiters.push(() => {
clearTimeout(timer)
resolve()
})
})
}
}
/** The result Sim delivered for `toolCallId`, if any resume carried it. */
resultFor(toolCallId: string): (ResumedResult & { at: number }) | undefined {
for (const resume of this.resumes) {
const result = resume.results.find((entry) => entry.callId === toolCallId)
if (result) return { ...result, at: resume.at }
}
return undefined
}
private async handle(request: IncomingMessage, response: ServerResponse): Promise<void> {
const path = new URL(request.url ?? '/', this.url).pathname
const raw = (await readBody(request)).toString('utf8')
const body: Record<string, unknown> = raw ? JSON.parse(raw) : {}
if (path === '/api/mothership' || path === '/api/copilot') {
const message = typeof body.message === 'string' ? body.message : ''
const streamId = typeof body.messageId === 'string' ? body.messageId : generateId()
const turn = new AgentTurn(body, streamId, response)
this.turns.push(turn)
const entry = [...this.scripts].find(([marker]) => message.includes(marker))
if (!entry) {
turn.complete('No script for this message.')
return
}
await entry[1](turn)
return
}
if (path === '/api/tools/resume') {
const streamId = typeof body.streamId === 'string' ? body.streamId : ''
const results = toArray<ResumedResult>(body.results)
const resume: Resume = { at: Date.now(), streamId, results }
this.resumes.push(resume)
for (const wake of this.waiters.splice(0)) wake()
const turn = this.turns.find((candidate) => candidate.streamId === streamId)
const resumed = new AgentTurn(turn?.body ?? body, streamId, response)
const marker = turn
? [...this.resumeScripts.keys()].find((key) => turn.message.includes(key))
: undefined
const onResume = marker ? this.resumeScripts.get(marker) : undefined
if (onResume) await onResume(resume, resumed)
else resumed.complete('Done.')
return
}
if (path === '/api/generate-chat-title') {
response.writeHead(200, { 'Content-Type': 'application/json' })
response.end(JSON.stringify({ title: 'Desktop tools E2E' }))
return
}
// Stop, cleanup and every other worker callback: acknowledge.
response.writeHead(200, { 'Content-Type': 'application/json' })
response.end(JSON.stringify({ settled: true }))
}
}
/** A seeded user signed in to Sim, with one workspace and named chats. */
export interface SeededUser {
userId: string
name: string
workspaceId: string
sessionId: string
/** The `better-auth.session_token` cookie value for the seeded session. */
cookie: string
chats: Record<string, string>
}
/** Direct database access for seeding and for asserting what Sim persisted. */
export class SimDatabase {
readonly sql: postgres.Sql
private readonly userIds: string[] = []
constructor(private readonly config: LiveSimConfig) {
this.sql = postgres(config.databaseUrl, { max: 4, onnotice: () => {} })
}
async close(): Promise<void> {
if (this.userIds.length > 0)
await this.sql`delete from "user" where id in ${this.sql(this.userIds)}`.catch(() => {})
await this.sql.end({ timeout: 5 })
}
/** A user with a workspace, a session as the desktop sign-in creates, and one chat per title. */
async seedUser(chatTitles: string[]): Promise<SeededUser> {
const userId = generateId()
const workspaceId = generateId()
const sessionId = generateId()
const token = generateShortId()
const name = `Desktop E2E ${userId.slice(0, 6)}`
const email = `${userId}@desktop-tools-e2e.test`
const chats: Record<string, string> = {}
await this.sql.begin(async (tx) => {
await tx`insert into "user" (id, name, email, normalized_email, email_verified, created_at, updated_at)
values (${userId}, ${name}, ${email}, ${email}, true, now(), now())`
await tx`insert into user_stats (id, user_id) values (${generateId()}, ${userId})`
await tx`insert into workspace (id, name, owner_id, billed_account_user_id)
values (${workspaceId}, 'Desktop tools E2E', ${userId}, ${userId})`
await tx`insert into permissions (id, user_id, entity_type, entity_id, permission_type)
values (${generateId()}, ${userId}, 'workspace', ${workspaceId}, 'admin')`
await tx`insert into session (id, token, user_id, user_agent, expires_at, created_at, updated_at)
values (${sessionId}, ${token}, ${userId}, 'Sim Desktop', now() + interval '1 day', now(), now())`
for (const title of chatTitles) {
const chatId = generateId()
chats[title] = chatId
await tx`insert into copilot_chats (id, user_id, workspace_id, type, title)
values (${chatId}, ${userId}, ${workspaceId}, 'mothership', ${title})`
}
})
this.userIds.push(userId)
const signature = createHmac('sha256', this.config.authSecret).update(token).digest('base64')
return {
userId,
name,
workspaceId,
sessionId,
cookie: encodeURIComponent(`${token}.${signature}`),
chats,
}
}
/** The persisted desktop calls of a chat's runs, oldest first. */
toolCalls(chatId: string) {
return this.sql<
{
toolCallId: string
toolName: string
status: string
claimedBy: string | null
error: string | null
result: unknown
permissionRequestedAt: Date | null
persistSeq: number | null
runId: string
}[]
>`select c.tool_call_id as "toolCallId", c.tool_name as "toolName", c.status,
c.claimed_by as "claimedBy", c.error, c.result,
c.permission_requested_at as "permissionRequestedAt", c.persist_seq as "persistSeq",
c.run_id as "runId"
from copilot_async_tool_calls c join copilot_runs r on r.id = c.run_id
where r.chat_id = ${chatId} order by c.created_at`
}
runs(chatId: string) {
return this.sql<{ id: string; status: string; desktopDeviceId: string | null }[]>`
select id, status, desktop_device_id as "desktopDeviceId" from copilot_runs
where chat_id = ${chatId} order by created_at`
}
/** Names of the files a workspace holds. */
async workspaceFileNames(workspaceId: string): Promise<string[]> {
const rows = await this.sql<{ name: string }[]>`
select original_name as name from workspace_files
where workspace_id = ${workspaceId} and context = 'workspace' and deleted_at is null
order by original_name`
return rows.map((row) => row.name)
}
/** Names of the file folders a workspace holds. */
async workspaceFolderNames(workspaceId: string): Promise<string[]> {
const rows = await this.sql<{ name: string }[]>`
select name from folder where workspace_id = ${workspaceId} and resource_type = 'file'
order by name`
return rows.map((row) => row.name)
}
}
/**
* Records every command Sim sends Redis (`MONITOR`), so a test can assert that a turn rang no
* desktop doorbell. Speaks just enough RESP for that.
*/
export class RedisMonitor {
readonly lines: string[] = []
private socket: Socket | undefined
constructor(private readonly url: string) {}
async start(): Promise<void> {
const target = new URL(this.url)
const socket = connect(Number(target.port || 6379), target.hostname)
this.socket = socket
await new Promise<void>((resolve, reject) => {
socket.once('connect', resolve)
socket.once('error', reject)
})
// Each command sent before MONITOR answers `+OK`; MONITOR's own `+OK` means it is recording.
let pendingAcks = target.password ? 2 : 1
let buffer = ''
const acknowledged = new Promise<void>((resolve, reject) => {
socket.on('data', (chunk) => {
buffer += chunk.toString('utf8')
const lines = buffer.split('\r\n')
buffer = lines.pop() ?? ''
for (const line of lines) {
if (pendingAcks > 0) {
if (line.startsWith('-')) reject(new Error(`Redis refused MONITOR: ${line}`))
else if (line === '+OK' && --pendingAcks === 0) resolve()
continue
}
if (line.startsWith('+')) this.lines.push(line)
}
})
})
if (target.password) socket.write(`AUTH ${decodeURIComponent(target.password)}\r\n`)
socket.write('MONITOR\r\n')
await acknowledged
}
/** Commands seen that publish to a channel whose name contains `channel`. */
publishesTo(channel: string): string[] {
return this.lines.filter(
(line) => /"publish"/i.test(line) && line.toLowerCase().includes(channel.toLowerCase())
)
}
stop(): void {
this.socket?.destroy()
}
}
apps/desktop/e2e/fixtures/worker-restart.config.ts
+10
View File
@@ -0,0 +1,10 @@
import { defineConfig } from '@playwright/test'
/** Runs only the worker-restart fixture, for `check-report.spec.ts`. */
export default defineConfig({
testDir: '.',
testMatch: 'worker-restart.ts',
workers: 1,
retries: 0,
reporter: [['line']],
})
apps/desktop/e2e/fixtures/worker-restart.ts
+26
View File
@@ -0,0 +1,26 @@
import { expect, test } from '@playwright/test'
import { recordCheck } from '../check-report'
/**
* Two checks in one file, the first failing, run by `check-report.spec.ts` in a Playwright run of
* its own. The failure makes Playwright start a new worker for the second, as it does mid-suite.
*/
const reportPath = process.env.WORKER_RESTART_REPORT_PATH
function record(name: string, status: string): void {
recordCheck(reportPath, 'worker-restart', {
name,
status,
durationMs: 0,
error: `worker ${test.info().workerIndex}`,
})
}
test('fails first', () => {
record('fails first', 'failed')
expect('failed').toBe('passed')
})
test('passes after the worker restarts', () => {
record('passes after the worker restarts', 'passed')
})
apps/desktop/e2e/local-files.spec.ts
+8 -3
View File
@@ -16,7 +16,8 @@ test('native file tools read and import through the installed preload without Si
const png =
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAusB9Y9Zl1sAAAAASUVORK5CYII='
writeFileSync(join(source, 'image.png'), Buffer.from(png, 'base64'))
let claimed = false
/** Calls the server saw claimed; like the server, only an import refuses a second claim. */
const claimed = new Set<string>()
const calls: Record<string, { toolName: string; args: Record<string, unknown> }> = {
text: { toolName: 'read_local_file', args: { path: join(source, 'report.txt') } },
image: { toolName: 'read_local_file', args: { path: join(source, 'image.png') } },
@@ -44,11 +45,14 @@ test('native file tools read and import through the installed preload without Si
for await (const chunk of request) body += chunk.toString()
const input = JSON.parse(body)
const call = calls[input.toolCallId]
if (!call || (input.claim && claimed)) {
if (
!call ||
(input.claim && call.toolName === 'import_local_files' && claimed.has(input.toolCallId))
) {
response.writeHead(call ? 409 : 403, { 'Content-Type': 'application/json' }).end('{}')
return
}
if (input.claim) claimed = true
if (input.claim) claimed.add(input.toolCallId)
response
.writeHead(200, { 'Content-Type': 'application/json' })
.end(JSON.stringify({ ...call, chatId: 'org-chat' }))
@@ -92,6 +96,7 @@ test('native file tools read and import through the installed preload without Si
ok: true,
data: { observations: [{ mediaType: 'image/png', data: png }] },
})
expect([...claimed]).toEqual(['text', 'image'])
const result = await invoke({ operation: 'manifest', toolCallId: 'import' })
if (!result.ok || result.data.kind !== 'manifest') throw new Error(JSON.stringify(result))
expect(result.data.targetWorkspaceId).toBe('target-workspace')
apps/desktop/e2e/terminal-cancel.spec.ts
+609
View File
@@ -0,0 +1,609 @@
import { execFileSync } from 'node:child_process'
import { chmodSync, mkdtempSync, readdirSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { type ElectronApplication, expect, type Page, test } from '@playwright/test'
import type { SimDesktopApi } from '@sim/desktop-bridge'
import { sleep } from '@sim/utils/helpers'
import { randomInt } from '@sim/utils/random'
import { recordCheck } from './check-report'
import { FixtureSim, launch, processRunning, registeredDevice, settled } from './executor-sim'
/**
* Stopping the agent's terminal commands in the real app, with real shells and a real tmux
* server: Stop, sign-out (by the web app's logout and by the session cookie going away), switching
* Terminal off, and the launch-time recovery of an interrupted sign-out. What is asserted is the
* machine's own state (which processes still run, which tmux panes still exist), never which
* calls were made.
*
* Agent commands in a plain shell ignore SIGHUP, SIGINT and SIGTERM, so closing the shell cannot
* end them by accident: only Sim's own stop, which escalates to SIGKILL on the command's process
* group, does. The user's commands ignore the same signals, so a stop that reached them would
* show. Each launch gets its own tmux server (TMUX_TMPDIR), which the app's shells and its tmux
* calls inherit. The tmux scenarios skip where tmux is not installed.
*/
const CHAT_PLAIN = 'chat-cancel-plain'
const CHAT_TMUX = 'chat-cancel-tmux'
const TMUX_SESSION = 'agent-e2e'
/** Commands that only a SIGKILL ends, so a shell closing around them proves nothing. */
const STUBBORN = "trap '' HUP INT TERM"
const sim = new FixtureSim()
/** The app's own log output for the current test, attached when it fails. */
const appOutput: string[] = []
/** The shells, sleeps and tmux processes running, for a failure about who started what. */
function processes(): string {
try {
return execFileSync('ps', ['-eo', 'pid,ppid,pgid,args'])
.toString()
.split('\n')
.filter((line) => /tmux|sleep|bash|zsh/.test(line))
.join('\n')
} catch (error) {
return String(error)
}
}
const REAL_TMUX = findTmux()
/**
* Unique to this run of the suite, so its processes are never confused with those of another run on
* the same machine (CI boxes run several at once).
*/
const RUN_NONCE = String(randomInt(0, 1_000_000)).padStart(6, '0')
/** A sleep that runs for about `seconds` and that only this test names. */
function slept(seconds: number): string {
return `sleep ${seconds}.${RUN_NONCE}`
}
/** The command line a stubborn sleep runs as, unique to one test by its duration. */
function stubborn(seconds: number): string {
return `bash -c "${STUBBORN}; ${slept(seconds)}"`
}
function findTmux(): string | null {
try {
return execFileSync('sh', ['-c', 'command -v tmux']).toString().trim() || null
} catch {
return null
}
}
/** One launch's own tmux server, and a way to read and reshape it as the user would. */
class TmuxServer {
readonly dir = mkdtempSync('/tmp/sim-tmux-')
run(args: string[]): string {
return execFileSync(REAL_TMUX ?? 'tmux', args, {
env: { ...process.env, TMUX_TMPDIR: this.dir },
}).toString()
}
/** Every pane: its id, its run tag (empty when untagged) and the command it was started with. */
panes(): Array<{ id: string; runId: string; command: string }> {
try {
return this.run([
'list-panes',
'-a',
'-F',
'#{pane_id}\t#{@sim-run-id}\t#{pane_start_command}',
])
.trim()
.split('\n')
.filter(Boolean)
.map((line) => {
const [id = '', runId = '', command = ''] = line.split('\t')
return { id, runId, command }
})
} catch {
return []
}
}
/** The pane an agent run opened, found by the run script it was started with. */
runPane(): { id: string; runId: string } | undefined {
return this.panes().find((pane) => pane.command.includes('run.sh'))
}
hasPane(id: string): boolean {
return this.panes().some((pane) => pane.id === id)
}
/** The user splits a pane of their own beside the agent's, running their own command. */
split(target: string, command: string): string {
return this.run(['split-window', '-d', '-t', target, '-P', '-F', '#{pane_id}', command]).trim()
}
clients(): number {
try {
return this.run(['list-clients']).trim().split('\n').filter(Boolean).length
} catch {
return 0
}
}
kill(): void {
try {
this.run(['kill-server'])
} catch {
// Already gone.
}
}
}
/** A `tmux` ahead of the real one on PATH that refuses pane options, as tmux before 3.0 does. */
function oldTmuxPath(): string {
const dir = mkdtempSync(join(tmpdir(), 'sim-old-tmux-'))
const binary = join(dir, 'tmux')
writeFileSync(
binary,
`#!/bin/sh
if [ "$1" = set-option ] && [ "$2" = -p ]; then
printf 'tmux: unknown option -- p\\nusage: set-option [-aFgosquw] [-t target-window] option [value]\\n' >&2
exit 1
fi
exec ${JSON.stringify(REAL_TMUX)} "$@"
`
)
chmodSync(binary, 0o755)
return `${dir}:${process.env.PATH ?? ''}`
}
function bridge(window: Page) {
return {
/** The user opens a terminal tab in a chat's terminal panel and starts a command in it. */
async userRuns(scope: string, command: string): Promise<void> {
const terminalId = await window.evaluate(async (scopeId) => {
const terminal = (globalThis as typeof globalThis & { simDesktop: SimDesktopApi })
.simDesktop.terminal
await terminal.activateScope(scopeId)
terminal.setVisible?.(true, scopeId)
terminal.setFocused(true, scopeId)
const opened = await terminal.openTerminal(undefined, scopeId)
return opened.activeTerminalId
}, scope)
if (!terminalId) throw new Error('No terminal tab opened')
// Typing is gated on a real keypress in the window, as it is for the panel's own input.
await window.keyboard.press('Shift')
await window.evaluate(
([id, text, scopeId]) =>
(
globalThis as typeof globalThis & { simDesktop: SimDesktopApi }
).simDesktop.terminal.write(id, text, scopeId),
[terminalId, `${command}\r`, scope] as const
)
},
/** The user puts a chat away: its terminals close, though tmux runs started in them go on. */
async putAway(scope: string): Promise<boolean> {
return window.evaluate(async (scopeId) => {
const terminal = (globalThis as typeof globalThis & { simDesktop: SimDesktopApi })
.simDesktop.terminal
await terminal.activateScope(scopeId)
return terminal.suspendScope(scopeId)
}, scope)
},
async setTerminalEnabled(enabled: boolean): Promise<void> {
await window.evaluate(
(value) =>
(
globalThis as typeof globalThis & { simDesktop: SimDesktopApi }
).simDesktop.settings.setPreference('terminalEnabled', value),
enabled
)
},
}
}
/** The agent runs a plain-shell command in a chat's terminal; resolves once it is running. */
async function agentRunsPlain(deviceId: string, seconds: number): Promise<string> {
const call = sim.issue(deviceId, CHAT_PLAIN, 'terminal', {
operation: 'run',
args: { command: stubborn(seconds), waitSeconds: 120 },
})
await expect.poll(() => processRunning(`${slept(seconds)}`), { timeout: 30_000 }).toBe(true)
return call
}
/**
* The agent's terminal attaches to tmux (the agent starts a tmux client in it), and the agent then
* runs a command, which tmux gives its own tagged pane. Resolves with the call and that pane.
*/
/** The agent starts a tmux client in its chat's terminal, so its runs go to tmux panes. */
async function agentAttachesTmux(deviceId: string, tmux: TmuxServer): Promise<void> {
const before = tmux.clients()
sim.issue(deviceId, CHAT_TMUX, 'terminal', {
operation: 'run',
args: { command: `tmux new-session -A -s ${TMUX_SESSION}`, waitSeconds: 2 },
})
await expect.poll(() => tmux.clients(), { timeout: 30_000 }).toBe(before + 1)
// The terminal's attachment is looked up again once its short cache lapses.
await sleep(3_500)
}
/**
* The agent runs a command in tmux, which gives it its own tagged pane; with a short
* `waitSeconds` the call hands it back as still running, with that pane.
*/
async function agentRunsInTmux(
deviceId: string,
tmux: TmuxServer,
seconds: number,
waitSeconds = 120
): Promise<{ call: string; pane: { id: string; runId: string } }> {
await agentAttachesTmux(deviceId, tmux)
// It ignores Ctrl-C, so stopping it takes closing its pane, the step that must spare the user's.
const call = sim.issue(deviceId, CHAT_TMUX, 'terminal', {
operation: 'run',
args: {
command: `bash -c "trap '' INT; echo collect-me; ${slept(seconds)}"`,
waitSeconds,
},
})
await expect.poll(() => processRunning(`${slept(seconds)}`), { timeout: 30_000 }).toBe(true)
await expect.poll(() => tmux.runPane(), { timeout: 10_000 }).toBeTruthy()
return { call, pane: tmux.runPane() as { id: string; runId: string } }
}
async function claimed(call: string): Promise<void> {
await expect.poll(() => sim.requireCall(call).claims, { timeout: 30_000 }).toBe(1)
}
test.describe('terminal cancel', () => {
let app: ElectronApplication | null = null
let tmux: TmuxServer
const leftovers: string[] = []
test.beforeAll(async () => {
await sim.start()
})
test.beforeEach(() => {
tmux = new TmuxServer()
})
test.afterEach(async () => {
const testInfo = test.info()
recordCheck(process.env.TERMINAL_CANCEL_REPORT_PATH, 'terminal-cancel', {
name: testInfo.title,
status: testInfo.status ?? 'unknown',
durationMs: testInfo.duration,
retry: testInfo.retry,
})
if (testInfo.status !== testInfo.expectedStatus) {
// What each call came back with, and the panes tmux held, explain most failures.
const calls = [...sim.calls.values()].map((call) => ({
toolCallId: call.toolCallId,
args: call.args,
status: call.status,
completions: call.completions.map(({ status, message, data }) => ({
status,
message,
data,
})),
}))
await testInfo.attach('calls', {
body: JSON.stringify({ calls, panes: tmux.panes() }, null, 2),
})
await testInfo.attach('app-output', { body: appOutput.join('').slice(-6_000) })
await testInfo.attach('processes', { body: processes() })
}
appOutput.length = 0
// Quitting with a command still running in a tab asks first (natively, on macOS), and nobody
// is there to answer, so a close that does not finish promptly ends the app instead.
const closing = app
const closed = await Promise.race([
closing?.close().then(
() => true,
() => true
) ?? Promise.resolve(true),
sleep(5_000).then(() => false),
])
if (!closed) closing?.process().kill('SIGKILL')
app = null
tmux.kill()
for (const pattern of leftovers.splice(0)) {
try {
execFileSync('pkill', ['-9', '-f', pattern])
} catch {
// Nothing left.
}
}
sim.reset()
})
test.afterAll(async () => {
await sim.stop()
})
async function start(
name: string,
env: Record<string, string> = {}
): Promise<{ window: Page; deviceId: string; userData: string }> {
const userData = mkdtempSync(join(tmpdir(), `sim-cancel-${name}-`))
const launched = await launch(sim, userData, { TMUX_TMPDIR: tmux.dir, ...env })
app = launched.app
launched.app.process().stdout?.on('data', (chunk: Buffer) => appOutput.push(chunk.toString()))
launched.app.process().stderr?.on('data', (chunk: Buffer) => appOutput.push(chunk.toString()))
const deviceId = await registeredDevice(sim)
await launched.window.goto(`${sim.origin}/workspace/ws-e2e/chat/${CHAT_PLAIN}`)
return { window: launched.window, deviceId, userData }
}
test('Stop ends the agent command in a plain shell and leaves the user command running', async () => {
const { window, deviceId } = await start('stop-plain')
leftovers.push(slept(701), slept(702))
// The agent's terminal comes first; the user's own tab opens beside it.
const call = await agentRunsPlain(deviceId, 701)
await bridge(window).userRuns(CHAT_PLAIN, stubborn(702))
await expect.poll(() => processRunning(slept(702)), { timeout: 30_000 }).toBe(true)
sim.stopCall(call)
await settled(sim, call, 20_000)
await expect.poll(() => processRunning(slept(701)), { timeout: 15_000 }).toBe(false)
expect(processRunning(slept(702))).toBe(true)
})
test('Stop interrupts a tagged tmux run and closes only its pane, never the user pane beside it', async () => {
test.skip(!REAL_TMUX, 'tmux is not installed on this machine')
const { deviceId } = await start('stop-tmux')
leftovers.push(slept(711), slept(712))
const { call, pane } = await agentRunsInTmux(deviceId, tmux, 711)
expect(pane.runId).not.toBe('')
const users = tmux.split(pane.id, stubborn(712))
sim.stopCall(call)
await settled(sim, call, 20_000)
await expect.poll(() => processRunning(slept(711)), { timeout: 15_000 }).toBe(false)
await expect.poll(() => tmux.hasPane(pane.id), { timeout: 10_000 }).toBe(false)
expect(tmux.hasPane(users)).toBe(true)
expect(processRunning(slept(712))).toBe(true)
})
test('signing out stops every agent command and leaves the user commands running', async () => {
const { window, deviceId } = await start('sign-out')
leftovers.push(slept(721), slept(722), slept(723), slept(724))
const plain = await agentRunsPlain(deviceId, 721)
await claimed(plain)
await bridge(window).userRuns(CHAT_PLAIN, stubborn(722))
await expect.poll(() => processRunning(slept(722)), { timeout: 30_000 }).toBe(true)
const run = REAL_TMUX ? await agentRunsInTmux(deviceId, tmux, 723) : null
const users = run ? tmux.split(run.pane.id, stubborn(724)) : null
// The web app signs out by navigating to its login page.
await window.goto(`${sim.origin}/login?fromLogout=true`)
await expect.poll(() => processRunning(slept(721)), { timeout: 30_000 }).toBe(false)
expect(processRunning(slept(722))).toBe(true)
if (run && users) {
await expect.poll(() => processRunning(slept(723)), { timeout: 15_000 }).toBe(false)
await expect.poll(() => tmux.hasPane(run.pane.id), { timeout: 10_000 }).toBe(false)
expect(tmux.hasPane(users)).toBe(true)
expect(processRunning(slept(724))).toBe(true)
}
})
test('a session ended by another account signing in stops every agent command and leaves the user pane running', async () => {
const { window, deviceId } = await start('account')
leftovers.push(slept(731), slept(733), slept(734))
await agentRunsPlain(deviceId, 731)
const run = REAL_TMUX ? await agentRunsInTmux(deviceId, tmux, 733) : null
const users = run ? tmux.split(run.pane.id, stubborn(734)) : null
if (users) await expect.poll(() => processRunning(slept(734)), { timeout: 15_000 }).toBe(true)
// Another account signing in replaces the session: the old session cookie goes away.
await window.goto(`${sim.origin}/session-ended`)
await expect.poll(() => processRunning(slept(731)), { timeout: 30_000 }).toBe(false)
if (run && users) {
await expect.poll(() => processRunning(slept(733)), { timeout: 15_000 }).toBe(false)
await expect.poll(() => tmux.hasPane(run.pane.id), { timeout: 10_000 }).toBe(false)
expect(tmux.hasPane(users)).toBe(true)
expect(processRunning(slept(734))).toBe(true)
}
})
test('switching Terminal off stops every agent command and leaves the user commands running', async () => {
const { window, deviceId } = await start('terminal-off')
leftovers.push(slept(741), slept(743), slept(744))
await agentRunsPlain(deviceId, 741)
const run = REAL_TMUX ? await agentRunsInTmux(deviceId, tmux, 743) : null
const users = run ? tmux.split(run.pane.id, stubborn(744)) : null
await bridge(window).setTerminalEnabled(false)
await expect.poll(() => processRunning(slept(741)), { timeout: 30_000 }).toBe(false)
if (run && users) {
await expect.poll(() => processRunning(slept(743)), { timeout: 15_000 }).toBe(false)
await expect.poll(() => tmux.hasPane(run.pane.id), { timeout: 10_000 }).toBe(false)
expect(tmux.hasPane(users)).toBe(true)
expect(processRunning(slept(744))).toBe(true)
}
})
test('signing out stops a tmux run whose chat was put away', async () => {
test.skip(!REAL_TMUX, 'tmux is not installed on this machine')
const { window, deviceId, userData } = await start('put-away')
leftovers.push(slept(801))
const { pane } = await agentRunsInTmux(deviceId, tmux, 801)
// No live terminal holds the run any more; only its record does.
expect(await bridge(window).putAway(CHAT_TMUX)).toBe(true)
await sleep(1_000)
expect(processRunning(slept(801))).toBe(true)
await window.goto(`${sim.origin}/login?fromLogout=true`)
await expect.poll(() => processRunning(slept(801)), { timeout: 30_000 }).toBe(false)
expect(tmux.hasPane(pane.id)).toBe(false)
await expect.poll(() => recordedRuns(userData), { timeout: 10_000 }).toEqual([])
})
/** The app dies without a word: its shells go with it, and its tmux runs outlive it. */
function crash(): void {
app?.process().kill('SIGKILL')
app = null
}
/** The runs this app has recorded for a later process to find. */
function recordedRuns(userData: string): string[] {
try {
return readdirSync(join(userData, 'terminal-runs'))
} catch {
return []
}
}
test('the launch after an interrupted sign-out stops the tmux run the previous app left running', async () => {
test.skip(!REAL_TMUX, 'tmux is not installed on this machine')
const { deviceId, userData } = await start('recovery')
leftovers.push(slept(751))
const { pane } = await agentRunsInTmux(deviceId, tmux, 751)
// The app dies mid sign-out: its recovery marker is on disk and the tmux run outlives it.
crash()
writeFileSync(
join(userData, 'account-data-teardown-required.json'),
JSON.stringify({ version: 2, kind: 'account', origin: sim.origin })
)
expect(processRunning(slept(751))).toBe(true)
app = (await launch(sim, userData, { TMUX_TMPDIR: tmux.dir })).app
await expect.poll(() => processRunning(slept(751)), { timeout: 30_000 }).toBe(false)
expect(tmux.hasPane(pane.id)).toBe(false)
await expect.poll(() => recordedRuns(userData), { timeout: 10_000 }).toEqual([])
})
test('the launch after a crash stops the tmux run its call can no longer report', async () => {
test.skip(!REAL_TMUX, 'tmux is not installed on this machine')
const { deviceId, userData } = await start('crash-live')
leftovers.push(slept(771), slept(772))
const { pane } = await agentRunsInTmux(deviceId, tmux, 771)
const users = tmux.split(pane.id, stubborn(772))
// Same user, plain crash: the journal settles the call as outcome unknown, and nothing will
// ever read what the command does.
crash()
expect(processRunning(slept(771))).toBe(true)
app = (await launch(sim, userData, { TMUX_TMPDIR: tmux.dir })).app
await expect.poll(() => processRunning(slept(771)), { timeout: 30_000 }).toBe(false)
expect(tmux.hasPane(pane.id)).toBe(false)
expect(tmux.hasPane(users)).toBe(true)
expect(processRunning(slept(772))).toBe(true)
await expect.poll(() => recordedRuns(userData), { timeout: 10_000 }).toEqual([])
})
test('the launch after a crash only forgets a tmux run that had already finished', async () => {
test.skip(!REAL_TMUX, 'tmux is not installed on this machine')
const { deviceId, userData } = await start('crash-finished')
leftovers.push(slept(6), slept(782))
// Handed back as still running, it then finishes on its own.
const { call, pane } = await agentRunsInTmux(deviceId, tmux, 6, 2)
await settled(sim, call)
const users = tmux.split(pane.id, stubborn(782))
await expect.poll(() => tmux.hasPane(pane.id), { timeout: 20_000 }).toBe(false)
expect(recordedRuns(userData)).toHaveLength(1)
crash()
app = (await launch(sim, userData, { TMUX_TMPDIR: tmux.dir })).app
await expect.poll(() => recordedRuns(userData), { timeout: 30_000 }).toEqual([])
expect(tmux.hasPane(users)).toBe(true)
expect(processRunning(slept(782))).toBe(true)
})
test('a run handed back as still going survives a quit and the relaunch, and can still be read', async () => {
test.skip(!REAL_TMUX, 'tmux is not installed on this machine')
const { deviceId, userData } = await start('collectable')
leftovers.push(slept(811))
const { call, pane } = await agentRunsInTmux(deviceId, tmux, 811, 2)
const handedBack = await settled(sim, call)
expect(handedBack.data).toMatchObject({ status: 'running', pane: pane.id })
// Quitting stops no tmux run; the relaunch leaves one the model may come back to.
const quitting = app
app = null
const quit = await Promise.race([
quitting?.close().then(() => true) ?? Promise.resolve(true),
sleep(10_000).then(() => false),
])
if (!quit) quitting?.process().kill('SIGKILL')
expect(processRunning(slept(811))).toBe(true)
app = (await launch(sim, userData, { TMUX_TMPDIR: tmux.dir })).app
// Same install, same device: it is back once its doorbell is open again.
const relaunched = await registeredDevice(sim)
await sleep(5_000)
expect(processRunning(slept(811))).toBe(true)
expect(tmux.hasPane(pane.id)).toBe(true)
expect(recordedRuns(userData)).toHaveLength(1)
// The model comes back to the pane it was handed.
await agentAttachesTmux(relaunched, tmux)
const read = sim.issue(relaunched, CHAT_TMUX, 'terminal', {
operation: 'read',
args: { pane: pane.id, lines: 50 },
})
const output = await settled(sim, read)
expect(output.status, output.message).toBe('success')
expect(JSON.stringify(output.data)).toContain('collect-me')
})
test("never touches a pane that took a recorded run's id after tmux restarted", async () => {
test.skip(!REAL_TMUX, 'tmux is not installed on this machine')
const { deviceId, userData } = await start('stale-record')
leftovers.push(slept(791), slept(792))
const { pane } = await agentRunsInTmux(deviceId, tmux, 791)
crash()
// tmux restarts, ending the run with it, and the user's own pane gets the run's old id.
tmux.kill()
// The old server takes a moment to let go of its socket.
await expect
.poll(
() => {
try {
tmux.run(['-f', '/dev/null', 'new-session', '-d', '-s', 'users', stubborn(792)])
return true
} catch {
return false
}
},
{ timeout: 10_000 }
)
.toBe(true)
for (let panes = 0; panes < 10 && !tmux.hasPane(pane.id); panes += 1) {
tmux.split('users', stubborn(792))
}
expect(processRunning(slept(791))).toBe(false)
app = (await launch(sim, userData, { TMUX_TMPDIR: tmux.dir })).app
await expect.poll(() => recordedRuns(userData), { timeout: 30_000 }).toEqual([])
expect(tmux.hasPane(pane.id)).toBe(true)
expect(processRunning(slept(792))).toBe(true)
})
test('on tmux without pane options a run goes ahead untracked, and Stop and sign-out leave it alone', async () => {
test.skip(!REAL_TMUX, 'tmux is not installed on this machine')
const { window, deviceId } = await start('untracked', { PATH: oldTmuxPath() })
leftovers.push(slept(761))
const { call, pane } = await agentRunsInTmux(deviceId, tmux, 761)
expect(pane.runId).toBe('')
sim.stopCall(call)
await settled(sim, call, 20_000)
await sleep(6_000)
expect(processRunning(slept(761))).toBe(true)
expect(tmux.hasPane(pane.id)).toBe(true)
await window.goto(`${sim.origin}/login?fromLogout=true`)
await sleep(8_000)
expect(processRunning(slept(761))).toBe(true)
expect(tmux.hasPane(pane.id)).toBe(true)
})
})
apps/desktop/package.json
+1
View File
@@ -60,6 +60,7 @@
"jsdom": "^26.0.0",
"postcss": "^8",
"postcss-load-config": "6.0.1",
"postgres": "^3.4.5",
"react": "19.2.4",
"react-dom": "19.2.4",
"typescript": "^7.0.2",
apps/desktop/src/main/browser-agent/driver.test.ts
+96
View File
@@ -79,6 +79,102 @@ describe('executeTool', () => {
driver = freshDriver()
})
/** The user presses a key in the page the agent drives, as the browser reports it. */
function typeInAgentPage(contents: WebContents) {
const calls: ReadonlyArray<readonly unknown[]> = vi.mocked(contents.on).mock.calls
const listener = calls.find(([event]) => event === 'before-input-event')?.[1]
if (typeof listener !== 'function') throw new Error('No input listener on the agent page')
listener({}, { type: 'keyDown', isAutoRepeat: false })
}
it('lets a call from the chat view act at once while the user works in the page', async () => {
await driver.executeTool('chat-test', 'browser_open_tab', {})
typeInAgentPage(session.requireTab().view.webContents)
expect(session.msSinceUserIntervention()).not.toBeNull()
vi.useFakeTimers()
try {
let settled = false
const opening = driver.executeTool('chat-test', 'browser_open_tab', {}).then((result) => {
settled = true
return result
})
await vi.advanceTimersByTimeAsync(100)
expect(settled).toBe(true)
expect((await opening).ok).toBe(true)
} finally {
vi.useRealTimers()
}
})
it('makes a background call wait until the user leaves the page alone', async () => {
await driver.executeTool('chat-test', 'browser_open_tab', {})
typeInAgentPage(session.requireTab().view.webContents)
vi.useFakeTimers()
try {
let settled = false
const opening = driver
.executeTool('chat-test', 'browser_open_tab', {}, 'background-1', undefined, {
background: true,
})
.then((result) => {
settled = true
return result
})
await vi.advanceTimersByTimeAsync(1_000)
expect(settled).toBe(false)
await vi.advanceTimersByTimeAsync(4_000)
expect((await opening).ok).toBe(true)
} finally {
vi.useRealTimers()
}
})
it('stops a background action before its next input once the user starts working mid-action', async () => {
await driver.executeTool('chat-test', 'browser_open_tab', {})
const contents = session.requireTab().view.webContents
// The user presses a key while the action reads the page before acting on it.
const probe = vi.mocked(contents.executeJavaScript).getMockImplementation()
vi.mocked(contents.executeJavaScript).mockImplementation(async (...args) => {
typeInAgentPage(contents)
return probe ? probe(...args) : undefined
})
const result = await driver.executeTool(
'chat-test',
'browser_press_key',
{ key: 'Enter' },
'background-key',
undefined,
{ background: true }
)
expect(result.ok).toBe(false)
expect(result.error).toContain('the user started working in this page')
})
it('never makes a background read wait for the user', async () => {
await driver.executeTool('chat-test', 'browser_open_tab', {})
typeInAgentPage(session.requireTab().view.webContents)
vi.useFakeTimers()
try {
let settled = false
void driver
.executeTool('chat-test', 'browser_list_tabs', {}, 'background-read', undefined, {
background: true,
})
.then(() => {
settled = true
})
await vi.advanceTimersByTimeAsync(100)
expect(settled).toBe(true)
} finally {
vi.useRealTimers()
}
})
it('returns ok:false instead of throwing for tool-level failures', async () => {
// No session exists, so any page-dependent tool fails with guidance.
const result = await driver.executeTool('chat-test', 'browser_click', { elementId: 1 })
apps/desktop/src/main/browser-agent/driver.ts
+81 -3
View File
@@ -19,6 +19,7 @@
import {
BROWSER_DATA_KINDS,
BROWSER_NAVIGATION_NATIVE_WATCHDOG_MS,
BROWSER_TOOL_OBSERVES_ONLY,
BROWSER_TOOL_QUEUE_WAIT_TIMEOUT_MS,
BROWSER_UPLOAD_MAX_FILES,
type BrowserDataKind,
@@ -32,7 +33,7 @@ import {
import type { BrowserDownloadsState, BrowserToolbarCommand } from '@sim/desktop-bridge'
import { createLogger } from '@sim/logger'
import { getErrorMessage } from '@sim/utils/errors'
import { sleep } from '@sim/utils/helpers'
import { interruptibleSleep, sleep } from '@sim/utils/helpers'
import { isRecordLike, omit, toArray, toRecord } from '@sim/utils/object'
import type { BrowserWindow, MenuItemConstructorOptions, WebContents, WebFrameMain } from 'electron'
import { Menu } from 'electron'
@@ -938,6 +939,12 @@ export function restoreBrowserScope(scopeId: string): BrowserTabsState {
})
}
/** Whether a chat's browser has a live page for a tool to act on. */
export function hasBrowserScopeSession(scopeId: string): boolean {
const resolved = resolveDriverScopeId(scopeId)
return session.withBrowserScope(resolved, () => session.hasSession())
}
/** Moves pending-new-chat driver and tab state to the server-issued chat id. */
export function migrateBrowserScope(fromScopeId: string, toScopeId: string): boolean {
const from = resolveDriverScopeId(fromScopeId)
@@ -5088,12 +5095,64 @@ function withNotices(result: unknown): unknown {
return { value: result, notices }
}
/** Quiet time after the user's last click or keystroke before the agent takes the page back. */
const USER_TAKEOVER_IDLE_MS = 4_000
/** The longest an action waits for the user to finish; past it, the action does not run. */
const USER_TAKEOVER_MAX_WAIT_MS = 30_000
const USER_KEPT_WORKING =
'Not run: the user kept working in this page, so this browser action never started and nothing was sent to the page. Ask the user whether they are done before acting in this page again.'
const USER_TOOK_OVER_MID_ACTION =
'Stopped: the user started working in this page, so this browser action stopped before sending its next input. Earlier steps may have taken effect; inspect the page before continuing.'
function isUserWorkingInPage(): boolean {
const since = session.msSinceUserIntervention()
return since !== null && since < USER_TAKEOVER_IDLE_MS
}
/**
* The user and the agent never act in the same page at once. While the user is clicking, typing
* or scrolling in the tab the agent drives, the agent's next action waits, marked as needing
* attention, and resumes once the user has left the page alone for a few seconds.
*/
async function yieldToUser(
toolCallId: string | undefined,
signal: AbortSignal,
maxWaitMs: number
): Promise<void> {
session.setAutomationNeedsAttention(true)
logger.info('Browser automation yielding to the user', { toolCallId })
// Bounded well inside the action's own deadline, so a user who keeps working gets the model a
// clear "not run" rather than a timeout with an unknown outcome.
const deadline = Date.now() + maxWaitMs
try {
while (isUserWorkingInPage()) {
if (Date.now() >= deadline) throw new ToolError(USER_KEPT_WORKING)
await interruptibleSleep(250, signal)
if (signal.aborted) throw new ToolError('This browser action was cancelled.')
}
} finally {
session.setAutomationNeedsAttention(false)
}
logger.info('Browser automation resumed after the user stopped', { toolCallId })
}
/** How a browser call reached the driver. */
interface BrowserToolExecutionOptions {
/**
* Run by the background executor while the user may be working in the same page, so it yields
* the page to them. A call from the chat view the user is watching never does: there the user
* steers the agent directly, as they always have.
*/
background?: boolean
}
export async function executeTool(
scopeId: string,
tool: BrowserToolName,
params: Record<string, unknown>,
toolCallId?: string,
authorizationBoundary?: BrowserToolQueueBoundary
authorizationBoundary?: BrowserToolQueueBoundary,
options: BrowserToolExecutionOptions = {}
): Promise<{ ok: boolean; result?: unknown; error?: string }> {
const resolvedScopeId = resolveDriverScopeId(scopeId)
if (authorizationBoundary) {
@@ -5184,17 +5243,36 @@ export async function executeTool(
session.setAutomationActive(true)
}
try {
// Only a background call acting on the page yields; reading it cannot collide with the user.
const yieldsToUser =
options.background === true &&
tool !== 'browser_request_takeover' &&
!BROWSER_TOOL_OBSERVES_ONLY[tool]
const watchdogMs = browserToolWatchdogMs(tool, params)
if (yieldsToUser && isUserWorkingInPage()) {
await yieldToUser(
toolCallId,
executionController.signal,
watchdogMs === null
? USER_TAKEOVER_MAX_WAIT_MS
: Math.min(USER_TAKEOVER_MAX_WAIT_MS, Math.floor(watchdogMs / 2))
)
}
const response = dialogResponse(tool, params)
state.dialogResponse = response
? { contents: session.requireAutomationTab().view.webContents, response }
: null
const executionEpoch = ++state.toolExecutionEpoch
const watchdogMs = browserToolWatchdogMs(tool, params)
const executionDeadline = watchdogMs === null ? undefined : Date.now() + watchdogMs
// Checked right before each input the action sends: a user who starts working in the
// page mid-action (between a batch's steps, during a click's target probes) wins it.
const assertCurrentExecution = () => {
if (state.toolExecutionEpoch !== executionEpoch) {
throw new ToolError('This browser action expired before it could dispatch input.')
}
if (yieldsToUser && isUserWorkingInPage()) {
throw new ToolError(USER_TOOK_OVER_MID_ACTION)
}
}
let actionOutcome: BrowserActionOutcome | undefined
const execution = withPostActionObservation(
apps/desktop/src/main/browser-agent/session.test.ts
+16
View File
@@ -1071,6 +1071,22 @@ describe('browser-agent session', () => {
expect(session.listTabs()).toHaveLength(1)
})
it('forgets the user input marker when the session closes, so a reused tab id starts clean', () => {
const agent = session.ensureAutomationTab()
const keyDown = (agent.view as unknown as MockView).webContents.on.mock.calls.find(
([eventName]) => eventName === 'before-input-event'
)?.[1]
if (typeof keyDown !== 'function') throw new Error('no before-input-event listener bound')
keyDown({ preventDefault: vi.fn() }, { type: 'keyDown', isAutoRepeat: false })
expect(session.msSinceUserIntervention()).not.toBeNull()
session.closeSession()
const reopened = session.ensureAutomationTab()
expect(reopened.id).toBe(agent.id)
expect(session.msSinceUserIntervention()).toBeNull()
})
it('opens, switches, and closes tabs with stable ids', () => {
const first = session.ensureTab()
const second = session.addTab()
apps/desktop/src/main/browser-agent/session.ts
+29
View File
@@ -298,6 +298,11 @@ interface BrowserScopeState {
*/
findingTabId: string | null
findingRequestId: number | null
/**
* When the user last clicked, scrolled or typed in the tab the agent drives, and in which tab:
* activity in a tab the agent no longer drives (closed, replaced) holds nothing up.
*/
userIntervention: { tabId: string; at: number } | null
}
function createBrowserScopeState(): BrowserScopeState {
@@ -319,6 +324,7 @@ function createBrowserScopeState(): BrowserScopeState {
automationNeedsAttention: false,
findingTabId: null,
findingRequestId: null,
userIntervention: null,
}
}
@@ -2368,6 +2374,7 @@ function initializeTabView(
return
}
const tab = tabs.find((entry) => entry.view.webContents === contents)
if (tab && tab.id === currentScope.automationTabId) noteUserIntervention(tab.id)
if (tab?.id === currentScope.activeTabId) {
currentScope.visibleTabUserSelected = true
if (mouse.type === 'mouseDown') tab.lastRealUserGestureAt = Date.now()
@@ -2484,6 +2491,14 @@ function initializeTabView(
'before-input-event',
bindToBrowserScope(scopeId, (event, input) => {
const tab = tabs.find((entry) => entry.view === view)
if (
!isDispatchingAgentInput(contents) &&
input.type === 'keyDown' &&
tab &&
tab.id === currentScope.automationTabId
) {
noteUserIntervention(tab.id)
}
if (!isDispatchingAgentInput(contents) && tab?.id === currentScope.activeTabId) {
currentScope.visibleTabUserSelected = true
if (input.type === 'keyDown' && !input.isAutoRepeat) tab.lastRealUserGestureAt = Date.now()
@@ -3117,8 +3132,21 @@ export function claimActiveTabForUser(): AgentTab | null {
return tab
}
/** The user just acted in the page the agent drives; the agent yields until they stop. */
function noteUserIntervention(tabId: string): void {
currentScope.userIntervention = { tabId, at: Date.now() }
}
/** How long ago the user last acted in the tab the agent drives now, or null if never. */
export function msSinceUserIntervention(): number | null {
const intervention = currentScope.userIntervention
if (!intervention || intervention.tabId !== currentScope.automationTabId) return null
return Math.max(0, Date.now() - intervention.at)
}
/** Explicit hand-back after takeover lets automation resume in the same page. */
export function returnAutomationTabToAgent(): void {
currentScope.userIntervention = null
if (currentScope.activeTabId === currentScope.automationTabId) {
currentScope.visibleTabUserSelected = false
}
@@ -3618,6 +3646,7 @@ function closeLiveTabs(): void {
currentScope.automationActive = false
currentScope.automationNeedsAttention = false
currentScope.visibleTabUserSelected = false
currentScope.userIntervention = null
clearFocusedBrowserTab()
}
apps/desktop/src/main/config.ts
+3
View File
@@ -104,6 +104,8 @@ export interface DesktopSettings {
/** Whether omnibox typing may request live Google search completions. */
browserSearchSuggestionsEnabled?: boolean
terminalEnabled?: boolean
/** Keep the machine awake while a chat is running desktop work in the background. */
preventSleepWhileRunning?: boolean
/** Device-wide browser page appearance; `app` follows Sim. */
browserTheme?: 'app' | 'light' | 'dark'
/** Device-wide default zoom for built-in browser pages. */
@@ -236,6 +238,7 @@ const DEFAULT_SETTINGS: DesktopSettings = {
browserEnabled: true,
browserSearchSuggestionsEnabled: true,
terminalEnabled: true,
preventSleepWhileRunning: true,
}
export interface ConfigStore {
apps/desktop/src/main/desktop-executor/approval-notifier.test.ts
+110
View File
@@ -0,0 +1,110 @@
import { describe, expect, it } from 'vitest'
import { createApprovalNotifier } from '@/main/desktop-executor/approval-notifier'
import type { DesktopApprovalItem } from '@/main/desktop-executor/executor'
function approval(toolCallId: string, chatId = 'chat-b'): DesktopApprovalItem {
return {
kind: 'approval_needed',
toolCallId,
toolName: 'terminal',
chatId,
chatTitle: 'Fix CI',
workspaceId: 'ws-1',
summary: 'rm -rf build && npm test',
}
}
function harness(options: { enabled?: boolean; focusedChatId?: string | null } = {}) {
/** Each OS notification the notifier made, and what the user would see of it now. */
const notifications: Array<{
options: { title: string; body: string; silent: boolean }
state: 'created' | 'shown' | 'closed'
click: () => void
}> = []
/** Routes the app was asked to open. */
const opened: Array<string | undefined> = []
let focusedChatId = options.focusedChatId ?? null
const notifier = createApprovalNotifier({
preferences: () => ({
notificationsEnabled: options.enabled ?? true,
notificationSounds: true,
}),
focusedChatId: () => focusedChatId,
openRoute: (route) => {
opened.push(route)
},
createNotification: (notificationOptions) => {
let click = () => {}
const notification = {
options: notificationOptions,
state: 'created' as 'created' | 'shown' | 'closed',
get click() {
return click
},
}
notifications.push(notification)
return {
show: () => {
notification.state = 'shown'
},
close: () => {
notification.state = 'closed'
},
on: (_event, listener) => {
click = listener
},
}
},
})
return {
notifier,
notifications,
opened,
focus: (chatId: string | null) => {
focusedChatId = chatId
},
}
}
describe('approval notifications', () => {
it('notifies once per waiting call and opens its chat, naming neither chat nor command', () => {
const { notifier, notifications, opened } = harness()
notifier.update([approval('call-1')])
notifier.update([approval('call-1')])
expect(notifications).toHaveLength(1)
expect(notifications[0]?.state).toBe('shown')
expect(notifications[0]?.options.body).not.toContain('rm -rf')
expect(notifications[0]?.options.body).not.toContain('Fix CI')
notifications[0]?.click()
expect(opened).toEqual(['/workspace/ws-1/chat/chat-b'])
})
it('closes the notification once the call is decided', () => {
const { notifier, notifications } = harness()
notifier.update([approval('call-1')])
notifier.update([])
expect(notifications[0]?.state).toBe('closed')
})
it('stays quiet for the chat the user is looking at, even after they leave it', () => {
const { notifier, notifications, focus } = harness({ focusedChatId: 'chat-b' })
notifier.update([approval('call-1')])
focus('chat-c')
notifier.update([approval('call-1')])
expect(notifications).toHaveLength(0)
})
it('stays quiet when notifications are switched off', () => {
const { notifier, notifications } = harness({ enabled: false })
notifier.update([approval('call-1')])
expect(notifications).toHaveLength(0)
})
})
apps/desktop/src/main/desktop-executor/approval-notifier.ts
+73
View File
@@ -0,0 +1,73 @@
/**
* Tells the user when a chat they are not looking at waits for their approval. The notification
* only opens the chat at its approval card; approving happens there, where the user sees what
* they are approving. It closes itself once the call is decided, and names neither the chat nor
* the command, since a notification can show on a locked screen.
*/
import type { DesktopApprovalItem } from '@/main/desktop-executor/executor'
interface ApprovalNotification {
show(): void
close(): void
on(event: 'click', listener: () => void): void
}
export interface ApprovalNotifierDeps {
/** Whether the user allows desktop notifications, and with sound. */
preferences: () => { notificationsEnabled: boolean; notificationSounds: boolean }
/** The chat the focused Sim window shows, if any; its own approvals need no notification. */
focusedChatId: () => string | null
openRoute: (route: string | undefined) => void
createNotification: (options: {
title: string
body: string
silent: boolean
}) => ApprovalNotification | null
}
/**
* Creates the notifier the executor feeds each inbox read's approval items to. `update` notifies
* once per newly waiting call and closes notifications for calls decided since; `clear` closes
* them all, for sign-out.
*/
export function createApprovalNotifier(deps: ApprovalNotifierDeps) {
/** Calls already brought to the user's attention, by notification or by being on screen. */
const shown = new Map<string, ApprovalNotification | null>()
return {
/** Shows a notification for each newly waiting call and closes those no longer waiting. */
update(items: DesktopApprovalItem[]): void {
const waiting = new Set(items.map((item) => item.toolCallId))
for (const [toolCallId, notification] of shown) {
if (waiting.has(toolCallId)) continue
notification?.close()
shown.delete(toolCallId)
}
const preferences = deps.preferences()
if (!preferences.notificationsEnabled) return
for (const item of items) {
if (shown.has(item.toolCallId)) continue
if (deps.focusedChatId() === item.chatId) {
shown.set(item.toolCallId, null)
continue
}
const notification = deps.createNotification({
title: 'Approval needed',
body: 'A chat is waiting for your approval.',
silent: !preferences.notificationSounds,
})
if (!notification) return
const route = item.workspaceId
? `/workspace/${encodeURIComponent(item.workspaceId)}/chat/${encodeURIComponent(item.chatId)}`
: undefined
notification.on('click', () => deps.openRoute(route))
notification.show()
shown.set(item.toolCallId, notification)
}
},
clear(): void {
for (const notification of shown.values()) notification?.close()
shown.clear()
},
}
}
apps/desktop/src/main/desktop-executor/client.test.ts
+86
View File
@@ -0,0 +1,86 @@
import { DESKTOP_IMPORT_TOKEN_HEADER } from '@sim/desktop-bridge'
import { describe, expect, it } from 'vitest'
import { createDesktopExecutorClient, UnsendableRequestError } from '@/main/desktop-executor/client'
describe('desktop executor client', () => {
it('refuses a request it cannot encode without sending anything', async () => {
let sent = 0
const client = createDesktopExecutorClient({
origin: () => 'https://sim.test',
fetch: async () => {
sent += 1
return Response.json({ outcome: 'recorded', status: 'completed' })
},
deviceId: '00000000-0000-4000-8000-000000000000',
})
const cyclic: Record<string, unknown> = {}
cyclic.self = cyclic
await expect(
client.complete({
toolCallId: 'call-1',
executionToken: 'token-1',
completion: { status: 'success', message: 'done', data: cyclic },
})
).rejects.toBeInstanceOf(UnsendableRequestError)
expect(sent).toBe(0)
})
it('never splits a character when it shortens a long completion message', async () => {
const sent: string[] = []
const client = createDesktopExecutorClient({
origin: () => 'https://sim.test',
fetch: async (_url, init) => {
sent.push(String(init.body))
return Response.json({ outcome: 'recorded', status: 'completed' })
},
deviceId: '00000000-0000-4000-8000-000000000000',
})
// An emoji straddles the cut point, so a cut by code units would leave half of it behind.
const message = `${'a'.repeat(9_996)}😀${'b'.repeat(10)}`
await client.complete({
toolCallId: 'call-1',
executionToken: 'token-1',
completion: { status: 'success', message },
})
const sentMessage: string = JSON.parse(sent[0] ?? '{}').message
const loneSurrogate = /[\uD800-\uDBFF](?![\uDC00-\uDFFF])|(?<![\uD800-\uDBFF])[\uDC00-\uDFFF]/
expect(loneSurrogate.test(sentMessage)).toBe(false)
})
it('sends an import entry with its execution token in a header, never in the URL', async () => {
const received: Array<{ url: string; token: string | null }> = []
const client = createDesktopExecutorClient({
origin: () => 'https://sim.test',
fetch: async (url, init) => {
received.push({ url, token: new Headers(init.headers).get(DESKTOP_IMPORT_TOKEN_HEADER) })
return Response.json({ id: 'file-1', name: 'notes.txt' })
},
deviceId: '00000000-0000-4000-8000-000000000000',
})
await client.importEntry(
{
call: {
toolCallId: 'call-1',
toolName: 'import_local_files',
args: {},
chatId: 'chat-1',
workspaceId: 'ws-1',
executionToken: 'secret-token-1',
},
kind: 'file',
sourceName: 'notes.txt',
relativePath: '',
content: new Blob(['hello']),
},
new AbortController().signal
)
expect(received).toHaveLength(1)
expect(received[0]?.token).toBe('secret-token-1')
expect(decodeURIComponent(received[0]?.url ?? '')).not.toContain('secret-token-1')
})
})
apps/desktop/src/main/desktop-executor/client.ts
+244
View File
@@ -0,0 +1,244 @@
/**
* HTTP client for Sim's background executor routes. Every request carries the app partition's
* own session cookie, which is the session the device registered under; Sim refuses any other.
*/
import { DESKTOP_IMPORT_TOKEN_HEADER } from '@sim/desktop-bridge'
import { getErrorMessage } from '@sim/utils/errors'
import { parseRetryAfter } from '@sim/utils/retry'
import { truncateAtCodePoint } from '@sim/utils/string'
import {
type ClaimedDesktopCall,
COMPLETION_MESSAGE_MAX_CHARS,
type DesktopCompletionOutcome,
type DesktopCompletionRequest,
type DesktopDeviceRegistration,
type DesktopExecutorTiming,
type DesktopImportEntryRequest,
type DesktopImportedEntry,
type DesktopInboxItem,
parseClaim,
parseCompletionOutcome,
parseImportedEntry,
parseInbox,
parseRegistration,
} from '@/main/desktop-executor/protocol'
const REQUEST_TIMEOUT_MS = 15_000
/** An import entry carries up to a 64 MB file, so it gets longer than a control request. */
const IMPORT_TIMEOUT_MS = 300_000
/** A request Sim answered with a failure, or that never got an answer (`status` 0). */
export class DeviceRequestError extends Error {
constructor(
readonly status: number,
message: string,
readonly retryAfterMs: number | null = null
) {
super(message)
this.name = 'DeviceRequestError'
}
/** The device's registration no longer holds for this session; it must register again. */
get unregistered(): boolean {
return this.status === 401
}
/** Worth retrying unchanged: no answer, a timeout, a server fault, or a rate limit. */
get transient(): boolean {
return this.status === 0 || this.status === 408 || this.status === 429 || this.status >= 500
}
}
/**
* A request this device could not even encode (a result holding a cycle or a BigInt). Nothing was
* sent, and sending it again cannot help.
*/
export class UnsendableRequestError extends Error {
constructor(message: string) {
super(message)
this.name = 'UnsendableRequestError'
}
}
function encode(body: Record<string, unknown>): string {
try {
return JSON.stringify(body)
} catch (error) {
throw new UnsendableRequestError(getErrorMessage(error))
}
}
type DeviceFetch = (url: string, init: RequestInit) => Promise<Response>
export interface DesktopExecutorClient {
register(registration: DesktopDeviceRegistration): Promise<DesktopExecutorTiming>
listInbox(): Promise<DesktopInboxItem[]>
/** Opens the doorbell; the caller reads the event stream and aborts it to close. */
openInboxStream(signal: AbortSignal): Promise<ReadableStream<Uint8Array>>
claim(toolCallId: string): Promise<ClaimedDesktopCall>
renewLease(toolCallId: string, executionToken: string): Promise<void>
complete(request: DesktopCompletionRequest): Promise<DesktopCompletionOutcome>
importEntry(
request: DesktopImportEntryRequest,
signal: AbortSignal
): Promise<DesktopImportedEntry>
}
interface DesktopExecutorClientOptions {
origin: () => string
fetch: DeviceFetch
deviceId: string
}
async function errorMessage(response: Response): Promise<string> {
const body = (await response.json().catch(() => null)) as { error?: unknown } | null
return typeof body?.error === 'string' ? body.error : `HTTP ${response.status}`
}
/**
* Creates the client for one install id. Each request times out on its own and fails with a
* {@link DeviceRequestError}, so the executor decides what to retry; responses are parsed before
* they are returned, and a malformed one is a 502.
*/
export function createDesktopExecutorClient(
options: DesktopExecutorClientOptions
): DesktopExecutorClient {
const { deviceId } = options
async function send(
method: 'GET' | 'POST' | 'PUT',
path: string,
body?: Record<string, unknown> | Blob,
signal?: AbortSignal,
timeoutMs = REQUEST_TIMEOUT_MS,
extraHeaders: Record<string, string> = {}
): Promise<Response> {
const timeout = AbortSignal.timeout(timeoutMs)
const raw = body instanceof Blob
const encoded = body === undefined ? undefined : raw ? body : encode(body)
let response: Response
try {
response = await options.fetch(`${options.origin()}${path}`, {
method,
credentials: 'include',
headers: {
Accept: 'application/json',
...(body
? { 'Content-Type': raw ? 'application/octet-stream' : 'application/json' }
: {}),
...extraHeaders,
},
...(encoded !== undefined ? { body: encoded } : {}),
signal: signal ? AbortSignal.any([signal, timeout]) : timeout,
})
} catch (error) {
throw new DeviceRequestError(0, getErrorMessage(error))
}
if (!response.ok) {
throw new DeviceRequestError(
response.status,
await errorMessage(response),
parseRetryAfter(response.headers.get('retry-after'))
)
}
return response
}
async function json(
method: 'GET' | 'POST',
path: string,
body?: Record<string, unknown>
): Promise<unknown> {
const response = await send(method, path, body)
return response.json().catch(() => null)
}
function malformed(route: string): DeviceRequestError {
return new DeviceRequestError(502, `Sim returned a malformed ${route} response`)
}
return {
async register(registration) {
const timing = parseRegistration(
await json('POST', '/api/desktop/devices', { ...registration })
)
if (!timing) throw malformed('registration')
return timing
},
async listInbox() {
const items = parseInbox(
await json('GET', `/api/desktop/inbox?deviceId=${encodeURIComponent(deviceId)}`)
)
if (!items) throw malformed('inbox')
return items
},
async openInboxStream(signal) {
let response: Response
try {
response = await options.fetch(
`${options.origin()}/api/desktop/inbox/stream?deviceId=${encodeURIComponent(deviceId)}`,
{
method: 'GET',
credentials: 'include',
headers: { Accept: 'text/event-stream' },
cache: 'no-store',
signal,
}
)
} catch (error) {
throw new DeviceRequestError(0, getErrorMessage(error))
}
if (!response.ok || !response.body) {
throw new DeviceRequestError(response.status, `Doorbell refused: HTTP ${response.status}`)
}
return response.body
},
async claim(toolCallId) {
const claimed = parseClaim(
toolCallId,
await json('POST', '/api/desktop/tool/claim', { deviceId, toolCallId })
)
if (!claimed) throw malformed('claim')
return claimed
},
async renewLease(toolCallId, executionToken) {
await send('POST', '/api/desktop/tool/lease', { deviceId, toolCallId, executionToken })
},
async complete({ toolCallId, executionToken, completion }) {
const outcome = parseCompletionOutcome(
await json('POST', '/api/desktop/tool/complete', {
deviceId,
toolCallId,
executionToken,
status: completion.status,
// Leaves room for the ellipsis, so a cut message still fits Sim's limit.
message: truncateAtCodePoint(completion.message, COMPLETION_MESSAGE_MAX_CHARS - 3),
...(completion.data !== undefined ? { data: completion.data } : {}),
})
)
if (!outcome) throw malformed('completion')
return outcome
},
async importEntry({ call, kind, sourceName, relativePath, content }, signal) {
const query = new URLSearchParams({
deviceId,
toolCallId: call.toolCallId,
kind,
sourceName,
relativePath,
})
const response = await send(
'PUT',
`/api/desktop/tool/import?${query}`,
content,
signal,
IMPORT_TIMEOUT_MS,
{ [DESKTOP_IMPORT_TOKEN_HEADER]: call.executionToken }
)
const entry = parseImportedEntry(await response.json().catch(() => null))
if (!entry) throw malformed('import')
return entry
},
}
}
apps/desktop/src/main/desktop-executor/doorbell.test.ts
+166
View File
@@ -0,0 +1,166 @@
import { describe, expect, it, vi } from 'vitest'
import { DeviceRequestError } from '@/main/desktop-executor/client'
import { InboxDoorbell, parseServerSentEvents } from '@/main/desktop-executor/doorbell'
const encoder = new TextEncoder()
/** A stream the test writes SSE text into, closing when told to or when its reader aborts. */
function controllableStream(signal: AbortSignal) {
let controller: ReadableStreamDefaultController<Uint8Array> | undefined
const stream = new ReadableStream<Uint8Array>({
start(c) {
controller = c
},
})
signal.addEventListener('abort', () => {
try {
controller?.error(new Error('aborted'))
} catch {}
})
return {
stream,
write: (text: string) => controller?.enqueue(encoder.encode(text)),
end: () => controller?.close(),
}
}
function harness(options: { staleAfterMs?: number; retryBaseMs?: number } = {}) {
const connections: ReturnType<typeof controllableStream>[] = []
const failures: DeviceRequestError[] = []
/** How many times the doorbell rang, and asked to register again. */
const counts = { rings: 0, unregistered: 0 }
const onRing = () => {
counts.rings += 1
}
const onUnregistered = () => {
counts.unregistered += 1
}
const doorbell = new InboxDoorbell({
client: {
openInboxStream: async (signal) => {
const failure = failures.shift()
if (failure) throw failure
const connection = controllableStream(signal)
connections.push(connection)
return connection.stream
},
},
onRing,
onUnregistered,
retryBaseMs: 5,
...options,
})
return { doorbell, connections, failures, counts }
}
describe('parseServerSentEvents', () => {
it('keeps a partial event for the next chunk and skips comments', () => {
const { events, rest } = parseServerSentEvents(
': heartbeat\n\nevent: inbox_changed\ndata: {"reason":"call"}\n\nevent: rot'
)
expect(events).toEqual([{ event: 'inbox_changed', data: '{"reason":"call"}' }])
expect(rest).toBe('event: rot')
})
})
describe('InboxDoorbell', () => {
it('rings on connect and on every inbox change', async () => {
const { doorbell, connections, counts } = harness()
doorbell.start()
await vi.waitFor(() => expect(counts.rings).toBe(1))
connections[0]?.write('event: inbox_changed\ndata: {"reason":"call"}\n\n')
await vi.waitFor(() => expect(counts.rings).toBe(2))
doorbell.stop()
})
it('opens a new connection when Sim rotates the stream', async () => {
const { doorbell, connections } = harness()
doorbell.start()
await vi.waitFor(() => expect(connections).toHaveLength(1))
connections[0]?.write('event: rotate\ndata: {}\n\n')
await vi.waitFor(() => expect(connections).toHaveLength(2))
doorbell.stop()
})
it('reconnects after the server drops the stream or refuses a connection', async () => {
const { doorbell, connections, failures, counts } = harness()
failures.push(new DeviceRequestError(0, 'offline'), new DeviceRequestError(502, 'deploy'))
doorbell.start()
await vi.waitFor(() => expect(connections).toHaveLength(1))
connections[0]?.end()
await vi.waitFor(() => expect(connections).toHaveLength(2))
expect(counts.rings).toBe(2)
doorbell.stop()
})
it('backs off when every stream ends as soon as it opens', async () => {
vi.useFakeTimers()
const opened: number[] = []
const doorbell = new InboxDoorbell({
client: {
openInboxStream: async () => {
opened.push(Date.now())
return new ReadableStream<Uint8Array>({ start: (controller) => controller.close() })
},
},
onRing: () => {},
onUnregistered: () => {},
retryBaseMs: 5,
})
try {
doorbell.start()
await vi.advanceTimersByTimeAsync(400)
doorbell.stop()
} finally {
vi.useRealTimers()
}
// A fixed 5 ms retry would open 80; doubling from 5 ms opens a handful.
expect(opened.length).toBeGreaterThan(2)
expect(opened.length).toBeLessThan(15)
})
it('replaces a connection that goes silent past the heartbeat', async () => {
const { doorbell, connections } = harness({ staleAfterMs: 30 })
doorbell.start()
await vi.waitFor(() => expect(connections.length).toBeGreaterThanOrEqual(2))
doorbell.stop()
})
it('reports a device Sim no longer recognizes', async () => {
const { doorbell, failures, counts } = harness()
failures.push(new DeviceRequestError(401, 'unregistered'))
doorbell.start()
await vi.waitFor(() => expect(counts.unregistered).toBeGreaterThan(0))
doorbell.stop()
})
it('reconnects at once on wake, without waiting out a backoff', async () => {
const { doorbell, connections, counts } = harness({ retryBaseMs: 60_000 })
doorbell.start()
await vi.waitFor(() => expect(connections).toHaveLength(1))
doorbell.wake()
await vi.waitFor(() => expect(connections).toHaveLength(2))
await vi.waitFor(() => expect(counts.rings).toBe(2))
doorbell.stop()
})
it('starts on wake after sleep stopped it', async () => {
const { doorbell, connections } = harness({ retryBaseMs: 60_000 })
doorbell.start()
await vi.waitFor(() => expect(connections).toHaveLength(1))
doorbell.stop()
doorbell.wake()
await vi.waitFor(() => expect(connections).toHaveLength(2))
doorbell.stop()
})
})
apps/desktop/src/main/desktop-executor/doorbell.ts
+186
View File
@@ -0,0 +1,186 @@
/**
* The inbox doorbell: a long-lived SSE stream on which Sim hints that this device's inbox
* changed. Hints only; the inbox read they trigger is the record, and the executor also reads
* it on a timer, so a lost event costs latency, never correctness. The stream reconnects with
* backoff, opens its replacement when Sim announces a rotation, and is replaced when it goes
* silent past the server's heartbeat.
*/
import { createLogger } from '@sim/logger'
import { getErrorMessage } from '@sim/utils/errors'
import { interruptibleSleep } from '@sim/utils/helpers'
import { backoffWithJitter } from '@sim/utils/retry'
import { type DesktopExecutorClient, DeviceRequestError } from '@/main/desktop-executor/client'
const logger = createLogger('DesktopExecutorDoorbell')
/** Sim heartbeats every 30 s; two missed ones mean the connection is gone. */
const STALE_STREAM_MS = 75_000
const RECONNECT_MAX_MS = 30_000
/** A stream that stayed open this long was healthy; its clean end starts backoff afresh. */
const HEALTHY_STREAM_MS = 60_000
const HANDSHAKE_TIMEOUT_MS = 15_000
interface DoorbellOptions {
client: Pick<DesktopExecutorClient, 'openInboxStream'>
/** The inbox may have changed: on every event, and on every (re)connect. */
onRing: () => void
onUnregistered: () => void
staleAfterMs?: number
retryBaseMs?: number
}
interface ServerSentEvent {
event: string
data: string
}
/** Parses complete SSE blocks out of `buffer`, returning them and the unparsed remainder. */
export function parseServerSentEvents(buffer: string): {
events: ServerSentEvent[]
rest: string
} {
const blocks = buffer.split(/\r?\n\r?\n/)
const rest = blocks.pop() ?? ''
const events: ServerSentEvent[] = []
for (const block of blocks) {
let event = 'message'
const data: string[] = []
for (const line of block.split(/\r?\n/)) {
if (line.startsWith(':')) continue
if (line.startsWith('event:')) event = line.slice(6).trim()
else if (line.startsWith('data:')) data.push(line.slice(5).trimStart())
}
if (data.length > 0 || event !== 'message') events.push({ event, data: data.join('\n') })
}
return { events, rest }
}
export class InboxDoorbell {
private running = false
/** Which loop is current; a loop left over from before a stop ends at its next turn. */
private loopGeneration = 0
/** The connection was dropped on purpose; open the next one without backing off. */
private reconnectNow = false
private connection: AbortController | null = null
private sleeper: AbortController | null = null
constructor(private readonly options: DoorbellOptions) {}
start(): void {
if (this.running) return
this.running = true
this.loopGeneration += 1
void this.loop(this.loopGeneration)
}
stop(): void {
this.running = false
this.loopGeneration += 1
this.connection?.abort()
this.sleeper?.abort()
}
/**
* After waking or coming back online: a stopped doorbell starts, and a running one drops its
* connection, which may be dead without knowing it, and reconnects at once.
*/
wake(): void {
if (!this.running) {
this.start()
return
}
this.reconnectNow = true
this.connection?.abort()
this.sleeper?.abort()
}
private async loop(generation: number): Promise<void> {
let attempt = 0
const current = () => this.loopGeneration === generation
while (current()) {
const openedAt = Date.now()
const rotated = await this.connectOnce().then(
(result) => {
if (result === 'rotated') {
attempt = 0
return true
}
// A stream something keeps closing right away is a failing connection, not a healthy one.
attempt = Date.now() - openedAt >= HEALTHY_STREAM_MS ? 0 : attempt + 1
return false
},
(error: unknown) => {
if (this.reconnectNow) {
this.reconnectNow = false
attempt = 0
return true
}
attempt += 1
if (error instanceof DeviceRequestError && error.unregistered) {
this.options.onUnregistered()
}
logger.info('Doorbell disconnected', {
attempt,
error: getErrorMessage(error),
})
return false
}
)
if (!current() || rotated) continue
this.sleeper = new AbortController()
await interruptibleSleep(
attempt === 0
? (this.options.retryBaseMs ?? 1_000)
: backoffWithJitter(attempt, null, {
baseMs: this.options.retryBaseMs ?? 1_000,
maxMs: RECONNECT_MAX_MS,
}),
this.sleeper.signal
)
this.sleeper = null
}
}
/** Holds one connection open; resolves when it ends cleanly or Sim asks for a rotation. */
private async connectOnce(): Promise<'ended' | 'rotated'> {
const connection = new AbortController()
this.connection = connection
const staleAfterMs = this.options.staleAfterMs ?? STALE_STREAM_MS
let staleTimer: ReturnType<typeof setTimeout> | null = null
const touch = () => {
if (staleTimer) clearTimeout(staleTimer)
staleTimer = setTimeout(() => connection.abort(), staleAfterMs)
}
try {
// A stream whose response never starts is as dead as one that went quiet.
const handshake = setTimeout(() => connection.abort(), HANDSHAKE_TIMEOUT_MS)
const stream = await this.options.client
.openInboxStream(connection.signal)
.finally(() => clearTimeout(handshake))
touch()
this.options.onRing()
const reader = stream.getReader()
const decoder = new TextDecoder()
let buffer = ''
for (;;) {
const { value, done } = await reader.read()
if (done) return 'ended'
touch()
buffer += decoder.decode(value, { stream: true })
const parsed = parseServerSentEvents(buffer)
buffer = parsed.rest
for (const event of parsed.events) {
if (event.event === 'rotate') {
void reader.cancel().catch(() => {})
return 'rotated'
}
if (event.event === 'inbox_changed') this.options.onRing()
}
}
} finally {
if (staleTimer) clearTimeout(staleTimer)
connection.abort()
if (this.connection === connection) this.connection = null
}
}
}
apps/desktop/src/main/desktop-executor/executor.test.ts
+925
View File
@@ -0,0 +1,925 @@
import type { DesktopToolCompletion } from '@sim/desktop-bridge/tool-results'
import { sleep } from '@sim/utils/helpers'
import { describe, expect, it, vi } from 'vitest'
import {
type DesktopExecutorClient,
DeviceRequestError,
UnsendableRequestError,
} from '@/main/desktop-executor/client'
import {
type DesktopApprovalItem,
DesktopExecutor,
type DesktopToolRunner,
} from '@/main/desktop-executor/executor'
import type { ExecutorJournal, JournalEntry } from '@/main/desktop-executor/journal'
import type {
ClaimedDesktopCall,
DesktopCompletionRequest,
DesktopInboxItem,
} from '@/main/desktop-executor/protocol'
/**
* Failure modes of the executor's state machine, each against fakes of its three boundaries:
* Sim's device routes, the encrypted journal, and the tools on this machine.
*/
interface Deferred<T> {
promise: Promise<T>
resolve(value: T): void
reject(error: unknown): void
}
function deferred<T>(): Deferred<T> {
let resolve: (value: T) => void = () => {}
let reject: (error: unknown) => void = () => {}
const promise = new Promise<T>((res, rej) => {
resolve = res
reject = rej
})
return { promise, resolve, reject }
}
function callItem(
toolCallId: string,
chatId: string,
toolName = 'browser_click'
): DesktopInboxItem {
return { kind: 'call', toolCallId, toolName, chatId, workspaceId: 'ws-1' }
}
const DONE: DesktopToolCompletion = { status: 'success', message: 'done', data: { ok: true } }
class FakeSim {
inbox: DesktopInboxItem[] = []
readonly claims: string[] = []
readonly renewals: string[] = []
readonly completions: DesktopCompletionRequest[] = []
claimError: ((toolCallId: string) => DeviceRequestError | null) | null = null
renewError: ((toolCallId: string) => DeviceRequestError | null) | null = null
completeErrors: DeviceRequestError[] = []
completionOutcome: 'recorded' | 'duplicate' | 'superseded' = 'recorded'
readonly client: DesktopExecutorClient = {
register: async () => {
throw new Error('not used')
},
listInbox: async () => [...this.inbox],
openInboxStream: async () => {
throw new Error('not used')
},
importEntry: async () => {
throw new Error('not used')
},
claim: async (toolCallId) => {
this.claims.push(toolCallId)
const error = this.claimError?.(toolCallId)
if (error) throw error
const item = this.inbox.find(
(entry): entry is Extract<DesktopInboxItem, { kind: 'call' }> =>
entry.kind === 'call' && entry.toolCallId === toolCallId
)
if (!item) throw new DeviceRequestError(404, 'gone')
this.inbox = this.inbox.filter((entry) => entry !== item)
return {
toolCallId,
toolName: item.toolName,
args: { step: toolCallId },
chatId: item.chatId,
workspaceId: item.workspaceId,
executionToken: `token-${toolCallId}`,
} satisfies ClaimedDesktopCall
},
renewLease: async (toolCallId) => {
this.renewals.push(toolCallId)
const error = this.renewError?.(toolCallId)
if (error) throw error
},
complete: async (request) => {
const error = this.completeErrors.shift()
if (error) throw error
this.completions.push(request)
return this.completionOutcome
},
}
}
class MemoryJournal implements ExecutorJournal {
readonly entries = new Map<string, JournalEntry>()
readonly history: JournalEntry[] = []
/** A transition the disk refuses, as a full disk or a failed encryption would. */
failOn: JournalEntry['state'] | null = null
async load() {
return [...this.entries.values()]
}
async put(entry: JournalEntry) {
if (entry.state === this.failOn) throw new Error('disk full')
this.entries.set(entry.toolCallId, entry)
this.history.push(entry)
}
async remove(toolCallId: string) {
this.entries.delete(toolCallId)
}
async clear() {
this.entries.clear()
}
}
class FakeRunner implements DesktopToolRunner {
readonly started: string[] = []
readonly cancelled: string[] = []
readonly pending = new Map<string, Deferred<DesktopToolCompletion>>()
/** When set, every call finishes at once with this completion. */
immediate: DesktopToolCompletion | null = null
onStart: ((call: ClaimedDesktopCall) => void) | null = null
/** An action that finishes with its full result even after being told to stop. */
ignoresAbort = false
async run(call: ClaimedDesktopCall, signal: AbortSignal) {
this.started.push(call.toolCallId)
this.onStart?.(call)
if (this.immediate) return this.immediate
const result = deferred<DesktopToolCompletion>()
this.pending.set(call.toolCallId, result)
signal.addEventListener('abort', () => {
if (!this.ignoresAbort)
result.resolve({ status: 'error', message: 'This browser action was cancelled.' })
})
return result.promise
}
async cancel(call: ClaimedDesktopCall) {
this.cancelled.push(call.toolCallId)
}
finish(toolCallId: string, completion: DesktopToolCompletion = DONE) {
const result = this.pending.get(toolCallId)
if (!result) throw new Error(`${toolCallId} is not running`)
result.resolve(completion)
}
}
function setup(
options: { leaseRenewMs?: number; maxHeldCalls?: number; deliveryAwakeLimitMs?: number } = {}
) {
/** Approval items the executor handed to the notifier, one array per inbox read. */
const approvals: DesktopApprovalItem[][] = []
const sim = new FakeSim()
const journal = new MemoryJournal()
const runner = new FakeRunner()
const onUnregistered = vi.fn()
const busy: boolean[] = []
/** Calls whose result the executor reported as reaching the model, in order. */
const delivered: string[] = []
/** Calls Sim finished with without their real result reaching the model, in order. */
const notDelivered: string[] = []
const executor = new DesktopExecutor({
client: sim.client,
journal,
runner,
leaseRenewMs: options.leaseRenewMs ?? 60_000,
retryBaseMs: 5,
onUnregistered,
onBusyChange: (value) => busy.push(value),
onApprovals: (items) => approvals.push(items),
onResultDelivered: (toolCallId) => delivered.push(toolCallId),
onResultNotDelivered: (toolCallId) => notDelivered.push(toolCallId),
...(options.maxHeldCalls ? { maxHeldCalls: options.maxHeldCalls } : {}),
...(options.deliveryAwakeLimitMs !== undefined
? { deliveryAwakeLimitMs: options.deliveryAwakeLimitMs }
: {}),
})
return {
sim,
journal,
runner,
executor,
onUnregistered,
busy,
approvals,
delivered,
notDelivered,
}
}
describe('claiming', () => {
it('claims an offered call, runs it from the server record, and reports with its token', async () => {
const { sim, journal, runner, executor } = setup()
runner.immediate = DONE
sim.inbox = [callItem('call-1', 'chat-a')]
await executor.reconcile()
await vi.waitFor(() => expect(sim.completions).toHaveLength(1))
expect(sim.completions[0]).toEqual({
toolCallId: 'call-1',
executionToken: 'token-call-1',
completion: DONE,
})
await vi.waitFor(() => expect(journal.entries.size).toBe(0))
expect(executor.heldCallCount()).toBe(0)
})
it('reports a result as reaching the model only once Sim takes it as the call own', async () => {
const { sim, journal, runner, executor, delivered, notDelivered } = setup()
runner.immediate = DONE
// Sim settled this call first: the result never reached the model.
sim.completionOutcome = 'superseded'
sim.inbox = [callItem('call-superseded', 'chat-a')]
await executor.reconcile()
await vi.waitFor(() => expect(notDelivered).toEqual(['call-superseded']))
expect(delivered).toEqual([])
// Taken by Sim, even with nothing written locally (no OS encryption, say).
sim.completionOutcome = 'recorded'
journal.failOn = 'result'
sim.inbox = [callItem('call-recorded', 'chat-a')]
await executor.reconcile()
await vi.waitFor(() => expect(delivered).toEqual(['call-recorded']))
expect(notDelivered).toEqual(['call-superseded'])
})
it('reports a result Sim settled first, while a failed send waited to retry, as never reaching the model', async () => {
const { sim, runner, executor, delivered, notDelivered } = setup()
// The terminal handed back a command still going, and the first send of that result fails.
runner.immediate = { status: 'success', message: 'running', data: { status: 'running' } }
sim.completeErrors = [new DeviceRequestError(503, 'unavailable')]
sim.inbox = [callItem('call-1', 'chat-a', 'terminal')]
await executor.reconcile()
await vi.waitFor(() => expect(runner.started).toEqual(['call-1']))
// The user pressed Stop meanwhile, so Sim settled the call before the retry reached it.
sim.completionOutcome = 'superseded'
sim.inbox = [{ kind: 'cancel', toolCallId: 'call-1' }]
await executor.reconcile()
await vi.waitFor(() => expect(notDelivered).toEqual(['call-1']))
expect(delivered).toEqual([])
})
it('reports a stand-in Sim holds as a duplicate as never reaching the model, within one app run', async () => {
const { sim, runner, executor, delivered, notDelivered } = setup()
// Too large to send, so Sim never took the real result; the stand-in's first answer was lost.
runner.immediate = DONE
sim.completeErrors = [
new DeviceRequestError(413, 'too large'),
new DeviceRequestError(0, 'connection reset'),
]
sim.completionOutcome = 'duplicate'
sim.inbox = [callItem('call-1', 'chat-a', 'terminal')]
await executor.reconcile()
await vi.waitFor(() => expect(notDelivered).toEqual(['call-1']))
expect(delivered).toEqual([])
})
it('reports a result Sim refused as never reaching the model', async () => {
const { sim, runner, executor, notDelivered } = setup()
runner.immediate = DONE
sim.completeErrors = [new DeviceRequestError(403, 'forbidden')]
sim.inbox = [callItem('call-refused', 'chat-a')]
await executor.reconcile()
await vi.waitFor(() => expect(notDelivered).toEqual(['call-refused']))
})
it('reports no result that stands in for what the action produced as reaching the model', async () => {
const { sim, runner, executor, delivered, notDelivered } = setup()
const standIns: DesktopToolCompletion[] = [
{ status: 'cancelled', message: 'Stopped.' },
{ status: 'error', message: 'Too large.', data: { resultOmitted: true } },
{ status: 'error', message: 'Not started.', data: { notStarted: true } },
{ status: 'error', message: 'Unknown.', data: { outcomeUnknown: true } },
]
for (const [index, completion] of standIns.entries()) {
runner.immediate = completion
sim.inbox = [callItem(`stand-in-${index}`, 'chat-a')]
await executor.reconcile()
await vi.waitFor(() => expect(sim.completions).toHaveLength(index + 1))
}
runner.immediate = DONE
sim.inbox = [callItem('real', 'chat-a')]
await executor.reconcile()
await vi.waitFor(() => expect(delivered).toEqual(['real']))
expect(notDelivered).toEqual(['stand-in-0', 'stand-in-1', 'stand-in-2', 'stand-in-3'])
})
it('claims a whole backlog at once, before any of it runs', async () => {
const { sim, runner, executor } = setup()
sim.inbox = [callItem('a-1', 'chat-a'), callItem('a-2', 'chat-a'), callItem('a-3', 'chat-a')]
await executor.reconcile()
expect(sim.claims).toEqual(['a-1', 'a-2', 'a-3'])
await vi.waitFor(() => expect(runner.started).toEqual(['a-1']))
})
it('runs one chat in inbox order and different chats side by side', async () => {
const { sim, runner, executor } = setup()
sim.inbox = [
callItem('a-1', 'chat-a'),
callItem('b-1', 'chat-b', 'terminal'),
callItem('a-2', 'chat-a'),
]
await executor.reconcile()
await vi.waitFor(() => expect(runner.started).toEqual(['a-1', 'b-1']))
runner.finish('b-1')
runner.finish('a-1')
await vi.waitFor(() => expect(runner.started).toEqual(['a-1', 'b-1', 'a-2']))
})
it('does not claim a call it already holds when the inbox lists it again', async () => {
const { sim, executor } = setup()
const item = callItem('call-1', 'chat-a')
sim.inbox = [item]
await executor.reconcile()
sim.inbox = [item]
await executor.reconcile()
expect(sim.claims).toEqual(['call-1'])
})
it.each([
[404, 'claimed elsewhere or settled'],
[403, 'still awaiting approval'],
[409, 'turn ended'],
])('drops a claim Sim refuses with %i (%s) without running anything', async (status) => {
const { sim, journal, runner, executor } = setup()
sim.claimError = () => new DeviceRequestError(status, 'refused')
sim.inbox = [callItem('call-1', 'chat-a')]
await executor.reconcile()
expect(runner.started).toEqual([])
expect(journal.entries.size).toBe(0)
expect(executor.heldCallCount()).toBe(0)
})
it('stops claiming at its in-flight cap and claims the rest once a slot frees', async () => {
const { sim, runner, executor } = setup({ maxHeldCalls: 2 })
sim.inbox = [callItem('a-1', 'chat-a'), callItem('b-1', 'chat-b'), callItem('c-1', 'chat-c')]
await executor.reconcile()
expect(sim.claims).toEqual(['a-1', 'b-1'])
await vi.waitFor(() => expect(runner.started).toContain('a-1'))
runner.finish('a-1')
await vi.waitFor(() => expect(executor.heldCallCount()).toBe(1))
await executor.reconcile()
expect(sim.claims).toEqual(['a-1', 'b-1', 'c-1'])
})
it('does not claim while paused for sleep', async () => {
const { sim, executor } = setup()
sim.inbox = [callItem('call-1', 'chat-a')]
executor.setPaused(true)
await executor.reconcile()
expect(sim.claims).toEqual([])
})
it('records that a call started before the action begins', async () => {
const { sim, journal, runner, executor } = setup()
let stateAtStart: JournalEntry['state'] | undefined
runner.onStart = (call) => {
stateAtStart = journal.entries.get(call.toolCallId)?.state
}
sim.inbox = [callItem('call-1', 'chat-a')]
await executor.reconcile()
await vi.waitFor(() => expect(stateAtStart).toBe('started'))
})
})
describe('leases', () => {
it('renews the lease while a call waits in its chat queue and while it runs', async () => {
const { sim, runner, executor } = setup({ leaseRenewMs: 20 })
sim.inbox = [callItem('a-1', 'chat-a'), callItem('a-2', 'chat-a')]
await executor.reconcile()
await vi.waitFor(() => {
expect(sim.renewals).toContain('a-1')
expect(sim.renewals).toContain('a-2')
})
expect(runner.started).toEqual(['a-1'])
runner.finish('a-1')
await vi.waitFor(() => expect(runner.started).toEqual(['a-1', 'a-2']))
runner.finish('a-2')
await vi.waitFor(() => expect(executor.heldCallCount()).toBe(0))
const renewalsAfterAck = sim.renewals.length
await sleep(80)
expect(sim.renewals.length).toBe(renewalsAfterAck)
})
it('stops the action when Sim revokes its lease', async () => {
const { sim, runner, executor } = setup({ leaseRenewMs: 20 })
sim.inbox = [callItem('call-1', 'chat-a')]
sim.renewError = () => new DeviceRequestError(410, 'revoked')
await executor.reconcile()
await vi.waitFor(() => expect(runner.cancelled).toEqual(['call-1']))
await vi.waitFor(() => expect(sim.completions.map((c) => c.toolCallId)).toEqual(['call-1']))
expect(executor.heldCallCount()).toBe(0)
})
})
describe('reporting', () => {
it('retries a result Sim did not acknowledge, without running the action again', async () => {
const { sim, journal, runner, executor } = setup()
runner.immediate = DONE
sim.completeErrors = [
new DeviceRequestError(0, 'offline'),
new DeviceRequestError(503, 'deploying'),
]
sim.inbox = [callItem('call-1', 'chat-a')]
await executor.reconcile()
await vi.waitFor(() => expect(journal.history.map((entry) => entry.state)).toContain('result'))
await vi.waitFor(() => expect(sim.completions).toHaveLength(1))
expect(runner.started).toEqual(['call-1'])
await vi.waitFor(() => expect(journal.entries.size).toBe(0))
})
it.each(['duplicate', 'superseded'] as const)(
'treats a %s completion as acknowledged',
async (outcome) => {
const { sim, journal, runner, executor } = setup()
runner.immediate = DONE
sim.completionOutcome = outcome
sim.inbox = [callItem('call-1', 'chat-a')]
await executor.reconcile()
await vi.waitFor(() => expect(journal.entries.size).toBe(0))
expect(sim.completions).toHaveLength(1)
}
)
})
describe('stopping', () => {
it('stops a running call the inbox lists as cancelled, and acknowledges it', async () => {
const { sim, runner, executor } = setup()
sim.inbox = [callItem('call-1', 'chat-a')]
await executor.reconcile()
await vi.waitFor(() => expect(runner.started).toEqual(['call-1']))
sim.inbox = [{ kind: 'cancel', toolCallId: 'call-1' }]
await executor.reconcile()
await vi.waitFor(() => expect(runner.cancelled).toEqual(['call-1']))
await vi.waitFor(() => expect(sim.completions.map((c) => c.toolCallId)).toEqual(['call-1']))
await vi.waitFor(() => expect(executor.heldCallCount()).toBe(0))
})
it('never starts a queued call that was stopped, and acknowledges the stop', async () => {
const { sim, runner, executor } = setup()
sim.inbox = [callItem('a-1', 'chat-a'), callItem('a-2', 'chat-a')]
await executor.reconcile()
await vi.waitFor(() => expect(runner.started).toEqual(['a-1']))
sim.inbox = [{ kind: 'cancel', toolCallId: 'a-2' }]
await executor.reconcile()
await vi.waitFor(() => expect(sim.completions.map((c) => c.toolCallId)).toEqual(['a-2']))
expect(sim.completions[0]?.completion.status).toBe('cancelled')
runner.finish('a-1')
await vi.waitFor(() => expect(executor.heldCallCount()).toBe(0))
expect(runner.started).toEqual(['a-1'])
})
it('sends nothing the stopped action produced, only the acknowledgement', async () => {
const { sim, runner, executor } = setup()
runner.ignoresAbort = true
sim.inbox = [callItem('call-1', 'chat-a', 'read_local_file')]
await executor.reconcile()
await vi.waitFor(() => expect(runner.started).toEqual(['call-1']))
sim.inbox = [{ kind: 'cancel', toolCallId: 'call-1' }]
await executor.reconcile()
await vi.waitFor(() => expect(runner.cancelled).toEqual(['call-1']))
runner.finish('call-1', { status: 'success', message: 'read', data: { text: 'secret' } })
await vi.waitFor(() => expect(sim.completions).toHaveLength(1))
expect(sim.completions[0]?.completion.status).toBe('cancelled')
expect(JSON.stringify(sim.completions[0])).not.toContain('secret')
})
it('ignores a cancel for a call it never held', async () => {
const { sim, runner, executor } = setup()
sim.inbox = [{ kind: 'cancel', toolCallId: 'someone-else' }]
await executor.reconcile()
expect(runner.cancelled).toEqual([])
expect(sim.completions).toEqual([])
})
})
describe('restarting', () => {
it('reports what each journaled call reached, and never runs any of them again', async () => {
const { sim, journal, runner, executor } = setup()
await journal.put({ toolCallId: 'claiming-1', state: 'claiming' })
await journal.put({ toolCallId: 'claimed-1', state: 'claimed', executionToken: 't-claimed' })
await journal.put({ toolCallId: 'started-1', state: 'started', executionToken: 't-started' })
await journal.put({
toolCallId: 'result-1',
state: 'result',
executionToken: 't-result',
completion: DONE,
})
await executor.recover()
await vi.waitFor(() => expect(sim.completions).toHaveLength(3))
const reported = new Map(sim.completions.map((request) => [request.toolCallId, request]))
expect(reported.get('claimed-1')?.completion.data).toMatchObject({ notStarted: true })
expect(reported.get('claimed-1')?.executionToken).toBe('t-claimed')
expect(reported.get('started-1')?.completion.data).toMatchObject({
outcomeUnknown: true,
doNotRetry: true,
})
expect(reported.get('result-1')?.completion).toEqual(DONE)
expect(reported.has('claiming-1')).toBe(false)
expect(runner.started).toEqual([])
await vi.waitFor(() => expect(journal.entries.size).toBe(0))
})
it('leaves open whether the model has a result when Sim already held one for the call', async () => {
const { sim, journal, executor, delivered, notDelivered } = setup()
// The previous run's send of the real result may have landed just before the app went down.
sim.completionOutcome = 'duplicate'
await journal.put({ toolCallId: 'started-1', state: 'started', executionToken: 't-started' })
await executor.recover()
await vi.waitFor(() => expect(journal.entries.size).toBe(0))
expect(sim.completions).toHaveLength(1)
expect(delivered).toEqual([])
expect(notDelivered).toEqual([])
})
it('still leaves a recovered result open when Sim held one, after parking it', async () => {
const { sim, journal, executor, onUnregistered, delivered, notDelivered } = setup()
sim.completeErrors = [new DeviceRequestError(401, 'unregistered')]
sim.completionOutcome = 'duplicate'
await journal.put({ toolCallId: 'started-1', state: 'started', executionToken: 't-started' })
await executor.recover()
await vi.waitFor(() => expect(onUnregistered).toHaveBeenCalledTimes(1))
// Registered again.
executor.resumeParked()
await vi.waitFor(() => expect(sim.completions).toHaveLength(1))
await vi.waitFor(() => expect(journal.entries.size).toBe(0))
expect(delivered).toEqual([])
expect(notDelivered).toEqual([])
})
it('reports a recovered result too large to send, then held as a duplicate, as never reaching the model', async () => {
const { sim, journal, executor, delivered, notDelivered } = setup()
// No run could have sent this result: it is too large every time.
sim.completeErrors = [new DeviceRequestError(413, 'too large')]
sim.completionOutcome = 'duplicate'
await journal.put({
toolCallId: 'result-1',
state: 'result',
executionToken: 't',
completion: DONE,
})
await executor.recover()
await vi.waitFor(() => expect(notDelivered).toEqual(['result-1']))
expect(delivered).toEqual([])
})
})
describe('delivery', () => {
it('retries a result whose request timed out', async () => {
const { sim, journal, executor } = setup()
sim.completeErrors = [new DeviceRequestError(408, 'request timeout')]
await journal.put({
toolCallId: 'r-1',
state: 'result',
executionToken: 't-1',
completion: DONE,
})
await executor.recover()
await vi.waitFor(() => expect(sim.completions).toHaveLength(1))
})
it('reports a result it cannot encode as outcome unknown, once, instead of retrying it', async () => {
const { sim, journal, executor } = setup()
const complete = sim.client.complete
let attempts = 0
sim.client.complete = async (request) => {
attempts += 1
if (request.completion.data && 'cyclic' in request.completion.data) {
throw new UnsendableRequestError('Converting circular structure to JSON')
}
return complete(request)
}
await journal.put({
toolCallId: 'r-1',
state: 'result',
executionToken: 't-1',
completion: { status: 'success', message: 'done', data: { cyclic: true } },
})
await executor.recover()
await vi.waitFor(() => expect(sim.completions).toHaveLength(1))
expect(attempts).toBe(2)
expect(sim.completions[0]?.completion).toMatchObject({
status: 'error',
data: { outcomeUnknown: true, doNotRetry: true },
})
})
it('holds a result Sim refuses as unregistered until the device registers again', async () => {
const { sim, journal, executor, onUnregistered } = setup()
sim.completeErrors = [new DeviceRequestError(401, 'unregistered')]
await journal.put({
toolCallId: 'r-1',
state: 'result',
executionToken: 't-1',
completion: DONE,
})
await executor.recover()
await vi.waitFor(() => expect(onUnregistered).toHaveBeenCalledTimes(1))
await sleep(40)
expect(onUnregistered).toHaveBeenCalledTimes(1)
expect(sim.completions).toHaveLength(0)
expect(journal.entries.get('r-1')?.state).toBe('result')
executor.resumeParked()
await vi.waitFor(() => expect(sim.completions).toHaveLength(1))
await vi.waitFor(() => expect(journal.entries.size).toBe(0))
})
})
describe('keeping the machine awake', () => {
it('is busy from the claim until Sim has the result', async () => {
const { sim, runner, executor, busy } = setup()
sim.inbox = [callItem('call-1', 'chat-a')]
await executor.reconcile()
await vi.waitFor(() => expect(runner.started).toEqual(['call-1']))
expect(busy).toEqual([true])
runner.finish('call-1')
await vi.waitFor(() => expect(busy).toEqual([true, false]))
expect(sim.completions).toHaveLength(1)
})
it('leaves nothing of the previous session behind when sign-out lands mid-recovery', async () => {
const { sim, journal, executor } = setup()
await journal.put({ toolCallId: 'c-1', state: 'claiming' })
await journal.put({
toolCallId: 'r-1',
state: 'result',
executionToken: 't-1',
completion: DONE,
})
const loaded = deferred<void>()
const load = journal.load.bind(journal)
journal.load = async () => {
await loaded.promise
return load()
}
const recovering = executor.recover()
const signingOut = executor.dispose()
loaded.resolve()
await Promise.all([recovering, signingOut])
await sleep(40)
expect(sim.completions).toEqual([])
expect(journal.entries.size).toBe(0)
})
it('goes idle at sign-out and stays silent when a recovered delivery settles afterwards', async () => {
const { sim, journal, executor, busy } = setup()
const answer = deferred<void>()
const complete = sim.client.complete
let sending = false
sim.client.complete = async (request) => {
sending = true
await answer.promise
return complete(request)
}
await journal.put({
toolCallId: 'r-1',
state: 'result',
executionToken: 't-1',
completion: DONE,
})
await executor.recover()
await vi.waitFor(() => expect(sending).toBe(true))
expect(busy).toEqual([true])
await executor.dispose()
expect(busy).toEqual([true, false])
answer.resolve()
await sleep(40)
expect(busy).toEqual([true, false])
})
it('lets the machine sleep once a result has kept failing to reach Sim, and keeps retrying', async () => {
const { sim, journal, executor, busy } = setup({ deliveryAwakeLimitMs: 20 })
sim.completeErrors = Array.from({ length: 6 }, () => new DeviceRequestError(503, 'deploying'))
await journal.put({
toolCallId: 'r-1',
state: 'result',
executionToken: 't-1',
completion: DONE,
})
await executor.recover()
await vi.waitFor(() => expect(busy).toEqual([true, false]))
expect(sim.completions).toHaveLength(0)
await vi.waitFor(() => expect(sim.completions).toHaveLength(1))
expect(busy).toEqual([true, false])
})
it('stays busy while a result a previous run left is still on its way to Sim', async () => {
const { sim, journal, executor, busy } = setup()
sim.completeErrors = [new DeviceRequestError(503, 'deploying')]
await journal.put({
toolCallId: 'result-1',
state: 'result',
executionToken: 't-result',
completion: DONE,
})
await executor.recover()
expect(busy).toEqual([true])
await vi.waitFor(() => expect(sim.completions).toHaveLength(1))
await vi.waitFor(() => expect(busy).toEqual([true, false]))
})
})
describe('registration', () => {
it('asks to register again when Sim no longer recognizes the device', async () => {
const { sim, executor, onUnregistered } = setup()
sim.claimError = () => new DeviceRequestError(401, 'unregistered')
sim.inbox = [callItem('call-1', 'chat-a')]
await executor.reconcile()
expect(onUnregistered).toHaveBeenCalled()
})
it('drops everything it holds on sign-out', async () => {
const { sim, journal, runner, executor } = setup()
sim.inbox = [callItem('call-1', 'chat-a')]
await executor.reconcile()
await vi.waitFor(() => expect(runner.started).toEqual(['call-1']))
await executor.dispose()
expect(runner.cancelled).toEqual(['call-1'])
expect(journal.entries.size).toBe(0)
expect(executor.heldCallCount()).toBe(0)
})
it('stops running actions at sign-out without waiting on a claim still in flight', async () => {
const { sim, journal, runner, executor } = setup()
sim.inbox = [callItem('call-1', 'chat-a')]
await executor.reconcile()
await vi.waitFor(() => expect(runner.started).toEqual(['call-1']))
const answer = deferred<void>()
const claim = sim.client.claim
sim.client.claim = async (toolCallId) => {
await answer.promise
return claim(toolCallId)
}
sim.inbox = [callItem('call-2', 'chat-b')]
const reading = executor.reconcile()
await vi.waitFor(() => expect(journal.entries.get('call-2')?.state).toBe('claiming'))
const signingOut = executor.dispose()
await vi.waitFor(() => expect(runner.cancelled).toEqual(['call-1']))
answer.resolve()
await Promise.all([reading, signingOut])
expect(runner.started).toEqual(['call-1'])
expect(executor.heldCallCount()).toBe(0)
expect(journal.entries.size).toBe(0)
})
it('still claims the calls of an inbox read whose notifier throws', async () => {
const sim = new FakeSim()
const runner = new FakeRunner()
runner.immediate = DONE
const executor = new DesktopExecutor({
client: sim.client,
journal: new MemoryJournal(),
runner,
leaseRenewMs: 60_000,
retryBaseMs: 5,
onUnregistered: () => {},
onApprovals: () => {
throw new Error('notifications are unavailable')
},
})
sim.inbox = [callItem('call-1', 'chat-a')]
await expect(executor.reconcile()).resolves.toBeUndefined()
await vi.waitFor(() => expect(runner.started).toEqual(['call-1']))
})
it('raises no approval from an inbox read that Sim answers after sign-out', async () => {
const { sim, executor, approvals } = setup()
const answer = deferred<void>()
const listInbox = sim.client.listInbox
sim.client.listInbox = async () => {
await answer.promise
return listInbox()
}
sim.inbox = [
{
kind: 'approval_needed',
toolCallId: 'gated-1',
toolName: 'terminal',
chatId: 'chat-a',
chatTitle: 'Fix CI',
workspaceId: 'ws-1',
summary: 'npm publish',
},
]
const reading = executor.reconcile()
const signingOut = executor.dispose()
answer.resolve()
await Promise.all([reading, signingOut])
expect(approvals).toEqual([])
})
it('does not keep a claim that Sim answers after sign-out', async () => {
const { sim, journal, runner, executor } = setup({ leaseRenewMs: 10 })
const answer = deferred<void>()
const claim = sim.client.claim
sim.client.claim = async (toolCallId) => {
await answer.promise
return claim(toolCallId)
}
sim.inbox = [callItem('call-1', 'chat-a')]
const reading = executor.reconcile()
await vi.waitFor(() => expect(journal.entries.get('call-1')?.state).toBe('claiming'))
const signingOut = executor.dispose()
answer.resolve()
await Promise.all([reading, signingOut])
await sleep(40)
expect(executor.heldCallCount()).toBe(0)
expect(runner.started).toEqual([])
expect(sim.renewals).toEqual([])
expect(journal.entries.size).toBe(0)
})
})
describe('recording', () => {
it('leaves a call on offer when it cannot record the claim', async () => {
const { sim, journal, executor } = setup()
journal.failOn = 'claiming'
sim.inbox = [callItem('call-1', 'chat-a')]
await executor.reconcile()
expect(sim.claims).toEqual([])
})
it('never starts an action it could not record, and reports it as not run', async () => {
const { sim, journal, runner, executor } = setup()
journal.failOn = 'started'
sim.inbox = [callItem('call-1', 'chat-a')]
await executor.reconcile()
await vi.waitFor(() => expect(sim.completions).toHaveLength(1))
expect(sim.completions[0]?.completion.data).toMatchObject({ notStarted: true })
expect(runner.started).toEqual([])
})
})
apps/desktop/src/main/desktop-executor/executor.ts
+589
View File
@@ -0,0 +1,589 @@
/**
* The background executor's state machine: takes the calls Sim offers this device, runs each in
* its chat's own browser, terminal or file scope, and delivers every result until Sim
* acknowledges it, whether or not any window shows that chat.
*
* A call is claimed as soon as it is offered, then waits in a local queue per chat and surface,
* so a busy chat never lets a backlog miss Sim's pickup window. Its lease is renewed from claim
* until Sim acknowledges the result. Each step is journaled before the step it guards, so a
* restart reports a call that never started as not started, one that did as outcome unknown, and
* a finished one with its real result; none is ever run twice.
*/
import type { DesktopToolCompletion } from '@sim/desktop-bridge/tool-results'
import { createLogger } from '@sim/logger'
import { getErrorMessage } from '@sim/utils/errors'
import { sleep } from '@sim/utils/helpers'
import { backoffWithJitter } from '@sim/utils/retry'
import {
type DesktopExecutorClient,
DeviceRequestError,
UnsendableRequestError,
} from '@/main/desktop-executor/client'
import type { ExecutorJournal, JournalEntry } from '@/main/desktop-executor/journal'
import type { ClaimedDesktopCall, DesktopInboxItem } from '@/main/desktop-executor/protocol'
const logger = createLogger('DesktopExecutor')
/** Claimed calls this device holds at once, across every chat. */
const DEFAULT_MAX_HELD_CALLS = 32
/** Ceiling on the wait between attempts to deliver one result. */
const DELIVERY_RETRY_MAX_MS = 30_000
/**
* How long a result that keeps failing to reach Sim holds the machine awake. Delivery keeps
* retrying after this; it just stops counting as work that needs the machine awake.
*/
const DELIVERY_AWAKE_LIMIT_MS = 10 * 60_000
const NOT_STARTED_AFTER_RESTART =
'Not run: this action never started, because the Sim desktop app restarted before it began. Nothing happened on the user’s computer. Do not retry it in this turn; tell the user, who can ask again.'
const OUTCOME_UNKNOWN_AFTER_RESTART =
'The Sim desktop app restarted while this action was running, so its result was lost. It may already have taken effect: inspect the current state before repeating it, and do not retry it automatically.'
const STOPPED_BEFORE_START = 'Stopped before the Sim desktop app started this action.'
const STOPPED_WHILE_RUNNING = 'Stopped while the Sim desktop app was running this action.'
const NOT_RECORDED =
'Not run: this action never started, because the Sim desktop app could not record it on the user’s computer first. Nothing happened on the user’s computer. Do not retry it in this turn; tell the user, who can ask again later.'
const RESULT_UNSENDABLE =
'The action finished, but its result could not be encoded to send back. It may have taken effect: inspect the current state before repeating it, and do not retry it automatically.'
const RESULT_TOO_LARGE =
'The action finished, but its result was too large to send back. Do not repeat a side-effecting action; inspect the current state instead.'
/** Runs one claimed call on this machine. Implementations never throw; failures are completions. */
export interface DesktopToolRunner {
run(call: ClaimedDesktopCall, signal: AbortSignal): Promise<DesktopToolCompletion>
/** Stops the action a running call started, through its surface's own cancel. */
cancel(call: ClaimedDesktopCall): Promise<void>
}
export type DesktopApprovalItem = Extract<DesktopInboxItem, { kind: 'approval_needed' }>
/**
* Whether a result hands the model what the action produced: not a stop, and not one standing in
* for an action that did not start, whose outcome is unknown, or whose output was too large to send.
*/
export function isDeliveredResult(completion: DesktopToolCompletion): boolean {
const data = completion.data
return (
completion.status !== 'cancelled' &&
data?.notStarted !== true &&
data?.outcomeUnknown !== true &&
data?.resultOmitted !== true
)
}
export interface DesktopExecutorOptions {
client: DesktopExecutorClient
journal: ExecutorJournal
runner: DesktopToolRunner
leaseRenewMs: number
/** Called when Sim no longer recognizes this device for the current session. */
onUnregistered: () => void
/** Every inbox read's calls that wait for the user's approval. */
onApprovals?: (items: DesktopApprovalItem[]) => void
/** Called whenever the number of held calls changes between zero and more. */
onBusyChange?: (busy: boolean) => void
/**
* Called once Sim has taken a call's real result ({@link isDeliveredResult}) as the call's own
* (recorded, or a duplicate of one it recorded): the model has it, so anything it hands back (a
* pane still running) is in use.
*/
onResultDelivered?: (toolCallId: string) => void
/**
* Called once Sim is done with a call without taking its real result: it settled the call
* first, refused the result, or recorded one that stands in for it. The model never learns of
* anything the action handed back as still going, so nothing will come back to it.
*/
onResultNotDelivered?: (toolCallId: string) => void
maxHeldCalls?: number
/** First delivery retry delay; tests shorten it. */
retryBaseMs?: number
/** How long a failing delivery keeps the machine awake; tests shorten it. */
deliveryAwakeLimitMs?: number
}
type HeldPhase = 'queued' | 'running' | 'reporting'
interface HeldCall {
call: ClaimedDesktopCall
phase: HeldPhase
controller: AbortController
renewTimer: ReturnType<typeof setInterval> | null
stopped: boolean
}
function surfaceOf(toolName: string): 'browser' | 'terminal' | 'files' {
if (toolName.startsWith('browser_')) return 'browser'
if (toolName === 'terminal') return 'terminal'
return 'files'
}
export class DesktopExecutor {
private readonly held = new Map<string, HeldCall>()
/** Each chat surface's tail, so calls in one chat run in the order they were offered. */
private readonly queues = new Map<string, Promise<void>>()
private reconciling: Promise<void> | null = null
private reconcileAgain = false
private paused = false
private disposed = false
private busy = false
/** The journal walk after a restart; sign-out waits for it before clearing the journal. */
private recovering: Promise<void> | null = null
/** Results a previous app run left, still on their way to Sim. */
private readonly recoveringIds = new Set<string>()
/**
* Results a previous app run left, until Sim answers for them, parked ones included: that run's
* send of the real result may already have landed.
*/
private readonly recoveredResults = new Set<string>()
/** Results that have failed to reach Sim for longer than {@link DELIVERY_AWAKE_LIMIT_MS}. */
private readonly stalledDeliveries = new Set<string>()
/** Results Sim refused because it no longer recognized this device; sent once it registers again. */
private readonly parked = new Map<
string,
{ executionToken: string; completion: DesktopToolCompletion }
>()
constructor(private readonly options: DesktopExecutorOptions) {}
heldCallCount(): number {
return this.held.size
}
/** Suspend stops new claims; held calls keep running and reporting. */
setPaused(paused: boolean): void {
this.paused = paused
}
/** Reports every call the journal says a previous run of the app left unfinished. */
async recover(): Promise<void> {
this.recovering = this.recoverEntries()
await this.recovering
}
private async recoverEntries(): Promise<void> {
const entries = await this.options.journal.load()
for (const entry of entries) {
// Signed out mid-recovery: the previous session's results stay unsent and are cleared.
if (this.disposed) return
if (entry.state === 'claiming') {
// Unknown whether the claim landed. If it did not, the inbox offers the call again. If it
// did, its token never reached this device, so nothing here can report it; Sim settles it
// as outcome unknown once its lease lapses, which is conservative because nothing ran.
await this.forget(entry.toolCallId)
continue
}
const completion: DesktopToolCompletion =
entry.state === 'result'
? entry.completion
: entry.state === 'claimed'
? {
status: 'error',
message: NOT_STARTED_AFTER_RESTART,
data: { error: NOT_STARTED_AFTER_RESTART, notStarted: true },
}
: {
status: 'error',
message: OUTCOME_UNKNOWN_AFTER_RESTART,
data: {
error: OUTCOME_UNKNOWN_AFTER_RESTART,
outcomeUnknown: true,
doNotRetry: true,
},
}
logger.info('Reporting a call left unfinished by the previous app run', {
toolCallId: entry.toolCallId,
state: entry.state,
})
// Held awake like a running call: the result exists only on this machine until Sim has it.
this.recoveringIds.add(entry.toolCallId)
this.recoveredResults.add(entry.toolCallId)
this.updateBusy()
void this.deliver(entry.toolCallId, entry.executionToken, completion).finally(() => {
this.recoveringIds.delete(entry.toolCallId)
this.updateBusy()
})
}
}
/** After registering again: sends the results parked while Sim did not recognize the device. */
resumeParked(): void {
const parked = [...this.parked]
this.parked.clear()
for (const [toolCallId, { executionToken, completion }] of parked) {
void this.deliver(toolCallId, executionToken, completion)
}
}
/**
* Reads the inbox and acts on it. Concurrent requests coalesce: one more read runs after the
* current one, so a doorbell that rings mid-read is never lost.
*/
reconcile(): Promise<void> {
if (this.reconciling) {
this.reconcileAgain = true
return this.reconciling
}
const run = async () => {
try {
do {
this.reconcileAgain = false
await this.reconcileOnce()
} while (this.reconcileAgain && !this.disposed)
} catch (error) {
// Callers fire and forget; one bad read must never become an unhandled rejection.
logger.error('Desktop inbox read failed unexpectedly', { error: getErrorMessage(error) })
} finally {
this.reconciling = null
}
}
this.reconciling = run()
return this.reconciling
}
/** Sign-out: stops every action and forgets every call; the session that owned them is gone. */
async dispose(): Promise<void> {
this.disposed = true
const stopping = this.dropHeld()
// A claim still in flight sees `disposed` once Sim answers and is never held; the journal is
// cleared only after it, so its `claiming` record does not outlive the session.
await this.reconciling?.catch(() => {})
await Promise.all([stopping, this.dropHeld()])
await this.recovering?.catch(() => {})
this.updateBusy()
await this.options.journal.clear()
}
/** Releases every held call and stops the actions already running. */
private async dropHeld(): Promise<void> {
const held = [...this.held.values()]
for (const entry of held) this.release(entry)
await Promise.allSettled(
held
.filter((entry) => entry.phase === 'running')
.map((entry) => {
entry.controller.abort()
return this.options.runner.cancel(entry.call)
})
)
}
private async reconcileOnce(): Promise<void> {
if (this.disposed) return
let items: DesktopInboxItem[]
try {
items = await this.options.client.listInbox()
} catch (error) {
this.noteRequestFailure('Could not read the desktop inbox', error)
return
}
// Signed out while the read was in flight: its approvals and calls belong to the old account.
if (this.disposed) return
for (const item of items) {
if (item.kind === 'cancel') void this.stop(item.toolCallId, 'Stopped by the user.')
}
try {
this.options.onApprovals?.(
items.filter((item): item is DesktopApprovalItem => item.kind === 'approval_needed')
)
} catch (error) {
// Notifications are a courtesy; the calls in this read still get claimed.
logger.warn('Could not notify about desktop approvals', { error: getErrorMessage(error) })
}
for (const item of items) {
if (item.kind !== 'call' || this.held.has(item.toolCallId)) continue
if (this.paused || this.disposed) return
if (this.held.size >= (this.options.maxHeldCalls ?? DEFAULT_MAX_HELD_CALLS)) return
await this.claim(item.toolCallId)
}
}
/** Writes a journal transition; false when it could not be made durable. */
private async record(entry: JournalEntry): Promise<boolean> {
try {
await this.options.journal.put(entry)
return true
} catch (error) {
logger.warn('Could not record a desktop call locally', {
toolCallId: entry.toolCallId,
state: entry.state,
error: getErrorMessage(error),
})
return false
}
}
private async forget(toolCallId: string): Promise<void> {
await this.options.journal.remove(toolCallId).catch((error) =>
logger.warn('Could not forget a desktop call locally', {
toolCallId,
error: getErrorMessage(error),
})
)
}
private async claim(toolCallId: string): Promise<void> {
// Unrecorded, a claim that then crashed could never be accounted for; leave it on offer.
if (!(await this.record({ toolCallId, state: 'claiming' }))) return
let call: ClaimedDesktopCall
try {
call = await this.options.client.claim(toolCallId)
} catch (error) {
await this.forget(toolCallId)
this.noteRequestFailure('Desktop call was not claimed', error, { toolCallId })
return
}
if (this.disposed) return
// Best effort: an unrecorded token leaves `claiming`, which recovery treats conservatively.
await this.record({ toolCallId, state: 'claimed', executionToken: call.executionToken })
const entry: HeldCall = {
call,
phase: 'queued',
controller: new AbortController(),
renewTimer: null,
stopped: false,
}
this.held.set(toolCallId, entry)
this.updateBusy()
entry.renewTimer = setInterval(() => void this.renew(entry), this.options.leaseRenewMs)
logger.info('Claimed a desktop call', {
toolCallId,
toolName: call.toolName,
chatId: call.chatId,
})
this.enqueue(entry)
}
private enqueue(entry: HeldCall): void {
const key = `${entry.call.chatId}:${surfaceOf(entry.call.toolName)}`
const tail = this.queues.get(key) ?? Promise.resolve()
const next = tail.then(() => this.execute(entry))
this.queues.set(key, next)
void next.finally(() => {
if (this.queues.get(key) === next) this.queues.delete(key)
})
}
private async execute(entry: HeldCall): Promise<void> {
if (entry.stopped || this.disposed) return
const { call } = entry
const recorded = await this.record({
toolCallId: call.toolCallId,
state: 'started',
executionToken: call.executionToken,
})
if (entry.stopped || this.disposed) return
entry.phase = 'reporting'
if (!recorded) {
// Running it unrecorded could let a crash report an action that ran as never started.
await this.deliver(call.toolCallId, call.executionToken, {
status: 'error',
message: NOT_RECORDED,
data: { error: NOT_RECORDED, notStarted: true },
})
this.release(entry)
return
}
entry.phase = 'running'
const completion = await this.options.runner.run(call, entry.controller.signal)
if (this.disposed || this.held.get(call.toolCallId) !== entry) return
entry.phase = 'reporting'
// A stopped call's result is only an acknowledgement: Sim already settled it, so nothing the
// action produced (page text, file contents) leaves the machine.
await this.deliver(
call.toolCallId,
call.executionToken,
entry.stopped ? { status: 'cancelled', message: STOPPED_WHILE_RUNNING } : completion
)
this.release(entry)
}
/**
* Delivers a result until Sim acknowledges it. Any answer Sim gives for the token acknowledges
* it: recorded, a duplicate of one already recorded, or superseded by Sim settling it first.
*/
private async deliver(
toolCallId: string,
executionToken: string,
completion: DesktopToolCompletion
): Promise<void> {
if (this.disposed) return
// Best effort: unrecorded, a crash reports the call from its `started` entry as outcome unknown.
await this.record({ toolCallId, state: 'result', executionToken, completion })
const sendingSince = Date.now()
try {
await this.sendResult(toolCallId, executionToken, completion, sendingSince)
} finally {
// Whoever holds the delivery (a held call, a recovered result) updates the busy state as it
// lets go; doing it here first would flash the machine awake again in between.
this.stalledDeliveries.delete(toolCallId)
}
}
private async sendResult(
toolCallId: string,
executionToken: string,
completion: DesktopToolCompletion,
sendingSince: number
): Promise<void> {
let pending = completion
let notDelivered = true
for (let attempt = 1; !this.disposed; attempt++) {
try {
const outcome = await this.options.client.complete({
toolCallId,
executionToken,
completion: pending,
})
logger.info('Desktop call result acknowledged', { toolCallId, outcome })
// Superseded: Sim settled the call first, so this result never reached the model.
const delivered = outcome !== 'superseded' && isDeliveredResult(pending)
if (delivered) this.options.onResultDelivered?.(toolCallId)
// A duplicate holds what this app run sent, unless the result was recovered unchanged from
// an earlier run, whose send of the real result may have landed before the restart.
const takenEarlier =
outcome === 'duplicate' && pending === completion && this.recoveredResults.has(toolCallId)
notDelivered = !delivered && !takenEarlier
break
} catch (error) {
// Encoding failed on this machine, so nothing was sent; the same data would fail again.
if (error instanceof UnsendableRequestError && pending.data !== undefined) {
pending = {
status: 'error',
message: RESULT_UNSENDABLE,
data: { error: RESULT_UNSENDABLE, outcomeUnknown: true, doNotRetry: true },
}
continue
}
if (!(error instanceof DeviceRequestError)) {
logger.error('Could not send a desktop call result; dropping it', {
toolCallId,
error: getErrorMessage(error),
})
break
}
if (error.unregistered) {
// Retrying cannot help until the device registers again; the journal keeps the result.
this.parked.set(toolCallId, { executionToken, completion: pending })
this.options.onUnregistered()
return
}
if (error.status === 413 && pending.data !== undefined) {
pending = {
status: pending.status,
message: RESULT_TOO_LARGE,
data: { error: RESULT_TOO_LARGE, resultOmitted: true },
}
continue
}
if (!error.transient) {
logger.warn('Sim refused a desktop call result; dropping it', {
toolCallId,
status: error.status,
})
break
}
if (
Date.now() - sendingSince >
(this.options.deliveryAwakeLimitMs ?? DELIVERY_AWAKE_LIMIT_MS) &&
!this.stalledDeliveries.has(toolCallId)
) {
// Still retried, but no longer a reason to keep the machine awake.
this.stalledDeliveries.add(toolCallId)
this.updateBusy()
}
await sleep(
backoffWithJitter(attempt, error.retryAfterMs, {
baseMs: this.options.retryBaseMs,
maxMs: DELIVERY_RETRY_MAX_MS,
})
)
}
}
this.recoveredResults.delete(toolCallId)
if (this.disposed) return
if (notDelivered) this.options.onResultNotDelivered?.(toolCallId)
await this.forget(toolCallId)
}
private async renew(entry: HeldCall): Promise<void> {
if (this.held.get(entry.call.toolCallId) !== entry) return
try {
await this.options.client.renewLease(entry.call.toolCallId, entry.call.executionToken)
} catch (error) {
if (error instanceof DeviceRequestError && error.status === 410) {
this.clearRenewal(entry)
await this.stop(entry.call.toolCallId, 'Sim no longer holds this call for this device.')
return
}
this.noteRequestFailure('Could not renew a desktop call lease', error, {
toolCallId: entry.call.toolCallId,
})
}
}
/**
* Stops a held call Sim settled without it. A queued call never starts; a running one is
* interrupted. Either way its result is still delivered, which acknowledges the stop.
*/
private async stop(toolCallId: string, reason: string): Promise<void> {
const entry = this.held.get(toolCallId)
if (!entry || entry.stopped) return
entry.stopped = true
logger.info('Stopping a desktop call', { toolCallId, phase: entry.phase, reason })
if (entry.phase === 'queued') {
entry.phase = 'reporting'
await this.deliver(toolCallId, entry.call.executionToken, {
status: 'cancelled',
message: STOPPED_BEFORE_START,
data: { error: STOPPED_BEFORE_START, notStarted: true },
})
this.release(entry)
return
}
if (entry.phase === 'running') {
entry.controller.abort()
await this.options.runner.cancel(entry.call).catch((error) =>
logger.warn('Could not stop a desktop action', {
toolCallId,
error: getErrorMessage(error),
})
)
}
}
private release(entry: HeldCall): void {
this.clearRenewal(entry)
if (this.held.get(entry.call.toolCallId) === entry) this.held.delete(entry.call.toolCallId)
this.updateBusy()
}
private clearRenewal(entry: HeldCall): void {
if (entry.renewTimer) clearInterval(entry.renewTimer)
entry.renewTimer = null
}
private updateBusy(): void {
// A disposed executor reports idle once, at dispose; a delivery that settles later must not
// speak for the executor that replaced it.
const awake = (toolCallId: string) => !this.stalledDeliveries.has(toolCallId)
const busy =
!this.disposed && ([...this.held.keys()].some(awake) || [...this.recoveringIds].some(awake))
if (busy === this.busy) return
this.busy = busy
this.options.onBusyChange?.(busy)
}
private noteRequestFailure(
message: string,
error: unknown,
context: Record<string, unknown> = {}
): void {
if (error instanceof DeviceRequestError && error.unregistered) {
this.options.onUnregistered()
}
logger.warn(message, {
...context,
...(error instanceof DeviceRequestError ? { status: error.status } : {}),
error: getErrorMessage(error),
})
}
}
apps/desktop/src/main/desktop-executor/journal.test.ts
+134
View File
@@ -0,0 +1,134 @@
import { mkdtemp, readFile, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { describe, expect, it, vi } from 'vitest'
vi.mock('electron', () => import('@/test/electron-mock'))
import { createExecutorJournal } from '@/main/desktop-executor/journal'
function testEncryption(available = true) {
return {
isEncryptionAvailable: vi.fn(() => available),
encryptString: vi.fn((value: string) => Buffer.from(`protected:${value}`, 'utf8')),
decryptString: vi.fn((value: Buffer) => value.toString('utf8').replace(/^protected:/, '')),
}
}
async function journalPath(): Promise<string> {
return join(await mkdtemp(join(tmpdir(), 'sim-executor-journal-')), 'journal.json')
}
const RESULT = {
toolCallId: 'call-1',
state: 'result' as const,
executionToken: 'token-1',
completion: { status: 'success' as const, message: 'done', data: { secret: 'page text' } },
}
describe('executor journal', () => {
it('survives a restart, encrypted at rest', async () => {
const filePath = await journalPath()
const encryption = testEncryption()
await createExecutorJournal(filePath, encryption).put(RESULT)
const raw = await readFile(filePath, 'utf8')
expect(raw).not.toContain('page text')
expect(raw).not.toContain('token-1')
await expect(createExecutorJournal(filePath, encryption).load()).resolves.toEqual([RESULT])
})
it('removes the file once every call is acknowledged', async () => {
const filePath = await journalPath()
const journal = createExecutorJournal(filePath, testEncryption())
await journal.put(RESULT)
await journal.remove('call-1')
await expect(readFile(filePath, 'utf8')).rejects.toMatchObject({ code: 'ENOENT' })
})
it('applies transitions in order, so an earlier write cannot resurrect a removed call', async () => {
const filePath = await journalPath()
const encryption = testEncryption()
const journal = createExecutorJournal(filePath, encryption)
await Promise.all([
journal.put({ toolCallId: 'call-1', state: 'claimed', executionToken: 'token-1' }),
journal.put(RESULT),
journal.remove('call-1'),
])
await expect(createExecutorJournal(filePath, encryption).load()).resolves.toEqual([])
})
it('writes nothing in plaintext when OS encryption is unavailable', async () => {
const filePath = await journalPath()
const journal = createExecutorJournal(filePath, testEncryption(false))
await journal.put(RESULT)
await expect(readFile(filePath, 'utf8')).rejects.toMatchObject({ code: 'ENOENT' })
})
it('reports a transition it could not make durable', async () => {
const encryption = testEncryption()
encryption.encryptString.mockImplementation(() => {
throw new Error('keychain locked')
})
await expect(
createExecutorJournal(await journalPath(), encryption).put(RESULT)
).rejects.toThrow('keychain locked')
})
it('keeps results within what a restart can read back, dropping the data of the excess', async () => {
const filePath = await journalPath()
const encryption = testEncryption()
const journal = createExecutorJournal(filePath, encryption)
const screenshot = 'x'.repeat(20 * 1024 * 1024)
for (const toolCallId of ['call-1', 'call-2']) {
await journal.put({
toolCallId,
state: 'result',
executionToken: `token-${toolCallId}`,
completion: { status: 'success', message: 'done', data: { screenshot } },
})
}
const restored = await createExecutorJournal(filePath, encryption).load()
expect(restored).toHaveLength(2)
const kept = restored.filter(
(entry) => entry.state === 'result' && entry.completion.data?.screenshot === screenshot
)
expect(kept).toHaveLength(1)
expect(JSON.stringify(restored)).toContain('resultOmitted')
})
it('budgets results by their encoded size, not their length in characters', async () => {
const filePath = await journalPath()
const encryption = testEncryption()
const journal = createExecutorJournal(filePath, encryption)
// Three bytes per character: within the budget by length, three times over it in bytes.
const pageText = '€'.repeat(6 * 1024 * 1024)
for (const toolCallId of ['call-1', 'call-2', 'call-3']) {
await journal.put({
toolCallId,
state: 'result',
executionToken: `token-${toolCallId}`,
completion: { status: 'success', message: 'done', data: { pageText } },
})
}
const restored = await createExecutorJournal(filePath, encryption).load()
expect(restored).toHaveLength(3)
const kept = restored.filter(
(entry) => entry.state === 'result' && entry.completion.data?.pageText === pageText
)
expect(kept).toHaveLength(1)
})
it('starts empty from a corrupt or foreign file instead of failing', async () => {
const filePath = await journalPath()
await writeFile(filePath, '{"version":1,"ciphertext":"not-base64-json"}')
await expect(createExecutorJournal(filePath, testEncryption()).load()).resolves.toEqual([])
})
})
apps/desktop/src/main/desktop-executor/journal.ts
+205
View File
@@ -0,0 +1,205 @@
/**
* The background executor's durable outbox: one entry per call this device took, advanced before
* each step it guards, so a restart knows exactly how far each call got.
*
* - `claiming`: a claim request is in flight; nothing ran.
* - `claimed`: this device owns the call (its token is recorded) but has not started it.
* - `started`: the action may have begun on this machine.
* - `result`: the action finished; its completion is waiting for Sim to acknowledge it.
*
* An acknowledged entry is removed. The file is encrypted with Electron safeStorage and written
* atomically, like every other account-bearing store in userData; without OS encryption nothing
* is written and the journal lives in memory only.
*/
import type { DesktopToolCompletion } from '@sim/desktop-bridge/tool-results'
import { createLogger } from '@sim/logger'
import { getErrorMessage } from '@sim/utils/errors'
import { isRecordLike } from '@sim/utils/object'
import { safeStorage } from 'electron'
import {
FileResourceLimitError,
readFileWithinLimit,
removeFileIfPresent,
writeJsonFileAtomically,
} from '@/main/atomic-json-file'
const logger = createLogger('DesktopExecutorJournal')
const JOURNAL_VERSION = 1
/** Results can carry a screenshot, so the bound is the size of a few of them. */
const MAX_JOURNAL_BYTES = 64 * 1024 * 1024
/**
* Result data kept on disk across all entries, in UTF-8 bytes. Encryption and base64 grow it by
* about a third, so a journal within this budget always reads back under {@link MAX_JOURNAL_BYTES}.
*/
const MAX_PERSISTED_RESULT_BYTES = 32 * 1024 * 1024
const RESULT_NOT_KEPT =
'The action finished, but its result was too large to keep on this computer. Do not repeat a side-effecting action; inspect the current state instead.'
/**
* The entries as written to disk: a result whose data would push the journal past its budget is
* kept as finished without its data, so the file never grows past what a restart can read.
*/
function boundedEntries(entries: Map<string, JournalEntry>): JournalEntry[] {
let budget = MAX_PERSISTED_RESULT_BYTES
return [...entries.values()].map((entry) => {
if (entry.state !== 'result' || entry.completion.data === undefined) return entry
const size = Buffer.byteLength(JSON.stringify(entry.completion.data), 'utf8')
if (size <= budget) {
budget -= size
return entry
}
return {
...entry,
completion: {
status: entry.completion.status,
message: RESULT_NOT_KEPT,
data: { error: RESULT_NOT_KEPT, resultOmitted: true },
},
}
})
}
export type JournalEntry =
| { toolCallId: string; state: 'claiming' }
| { toolCallId: string; state: 'claimed' | 'started'; executionToken: string }
| {
toolCallId: string
state: 'result'
executionToken: string
completion: DesktopToolCompletion
}
export interface ExecutorJournal {
load(): Promise<JournalEntry[]>
put(entry: JournalEntry): Promise<void>
remove(toolCallId: string): Promise<void>
clear(): Promise<void>
}
interface EncryptionProvider {
isEncryptionAvailable(): boolean
encryptString(value: string): Buffer
decryptString(value: Buffer): string
}
function encryptionAvailable(encryption: EncryptionProvider): boolean {
try {
return encryption.isEncryptionAvailable()
} catch {
return false
}
}
function isCompletion(value: unknown): value is DesktopToolCompletion {
return (
isRecordLike(value) &&
(value.status === 'success' || value.status === 'error' || value.status === 'cancelled') &&
typeof value.message === 'string' &&
(value.data === undefined || isRecordLike(value.data))
)
}
function parseEntry(value: unknown): JournalEntry | null {
if (!isRecordLike(value) || typeof value.toolCallId !== 'string' || !value.toolCallId) return null
const { toolCallId } = value
if (value.state === 'claiming') return { toolCallId, state: 'claiming' }
if (typeof value.executionToken !== 'string' || !value.executionToken) return null
const { executionToken } = value
if (value.state === 'claimed' || value.state === 'started') {
return { toolCallId, state: value.state, executionToken }
}
if (value.state === 'result' && isCompletion(value.completion)) {
return { toolCallId, state: 'result', executionToken, completion: value.completion }
}
return null
}
export function createExecutorJournal(
filePath: string,
encryption: EncryptionProvider = safeStorage
): ExecutorJournal {
const entries = new Map<string, JournalEntry>()
let mutationTail = Promise.resolve()
const enqueue = (operation: () => Promise<void>): Promise<void> => {
const result = mutationTail.then(operation)
mutationTail = result.catch(() => undefined)
return result
}
/**
* Rewrites the whole journal. A failure rejects, so the caller knows the transition is not
* durable; the in-memory journal stays current and the next transition rewrites it all.
*/
const persist = async (): Promise<void> => {
if (!encryptionAvailable(encryption)) return
if (entries.size === 0) {
await removeFileIfPresent(filePath)
return
}
const payload = JSON.stringify({ version: JOURNAL_VERSION, entries: boundedEntries(entries) })
const ciphertext = encryption.encryptString(payload).toString('base64')
await writeJsonFileAtomically(filePath, { version: JOURNAL_VERSION, ciphertext })
}
return {
async load() {
await enqueue(async () => {
entries.clear()
if (!encryptionAvailable(encryption)) return
let raw: Buffer
try {
raw = await readFileWithinLimit(filePath, MAX_JOURNAL_BYTES)
} catch (error) {
if ((error as NodeJS.ErrnoException).code === 'ENOENT') return
logger.warn('Could not read the executor journal', {
error: error instanceof FileResourceLimitError ? 'too large' : getErrorMessage(error),
})
return
}
try {
const envelope = JSON.parse(raw.toString('utf8')) as unknown
if (
!isRecordLike(envelope) ||
envelope.version !== JOURNAL_VERSION ||
typeof envelope.ciphertext !== 'string'
) {
return
}
const payload = JSON.parse(
encryption.decryptString(Buffer.from(envelope.ciphertext, 'base64'))
) as unknown
if (!isRecordLike(payload) || !Array.isArray(payload.entries)) return
for (const candidate of payload.entries) {
const entry = parseEntry(candidate)
if (entry) entries.set(entry.toolCallId, entry)
}
} catch (error) {
logger.warn('Discarding an unreadable executor journal', {
error: getErrorMessage(error),
})
}
})
return [...entries.values()]
},
put(entry) {
return enqueue(async () => {
entries.set(entry.toolCallId, entry)
await persist()
})
},
remove(toolCallId) {
return enqueue(async () => {
if (!entries.delete(toolCallId)) return
await persist()
})
},
clear() {
return enqueue(async () => {
entries.clear()
await removeFileIfPresent(filePath)
})
},
}
}
apps/desktop/src/main/desktop-executor/protocol.ts
+206
View File
@@ -0,0 +1,206 @@
/**
* The device side of Sim's background executor protocol (`/api/desktop/devices`, `/inbox`,
* `/inbox/stream`, `/tool/claim`, `/tool/lease`, `/tool/complete`, `/tool/import`). Sim's contracts are the
* source of truth; responses are parsed defensively here because a malformed one must never
* reach a tool.
*/
import { isDesktopScopeId } from '@sim/desktop-bridge'
import type { DesktopToolCompletion } from '@sim/desktop-bridge/tool-results'
import { isRecordLike } from '@sim/utils/object'
/** The executor protocol version this build speaks. */
export const DESKTOP_EXECUTOR_PROTOCOL_VERSION = 1
/** Sim refuses a longer completion message, so longer ones are cut before they are sent. */
export const COMPLETION_MESSAGE_MAX_CHARS = 10_000
export interface DesktopDeviceRegistration {
deviceId: string
name: string
appVersion: string
platform: string
capabilities: { executor: number; browser: boolean; terminal: boolean; localFiles: boolean }
}
/** Sim's answer to a registration. `enabled: false` means no new turn will bind to this device. */
export interface DesktopExecutorTiming {
enabled: boolean
protocolVersion: number
leaseMs: number
leaseRenewMs: number
reconcileMs: number
}
export type DesktopInboxItem =
| {
kind: 'call'
toolCallId: string
toolName: string
chatId: string
workspaceId: string | null
}
| {
kind: 'approval_needed'
toolCallId: string
toolName: string
chatId: string
chatTitle: string | null
workspaceId: string | null
summary: string | null
}
| { kind: 'cancel'; toolCallId: string }
/** A call this device now owns: the server's canonical arguments and the token that fences it. */
export interface ClaimedDesktopCall {
toolCallId: string
toolName: string
args: Record<string, unknown>
chatId: string
workspaceId: string | null
executionToken: string
}
export type DesktopCompletionOutcome = 'recorded' | 'duplicate' | 'superseded'
/**
* One entry of a claimed import, stored under the call's target folder: `sourceName` is the
* import source's own name and `relativePath` the entry's place inside it (`''` for the source).
*/
export interface DesktopImportEntryRequest {
call: ClaimedDesktopCall
kind: 'file' | 'directory'
sourceName: string
relativePath: string
/** A file's bytes; a directory has none. */
content?: Blob
}
/** What Sim stored an import entry as: the file, or the folder it reused or created. */
export interface DesktopImportedEntry {
id: string
name: string
}
export interface DesktopCompletionRequest {
toolCallId: string
executionToken: string
completion: DesktopToolCompletion
}
function positiveInteger(value: unknown): value is number {
return typeof value === 'number' && Number.isInteger(value) && value > 0
}
function nullableString(value: unknown): value is string | null {
return value === null || typeof value === 'string'
}
export function parseRegistration(body: unknown): DesktopExecutorTiming | null {
if (
!isRecordLike(body) ||
typeof body.enabled !== 'boolean' ||
!positiveInteger(body.protocolVersion) ||
!positiveInteger(body.leaseMs) ||
!positiveInteger(body.leaseRenewMs) ||
!positiveInteger(body.reconcileMs)
) {
return null
}
return {
enabled: body.enabled,
protocolVersion: body.protocolVersion,
leaseMs: body.leaseMs,
leaseRenewMs: body.leaseRenewMs,
reconcileMs: body.reconcileMs,
}
}
function parseInboxItem(value: unknown): DesktopInboxItem | null {
if (!isRecordLike(value) || typeof value.toolCallId !== 'string' || !value.toolCallId) return null
const { toolCallId } = value
if (value.kind === 'cancel') return { kind: 'cancel', toolCallId }
if (
typeof value.toolName !== 'string' ||
typeof value.chatId !== 'string' ||
!isDesktopScopeId(value.chatId) ||
!nullableString(value.workspaceId)
) {
return null
}
if (value.kind === 'call') {
return {
kind: 'call',
toolCallId,
toolName: value.toolName,
chatId: value.chatId,
workspaceId: value.workspaceId,
}
}
if (
value.kind === 'approval_needed' &&
nullableString(value.chatTitle) &&
nullableString(value.summary)
) {
return {
kind: 'approval_needed',
toolCallId,
toolName: value.toolName,
chatId: value.chatId,
chatTitle: value.chatTitle,
workspaceId: value.workspaceId,
summary: value.summary,
}
}
return null
}
/** Unknown item kinds are skipped, so a newer Sim can add one without breaking this build. */
export function parseInbox(body: unknown): DesktopInboxItem[] | null {
if (!isRecordLike(body) || !Array.isArray(body.items)) return null
return body.items.flatMap((item) => {
const parsed = parseInboxItem(item)
return parsed ? [parsed] : []
})
}
export function parseClaim(toolCallId: string, body: unknown): ClaimedDesktopCall | null {
if (
!isRecordLike(body) ||
typeof body.toolName !== 'string' ||
!isRecordLike(body.args) ||
typeof body.chatId !== 'string' ||
!isDesktopScopeId(body.chatId) ||
!nullableString(body.workspaceId) ||
typeof body.executionToken !== 'string' ||
!body.executionToken
) {
return null
}
return {
toolCallId,
toolName: body.toolName,
args: body.args,
chatId: body.chatId,
workspaceId: body.workspaceId,
executionToken: body.executionToken,
}
}
export function parseCompletionOutcome(body: unknown): DesktopCompletionOutcome | null {
if (!isRecordLike(body)) return null
return body.outcome === 'recorded' ||
body.outcome === 'duplicate' ||
body.outcome === 'superseded'
? body.outcome
: null
}
/**
* Reads Sim's answer to an import entry: the id and name it stored the entry as. Null when either
* is missing or empty, which the client reports as a malformed response.
*/
export function parseImportedEntry(body: unknown): DesktopImportedEntry | null {
if (!isRecordLike(body) || typeof body.id !== 'string' || typeof body.name !== 'string')
return null
return body.id && body.name ? { id: body.id, name: body.name } : null
}
apps/desktop/src/main/desktop-executor/runner.test.ts
+298
View File
@@ -0,0 +1,298 @@
import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import type { TerminalToolResponse } from '@sim/terminal-protocol'
import { afterEach, describe, expect, it, vi } from 'vitest'
import { DeviceRequestError } from '@/main/desktop-executor/client'
import type {
ClaimedDesktopCall,
DesktopImportEntryRequest,
} from '@/main/desktop-executor/protocol'
import { createDesktopToolRunner, type DesktopToolRunnerDeps } from '@/main/desktop-executor/runner'
import { executeLocalFileRequest } from '@/main/local-files'
function terminalCall(toolCallId: string, operation: string): ClaimedDesktopCall {
return {
toolCallId,
toolName: 'terminal',
args: { operation, args: {} },
chatId: 'chat-b',
workspaceId: 'ws-1',
executionToken: `token-${toolCallId}`,
}
}
function runner(overrides: Partial<DesktopToolRunnerDeps> = {}) {
return createDesktopToolRunner({
preferences: () => ({ browserEnabled: true, terminalEnabled: true }),
accountDataAvailable: () => true,
browser: {
executeTool: vi.fn(),
cancelTool: vi.fn(),
hasSession: () => true,
restoreScope: vi.fn(),
},
terminal: { executeTool: vi.fn(), cancelTool: vi.fn(async () => true) },
localFiles: {
request: (call, request) =>
executeLocalFileRequest(request, { toolName: call.toolName, args: call.args }),
},
imports: { importEntry: vi.fn() },
localFilesystem: { handle: vi.fn(), vfsRoot: () => 'user-local/x--1' },
...overrides,
})
}
function runnerWithTerminal(executeTool: (toolCallId: string) => Promise<TerminalToolResponse>) {
return runner({
terminal: {
executeTool: (_scope, toolCallId) => executeTool(toolCallId),
cancelTool: vi.fn(async () => true),
},
})
}
afterEach(() => {
vi.useRealTimers()
})
describe('background terminal calls', () => {
it('holds a chat terminal for an operation that outlived its deadline', async () => {
vi.useFakeTimers()
let releaseWedged: (response: TerminalToolResponse) => void = () => {}
const started: string[] = []
const runner = runnerWithTerminal((toolCallId) => {
started.push(toolCallId)
if (toolCallId === 'wedged')
return new Promise((resolve) => {
releaseWedged = resolve
})
return Promise.resolve({ ok: true, result: { output: '' } })
})
const first = runner.run(terminalCall('wedged', 'input'), new AbortController().signal)
await vi.advanceTimersByTimeAsync(15_000)
const timedOut = await first
expect(timedOut.status).toBe('error')
expect(timedOut.data).toMatchObject({ outcomeUnknown: true, doNotRetry: true })
const second = runner.run(terminalCall('next', 'read'), new AbortController().signal)
await vi.advanceTimersByTimeAsync(0)
expect(started).toEqual(['wedged'])
releaseWedged({ ok: true, result: {} })
expect((await second).status).toBe('success')
expect(started).toEqual(['wedged', 'next'])
})
it('never starts a terminal operation stopped while it waited for the chat terminal', async () => {
vi.useFakeTimers()
let releaseWedged: (response: TerminalToolResponse) => void = () => {}
const started: string[] = []
const runner = runnerWithTerminal((toolCallId) => {
started.push(toolCallId)
if (toolCallId === 'wedged')
return new Promise((resolve) => {
releaseWedged = resolve
})
return Promise.resolve({ ok: true, result: { output: '' } })
})
const first = runner.run(terminalCall('wedged', 'input'), new AbortController().signal)
await vi.advanceTimersByTimeAsync(15_000)
await first
const stop = new AbortController()
const second = runner.run(terminalCall('stopped', 'run'), stop.signal)
await vi.advanceTimersByTimeAsync(0)
stop.abort()
const completion = await second
releaseWedged({ ok: true, result: {} })
await vi.advanceTimersByTimeAsync(0)
expect(completion.status).toBe('error')
expect(started).toEqual(['wedged'])
})
})
describe('local file calls', () => {
it('names a passing storage state, not a setting, when local files are out of reach', async () => {
const completion = await runner({ accountDataAvailable: () => false }).run(
{
toolCallId: 'read-1',
toolName: 'read_local_file',
args: { path: '~/notes.txt' },
chatId: 'chat-b',
workspaceId: 'ws-1',
executionToken: 'token-read-1',
},
new AbortController().signal
)
expect(completion.data).toMatchObject({ notStarted: true })
expect(completion.message).toContain('cannot reach local files')
expect(completion.message).not.toContain('settings')
})
})
describe('background imports', () => {
const roots: string[] = []
afterEach(async () => {
await Promise.all(roots.splice(0).map((root) => rm(root, { recursive: true, force: true })))
})
/** A `Reports` folder holding `q3/summary.txt` and `notes.txt`. */
async function reportsFolder(): Promise<string> {
const root = await mkdtemp(join(tmpdir(), 'sim-runner-import-'))
roots.push(root)
const reports = join(root, 'Reports')
await mkdir(join(reports, 'q3'), { recursive: true })
await writeFile(join(reports, 'q3', 'summary.txt'), 'quarterly numbers')
await writeFile(join(reports, 'notes.txt'), 'remember')
return reports
}
function importCall(path: string): ClaimedDesktopCall {
return {
toolCallId: 'import-1',
toolName: 'import_local_files',
args: { path, targetWorkspaceId: 'ws-1', folderId: 'folder-1' },
chatId: 'chat-b',
workspaceId: 'ws-1',
executionToken: 'token-import-1',
}
}
/**
* A fake Sim import route: records every entry it stores, with each file's bytes as text, and
* the token each request presented. `answer` can refuse a request instead.
*/
function recordingSim(answer?: (request: DesktopImportEntryRequest) => Error | null) {
const stored: Array<{ kind: string; relativePath: string; text?: string }> = []
const tokens: string[] = []
let requests = 0
const importEntry = async (request: DesktopImportEntryRequest) => {
requests += 1
tokens.push(request.call.executionToken)
const refusal = answer?.(request)
if (refusal) throw refusal
stored.push({
kind: request.kind,
relativePath: request.relativePath,
...(request.content ? { text: await request.content.text() } : {}),
})
return { id: `id-${stored.length}`, name: request.relativePath || request.sourceName }
}
return { stored, tokens, importEntry, requests: () => requests }
}
it("stores a folder's tree in Sim, each file with the bytes on disk", async () => {
const sim = recordingSim()
const completion = await runner({ imports: { importEntry: sim.importEntry } }).run(
importCall(await reportsFolder()),
new AbortController().signal
)
expect(completion.status).toBe('success')
expect(sim.stored).toEqual([
{ kind: 'directory', relativePath: '' },
{ kind: 'file', relativePath: 'notes.txt', text: 'remember' },
{ kind: 'directory', relativePath: 'q3' },
{ kind: 'file', relativePath: 'q3/summary.txt', text: 'quarterly numbers' },
])
expect(new Set(sim.tokens)).toEqual(new Set(['token-import-1']))
expect(completion.data).toMatchObject({
success: true,
workspaceId: 'ws-1',
folders: [
{ id: 'id-1', relativePath: '' },
{ id: 'id-3', relativePath: 'q3' },
],
files: [
{ id: 'id-2', relativePath: 'notes.txt' },
{ id: 'id-4', relativePath: 'q3/summary.txt' },
],
})
})
it('reports what landed when an import stops part way, and not to retry it', async () => {
const sim = recordingSim((request) =>
request.relativePath === 'q3/summary.txt' ? new Error('Sim refused the entry') : null
)
const completion = await runner({ imports: { importEntry: sim.importEntry } }).run(
importCall(await reportsFolder()),
new AbortController().signal
)
expect(completion.status).toBe('error')
expect(completion.data).toMatchObject({
success: false,
partial: true,
doNotRetry: true,
outcomeUnknown: true,
error: 'Sim refused the entry',
files: [{ id: 'id-2', relativePath: 'notes.txt' }],
})
})
it('stores nothing more once the call is stopped', async () => {
const controller = new AbortController()
const sim = recordingSim(() => {
controller.abort()
return null
})
const completion = await runner({ imports: { importEntry: sim.importEntry } }).run(
importCall(await reportsFolder()),
controller.signal
)
expect(completion.status).toBe('error')
expect(sim.stored).toEqual([{ kind: 'directory', relativePath: '' }])
})
it("waits out Sim's rate limit instead of failing the import part way", async () => {
let limited = false
const sim = recordingSim((request) => {
if (request.relativePath !== 'notes.txt' || limited) return null
limited = true
return new DeviceRequestError(429, 'Too many requests', 10)
})
const completion = await runner({ imports: { importEntry: sim.importEntry } }).run(
importCall(await reportsFolder()),
new AbortController().signal
)
expect(completion.status).toBe('success')
expect(sim.stored.map((entry) => entry.relativePath)).toEqual([
'',
'notes.txt',
'q3',
'q3/summary.txt',
])
})
it('reports an import the rate limit never let start as safe to ask for again', async () => {
const sim = recordingSim(() => new DeviceRequestError(429, 'Too many requests', 1))
const completion = await runner({ imports: { importEntry: sim.importEntry } }).run(
importCall(await reportsFolder()),
new AbortController().signal
)
expect(completion.status).toBe('error')
expect(completion.data).toMatchObject({ partial: false, files: [], folders: [] })
expect(completion.data).not.toHaveProperty('doNotRetry')
expect(completion.data).not.toHaveProperty('outcomeUnknown')
})
it('fails without storing anything when the source cannot be read', async () => {
const sim = recordingSim()
const completion = await runner({ imports: { importEntry: sim.importEntry } }).run(
importCall(join(tmpdir(), 'sim-runner-import-missing', 'Reports')),
new AbortController().signal
)
expect(completion.status).toBe('error')
expect(sim.requests()).toBe(0)
expect(completion.data).toMatchObject({ workspaceId: 'ws-1', partial: false })
})
})
apps/desktop/src/main/desktop-executor/runner.ts
+377
View File
@@ -0,0 +1,377 @@
/**
* Runs one claimed call on this machine, in the scope of the chat Sim says it belongs to: that
* chat's built-in browser tabs, its terminals, or the user's granted folders. Arguments always
* come from Sim's record of the call. Every outcome, including a refusal, is a completion built
* by the same projections the chat view uses, so the model sees one shape either way.
*/
import {
type BrowserToolName,
browserToolRendererTimeoutMs,
isCurrentBrowserToolName,
} from '@sim/browser-protocol'
import type {
DesktopLocalFileRequest,
DesktopLocalFileResponse,
LocalFilesystemRequest,
LocalFilesystemResponse,
} from '@sim/desktop-bridge'
import { runUserLocalFilesystemTool } from '@sim/desktop-bridge/local-filesystem-tools'
import {
assertImportableManifest,
browserSessionClosedCompletion,
browserToolCompletion,
browserToolFailure,
browserToolNeedsLivePage,
browserToolTimeoutMessage,
type DesktopLocalFileImportResult,
type DesktopToolCompletion,
localFileImportCompletion,
localFileImportFailure,
localFileReadCompletion,
localFilesystemToolCompletion,
readImportEntry,
terminalOperationTimeoutMs,
terminalToolCompletion,
terminalToolFailure,
} from '@sim/desktop-bridge/tool-results'
import { createLogger } from '@sim/logger'
import {
isTerminalOperation,
type TerminalOperation,
type TerminalToolArgs,
type TerminalToolResponse,
} from '@sim/terminal-protocol'
import { getErrorMessage } from '@sim/utils/errors'
import { interruptibleSleep } from '@sim/utils/helpers'
import { isRecordLike } from '@sim/utils/object'
import { backoffWithJitter } from '@sim/utils/retry'
import { DeviceRequestError } from '@/main/desktop-executor/client'
import type { DesktopToolRunner } from '@/main/desktop-executor/executor'
import type {
ClaimedDesktopCall,
DesktopImportEntryRequest,
DesktopImportedEntry,
} from '@/main/desktop-executor/protocol'
const logger = createLogger('DesktopExecutorRunner')
const USER_LOCAL_TOOLS: ReadonlySet<string> = new Set(['read', 'grep', 'glob'])
/** A rate-limited import entry is retried this many times; Sim never stored a refused one. */
const IMPORT_RATE_LIMIT_ATTEMPTS = 8
const IMPORT_RATE_LIMIT_MAX_WAIT_MS = 30_000
/** The model learns a call never ran because a surface is switched off on this machine. */
function surfaceOff(surface: string): DesktopToolCompletion {
const message = `Not run: this action never started, because ${surface} is switched off in the Sim desktop app’s settings. Nothing happened on the user’s computer. Do not retry it in this turn; continue without it, or ask the user to switch it on.`
return { status: 'error', message, data: { error: message, notStarted: true } }
}
/** Local files are out of reach while the app has no usable account storage, a passing state. */
function localAccessUnavailable(): DesktopToolCompletion {
const message =
'Not run: this action never started, because the Sim desktop app cannot reach local files right now (it is signing out, switching account, or its storage is unavailable). Nothing happened on the user’s computer. Do not retry it in this turn; tell the user, who can ask again once the app is ready.'
return { status: 'error', message, data: { error: message, notStarted: true } }
}
function unsupported(toolName: string): DesktopToolCompletion {
const message = `Not run: this action never started, because this version of the Sim desktop app cannot run ${toolName} in the background. Nothing happened on the user’s computer. Do not retry it in this turn; tell the user to update the Sim desktop app.`
return { status: 'error', message, data: { error: message, notStarted: true } }
}
export interface DesktopToolRunnerDeps {
preferences: () => { browserEnabled: boolean; terminalEnabled: boolean }
/** False while account-bearing local storage is unavailable; local tools then refuse. */
accountDataAvailable: () => boolean
browser: {
executeTool(
scopeId: string,
tool: BrowserToolName,
params: Record<string, unknown>,
toolCallId: string
): Promise<{ ok: boolean; result?: unknown; error?: string }>
cancelTool(scopeId: string, toolCallId: string): boolean
hasSession(scopeId: string): boolean
restoreScope(scopeId: string): void
}
terminal: {
executeTool(
scope: string,
toolCallId: string,
operation: TerminalOperation,
args: TerminalToolArgs
): Promise<TerminalToolResponse>
cancelTool(scope: string, toolCallId: string): Promise<boolean>
}
/** Reads a `read_local_file` or `import_local_files` source, authorized by the call itself. */
localFiles: {
request(
call: ClaimedDesktopCall,
request: DesktopLocalFileRequest
): Promise<DesktopLocalFileResponse>
}
imports: {
importEntry(
request: DesktopImportEntryRequest,
signal: AbortSignal
): Promise<DesktopImportedEntry>
}
localFilesystem: {
handle(request: LocalFilesystemRequest): Promise<LocalFilesystemResponse>
vfsRoot(mount: { id: string; name: string }): string
}
}
/** A terminal operation that outlived its deadline may still land, as a browser action may. */
function terminalTimedOut(timeoutMs: number): DesktopToolCompletion {
const message = `The terminal did not respond within ${timeoutMs}ms. The operation may still be running and take effect: do not retry it automatically; check the terminal's state first.`
return {
status: 'error',
message,
data: { error: message, outcomeUnknown: true, doNotRetry: true },
}
}
/** Resolves once the signal aborts, which may be never. */
function untilAborted(signal: AbortSignal): Promise<void> {
if (signal.aborted) return Promise.resolve()
return new Promise((resolve) => signal.addEventListener('abort', () => resolve(), { once: true }))
}
/** Resolves with the work's result, or with `onTimeout`'s once `timeoutMs` passes first. */
async function withDeadline<T>(
work: Promise<T>,
timeoutMs: number | null,
onTimeout: () => T
): Promise<T> {
if (timeoutMs === null) return work
// Aborted once the race settles, which cancels the sleep; a cancelled sleep never times out.
const settled = new AbortController()
const timeout = interruptibleSleep(timeoutMs, settled.signal).then(() =>
settled.signal.aborted ? new Promise<T>(() => {}) : onTimeout()
)
try {
return await Promise.race([work, timeout])
} finally {
settled.abort()
}
}
export function createDesktopToolRunner(deps: DesktopToolRunnerDeps): DesktopToolRunner {
/** Each chat's last terminal operation, until it actually settles. */
const unsettledTerminalWork = new Map<string, Promise<void>>()
async function runBrowser(
call: ClaimedDesktopCall,
tool: BrowserToolName
): Promise<DesktopToolCompletion> {
if (!deps.preferences().browserEnabled) return surfaceOff('the built-in browser')
if (browserToolNeedsLivePage(tool) && !deps.browser.hasSession(call.chatId)) {
try {
deps.browser.restoreScope(call.chatId)
} catch (error) {
logger.warn('Could not restore a chat browser before a background call', {
toolCallId: call.toolCallId,
error: getErrorMessage(error),
})
}
if (!deps.browser.hasSession(call.chatId)) return browserSessionClosedCompletion()
}
const timeoutMs =
tool === 'browser_request_takeover' ? null : browserToolRendererTimeoutMs(tool, call.args)
return withDeadline(
deps.browser.executeTool(call.chatId, tool, call.args, call.toolCallId).then((response) =>
response.ok
? browserToolCompletion(tool, response.result)
: browserToolFailure(response.error ?? 'The browser action failed.', {
sessionClosed: !deps.browser.hasSession(call.chatId),
})
),
timeoutMs,
() => {
deps.browser.cancelTool(call.chatId, call.toolCallId)
return browserToolFailure(browserToolTimeoutMessage(timeoutMs ?? 0), {
outcomeUnknown: true,
})
}
)
}
async function runTerminal(
call: ClaimedDesktopCall,
signal: AbortSignal
): Promise<DesktopToolCompletion> {
if (!deps.preferences().terminalEnabled) return surfaceOff('the terminal')
const { operation } = call.args
if (!isTerminalOperation(operation)) {
return terminalToolFailure(
`Unknown terminal operation: ${String(operation)}`,
'INVALID_REQUEST'
)
}
const args = isRecordLike(call.args.args) ? (call.args.args as TerminalToolArgs) : {}
const timeoutMs = terminalOperationTimeoutMs(operation)
// An operation reported as unresponsive may still land; the chat's next one waits for it, so
// two never act on the same terminals at once.
const previous = unsettledTerminalWork.get(call.chatId)
if (previous) await Promise.race([previous, untilAborted(signal)])
// Stopped while it waited: the terminal never heard of it, so its own cancel cannot reach it.
if (signal.aborted) {
return terminalToolFailure('The terminal action was stopped before it started.', 'CANCELLED')
}
const operationDone = deps.terminal.executeTool(call.chatId, call.toolCallId, operation, args)
const settled = operationDone.then(
() => undefined,
() => undefined
)
unsettledTerminalWork.set(call.chatId, settled)
void settled.then(() => {
if (unsettledTerminalWork.get(call.chatId) === settled)
unsettledTerminalWork.delete(call.chatId)
})
return withDeadline(operationDone.then(terminalToolCompletion), timeoutMs, () =>
terminalTimedOut(timeoutMs ?? 0)
)
}
async function runUserLocal(
call: ClaimedDesktopCall,
signal: AbortSignal
): Promise<DesktopToolCompletion> {
try {
const data = await runUserLocalFilesystemTool(call.toolCallId, call.toolName, call.args, {
invoke: (request) => deps.localFilesystem.handle(request),
vfsRoot: deps.localFilesystem.vfsRoot,
signal,
})
return localFilesystemToolCompletion({ ok: true, data })
} catch (error) {
return localFilesystemToolCompletion({ ok: false, error: getErrorMessage(error) })
}
}
/**
* Imports the call's source into its workspace one entry at a time: each directory as a folder,
* each file read in chunks and checked against the manifest that listed it.
*/
/**
* Sends one entry, waiting out Sim's rate limit: a large tree can outrun it, and a 429 means
* nothing was stored, so sending the entry again cannot duplicate it.
*/
async function importEntry(
request: DesktopImportEntryRequest,
signal: AbortSignal
): Promise<DesktopImportedEntry> {
for (let attempt = 1; ; attempt++) {
try {
return await deps.imports.importEntry(request, signal)
} catch (error) {
if (
!(error instanceof DeviceRequestError) ||
error.status !== 429 ||
attempt >= IMPORT_RATE_LIMIT_ATTEMPTS
) {
throw error
}
await interruptibleSleep(
backoffWithJitter(attempt, error.retryAfterMs, { maxMs: IMPORT_RATE_LIMIT_MAX_WAIT_MS }),
signal
)
signal.throwIfAborted()
}
}
}
async function runImport(
call: ClaimedDesktopCall,
signal: AbortSignal
): Promise<DesktopToolCompletion> {
const files: DesktopLocalFileImportResult['files'] = []
const folders: DesktopLocalFileImportResult['folders'] = []
const targetWorkspaceId =
typeof call.args.targetWorkspaceId === 'string' ? call.args.targetWorkspaceId : ''
const read = (request: DesktopLocalFileRequest) => deps.localFiles.request(call, request)
try {
const response = await read({ operation: 'manifest', toolCallId: call.toolCallId })
if (!response.ok) throw new Error(response.error)
if (response.data.kind !== 'manifest') throw new Error('Unexpected file manifest response.')
const manifest = response.data
assertImportableManifest(manifest)
for (const entry of manifest.entries) {
signal.throwIfAborted()
const target = { call, sourceName: manifest.name, relativePath: entry.relativePath }
if (entry.kind === 'directory') {
const folder = await importEntry({ ...target, kind: 'directory' }, signal)
folders.push({ id: folder.id, relativePath: entry.relativePath })
continue
}
const parts = await readImportEntry(call.toolCallId, entry, read, signal)
const file = await importEntry(
{ ...target, kind: 'file', content: new Blob(parts) },
signal
)
files.push({ id: file.id, name: file.name, relativePath: entry.relativePath })
}
return localFileImportCompletion({
success: true,
workspaceId: manifest.targetWorkspaceId,
files,
folders,
})
} catch (error) {
return localFileImportCompletion(
localFileImportFailure(
{ workspaceId: targetWorkspaceId, files, folders },
getErrorMessage(error),
// A rate limit that outlasted every retry refused the entry outright: nothing of it landed.
{ outcomeKnown: error instanceof DeviceRequestError && error.status === 429 }
)
)
}
}
return {
async run(call, signal) {
try {
if (isCurrentBrowserToolName(call.toolName)) return await runBrowser(call, call.toolName)
if (call.toolName === 'terminal') return await runTerminal(call, signal)
if (!deps.accountDataAvailable()) return localAccessUnavailable()
if (call.toolName === 'read_local_file') {
return localFileReadCompletion(
await deps.localFiles.request(call, {
operation: 'read',
toolCallId: call.toolCallId,
})
)
}
if (call.toolName === 'import_local_files') return await runImport(call, signal)
if (USER_LOCAL_TOOLS.has(call.toolName)) return await runUserLocal(call, signal)
return unsupported(call.toolName)
} catch (error) {
logger.error('Background desktop call failed unexpectedly', {
toolCallId: call.toolCallId,
toolName: call.toolName,
error: getErrorMessage(error),
})
const message = `The Sim desktop app could not finish this action: ${getErrorMessage(error)}`
return {
status: 'error',
message,
data: { error: message, outcomeUnknown: true, doNotRetry: true },
}
}
},
async cancel(call) {
if (isCurrentBrowserToolName(call.toolName)) {
deps.browser.cancelTool(call.chatId, call.toolCallId)
return
}
if (call.toolName === 'terminal') {
await deps.terminal.cancelTool(call.chatId, call.toolCallId)
return
}
if (USER_LOCAL_TOOLS.has(call.toolName)) {
await deps.localFilesystem.handle({ operation: 'cancel', requestId: call.toolCallId })
}
},
}
}
apps/desktop/src/main/desktop-executor/service.test.ts
+360
View File
@@ -0,0 +1,360 @@
import { chmod, mkdir, mkdtemp } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { sleep } from '@sim/utils/helpers'
import { describe, expect, it, vi } from 'vitest'
vi.mock('electron', () => import('@/test/electron-mock'))
/** How many journal loads still fail, like a read that throws. */
const journalFaults = vi.hoisted(() => ({ loads: 0 }))
vi.mock('@/main/desktop-executor/journal', async () => {
const actual = await vi.importActual<typeof import('@/main/desktop-executor/journal')>(
'@/main/desktop-executor/journal'
)
return {
...actual,
createExecutorJournal: (...args: Parameters<typeof actual.createExecutorJournal>) => {
const journal = actual.createExecutorJournal(...args)
return {
...journal,
load: () => {
if (journalFaults.loads <= 0) return journal.load()
journalFaults.loads -= 1
return Promise.reject(new Error('The disk went away.'))
},
}
},
}
})
import { net } from 'electron'
import { DesktopExecutor } from '@/main/desktop-executor/executor'
import { createExecutorJournal } from '@/main/desktop-executor/journal'
import { createDesktopExecutorService, deviceName } from '@/main/desktop-executor/service'
/** Sim's device routes, with registration answers held until the test releases them. */
function fakeSim(protocolVersion = 1) {
const requests: string[] = []
/** Calls the inbox offers; a claim answers for the first one. */
const offered: string[] = []
/**
* Answers to pending registrations: enabled or not, an HTTP status Sim fails with, or
* `'offline'` for a request that never reached Sim.
*/
const registrations: Array<(answer: boolean | number | 'offline') => void> = []
const fetch = vi.fn(async (url: string, init: RequestInit): Promise<Response> => {
const path = new URL(url).pathname
requests.push(`${init.method} ${path}`)
if (path === '/api/desktop/devices') {
const answer = await new Promise<boolean | number | 'offline'>((resolve) =>
registrations.push(resolve)
)
if (answer === 'offline') throw new TypeError('fetch failed')
if (typeof answer === 'number') {
return Response.json({ error: 'Not found' }, { status: answer })
}
const enabled = answer
return Response.json({
enabled,
protocolVersion,
leaseMs: 60_000,
leaseRenewMs: 20_000,
reconcileMs: 10_000,
})
}
if (path === '/api/desktop/inbox') {
return Response.json({
items: offered.map((toolCallId) => ({
kind: 'call',
toolCallId,
toolName: 'terminal',
chatId: 'chat-a',
workspaceId: 'ws-1',
createdAt: new Date().toISOString(),
})),
})
}
if (path === '/api/desktop/tool/claim') {
const toolCallId = offered.shift()
if (!toolCallId) return Response.json({ error: 'gone' }, { status: 404 })
return Response.json({
toolName: 'terminal',
args: { operation: 'run', args: { command: 'sleep 600' } },
chatId: 'chat-a',
workspaceId: 'ws-1',
executionToken: `token-${toolCallId}`,
})
}
if (path === '/api/desktop/tool/lease') return Response.json({ renewed: true })
if (path === '/api/desktop/tool/complete') return Response.json({ outcome: 'recorded' })
return new Promise<Response>((_resolve, reject) =>
init.signal?.addEventListener('abort', () => reject(new Error('aborted')))
)
})
return { fetch, requests, registrations, offered }
}
async function service(protocolVersion = 1, userDataPath?: string) {
const sim = fakeSim(protocolVersion)
/** What the sleep blocker was told, in order. */
const busy: boolean[] = []
const desktopExecutor = createDesktopExecutorService({
userDataPath: userDataPath ?? (await mkdtemp(join(tmpdir(), 'sim-executor-service-'))),
origin: () => 'https://sim.test',
appSession: () => ({ fetch: sim.fetch }),
preferences: () => ({ browserEnabled: true, terminalEnabled: true }),
accountDataAvailable: () => true,
// A command that runs until it is stopped.
runner: {
run: (_call, signal) =>
new Promise((resolve) =>
signal.addEventListener('abort', () =>
resolve({ status: 'cancelled', message: 'Stopped.' })
)
),
cancel: async () => {},
},
onBusyChange: (value) => busy.push(value),
})
return { sim, desktopExecutor, busy }
}
describe('results recovery will hand to the model', () => {
it('names the calls whose real result the journal holds for recovery to send', async () => {
const userData = await mkdtemp(join(tmpdir(), 'sim-executor-service-'))
const journal = createExecutorJournal(join(userData, 'desktop-executor-journal.json'))
await journal.put({ toolCallId: 'claimed', state: 'claimed', executionToken: 't1' })
await journal.put({ toolCallId: 'started', state: 'started', executionToken: 't2' })
await journal.put({
toolCallId: 'unknown',
state: 'result',
executionToken: 't3',
completion: { status: 'error', message: 'x', data: { outcomeUnknown: true } },
})
await journal.put({
toolCallId: 'not-started',
state: 'result',
executionToken: 't4',
completion: { status: 'error', message: 'x', data: { notStarted: true } },
})
await journal.put({
toolCallId: 'stopped',
state: 'result',
executionToken: 't6',
completion: { status: 'cancelled', message: 'Stopped.' },
})
await journal.put({
toolCallId: 'too-large',
state: 'result',
executionToken: 't7',
completion: { status: 'error', message: 'x', data: { resultOmitted: true } },
})
await journal.put({
toolCallId: 'handed-back',
state: 'result',
executionToken: 't5',
completion: { status: 'success', message: 'running', data: { status: 'running' } },
})
const { desktopExecutor } = await service(1, userData)
expect([...(await desktopExecutor.pendingResults())]).toEqual(['handed-back'])
})
it('counts an unreadable journal as holding none', async () => {
const userData = await mkdtemp(join(tmpdir(), 'sim-executor-service-'))
// A directory where the journal file should be: every read of it fails.
await mkdir(join(userData, 'desktop-executor-journal.json'))
const { desktopExecutor } = await service(1, userData)
expect([...(await desktopExecutor.pendingResults())]).toEqual([])
})
it('names what the journal held before recovery sent it, even when asked after', async () => {
const userData = await mkdtemp(join(tmpdir(), 'sim-executor-service-'))
const path = join(userData, 'desktop-executor-journal.json')
await createExecutorJournal(path).put({
toolCallId: 'handed-back',
state: 'result',
executionToken: 't1',
completion: { status: 'success', message: 'running', data: { status: 'running' } },
})
// Recovery returns only once Sim has the result and the journal no longer holds it, so a
// snapshot taken any time after recovery started would come back empty.
const recover = DesktopExecutor.prototype.recover
const recovered = vi
.spyOn(DesktopExecutor.prototype, 'recover')
.mockImplementation(async function (this: DesktopExecutor) {
await recover.call(this)
await vi.waitFor(async () => expect(await createExecutorJournal(path).load()).toEqual([]))
})
try {
const { sim, desktopExecutor } = await service(1, userData)
desktopExecutor.start()
await vi.waitFor(() => expect(sim.registrations).toHaveLength(1))
sim.registrations[0]?.(true)
await vi.waitFor(() => expect(sim.requests).toContain('POST /api/desktop/tool/complete'))
await vi.waitFor(async () => expect(await createExecutorJournal(path).load()).toEqual([]))
expect([...(await desktopExecutor.pendingResults())]).toEqual(['handed-back'])
await desktopExecutor.signOut()
} finally {
recovered.mockRestore()
}
})
it('counts a journal that cannot be loaded at all as holding none, and still starts', async () => {
journalFaults.loads = 1
const { sim, desktopExecutor } = await service()
expect([...(await desktopExecutor.pendingResults())]).toEqual([])
desktopExecutor.start()
await vi.waitFor(() => expect(sim.registrations).toHaveLength(1))
sim.registrations[0]?.(true)
await vi.waitFor(() => expect(sim.requests).toContain('GET /api/desktop/inbox'))
await desktopExecutor.signOut()
})
})
describe('desktop executor registration', () => {
it('offers the device for binding once Sim enables it', async () => {
const { sim, desktopExecutor } = await service()
desktopExecutor.start()
await vi.waitFor(() => expect(sim.registrations).toHaveLength(1))
sim.registrations[0]?.(true)
await vi.waitFor(() => expect(desktopExecutor.getDevice()).not.toBeNull())
await vi.waitFor(() => expect(sim.requests).toContain('GET /api/desktop/inbox'))
})
it('never resumes for a registration that Sim answers after sign-out', async () => {
const { sim, desktopExecutor } = await service()
desktopExecutor.start()
await vi.waitFor(() => expect(sim.registrations).toHaveLength(1))
await desktopExecutor.signOut()
sim.registrations[0]?.(true)
await sleep(50)
expect(desktopExecutor.getDevice()).toBeNull()
expect(sim.requests).not.toContain('GET /api/desktop/inbox')
expect(sim.requests).not.toContain('GET /api/desktop/inbox/stream')
})
it('offers no binding to a Sim that speaks another protocol version', async () => {
const { sim, desktopExecutor } = await service(2)
desktopExecutor.start()
await vi.waitFor(() => expect(sim.registrations).toHaveLength(1))
sim.registrations[0]?.(true)
await sleep(100)
expect(desktopExecutor.getDevice()).toBeNull()
expect(sim.requests.filter((request) => request.includes('/api/desktop/inbox'))).toEqual([])
})
it('stays dormant while Sim has the executor off: no inbox and no doorbell', async () => {
const { sim, desktopExecutor } = await service()
desktopExecutor.start()
await vi.waitFor(() => expect(sim.registrations).toHaveLength(1))
sim.registrations[0]?.(false)
await sleep(100)
expect(desktopExecutor.getDevice()).toBeNull()
expect(sim.requests.filter((request) => request.includes('/api/desktop/inbox'))).toEqual([])
})
it('stays dormant against a Sim without the executor routes, checking back only slowly', async () => {
const { sim, desktopExecutor } = await service()
vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout', 'setInterval', 'clearInterval'] })
try {
desktopExecutor.start()
await vi.waitFor(() => expect(sim.registrations).toHaveLength(1))
sim.registrations[0]?.(404)
await vi.advanceTimersByTimeAsync(60_000)
expect(sim.registrations).toHaveLength(1)
expect(sim.requests.filter((request) => request.includes('/api/desktop/inbox'))).toEqual([])
await vi.advanceTimersByTimeAsync(15 * 60_000)
expect(sim.registrations).toHaveLength(2)
} finally {
vi.useRealTimers()
}
})
it('registers again as soon as the network returns, not after its backoff', async () => {
const { sim, desktopExecutor } = await service()
vi.mocked(net.isOnline).mockReturnValue(false)
vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout', 'setInterval', 'clearInterval'] })
try {
desktopExecutor.start()
// Four failures with no network leave the next attempt at least 12.8 s away.
for (let failure = 0; failure < 4; failure += 1) {
await vi.waitFor(() => expect(sim.registrations).toHaveLength(failure + 1))
sim.registrations[failure]?.('offline')
await vi.advanceTimersByTimeAsync(failure < 3 ? 10_000 : 0)
}
expect(sim.registrations).toHaveLength(4)
vi.mocked(net.isOnline).mockReturnValue(true)
await vi.advanceTimersByTimeAsync(2_500)
expect(sim.registrations).toHaveLength(5)
} finally {
vi.useRealTimers()
vi.mocked(net.isOnline).mockReturnValue(true)
}
})
it('registers with no install id it could not save', async () => {
const userData = await mkdtemp(join(tmpdir(), 'sim-executor-service-'))
await chmod(userData, 0o500)
try {
const { sim, desktopExecutor } = await service(1, userData)
desktopExecutor.start()
await sleep(200)
expect(sim.requests.filter((request) => request.includes('/api/desktop/devices'))).toEqual([])
} finally {
await chmod(userData, 0o700)
}
})
it('keeps its install id through a read failure instead of minting a new one', async () => {
const userData = await mkdtemp(join(tmpdir(), 'sim-executor-service-'))
// A path that exists but cannot be read as a file: not "no id yet".
await mkdir(join(userData, 'desktop-executor-device.json'))
const { sim, desktopExecutor } = await service(1, userData)
desktopExecutor.start()
await sleep(200)
expect(sim.requests.filter((request) => request.includes('/api/desktop/devices'))).toEqual([])
})
it('releases the sleep blocker at sign-out while a call is still running', async () => {
const { sim, desktopExecutor, busy } = await service()
sim.offered.push('call-1')
desktopExecutor.start()
await vi.waitFor(() => expect(sim.registrations).toHaveLength(1))
sim.registrations[0]?.(true)
await vi.waitFor(() => expect(busy).toEqual([true]))
await desktopExecutor.signOut()
expect(busy).toEqual([true, false])
})
})
describe('device name', () => {
it('fits a long hostname within what Sim accepts at registration', () => {
expect(deviceName(`${'studio-'.repeat(40)}.local`).length).toBeLessThanOrEqual(128)
expect(deviceName('Studio-Mac.local')).toBe('Studio-Mac')
})
})
apps/desktop/src/main/desktop-executor/service.ts
+504
View File
@@ -0,0 +1,504 @@
/**
* Runs the background executor for as long as the app is signed in: registers this install as a
* device, keeps the inbox doorbell open, reads the inbox on every ring and on Sim's reconcile
* timer, and follows sleep, wake and network changes. One per process, which the single-instance
* lock makes one per machine.
*/
import { hostname } from 'node:os'
import { join } from 'node:path'
import type { DesktopExecutorDevice } from '@sim/desktop-bridge'
import { createLogger } from '@sim/logger'
import { getErrorMessage } from '@sim/utils/errors'
import { generateId } from '@sim/utils/id'
import { isRecordLike } from '@sim/utils/object'
import { randomFloat } from '@sim/utils/random'
import { backoffWithJitter } from '@sim/utils/retry'
import { truncate } from '@sim/utils/string'
import type { Session } from 'electron'
import { app, net, powerMonitor } from 'electron'
import {
readFileWithinLimit,
removeFileIfPresent,
writeJsonFileAtomically,
} from '@/main/atomic-json-file'
import {
createDesktopExecutorClient,
type DesktopExecutorClient,
DeviceRequestError,
} from '@/main/desktop-executor/client'
import { InboxDoorbell } from '@/main/desktop-executor/doorbell'
import {
type DesktopApprovalItem,
DesktopExecutor,
type DesktopToolRunner,
isDeliveredResult,
} from '@/main/desktop-executor/executor'
import { createExecutorJournal } from '@/main/desktop-executor/journal'
import {
DESKTOP_EXECUTOR_PROTOCOL_VERSION,
type DesktopExecutorTiming,
type DesktopImportEntryRequest,
type DesktopImportedEntry,
} from '@/main/desktop-executor/protocol'
const logger = createLogger('DesktopExecutorService')
/** Batches the burst of cookie and preference events a sign-in produces into one registration. */
const REGISTRATION_DEBOUNCE_MS = 1_000
const REGISTRATION_RETRY_MAX_MS = 5 * 60_000
/** How often a device Sim has not enabled checks whether that changed. */
const DORMANT_RECHECK_MS = 15 * 60_000
const ONLINE_POLL_MS = 2_000
/** Spreads reconcile reads so devices that woke together do not read together. */
const RECONCILE_JITTER = 0.2
export interface DesktopExecutorServiceDeps {
userDataPath: string
origin: () => string
appSession: () => Pick<Session, 'fetch'>
preferences: () => { browserEnabled: boolean; terminalEnabled: boolean }
accountDataAvailable: () => boolean
runner: DesktopToolRunner
onApprovals?: (items: DesktopApprovalItem[]) => void
/** Whether any chat has desktop work claimed on this machine changed. */
onBusyChange?: (busy: boolean) => void
/** Sim has taken a call's real result as the call's own, so the model has it. */
onResultDelivered?: (toolCallId: string) => void
/** Sim is done with a call whose real result the model never got. */
onResultNotDelivered?: (toolCallId: string) => void
}
export interface DesktopExecutorService {
start(): void
/** Re-registers after a sign-in, a session change, or a change to what this device can run. */
refreshRegistration(): void
getDevice(): DesktopExecutorDevice | null
/**
* The calls whose real result ({@link isDeliveredResult}) is in the journal and not yet
* acknowledged, so recovery will hand it to the model. Read once, before recovery can change the
* journal, whenever it is asked; empty when the journal cannot be read.
*/
pendingResults(): Promise<Set<string>>
/** Stores one entry of a claimed import, as this device's registered session. */
importEntry(
request: DesktopImportEntryRequest,
signal: AbortSignal
): Promise<DesktopImportedEntry>
/** Sign-out: stops every action, forgets every call, and retires this install id. */
signOut(): Promise<void>
}
/** The install id, kept in userData; a new one is minted after sign-out or a conflict. */
async function readInstallId(filePath: string): Promise<string | null> {
let raw: string
try {
raw = (await readFileWithinLimit(filePath, 4096)).toString('utf8')
} catch (error) {
// Only a missing file means no id yet. Any other failure could be passing, and minting a new
// id over it would orphan the calls bound to this one, so it is left to the next attempt.
if (isRecordLike(error) && error.code === 'ENOENT') return null
throw error
}
try {
const parsed: unknown = JSON.parse(raw)
return isRecordLike(parsed) && typeof parsed.deviceId === 'string' ? parsed.deviceId : null
} catch {
// Unreadable contents can never yield the id; a fresh one replaces them.
return null
}
}
/** Sim refuses a longer device name at registration. */
const DEVICE_NAME_MAX_CHARS = 128
/** The machine's name as the user knows it, within what Sim accepts. */
export function deviceName(host = hostname()): string {
const name = host.replace(/\.local$/, '').trim() || 'Sim desktop'
return truncate(name, DEVICE_NAME_MAX_CHARS - 3)
}
export function createDesktopExecutorService(
deps: DesktopExecutorServiceDeps
): DesktopExecutorService {
const identityPath = join(deps.userDataPath, 'desktop-executor-device.json')
const journal = createExecutorJournal(join(deps.userDataPath, 'desktop-executor-journal.json'))
let deviceId: string | null = null
let device: DesktopExecutorDevice | null = null
let timing: DesktopExecutorTiming | null = null
let client: DesktopExecutorClient | null = null
let executor: DesktopExecutor | null = null
let doorbell: InboxDoorbell | null = null
let reconcileTimer: ReturnType<typeof setTimeout> | null = null
let registrationTimer: ReturnType<typeof setTimeout> | null = null
let registering = false
let registerAgain = false
let registrationAttempt = 0
/** Logged once per run of missing routes, not on every recheck. */
let routesMissingNoted = false
/** The last registration failed with no answer from Sim at all. */
let registrationFailedOffline = false
let suspended = false
let started = false
/** The journal's pending real results as they stood before this process's recovery. */
let pendingSnapshot: Promise<Set<string>> | null = null
const pendingResultsSnapshot = (): Promise<Set<string>> => {
// An unreadable journal loads as empty, so it holds none.
pendingSnapshot ??= journal
.load()
.then(
(entries) =>
new Set(
entries.flatMap((entry) =>
entry.state === 'result' && isDeliveredResult(entry.completion)
? [entry.toolCallId]
: []
)
)
)
// The journal reads an unreadable file as empty, so this is for a load that throws anyway:
// it holds none, and the executor still starts.
.catch((error: unknown) => {
logger.warn('Could not read the executor journal for pending results', {
error: getErrorMessage(error),
})
return new Set<string>()
})
return pendingSnapshot
}
/** Bumped on sign-out, so work started for the previous session cannot resume it. */
let generation = 0
let executorDeviceId: string | null = null
async function installId(): Promise<string> {
if (deviceId) return deviceId
deviceId = (await readInstallId(identityPath)) ?? (await rotateInstallId())
return deviceId
}
/**
* Mints a new install id and makes it durable before anything uses it: an id the next launch
* would not find again could never resume the calls bound to it. Fails if it cannot be saved.
*/
async function rotateInstallId(): Promise<string> {
const next = generateId()
deviceId = null
await writeJsonFileAtomically(identityPath, { deviceId: next })
deviceId = next
return next
}
/**
* Retires the current install id for good: a new one replaces it, or, if that cannot be saved,
* the old one is removed so no later launch or sign-in can pick it up again. False when neither
* worked and the old id is still on disk.
*/
async function retireInstallId(): Promise<boolean> {
try {
await rotateInstallId()
return true
} catch (error) {
logger.warn('Could not save a new desktop install id', { error: getErrorMessage(error) })
}
try {
await removeFileIfPresent(identityPath)
return true
} catch (error) {
logger.warn('Could not remove the retired desktop install id', {
error: getErrorMessage(error),
})
return false
}
}
function fetchWithAppSession(url: string, init: RequestInit): Promise<Response> {
return deps.appSession().fetch(url, init)
}
function scheduleReconcile(): void {
if (reconcileTimer) clearTimeout(reconcileTimer)
// Stopped loops (a dormant or unrecognized device) stay stopped until registration restarts them.
if (!timing || !executor || !doorbell) return
const delay = timing.reconcileMs * (1 - RECONCILE_JITTER + randomFloat() * RECONCILE_JITTER)
reconcileTimer = setTimeout(() => {
reconcileTimer = null
void executor?.reconcile().finally(scheduleReconcile)
}, delay)
}
function ring(): void {
if (suspended) return
void executor?.reconcile()
}
/**
* Sim stopped recognizing this device. With the executor on, the session changed under it, so
* it registers again at once. With it off, Sim never recorded the device, so it stays quiet and
* checks back later in case the executor is switched on.
*/
function handleUnrecognized(): void {
if (device) {
scheduleRegistration()
return
}
stopLoops()
// Every refused request lands here; re-arming each time would push the recheck out forever.
if (!registrationTimer) scheduleRegistration(DORMANT_RECHECK_MS)
}
function stopLoops(): void {
doorbell?.stop()
doorbell = null
if (reconcileTimer) clearTimeout(reconcileTimer)
reconcileTimer = null
}
/** Builds the executor for this install the first time Sim recognizes it. */
async function startExecutor(
id: string,
nextTiming: DesktopExecutorTiming,
registrationGeneration: number
): Promise<void> {
if (executor && executorDeviceId !== id) await resetExecutor()
timing = nextTiming
if (!executor || !client) {
executorDeviceId = id
client = createDesktopExecutorClient({
origin: deps.origin,
fetch: fetchWithAppSession,
deviceId: id,
})
executor = new DesktopExecutor({
client,
journal,
runner: deps.runner,
leaseRenewMs: nextTiming.leaseRenewMs,
onUnregistered: handleUnrecognized,
...(deps.onApprovals ? { onApprovals: deps.onApprovals } : {}),
...(deps.onBusyChange ? { onBusyChange: deps.onBusyChange } : {}),
...(deps.onResultDelivered ? { onResultDelivered: deps.onResultDelivered } : {}),
...(deps.onResultNotDelivered ? { onResultNotDelivered: deps.onResultNotDelivered } : {}),
})
// Recovery rewrites the journal, so what it held before is read first.
await pendingResultsSnapshot()
await executor.recover()
// Signed out while recovering: sign-out already disposed this executor.
if (registrationGeneration !== generation || !executor) return
} else {
executor.resumeParked()
}
if (!doorbell) {
doorbell = new InboxDoorbell({
client,
onRing: ring,
onUnregistered: handleUnrecognized,
})
if (!suspended) doorbell.start()
}
if (!suspended) void executor.reconcile()
scheduleReconcile()
}
async function register(): Promise<void> {
if (!deps.accountDataAvailable()) return
const registrationGeneration = generation
let id: string
try {
id = await installId()
} catch (error) {
registrationAttempt += 1
logger.warn('Could not read the desktop install id', { error: getErrorMessage(error) })
scheduleRegistration(
backoffWithJitter(registrationAttempt, null, {
baseMs: 2_000,
maxMs: REGISTRATION_RETRY_MAX_MS,
})
)
return
}
const preferences = deps.preferences()
const registrationClient = createDesktopExecutorClient({
origin: deps.origin,
fetch: fetchWithAppSession,
deviceId: id,
})
try {
const nextTiming = await registrationClient.register({
deviceId: id,
name: deviceName(),
appVersion: app.getVersion(),
platform: `${process.platform}-${process.arch}`,
capabilities: {
executor: DESKTOP_EXECUTOR_PROTOCOL_VERSION,
browser: preferences.browserEnabled,
terminal: preferences.terminalEnabled,
localFiles: deps.accountDataAvailable(),
},
})
if (registrationGeneration !== generation || id !== deviceId) return
registrationAttempt = 0
registrationFailedOffline = false
routesMissingNoted = false
// A Sim that speaks another protocol version gets no new turns bound to this device.
device =
nextTiming.enabled && nextTiming.protocolVersion === DESKTOP_EXECUTOR_PROTOCOL_VERSION
? { deviceId: id, protocolVersion: DESKTOP_EXECUTOR_PROTOCOL_VERSION }
: null
logger.info('Desktop executor registered', { enabled: nextTiming.enabled })
// Off on this Sim, and nothing here to finish: stay dormant. No inbox, no doorbell; only
// a slow recheck, so switching the executor on in Sim reaches this device without a relaunch.
if (!device && !executor && (await journal.load()).length === 0) {
if (registrationGeneration !== generation) return
scheduleRegistration(DORMANT_RECHECK_MS)
return
}
// Off on this Sim mid-session, or results from a previous run to deliver: a turn already
// bound here still finishes, so the executor serves the inbox; no new turn binds.
await startExecutor(id, nextTiming, registrationGeneration)
} catch (error) {
if (registrationGeneration !== generation) return
device = null
if (error instanceof DeviceRequestError && error.status === 409) {
// The id belongs to another account (a copied profile); this install takes a new one.
logger.warn('Desktop install id is registered to another account; minting a new one')
await resetExecutor()
if (await retireInstallId()) {
scheduleRegistration(0)
} else {
// The conflicting id is still on disk: retrying at once would only conflict again.
registrationAttempt += 1
scheduleRegistration(
backoffWithJitter(registrationAttempt, null, {
baseMs: 2_000,
maxMs: REGISTRATION_RETRY_MAX_MS,
})
)
}
return
}
if (error instanceof DeviceRequestError && error.unregistered) {
// Signed out: the next sign-in's session change registers again.
stopLoops()
return
}
if (error instanceof DeviceRequestError && (error.status === 404 || error.status === 405)) {
// A Sim without the executor routes (older, or self-hosted): dormant, with only the slow
// recheck, so an upgrade of Sim reaches this device without a relaunch.
stopLoops()
if (!routesMissingNoted) {
routesMissingNoted = true
logger.info('Sim does not offer the desktop background executor; staying dormant')
}
scheduleRegistration(DORMANT_RECHECK_MS)
return
}
registrationFailedOffline = error instanceof DeviceRequestError && error.status === 0
registrationAttempt += 1
logger.warn('Desktop executor registration failed', {
attempt: registrationAttempt,
error: getErrorMessage(error),
})
scheduleRegistration(
backoffWithJitter(registrationAttempt, null, {
baseMs: 2_000,
maxMs: REGISTRATION_RETRY_MAX_MS,
})
)
}
}
function scheduleRegistration(delayMs = REGISTRATION_DEBOUNCE_MS): void {
if (!started) return
if (registrationTimer) clearTimeout(registrationTimer)
registrationTimer = setTimeout(() => {
registrationTimer = null
void runRegistration()
}, delayMs)
}
/** One registration at a time; a trigger that lands mid-flight registers once more after it. */
async function runRegistration(): Promise<void> {
if (registering) {
registerAgain = true
return
}
registering = true
try {
do {
registerAgain = false
await register()
} while (registerAgain)
} finally {
registering = false
}
}
async function resetExecutor(): Promise<void> {
stopLoops()
const current = executor
executor = null
executorDeviceId = null
client = null
timing = null
device = null
await current?.dispose()
}
function wake(): void {
suspended = false
executor?.setPaused(false)
doorbell?.wake()
void executor?.reconcile()
scheduleReconcile()
}
function watchPowerAndNetwork(): void {
powerMonitor.on('suspend', () => {
suspended = true
executor?.setPaused(true)
doorbell?.stop()
})
powerMonitor.on('resume', wake)
let online = net.isOnline()
setInterval(() => {
const now = net.isOnline()
if (now && !online) {
wake()
// A registration that failed for want of a network need not wait out its backoff.
if (registrationFailedOffline) scheduleRegistration(0)
}
online = now
}, ONLINE_POLL_MS).unref?.()
}
return {
start() {
if (started) return
started = true
watchPowerAndNetwork()
scheduleRegistration(0)
},
refreshRegistration() {
scheduleRegistration()
},
getDevice() {
return device
},
pendingResults() {
return pendingResultsSnapshot()
},
importEntry(request, signal) {
if (!client) throw new Error('The Sim desktop app is not signed in to Sim.')
return client.importEntry(request, signal)
},
async signOut() {
// A registration in flight now answers for a session that is gone; it must not restart.
generation += 1
if (registrationTimer) clearTimeout(registrationTimer)
registrationTimer = null
await resetExecutor()
await journal.clear()
await retireInstallId()
},
}
}
apps/desktop/src/main/desktop-executor/sleep-blocker.test.ts
+62
View File
@@ -0,0 +1,62 @@
import { describe, expect, it } from 'vitest'
import { createSleepBlocker } from '@/main/desktop-executor/sleep-blocker'
function harness(enabled = true) {
let nextId = 1
/** The blockers the OS holds now, by id, with the kind each one prevents. */
const active = new Map<number, string>()
const powerSaveBlocker = {
start: (type: 'prevent-app-suspension') => {
const id = nextId++
active.set(id, type)
return id
},
stop: (id: number) => {
active.delete(id)
},
isStarted: (id: number) => active.has(id),
}
let preference = enabled
const blocker = createSleepBlocker({ enabled: () => preference, powerSaveBlocker })
return {
blocker,
active,
setPreference: (value: boolean) => {
preference = value
blocker.refresh()
},
}
}
describe('keeping the machine awake for background work', () => {
it('holds one blocker only while a chat has work running', () => {
const { blocker, active } = harness()
blocker.setBusy(true)
blocker.setBusy(true)
expect([...active.values()]).toEqual(['prevent-app-suspension'])
blocker.setBusy(false)
expect(active.size).toBe(0)
})
it('never holds one while the user has switched it off, and follows the switch live', () => {
const { blocker, active, setPreference } = harness(false)
blocker.setBusy(true)
expect(active.size).toBe(0)
setPreference(true)
expect(active.size).toBe(1)
setPreference(false)
expect(active.size).toBe(0)
})
it('does not start one for an idle machine when the switch turns on', () => {
const { active, setPreference } = harness(false)
setPreference(true)
expect(active.size).toBe(0)
})
})
apps/desktop/src/main/desktop-executor/sleep-blocker.ts
+50
View File
@@ -0,0 +1,50 @@
/**
* Keeps the machine from idle-sleeping while a chat has desktop work running in the background,
* and only then. It cannot keep a closed laptop awake: the OS sleeps it anyway, and the work
* resumes reporting when it wakes.
*/
interface PowerSaveBlocker {
start(type: 'prevent-app-suspension'): number
stop(id: number): void
isStarted(id: number): boolean
}
interface SleepBlockerDeps {
/** The user's "prevent sleep while a chat is running" switch. */
enabled: () => boolean
powerSaveBlocker: PowerSaveBlocker
}
/**
* Holds one `prevent-app-suspension` power blocker while the executor is busy and the switch is
* on, and releases it as soon as either stops being true. `setBusy` follows the executor;
* `refresh` follows the switch.
*/
export function createSleepBlocker(deps: SleepBlockerDeps) {
let busy = false
let blockerId: number | null = null
const apply = () => {
const wanted = busy && deps.enabled()
const held = blockerId !== null && deps.powerSaveBlocker.isStarted(blockerId)
if (wanted && !held) {
blockerId = deps.powerSaveBlocker.start('prevent-app-suspension')
} else if (!wanted && blockerId !== null) {
if (held) deps.powerSaveBlocker.stop(blockerId)
blockerId = null
}
}
return {
/** Whether any chat has desktop work claimed on this machine. */
setBusy(next: boolean): void {
busy = next
apply()
},
/** Re-applies the user's switch to the work running now. */
refresh(): void {
apply()
},
}
}
apps/desktop/src/main/desktop-settings.test.ts
+92
View File
@@ -0,0 +1,92 @@
import { beforeEach, describe, expect, it, vi } from 'vitest'
vi.mock('electron', () => import('@/test/electron-mock'))
import type { BrowserWindow } from 'electron'
import { createDesktopSettingsService } from '@/main/desktop-settings'
import { Notification } from '@/test/electron-mock'
function service(window: { url: string; focused: boolean } | null) {
const config = new Map<string, unknown>()
return createDesktopSettingsService({
config: {
get: (key: string) => config.get(key),
set: (key: string, value: unknown) => config.set(key, value),
flush: () => true,
} as never,
getMainWindow: () =>
window
? ({
isFocused: () => window.focused,
webContents: { getURL: () => window.url },
} as unknown as BrowserWindow)
: null,
openMainWindowAt: vi.fn(),
setAutoDownloadUpdates: vi.fn(),
setTrayEnabled: vi.fn(),
setBrowserEnabled: vi.fn(),
setTerminalEnabled: vi.fn(),
setBrowserTheme: vi.fn(),
setBrowserDefaultZoom: vi.fn(),
setTerminalDefaultZoom: vi.fn(),
setPreventSleepWhileRunning: vi.fn(),
getDefaultBrowserDownloadDirectory: () => '/tmp',
chooseBrowserDownloadDirectory: async () => null,
})
}
const BACKGROUND_CHAT = '/workspace/ws-1/chat/chat-b'
describe('desktop notifications in the default "background only" mode', () => {
beforeEach(() => {
Notification.instances.length = 0
})
it('announces a chat in the background while the user works in another one', () => {
const shown = service({
url: 'https://sim.ai/workspace/ws-1/chat/chat-c',
focused: true,
}).notify({
title: 'Fix CI',
body: 'Sim finished responding.',
route: BACKGROUND_CHAT,
background: true,
})
expect(shown).toBe(true)
expect(Notification.instances).toHaveLength(1)
})
it('stays quiet about the chat the focused window shows', () => {
const shown = service({ url: `https://sim.ai${BACKGROUND_CHAT}`, focused: true }).notify({
title: 'Fix CI',
body: 'Sim finished responding.',
route: BACKGROUND_CHAT,
background: true,
})
expect(shown).toBe(false)
expect(Notification.instances).toHaveLength(0)
})
it('stays quiet about the app as a whole while it is focused', () => {
const shown = service({ url: 'https://sim.ai/workspace/ws-1/home', focused: true }).notify({
title: 'Update ready',
body: 'Restart to update.',
})
expect(shown).toBe(false)
})
it('holds back an ordinary notification whenever the window is focused, wherever it is', () => {
// The workflow panel's chat completes while the user watches it on the workflow page.
const shown = service({ url: 'https://sim.ai/workspace/ws-1/w/wf-1', focused: true }).notify({
title: 'Task complete',
body: 'Sim finished responding.',
route: BACKGROUND_CHAT,
})
expect(shown).toBe(false)
expect(Notification.instances).toHaveLength(0)
})
})
apps/desktop/src/main/desktop-settings.ts
+32 -1
View File
@@ -37,6 +37,7 @@ export interface DesktopSettingsService {
getPreferences(): DesktopPreferences
setPreference(key: DesktopPreferenceKey, value: boolean): DesktopPreferences
setBrowserSearchSuggestionsEnabled(enabled: boolean): DesktopPreferences
setPreventSleepWhileRunning(enabled: boolean): DesktopPreferences
setAppearancePreference(
key: DesktopAppearanceSettingKey,
value: DesktopAppearanceTheme
@@ -60,6 +61,8 @@ interface DesktopSettingsServiceDeps {
setBrowserEnabled: (enabled: boolean) => void
/** Ends every open agent shell when the surface is turned off. */
setTerminalEnabled: (enabled: boolean) => void
/** Starts or stops keeping the machine awake for background work already running. */
setPreventSleepWhileRunning: (enabled: boolean) => void
/** Repaints current browser tabs when their persisted appearance changes. */
setBrowserTheme: (theme: DesktopAppearanceTheme) => void
/** Applies a new default zoom to current and future browser tabs. */
@@ -93,6 +96,7 @@ function readPreferences(
browserEnabled: config.get('browserEnabled') ?? true,
browserSearchSuggestionsEnabled: config.get('browserSearchSuggestionsEnabled') ?? true,
terminalEnabled: config.get('terminalEnabled') ?? true,
preventSleepWhileRunning: config.get('preventSleepWhileRunning') ?? true,
browserTheme: isDesktopAppearanceTheme(browserTheme) ? browserTheme : 'app',
browserDefaultZoom: isDesktopZoomPercent(browserDefaultZoom) ? browserDefaultZoom : 100,
browserDownloadDirectory:
@@ -105,6 +109,21 @@ function readPreferences(
}
}
/**
* Whether a focused window already shows what a notification is about. One without a route is
* about the app as a whole, which a focused window always shows; one about a chat is news only
* while that chat is in the background.
*/
function showsRoute(window: BrowserWindow, route: string | undefined): boolean {
if (!route) return true
try {
const shown = new URL(window.webContents.getURL()).pathname
return shown === new URL(route, 'https://sim.invalid').pathname
} catch {
return false
}
}
/**
* Owns device preferences and their native side effects. Renderer code can
* request a change, but only this main-process service touches login items,
@@ -160,6 +179,12 @@ export function createDesktopSettingsService(
deps.config.flush()
return read()
},
setPreventSleepWhileRunning(enabled) {
deps.config.set('preventSleepWhileRunning', enabled)
deps.config.flush()
deps.setPreventSleepWhileRunning(enabled)
return read()
},
setAppearancePreference(key, value) {
const previousBrowserTheme = key === 'browserTheme' ? read().browserTheme : undefined
deps.config.set(key, value)
@@ -203,7 +228,13 @@ export function createDesktopSettingsService(
return false
}
const window = deps.getMainWindow()
if (preferences.notificationsOnlyWhenUnfocused && window?.isFocused()) {
// A focused window holds back every notification, as it always has. Only a background
// chat's completion is shown anyway when the focused window is somewhere else.
if (
preferences.notificationsOnlyWhenUnfocused &&
window?.isFocused() &&
(!payload.background || showsRoute(window, payload.route))
) {
return false
}
apps/desktop/src/main/index.ts
+142 -13
View File
@@ -2,7 +2,17 @@ import { join } from 'node:path'
import { createLogger } from '@sim/logger'
import { getErrorMessage } from '@sim/utils/errors'
import type { OpenDialogOptions, Session, WebContents } from 'electron'
import { app, BrowserWindow, crashReporter, dialog, net, session, shell } from 'electron'
import {
app,
BrowserWindow,
crashReporter,
dialog,
Notification,
net,
powerSaveBlocker,
session,
shell,
} from 'electron'
import {
beginAccountDataTeardown,
completeDeploymentScopedTeardown,
@@ -17,9 +27,13 @@ import {
import { newChatRoute, settingsRoute } from '@/main/app-routes'
import {
activateBrowserScope as activateAgentBrowserScope,
cancelTool as cancelAgentBrowserTool,
clearBrowserProfile as clearAgentBrowserProfile,
closeBrowserSession as closeAgentBrowserSession,
executeTool as executeAgentBrowserTool,
hasBrowserScopeSession,
initDriver as initBrowserAgentDriver,
restoreBrowserScope as restoreAgentBrowserScope,
} from '@/main/browser-agent/driver'
import {
canReportPanelBounds,
@@ -47,6 +61,10 @@ import {
import { attachContextMenu } from '@/main/context-menu'
import { attachCspFallback } from '@/main/csp'
import { DesktopChatSessionStore } from '@/main/desktop-chat-session-store'
import { createApprovalNotifier } from '@/main/desktop-executor/approval-notifier'
import { createDesktopToolRunner } from '@/main/desktop-executor/runner'
import { createDesktopExecutorService } from '@/main/desktop-executor/service'
import { createSleepBlocker } from '@/main/desktop-executor/sleep-blocker'
import { createDesktopSettingsService } from '@/main/desktop-settings'
import { attachDownloadHandling } from '@/main/downloads'
import { createAuthFlow, createConnectFlow, createHandoffManager } from '@/main/handoff'
@@ -56,7 +74,8 @@ import {
} from '@/main/help-search'
import { registerIpcHandlers } from '@/main/ipc'
import { attachLoadHealth, type LoadHealthHandle } from '@/main/load-health'
import { LocalFilesystemService } from '@/main/local-filesystem'
import { executeLocalFileRequest } from '@/main/local-files'
import { LocalFilesystemService, mountVfsRoot } from '@/main/local-filesystem'
import { createEncryptedLocalFilesystemGrantStore } from '@/main/local-filesystem-grant-store'
import {
attachLocalPageProtocol,
@@ -75,12 +94,14 @@ import {
createSessionLifecycleCoordinator,
decideStartRoute,
handleConnectIntercept,
isSessionCookieName,
readSessionUserId,
resolveStartRoute,
} from '@/main/session-lifecycle'
import { setShellTheme } from '@/main/shell-theme'
import { attachTelemetryPolicy } from '@/main/telemetry-policy'
import { TerminalRegistry } from '@/main/terminal/registry'
import { createRunLedger } from '@/main/terminal/run-ledger'
import { installTray, type TrayHandle } from '@/main/tray'
import { checkForUpdatesInteractive, initUpdater, type UpdaterHandle } from '@/main/updater'
import { installBrowserUserAgent } from '@/main/user-agent'
@@ -147,15 +168,20 @@ function main(): void {
),
})
const scopeEvents = new ScopedEventRouter()
const terminal = new TerminalRegistry({
load: (scopeId) => desktopChatSessions.getTerminal(processOrigin, scopeId) ?? undefined,
save: (scopeId, snapshot) => desktopChatSessions.setTerminal(processOrigin, scopeId, snapshot),
migrate: (fromScopeId, toScopeId) =>
desktopChatSessions.migrateTerminal(processOrigin, fromScopeId, toScopeId),
disposeScope: (scopeId) => {
desktopChatSessions.deleteScope(processOrigin, scopeId)
const terminal = new TerminalRegistry(
{
load: (scopeId) => desktopChatSessions.getTerminal(processOrigin, scopeId) ?? undefined,
save: (scopeId, snapshot) =>
desktopChatSessions.setTerminal(processOrigin, scopeId, snapshot),
migrate: (fromScopeId, toScopeId) =>
desktopChatSessions.migrateTerminal(processOrigin, fromScopeId, toScopeId),
disposeScope: (scopeId) => {
desktopChatSessions.deleteScope(processOrigin, scopeId)
},
},
})
undefined,
createRunLedger(join(userDataPath, 'terminal-runs'))
)
const preloadPath = join(__dirname, 'preload.cjs')
const windows = new Set<BrowserWindow>()
@@ -306,7 +332,22 @@ function main(): void {
// Shells are account-scoped runtime state. Leaving them alive across
// sign-out would stream the previous account's output into the next
// renderer and keep its local processes running invisibly.
{ label: 'terminal sessions', clear: () => terminal.dispose() },
{
label: 'background executor',
clear: async () => {
approvalNotifier.clear()
await desktopExecutor.signOut()
},
},
{
label: 'terminal sessions',
// Agent commands are stopped by their own process groups first: closing a shell only
// hangs up its foreground, and a tmux run outlives the Sim terminal entirely.
clear: async () => {
await terminal.stopAgentCommands()
terminal.dispose()
},
},
{ label: 'task resource state', clear: clearDesktopChatSessions },
{ label: 'local filesystem grants', clear: () => localFilesystem.forgetAll() },
]
@@ -501,10 +542,19 @@ function main(): void {
// pinned strip survive, so switching back on resumes rather than restarts.
setBrowserEnabled: (enabled) => {
if (!enabled) closeAgentBrowserSession()
desktopExecutor.refreshRegistration()
},
setTerminalEnabled: (enabled) => {
if (!enabled) terminal.dispose()
// Agent commands are stopped by their process groups first; a tmux run outlives its shell.
if (!enabled) {
void terminal.stopAgentCommands().then(() => {
// Switched back on while the commands stopped: the shells opened since are the user's.
if (!desktopSettings.getPreferences().terminalEnabled) terminal.dispose()
})
}
desktopExecutor.refreshRegistration()
},
setPreventSleepWhileRunning: () => sleepBlocker.refresh(),
setBrowserTheme: setAgentBrowserTheme,
setBrowserDefaultZoom: setAgentBrowserDefaultZoom,
setTerminalDefaultZoom: (zoom) => {
@@ -531,6 +581,65 @@ function main(): void {
},
})
const sleepBlocker = createSleepBlocker({
enabled: () => desktopSettings.getPreferences().preventSleepWhileRunning ?? true,
powerSaveBlocker,
})
const approvalNotifier = createApprovalNotifier({
preferences: () => desktopSettings.getPreferences(),
focusedChatId: () => {
const win = focusedAppWindow()
if (!win) return null
const route = routeFromAppUrl(win.webContents.getURL())
return route ? (/\/chat\/([^/?#]+)/.exec(route)?.[1] ?? null) : null
},
openRoute: (route) => void openMainWindowAt(route),
createNotification: (options) =>
Notification.isSupported() ? new Notification(options) : null,
})
const desktopExecutor = createDesktopExecutorService({
userDataPath,
origin: appOrigin,
appSession: ensureAppSession,
preferences: () => desktopSettings.getPreferences(),
accountDataAvailable,
onApprovals: (items) => approvalNotifier.update(items),
onBusyChange: (busy) => sleepBlocker.setBusy(busy),
// A result the model has makes a tmux run it handed back as still going collectable across a
// restart.
onResultDelivered: (toolCallId) => terminal.markRunDelivered(toolCallId),
// A result the model never got leaves such a run with no one to come back to it, so it is
// stopped, as the launch sweep stops a previous process's.
onResultNotDelivered: (toolCallId) => void terminal.stopUndeliveredRun(toolCallId),
runner: createDesktopToolRunner({
preferences: () => desktopSettings.getPreferences(),
accountDataAvailable,
browser: {
executeTool: (scopeId, tool, params, toolCallId) =>
executeAgentBrowserTool(scopeId, tool, params, toolCallId, undefined, {
background: true,
}),
cancelTool: cancelAgentBrowserTool,
hasSession: hasBrowserScopeSession,
restoreScope: restoreAgentBrowserScope,
},
terminal,
localFiles: {
request: (call, request) =>
executeLocalFileRequest(request, { toolName: call.toolName, args: call.args }),
},
imports: {
importEntry: (request, signal) => desktopExecutor.importEntry(request, signal),
},
localFilesystem: {
handle: (request) => localFilesystem.handle(request),
vfsRoot: mountVfsRoot,
},
}),
})
const serverWindow = createServerWindow({
config,
defaultOrigin: DEFAULT_ORIGIN,
@@ -674,7 +783,14 @@ function main(): void {
...(kind === 'account' && origin
? [
{ label: 'sign-in handoff state', clear: () => handoff.clear() },
{ label: 'terminal sessions', clear: () => terminal.dispose() },
{ label: 'background executor', clear: () => desktopExecutor.signOut() },
{
label: 'terminal sessions',
clear: async () => {
await terminal.stopAgentCommands()
terminal.dispose()
},
},
{ label: 'task resource state', clear: clearDesktopChatSessions },
{
label: 'app session storage',
@@ -700,6 +816,11 @@ function main(): void {
}
}
// The same user's tmux runs from a previous process: one whose pane the model has, or will get
// from recovery, is left to the model, which may come back to it; any other has nothing left
// to collect what it does, so it is stopped, while its pane still carries its tag.
void desktopExecutor.pendingResults().then((pending) => terminal.stopUncollectableRuns(pending))
if (!accountDataAvailable()) {
logger.warn(
'Account-bearing browser, terminal, and local filesystem APIs are unavailable until local recovery succeeds'
@@ -842,7 +963,15 @@ function main(): void {
getConfiguration: () => serverWindow.getConfiguration(),
setOrigin: (origin) => serverWindow.setOrigin(origin),
},
getExecutorDevice: () => desktopExecutor.getDevice(),
})
if (accountDataAvailable()) {
// A sign-in, or a session rotation, binds the device to the new session.
ensureAppSession().cookies.on('changed', (_event, cookie, _cause, removed) => {
if (!removed && isSessionCookieName(cookie.name)) desktopExecutor.refreshRegistration()
})
desktopExecutor.start()
}
await ensureMainWindow()
installApplicationMenu({
config,
apps/desktop/src/main/ipc.test.ts
+4 -2
View File
@@ -289,6 +289,7 @@ describe('registerIpcHandlers', () => {
mockCoordinator.fillCredential.mockClear()
deps = {
appOrigin: () => APP,
getExecutorDevice: () => null,
allowHttpLocalhost: () => false,
accountDataAvailable: () => true,
isLocalPageUrl,
@@ -310,6 +311,7 @@ describe('registerIpcHandlers', () => {
getPreferences: vi.fn(() => DEFAULT_DESKTOP_PREFERENCES),
setPreference: vi.fn(),
setBrowserSearchSuggestionsEnabled: vi.fn(),
setPreventSleepWhileRunning: vi.fn(),
setAppearancePreference: vi.fn(),
setBrowserDefaultZoom: vi.fn(),
setTerminalDefaultZoom: vi.fn(),
@@ -485,7 +487,7 @@ describe('registerIpcHandlers', () => {
expect(mounts).not.toHaveBeenCalled()
expect(fetchAuthorization).toHaveBeenCalledWith(
`${APP}/api/desktop/tool/authorize`,
expect.objectContaining({ body: JSON.stringify({ toolCallId: 'tool-native' }) })
expect.objectContaining({ body: JSON.stringify({ toolCallId: 'tool-native', claim: true }) })
)
expect(
await handler?.(evilEvent, { operation: 'read', toolCallId: 'tool-native' })
@@ -569,7 +571,7 @@ describe('registerIpcHandlers', () => {
expect.objectContaining({
method: 'POST',
credentials: 'include',
body: JSON.stringify({ toolCallId: 'tool-1' }),
body: JSON.stringify({ toolCallId: 'tool-1', claim: true }),
})
)
})
apps/desktop/src/main/ipc.ts
+31 -4
View File
@@ -9,6 +9,7 @@ import {
isCurrentBrowserToolName,
} from '@sim/browser-protocol'
import {
type DesktopExecutorDevice,
type DesktopNotificationPayload,
type DesktopServerChangeResult,
type DesktopServerConfiguration,
@@ -304,10 +305,11 @@ export function parseDesktopNotificationPayload(raw: unknown): DesktopNotificati
if (typeof raw !== 'object' || raw === null) {
return null
}
const { title, body, route } = raw as {
const { title, body, route, background } = raw as {
title?: unknown
body?: unknown
route?: unknown
background?: unknown
}
if (
typeof title !== 'string' ||
@@ -322,7 +324,12 @@ export function parseDesktopNotificationPayload(raw: unknown): DesktopNotificati
if (route !== undefined && (typeof route !== 'string' || !isSafeInternalPath(route))) {
return null
}
return { title, body, ...(route !== undefined ? { route } : {}) }
return {
title,
body,
...(route !== undefined ? { route } : {}),
...(background === true ? { background: true } : {}),
}
}
export interface IpcDeps {
@@ -374,6 +381,8 @@ export interface IpcDeps {
getConfiguration: () => DesktopServerConfiguration
setOrigin: (origin: string) => Promise<DesktopServerChangeResult>
}
/** The device a new chat turn may bind to, or null while Sim has not enabled one. */
getExecutorDevice: () => DesktopExecutorDevice | null
}
/**
@@ -578,10 +587,13 @@ async function authorizeLocalFilesystemTool(
request: unknown
): Promise<boolean> {
if (typeof request !== 'object' || request === null) return false
// Claimed like an import: the server sees the read picked up, and refuses one it already
// failed as never started.
const authorization = await fetchDesktopToolAuthorization(
event,
deps,
(request as { requestId?: unknown }).requestId
(request as { requestId?: unknown }).requestId,
true
)
return authorization
? deps.localFilesystem.isAuthorizedClientToolRequest(request, authorization)
@@ -768,6 +780,12 @@ export function registerIpcHandlers(deps: IpcDeps): void {
},
handler: (request) => deps.localFilesystem.handle(request),
},
'desktop-executor:get-device': {
kind: 'invoke',
gate: 'app-origin',
denied: null,
handler: () => deps.getExecutorDevice(),
},
'desktop:settings:get': {
kind: 'invoke',
gate: 'app-origin',
@@ -793,6 +811,15 @@ export function registerIpcHandlers(deps: IpcDeps): void {
? deps.settings.setBrowserSearchSuggestionsEnabled(enabled)
: deps.settings.getPreferences(),
},
'desktop:settings:set-prevent-sleep': {
kind: 'invoke',
gate: 'app-origin',
denied: null,
handler: (enabled) =>
typeof enabled === 'boolean'
? deps.settings.setPreventSleepWhileRunning(enabled)
: deps.settings.getPreferences(),
},
'desktop:settings:set-appearance': {
kind: 'invoke',
gate: 'app-origin',
@@ -2113,7 +2140,7 @@ export function registerIpcHandlers(deps: IpcDeps): void {
event,
deps,
request.toolCallId,
request.operation === 'manifest',
request.operation === 'manifest' || request.operation === 'read',
(status) => {
failureStatus = status
}
apps/desktop/src/main/local-filesystem.test.ts
+42
View File
@@ -107,6 +107,27 @@ describe('LocalFilesystemService', () => {
})
})
it('trims long runs of slashes in grep and glob paths in linear time', async () => {
const granted = await mount(service)
const slashes = `${'/'.repeat(200_000)}x`
const startedAt = performance.now()
expect(
service.isAuthorizedClientToolRequest(
{ operation: 'grep', uri: granted.uri, pattern: 'TODO', requestId: 'grep-tool' },
{ toolName: 'grep', args: { path: slashes, pattern: 'TODO' } }
)
).toBe(false)
await service.handle({
operation: 'glob',
uri: granted.uri,
pattern: '**/*.ts',
pathPrefix: slashes,
})
expect(performance.now() - startedAt).toBeLessThan(1000)
})
it('binds privileged client reads and searches to server-persisted tool args', async () => {
const granted = await mount(service)
const vfsRoot = `user-local/${encodeURIComponent(granted.name)}--${granted.id}`
@@ -236,6 +257,27 @@ describe('LocalFilesystemService', () => {
).toBe(false)
})
it('hands back nothing from a folder forgotten while the read ran', async () => {
const granted = await mount(service)
const reading = service.handle({ operation: 'read', uri: `${granted.uri}README.md` })
const searching = service.handle({ operation: 'grep', uri: granted.uri, pattern: 'hello' })
await service.handle({ operation: 'forget_mount', uri: granted.uri })
await expect(reading).resolves.toMatchObject({ ok: false, code: 'MOUNT_NOT_FOUND' })
await expect(searching).resolves.toMatchObject({ ok: false, code: 'MOUNT_NOT_FOUND' })
})
it('keeps a read whose folder the user selected again while it ran', async () => {
const granted = await mount(service)
const reading = service.handle({ operation: 'read', uri: `${granted.uri}README.md` })
const again = await mount(service)
expect(again.uri).toBe(granted.uri)
await expect(reading).resolves.toMatchObject({ ok: true })
})
it('rejects unknown mounts and symlinks that escape the selected directory', async () => {
const granted = await mount(service)
const outside = await mkdtemp(join(tmpdir(), 'sim-localfs-outside-'))
apps/desktop/src/main/local-filesystem.ts
+29 -10
View File
@@ -19,7 +19,7 @@ import {
} from '@sim/desktop-bridge/local-filesystem-limits'
import { generateId } from '@sim/utils/id'
import { isRecordLike } from '@sim/utils/object'
import { escapeRegExp, truncate } from '@sim/utils/string'
import { escapeRegExp, stripTrailingSlashes, truncate } from '@sim/utils/string'
import { app, dialog, shell } from 'electron'
import micromatch from 'micromatch'
import safeRegex from 'safe-regex2'
@@ -75,6 +75,22 @@ class LocalFilesystemError extends Error {
}
}
function mountNotFound(): LocalFilesystemError {
return new LocalFilesystemError(
'MOUNT_NOT_FOUND',
'That local folder is no longer available. Select it again.'
)
}
/** Operations that read inside one granted folder, named by the request's `uri`. */
const GRANT_SCOPED_OPERATIONS: ReadonlySet<string> = new Set([
'list',
'glob',
'read',
'grep',
'stat',
])
interface GrantedMount extends LocalFilesystemMount {
rootPath: string
bookmark?: string
@@ -157,7 +173,8 @@ function normalizeVfsDisplaySegment(segment: string): string {
.replace(/\s+/g, ' ')
}
function mountVfsRoot(mount: GrantedMount): string {
/** The `user-local/<name>--<id>` directory a granted folder appears under in the model's VFS. */
export function mountVfsRoot(mount: Pick<GrantedMount, 'id' | 'name'>): string {
return `user-local/${encodeURIComponent(normalizeVfsDisplaySegment(mount.name))}--${mount.id}`
}
@@ -420,7 +437,13 @@ export class LocalFilesystemService {
}
let data: LocalFilesystemData
// Reads and searches answer only while their folder is still granted: one forgotten while
// they ran must not hand back what they found in it.
let grant: GrantedMount | null = null
try {
if (GRANT_SCOPED_OPERATIONS.has(request.operation)) {
grant = this.parseUri(this.requiredUri(request)).mount
}
switch (request.operation) {
case 'mount_directory':
data = await this.mountDirectory()
@@ -470,6 +493,7 @@ export class LocalFilesystemService {
this.activeRequests.delete(requestId)
}
}
if (grant && this.mounts.get(grant.id)?.rootPath !== grant.rootPath) throw mountNotFound()
return { ok: true, data }
} catch (error) {
const safe = safeError(error)
@@ -532,7 +556,7 @@ export class LocalFilesystemService {
// request carrying them is the renderer searching for something the
// model did not ask for, or hiding results it believes are complete.
if (request.query !== undefined || request.include !== undefined) return false
const rawPath = typeof args.path === 'string' ? args.path.replace(/\/+$/, '') : ''
const rawPath = typeof args.path === 'string' ? stripTrailingSlashes(args.path) : ''
const uriAllowed =
rawPath === 'user-local'
? [...this.mounts.values()].some((mount) => request.uri === mount.uri)
@@ -889,12 +913,7 @@ export class LocalFilesystemService {
}
const mount = this.mounts.get(parsed.hostname)
if (!mount) {
throw new LocalFilesystemError(
'MOUNT_NOT_FOUND',
'That local folder is no longer available. Select it again.'
)
}
if (!mount) throw mountNotFound()
const encodedSegments = parsed.pathname.split('/').filter(Boolean)
const segments = encodedSegments.map((segment) => {
@@ -989,7 +1008,7 @@ export class LocalFilesystemService {
if (rawPathPrefix !== undefined && typeof rawPathPrefix !== 'string') {
throw new LocalFilesystemError('INVALID_REQUEST', 'pathPrefix must be a string.')
}
const pathPrefix = typeof rawPathPrefix === 'string' ? rawPathPrefix.replace(/\/+$/, '') : ''
const pathPrefix = typeof rawPathPrefix === 'string' ? stripTrailingSlashes(rawPathPrefix) : ''
const matcher = compileGlob(pattern)
const resolvedPath = await this.resolveUri(uri)
const baseStat = await stat(resolvedPath.realPath)
apps/desktop/src/main/terminal/index.ts
+343 -64
View File
@@ -14,11 +14,10 @@ import { homedir } from 'node:os'
import type { TerminalShortcutCommand } from '@sim/desktop-bridge'
import { createLogger } from '@sim/logger'
import {
DEFAULT_RUN_WAIT_MS,
isTerminalControlKey,
MAX_INPUT_KEYS,
MAX_RUN_WAIT_MS,
MAX_TOOL_OUTPUT_CHARS,
resolveRunWaitMs,
type TerminalCommandEvent,
type TerminalControlKey,
type TerminalCwdResult,
@@ -38,20 +37,26 @@ import {
isResourceTabSelectionShortcut,
resourceTabTargetIndex,
} from '@/main/resource-shortcuts'
import { elide, TerminalSession } from '@/main/terminal/session'
import { readForegroundProcessGroup, signalProcessGroup } from '@/main/terminal/process-group'
import type { RunLedger } from '@/main/terminal/run-ledger'
import { elide, type ShellStartupBounds, TerminalSession } from '@/main/terminal/session'
import {
activePane,
awaitRun,
capturePane,
closeRunWindow,
closeRunPane,
isRunComplete,
isTmuxUnavailable,
killPane,
listPanes,
pollRun,
type RecordedRun,
resolveAttachment,
runPaneState,
sendKey,
sendText,
startRun,
stopRun,
TMUX_KEY_NAMES,
type TmuxAttachment,
type TmuxRunHandle,
@@ -60,11 +65,15 @@ import {
const logger = createLogger('DesktopTerminal')
/**
* How long to let a just-spawned shell finish its startup files before
* concluding it has no integration. Generous because a heavy `.zshrc`
* (nvm, pyenv, starship) can take a while on a cold start.
* How long a just-spawned shell gets to reach its first prompt. It begins our startup files
* before anything of the user's runs, so one that has not within seconds was never instrumented.
* The user's own files get far longer, since a heavy `.zshrc` (oh-my-zsh, nvm, pyenv) is slow on
* a cold start and slower on a busy machine; past that, it is waiting on something.
*/
const SHELL_INTEGRATION_TIMEOUT_MS = 8_000
const SHELL_STARTUP_BOUNDS: ShellStartupBounds = { unstartedMs: 8_000, startingMs: 30_000 }
/** Enough of a stalled startup's screen to show what it is waiting on. */
const STARTUP_SCREEN_LINES = 20
/** Grace for a program to react to input before its screen is worth reading. */
const INPUT_ECHO_MS = 250
@@ -109,14 +118,6 @@ const HANDOFF_MAX_MS = 12 * 60 * 60 * 1000
*/
const HANDOFF_SETTLE_MS = 5_000
/** How long to hold the turn before handing a still-running command back. */
function resolveWaitMs(waitSeconds: number | undefined): number {
const requested = Number(waitSeconds)
return Number.isFinite(requested) && requested > 0
? Math.min(requested * 1000, MAX_RUN_WAIT_MS)
: DEFAULT_RUN_WAIT_MS
}
function elideOutput(value: string): { text: string; truncated: boolean } {
return elide(value, MAX_TOOL_OUTPUT_CHARS)
}
@@ -159,8 +160,61 @@ export interface TerminalServiceOptions {
*/
loadCwd?(): string | undefined
canSpawn?(): boolean
/** Reads and signals a terminal's foreground process group; the OS's by default. */
processGroups?: TerminalProcessGroups
/** Where tagged tmux runs are recorded, so a later process can still stop them. */
runLedger?: RunLedger
}
interface TerminalProcessGroups {
foreground(shellPid: number): Promise<number | null>
signal(pgid: number, signal: NodeJS.Signals): void
}
const OS_PROCESS_GROUPS: TerminalProcessGroups = {
foreground: readForegroundProcessGroup,
signal: signalProcessGroup,
}
/**
* One tool call's Stop: remembered if it arrives before the call's command starts, and handed to
* whatever is running once it does.
*/
interface StopLatch {
/** Aborts on Stop; work not yet started checks it, and input stops between keystrokes. */
readonly signal: AbortSignal
stopRunning: (() => Promise<void>) | null
}
function stoppedBeforeStart(): TerminalError {
return new TerminalError(
'CANCELLED',
'Stopped before the command started, so nothing ran in the terminal.'
)
}
function stoppedPartWay(): TerminalError {
return new TerminalError(
'CANCELLED',
'Stopped part way through the input; some of it may already have reached the terminal.'
)
}
/** Operations that change a terminal; a Stop that lands before one starts means it never does. */
const TERMINAL_CHANGING_OPERATIONS: ReadonlySet<TerminalOperation> = new Set([
'close',
'handoff',
'input',
'kill',
'run',
])
/**
* How long a stopped command gets to exit after each escalation: Ctrl-C first, as the user would
* press it, then SIGTERM and finally SIGKILL to its process group.
*/
const STOP_ESCALATION_MS = 2_000
export class TerminalService {
/** Insertion-ordered, which is also the tab order the user sees. */
private readonly sessions = new Map<string, TerminalSession>()
@@ -202,6 +256,17 @@ export class TerminalService {
* Held here so the terminal's own lifecycle can reclaim them.
*/
private readonly pendingRuns = new Map<string, TmuxRunHandle[]>()
/**
* Agent runs still going in tmux after their Sim terminal closed: the tmux session outlives the
* tab, but sign-out must still stop them.
*/
private readonly orphanedRuns = new Map<TmuxRunHandle, NodeJS.ProcessEnv>()
/** Runs a `run` call is still waiting on; their files are read when the wait ends. */
private readonly awaitedRuns = new Set<TmuxRunHandle>()
/** Awaited runs released meanwhile (their terminal closed); their files go once the wait ends. */
private readonly releasedAwaitedRuns = new Set<TmuxRunHandle>()
/** How to stop each tool call still in flight, so Stop interrupts exactly what it started. */
private readonly toolStops = new Map<string, () => Promise<void>>()
constructor(private readonly options: TerminalServiceOptions = {}) {}
@@ -421,33 +486,76 @@ export class TerminalService {
}
/**
* Removes the temp directories of tracked runs that have since finished.
* Removes the temp directories of tracked runs that have since finished, or whose pane is gone
* (the user closed it, or tmux restarted): nothing will ever write their status, and their ids
* may already belong to the user's own panes.
*
* Called when a new run starts on the same terminal, which is the one moment
* the service is already doing run bookkeeping — a dedicated reaper timer
* would be a subsystem to own for something this cheap. A run still going is
* left alone: its `tee` is still appending to that directory.
*/
private reapFinishedRuns(terminalId: string): void {
const pending = this.pendingRuns.get(terminalId)
if (!pending) return
const stillRunning: TmuxRunHandle[] = []
for (const handle of pending) {
if (isRunComplete(handle)) handle.dispose()
else stillRunning.push(handle)
private async reapFinishedRuns(terminalId: string, env: NodeJS.ProcessEnv): Promise<void> {
// A closed tab's run whose pane has since gone (its command ended) needs no stopping.
for (const [handle, orphanEnv] of this.orphanedRuns) {
if ((await runPaneState(handle, orphanEnv)) === 'gone') {
this.orphanedRuns.delete(handle)
this.forgetRun(handle)
}
}
if (stillRunning.length === 0) this.pendingRuns.delete(terminalId)
else this.pendingRuns.set(terminalId, stillRunning)
for (const handle of this.pendingRuns.get(terminalId) ?? []) {
if (this.awaitedRuns.has(handle)) continue
const complete = isRunComplete(handle)
if (complete || (await runPaneState(handle, env)) === 'gone') {
// A pane kept open after its command ended (`remain-on-exit`) closes with its run.
if (complete) await closeRunPane(handle, env)
this.untrackRun(terminalId, handle)
this.forgetRun(handle)
handle.dispose()
}
}
}
/** Drops a run's record once nothing of it is left for any process to stop. */
private forgetRun(handle: TmuxRunHandle): void {
if (handle.runId) this.options.runLedger?.forget(handle.runId)
}
/** Removes a run's files now, or once the call still reading them is done with them. */
private releaseRun(handle: TmuxRunHandle): void {
if (this.awaitedRuns.has(handle)) this.releasedAwaitedRuns.add(handle)
else handle.dispose()
}
private untrackRun(terminalId: string, handle: TmuxRunHandle): void {
const remaining = (this.pendingRuns.get(terminalId) ?? []).filter((entry) => entry !== handle)
if (remaining.length === 0) this.pendingRuns.delete(terminalId)
else this.pendingRuns.set(terminalId, remaining)
}
/**
* Releases every tracked run for a terminal, finished or not. The terminal is
* going away, so nothing will ever read these files again.
*/
private releasePendingRuns(terminalId: string): void {
private releasePendingRuns(terminalId: string, env?: NodeJS.ProcessEnv): void {
const pending = this.pendingRuns.get(terminalId)
if (!pending) return
for (const handle of pending) handle.dispose()
for (const handle of pending) {
// A finished run's pane may still be open (`remain-on-exit`): it is closed, while still the
// run's, before the record goes, and its files go only after that check, which an untracked
// run needs them for. Without the shell's environment the record stays, and the next sweep
// closes it. An untracked run is never stopped, so it is not kept either.
if (isRunComplete(handle) && env) {
void closeRunPane(handle, env)
.then(async () => {
if ((await runPaneState(handle, env)) === 'gone') this.forgetRun(handle)
})
.finally(() => this.releaseRun(handle))
continue
}
if (env && handle.runId !== null && !isRunComplete(handle)) this.orphanedRuns.set(handle, env)
this.releaseRun(handle)
}
this.pendingRuns.delete(terminalId)
}
@@ -461,10 +569,11 @@ export class TerminalService {
const closedCwd = session.currentCwd
const order = [...this.sessions.keys()]
const index = order.indexOf(terminalId)
const env = session.env
session.dispose()
this.sessions.delete(terminalId)
this.tmuxCache.delete(terminalId)
this.releasePendingRuns(terminalId)
this.releasePendingRuns(terminalId, env)
this.rememberClosed(closedCwd)
// Nothing is left for the user to hold on to; the next shell the agent
@@ -758,7 +867,7 @@ export class TerminalService {
this.sessions.clear()
this.tmuxCache.clear()
for (const handles of this.pendingRuns.values()) {
for (const handle of handles) handle.dispose()
for (const handle of handles) this.releaseRun(handle)
}
this.pendingRuns.clear()
this.activeId = null
@@ -775,8 +884,17 @@ export class TerminalService {
operation: TerminalOperation,
args: TerminalToolArgs
): Promise<TerminalToolResponse> {
// A Stop can arrive before the command exists (while the shell or tmux is still being
// resolved); the latch carries it to the moment the command would start.
const halt = new AbortController()
const latch: StopLatch = { signal: halt.signal, stopRunning: null }
const stop = async () => {
halt.abort()
await latch.stopRunning?.()
}
this.toolStops.set(toolCallId, stop)
try {
const result = await this.dispatch(toolCallId, operation, args ?? {})
const result = await this.dispatch(toolCallId, operation, args ?? {}, latch)
return { ok: true, result }
} catch (error) {
if (error instanceof TerminalError) {
@@ -786,13 +904,95 @@ export class TerminalService {
const message = (error as Error).message
logger.error('Terminal operation failed', { toolCallId, operation, error: message })
return { ok: false, error: message }
} finally {
if (this.toolStops.get(toolCallId) === stop) this.toolStops.delete(toolCallId)
}
}
/**
* Stops a tool call still in flight: interrupts the command a `run` started (escalating to its
* process group if Ctrl-C does not end it) or ends a handoff. The call then returns its result
* as usual. False when this service is not running that call.
*/
async cancelTool(toolCallId: string): Promise<boolean> {
const stop = this.toolStops.get(toolCallId)
if (!stop) return false
await stop()
return true
}
/**
* Stops every command the agent started that is still running, for sign-out: a plain shell's
* agent command by its own process group, as Stop does, and every tmux run window still going.
* A command the user started is not the agent's and is left alone.
*/
async stopAgentCommands(): Promise<void> {
const stops: Promise<void>[] = []
for (const session of this.sessions.values()) {
const toolCallId = session.agentCommandToolCallId
if (toolCallId) stops.push(this.stopCommand(session, toolCallId))
for (const handle of this.pendingRuns.get(session.terminalId) ?? []) {
if (!isRunComplete(handle)) stops.push(stopRun(handle, session.env, STOP_ESCALATION_MS))
}
}
for (const [handle, env] of this.orphanedRuns) {
stops.push(stopRun(handle, env, STOP_ESCALATION_MS))
}
this.orphanedRuns.clear()
await Promise.allSettled(stops)
}
/** Stops the command a plain shell's `run` handed back as still going, if it still is. */
async stopAgentCommand(toolCallId: string): Promise<void> {
await Promise.allSettled(
[...this.sessions.values()].map((session) => this.stopCommand(session, toolCallId))
)
}
/** Waits for the command a run started to end, up to `ms`. */
private async commandEnds(
session: TerminalSession,
toolCallId: string,
ms: number
): Promise<boolean> {
const deadline = Date.now() + ms
while (session.agentCommandToolCallId === toolCallId) {
if (Date.now() >= deadline || !session.alive) {
return session.agentCommandToolCallId !== toolCallId
}
await sleep(50)
}
return true
}
/**
* Interrupts the command a run started. Escalation is bound to that command's own process
* group, read while it still holds the foreground, and each signal is sent only while the same
* call and the same group still hold it, so a command the user starts meanwhile is never hit.
*/
private async stopCommand(session: TerminalSession, toolCallId: string): Promise<void> {
if (session.agentCommandToolCallId !== toolCallId) return
const groups = this.options.processGroups ?? OS_PROCESS_GROUPS
const pgid = await groups.foreground(session.pid)
if (session.agentCommandToolCallId !== toolCallId) return
session.kill('SIGINT')
for (const escalation of ['SIGTERM', 'SIGKILL'] as const) {
if (await this.commandEnds(session, toolCallId, STOP_ESCALATION_MS)) return
if (pgid === null || (await groups.foreground(session.pid)) !== pgid) return
if (session.agentCommandToolCallId !== toolCallId) return
logger.info('Stopped command ignored the previous signal; escalating', {
toolCallId,
signal: escalation,
})
groups.signal(pgid, escalation)
}
}
private async dispatch(
toolCallId: string,
operation: TerminalOperation,
args: TerminalToolArgs
args: TerminalToolArgs,
latch: StopLatch
): Promise<unknown> {
switch (operation) {
case 'list':
@@ -816,6 +1016,10 @@ export class TerminalService {
// A tab either has tmux attached or it does not, and every operation below
// behaves differently depending on which.
const tmux = await this.resolveTmux(session)
// A Stop that landed while the session resolved: nothing that changes the terminal starts.
if (latch.signal.aborted && TERMINAL_CHANGING_OPERATIONS.has(operation)) {
throw stoppedBeforeStart()
}
switch (operation) {
case 'cwd':
@@ -833,6 +1037,7 @@ export class TerminalService {
)
}
const target = await this.resolvePane(tmux.session, args, session)
if (latch.signal.aborted) throw stoppedBeforeStart()
const killed = await killPane(target, session.env)
if (!killed.ok) {
throw new TerminalError(
@@ -847,6 +1052,8 @@ export class TerminalService {
}
}
case 'handoff':
if (latch.signal.aborted) throw stoppedBeforeStart()
latch.stopRunning = async () => this.finishHandoff(session.terminalId)
return this.handoff(session, args)
case 'panes': {
if (!tmux) {
@@ -863,8 +1070,8 @@ export class TerminalService {
}
case 'run':
return tmux
? this.runInTmux(session, tmux.session, args)
: this.run(toolCallId, session, args)
? this.runInTmux(toolCallId, session, tmux.session, args, latch)
: this.run(toolCallId, session, args, latch)
case 'read': {
const requested = Number(args.lines)
const lines = Number.isFinite(requested) && requested > 0 ? requested : 200
@@ -888,8 +1095,8 @@ export class TerminalService {
}
case 'input':
return tmux
? this.inputToTmux(session, tmux.session, args)
: this.inputToShell(session, args)
? this.inputToTmux(session, tmux.session, args, latch.signal)
: this.inputToShell(session, args, latch.signal)
case 'kill': {
const signal =
args.signal === 'SIGTERM' || args.signal === 'SIGKILL' || args.signal === 'SIGINT'
@@ -900,6 +1107,7 @@ export class TerminalService {
// whole session rather than stopping the one thing they asked about.
if (tmux) {
const target = await this.resolvePane(tmux.session, args, session)
if (latch.signal.aborted) throw stoppedBeforeStart()
await sendKey(target, signal === 'SIGKILL' ? 'C-\\' : 'C-c', session.env)
return { signal, terminalId: session.terminalId, pane: target }
}
@@ -1015,15 +1223,18 @@ export class TerminalService {
private async inputToTmux(
terminal: TerminalSession,
session: string,
args: TerminalToolArgs
args: TerminalToolArgs,
signal: AbortSignal
): Promise<unknown> {
const target = await this.resolvePane(session, args, terminal)
if (signal.aborted) throw stoppedBeforeStart()
const keys = requestedKeys(args)
if (keys.length > 0) {
for (let index = 0; index < keys.length; index += 1) {
// Paced like the pty path: a pane redraws between presses, so a batch
// lands where the same keys pressed by hand would.
if (index > 0) await sleep(TMUX_KEY_GAP_MS)
if (signal.aborted) throw stoppedPartWay()
await sendKey(target, TMUX_KEY_NAMES[keys[index]] ?? keys[index], terminal.env)
}
} else if (typeof args.text === 'string') {
@@ -1031,6 +1242,7 @@ export class TerminalService {
// Enter is a separate send-keys for the same reason it is a separate pty
// write: a program reading one chunk treats text plus a carriage return
// as text, and the message sits unsubmitted.
if (signal.aborted) throw stoppedPartWay()
if (/[\r\n]$/.test(args.text)) await sendKey(target, 'Enter', terminal.env)
} else {
throw new TerminalError('INVALID_REQUEST', 'input needs `text`, `key`, or `keys`.')
@@ -1046,7 +1258,11 @@ export class TerminalService {
}
}
private async inputToShell(session: TerminalSession, args: TerminalToolArgs): Promise<unknown> {
private async inputToShell(
session: TerminalSession,
args: TerminalToolArgs,
signal: AbortSignal
): Promise<unknown> {
// Input is only ever delivered to a program that already holds the
// foreground. At a bare shell prompt these bytes would be a command
// line, and running commands that way would bypass the capture and
@@ -1061,14 +1277,17 @@ export class TerminalService {
// lets the model assume its message went through and start waiting on
// a reply to text still sitting unsubmitted in a composer; the screen
// is the evidence of what the program actually did with the input.
if (signal.aborted) throw stoppedBeforeStart()
const keys = requestedKeys(args)
if (keys.length > 0) {
await session.pressKeys(keys)
await session.pressKeys(keys, signal)
if (signal.aborted) throw stoppedPartWay()
await sleep(INPUT_ECHO_MS)
return { sent: keys.join(', '), ...(await session.readScrollback(INPUT_SCREEN_LINES)) }
}
if (typeof args.text === 'string') {
await session.type(args.text)
await session.type(args.text, signal)
if (signal.aborted) throw stoppedPartWay()
await sleep(INPUT_ECHO_MS)
return { sent: args.text, ...(await session.readScrollback(INPUT_SCREEN_LINES)) }
}
@@ -1085,30 +1304,68 @@ export class TerminalService {
* see through tmux.
*/
private async runInTmux(
toolCallId: string,
terminal: TerminalSession,
session: string,
args: TerminalToolArgs
args: TerminalToolArgs,
latch: StopLatch
): Promise<unknown> {
const command = typeof args.command === 'string' ? args.command.trim() : ''
if (!command) throw new TerminalError('INVALID_REQUEST', 'run needs a `command`.')
if (latch.signal.aborted) throw stoppedBeforeStart()
const started = Date.now()
this.reapFinishedRuns(terminal.terminalId)
const handle = await startRun(session, command, terminal.currentCwd, terminal.env)
await this.reapFinishedRuns(terminal.terminalId, terminal.env)
const ledger = this.options.runLedger
const handle = await startRun(session, command, terminal.currentCwd, terminal.env, {
...(ledger
? {
beforeStart: (run: RecordedRun) =>
ledger.record({ ...run, callId: toolCallId, state: 'started' }),
}
: {}),
})
if ('error' in handle) throw new TerminalError('SPAWN_FAILED', handle.error)
// Tracked from the moment its window exists, so sign-out can stop it even mid-wait.
const pending = this.pendingRuns.get(terminal.terminalId)
if (pending) pending.push(handle)
else this.pendingRuns.set(terminal.terminalId, [handle])
this.awaitedRuns.add(handle)
const waitMs = resolveWaitMs(args.waitSeconds)
const outcome = await awaitRun(handle, waitMs)
if (outcome.done) {
await closeRunWindow(handle, terminal.env)
handle.dispose()
} else {
// Still going, and nothing polls the status file again — `read` captures
// the pane instead.
const pending = this.pendingRuns.get(terminal.terminalId)
if (pending) pending.push(handle)
else this.pendingRuns.set(terminal.terminalId, [handle])
const waitMs = resolveRunWaitMs(args.waitSeconds)
// Inside tmux a stop arrives as Ctrl-C in the run's own window; closing that window hangs up
// anything that ignored it. The wait ends with the stop, since a closed window never writes
// the run's exit status.
let endWait: () => void = () => {}
const stopped = new Promise<void>((resolve) => {
endWait = resolve
})
latch.stopRunning = async () => {
await stopRun(handle, terminal.env, STOP_ESCALATION_MS)
endWait()
}
// A Stop that landed while the run window opened applies now.
if (latch.signal.aborted) void latch.stopRunning()
const outcome = await Promise.race([
awaitRun(handle, waitMs),
// A stopped run's closed pane never writes its status. An untracked run is never stopped,
// so it is still going unless its status says otherwise.
stopped.then(() =>
handle.runId === null ? pollRun(handle) : { ...pollRun(handle), done: true }
),
]).finally(() => {
this.awaitedRuns.delete(handle)
if (this.releasedAwaitedRuns.delete(handle)) handle.dispose()
})
if (outcome.done) {
await closeRunPane(handle, terminal.env)
this.untrackRun(terminal.terminalId, handle)
this.forgetRun(handle)
handle.dispose()
}
// Still going, it stays tracked, and nothing polls the status file again: `read` captures
// the pane instead. Its record is marked handed back only once that result reaches the model;
// see `TerminalRegistry.markRunDelivered`.
const { text, truncated } = elideOutput(outcome.output)
return {
@@ -1119,7 +1376,7 @@ export class TerminalService {
durationMs: Date.now() - started,
cwd: terminal.currentCwd,
terminalId: terminal.terminalId,
pane: handle.window,
pane: handle.pane,
truncated,
}
}
@@ -1127,20 +1384,15 @@ export class TerminalService {
private async run(
toolCallId: string,
session: TerminalSession,
args: TerminalToolArgs
args: TerminalToolArgs,
latch: StopLatch
): Promise<unknown> {
const command = typeof args.command === 'string' ? args.command.trim() : ''
if (!command) {
throw new TerminalError('INVALID_REQUEST', 'run needs a `command`.')
}
if (!session.hasShellIntegration) {
await session.waitForShellIntegration(SHELL_INTEGRATION_TIMEOUT_MS)
}
if (!session.hasShellIntegration) {
throw new TerminalError(
'NO_SHELL_INTEGRATION',
'This shell did not load Sim shell integration, so command boundaries and exit codes cannot be determined. Ask the user to run the command themselves, or use a bash/zsh session.'
)
await this.awaitShellStartup(session, latch)
}
if (session.isBusy) {
throw new TerminalError(
@@ -1149,7 +1401,34 @@ export class TerminalService {
)
}
return session.runCommand(command, toolCallId, resolveWaitMs(args.waitSeconds))
if (latch.signal.aborted) throw stoppedBeforeStart()
latch.stopRunning = () => this.stopCommand(session, toolCallId)
return session.runCommand(command, toolCallId, resolveRunWaitMs(args.waitSeconds))
}
/** Waits for a shell's first prompt, refusing the run with what it is doing if none comes. */
private async awaitShellStartup(session: TerminalSession, latch: StopLatch): Promise<void> {
const readiness = await session.waitForShellIntegration(SHELL_STARTUP_BOUNDS, latch.signal)
switch (readiness) {
case 'ready':
return
case 'stopped':
throw stoppedBeforeStart()
case 'exited':
throw new TerminalError('SESSION_CLOSED', 'The shell exited before it reached a prompt.')
case 'starting': {
const screen = (await session.readScrollback(STARTUP_SCREEN_LINES)).output.trim()
throw new TerminalError(
'NO_SHELL_INTEGRATION',
`The shell began its startup files over ${SHELL_STARTUP_BOUNDS.startingMs / 1000} s ago but never reached a prompt Sim can track, so nothing was run. Its screen:\n${screen || '(empty)'}\nIf a startup file is waiting for an answer, ask the user to answer it in that terminal (terminalId ${session.terminalId}), then run the command again. If the screen shows a prompt, a startup file replaced the shell (such as exec tmux or exec fish), so ask the user to run the command themselves.`
)
}
case 'not-instrumented':
throw new TerminalError(
'NO_SHELL_INTEGRATION',
'This shell did not load Sim shell integration, so command boundaries and exit codes cannot be determined. Ask the user to run the command themselves, or use a bash/zsh session.'
)
}
}
private spawn(
apps/desktop/src/main/terminal/pending-runs.test.ts
+5 -1
View File
@@ -20,17 +20,21 @@ vi.mock('@/main/terminal/tmux', () => ({
exitCode: tmuxStub.done ? 0 : null,
})),
capturePane: vi.fn(async () => ({ ok: true, stdout: '', stderr: '' })),
closeRunWindow: vi.fn(async () => undefined),
closeRunPane: vi.fn(async () => undefined),
isRunComplete: vi.fn((handle: { window: string }) => tmuxStub.complete.has(handle.window)),
runPaneState: vi.fn(async () => 'ours'),
isTmuxUnavailable: vi.fn(() => false),
killPane: vi.fn(async () => ({ ok: true, stdout: '', stderr: '' })),
listPanes: vi.fn(async () => []),
resolveAttachment: vi.fn(async () => ({ session: 'sess' })),
sendKey: vi.fn(async () => ({ ok: true, stdout: '', stderr: '' })),
sendText: vi.fn(async () => ({ ok: true, stdout: '', stderr: '' })),
stopRun: vi.fn(async () => undefined),
startRun: vi.fn(async () => {
const handle = {
window: `sess:${tmuxStub.handles.length}`,
pane: `%${tmuxStub.handles.length}`,
runId: `run-${tmuxStub.handles.length}`,
outPath: '/tmp/fake/out',
statusPath: '/tmp/fake/status',
dispose: vi.fn(),
apps/desktop/src/main/terminal/process-group.ts
+62
View File
@@ -0,0 +1,62 @@
/**
* The process group a terminal's tty currently delivers keyboard signals to: the whole pipeline
* the user would stop with Ctrl-C, including its children, and never the shell itself.
*/
import { spawn } from 'node:child_process'
import { createLogger } from '@sim/logger'
import { getErrorMessage } from '@sim/utils/errors'
const logger = createLogger('DesktopTerminalProcessGroup')
const LOOKUP_TIMEOUT_MS = 2_000
/**
* The tty's foreground process group, read from `ps`. Null when it cannot be read, or when the
* shell itself holds the foreground (it is sitting at its prompt).
*/
export function readForegroundProcessGroup(shellPid: number): Promise<number | null> {
if (!Number.isInteger(shellPid) || shellPid <= 0) return Promise.resolve(null)
return new Promise((resolve) => {
let child: ReturnType<typeof spawn>
try {
child = spawn('ps', ['-o', 'tpgid=', '-p', String(shellPid)], {
stdio: ['ignore', 'pipe', 'ignore'],
})
} catch {
resolve(null)
return
}
let stdout = ''
let settled = false
const finish = (value: number | null) => {
if (settled) return
settled = true
clearTimeout(timer)
resolve(value)
}
const timer = setTimeout(() => {
child.kill('SIGKILL')
finish(null)
}, LOOKUP_TIMEOUT_MS)
child.stdout?.on('data', (chunk: Buffer) => {
stdout += chunk.toString()
})
child.on('error', () => finish(null))
child.on('close', () => {
const pgid = Number.parseInt(stdout.trim(), 10)
finish(Number.isInteger(pgid) && pgid > 0 && pgid !== shellPid ? pgid : null)
})
})
}
/** Sends `signal` to every process in one group. */
export function signalProcessGroup(pgid: number, signal: NodeJS.Signals): void {
try {
process.kill(-pgid, signal)
} catch (error) {
logger.warn('Could not signal a terminal process group', {
signal,
error: getErrorMessage(error),
})
}
}
apps/desktop/src/main/terminal/registry-run-ledger.test.ts
+171
View File
@@ -0,0 +1,171 @@
import { mkdtempSync, rmSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { afterEach, describe, expect, it, vi } from 'vitest'
vi.mock('electron', () => import('@/test/electron-mock'))
/**
* Each recorded run's pane as tmux has it: running, dead after its command ended
* (`remain-on-exit`), stopped by Sim, already gone, or one tmux cannot answer for. A pane not
* listed is running.
*/
const { panes } = vi.hoisted(() => ({
panes: new Map<string, 'running' | 'dead' | 'stopped' | 'gone' | 'unknown'>(),
}))
vi.mock('@/main/terminal/tmux', async () => {
const actual =
await vi.importActual<typeof import('@/main/terminal/tmux')>('@/main/terminal/tmux')
const state = (runId: string) => panes.get(runId) ?? 'running'
return {
...actual,
recordedRunState: async (run: { runId: string }) => {
const pane = state(run.runId)
if (pane === 'running') return 'ours'
if (pane === 'dead') return 'finished'
return pane === 'unknown' ? 'unknown' : 'gone'
},
stopRecordedRun: async (run: { runId: string }) => {
if (state(run.runId) === 'unknown') return 'unknown'
if (state(run.runId) === 'running' || state(run.runId) === 'dead') {
panes.set(run.runId, 'stopped')
}
return 'gone'
},
}
})
import { TerminalRegistry } from '@/main/terminal/registry'
import { createRunLedger, type RunState } from '@/main/terminal/run-ledger'
const dirs: string[] = []
function ledgerDir(): string {
const dir = mkdtempSync(join(tmpdir(), 'sim-registry-ledger-'))
dirs.push(dir)
return join(dir, 'terminal-runs')
}
function run(runId: string, pane: string, state: RunState = 'started') {
return { runId, pane, socket: '/tmp/tmux-501/default', callId: `call-${runId}`, state }
}
afterEach(() => {
panes.clear()
for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true })
})
describe('stopping recorded tmux runs', () => {
it('forgets each run once nothing of it is left, and keeps one tmux could not answer for', async () => {
const dir = ledgerDir()
const previous = createRunLedger(dir)
previous.record(run('stopped', '%1'))
previous.record(run('unanswered', '%2'))
panes.set('unanswered', 'unknown')
// Signing out ends a run whose result the model has, too.
const ledger = createRunLedger(dir)
await new TerminalRegistry(undefined, undefined, ledger).stopAgentCommands()
expect(ledger.list().map((record) => record.runId)).toEqual(['unanswered'])
})
it("at launch, leaves the same user's runs the model can come back to, and stops the rest", async () => {
const dir = ledgerDir()
const previous = createRunLedger(dir)
// Sim took its result: the model has the pane.
previous.record(run('handed-back', '%1', 'delivered'))
previous.record(run('never-handed-back', '%2'))
// Its result is in the journal, unacknowledged: recovery will hand the pane to the model.
previous.record(run('on-its-way', '%3'))
previous.record(run('handed-back-and-gone', '%4', 'delivered'))
panes.set('handed-back-and-gone', 'gone')
// Handed back, but its command has since ended: only its dead pane is left.
previous.record(run('handed-back-and-done', '%5', 'delivered'))
panes.set('handed-back-and-done', 'dead')
const ledger = createRunLedger(dir)
await new TerminalRegistry(undefined, undefined, ledger).stopUncollectableRuns(
new Set(['call-on-its-way'])
)
expect(Object.fromEntries(panes)).toEqual({
'never-handed-back': 'stopped',
'handed-back-and-gone': 'gone',
'handed-back-and-done': 'stopped',
})
expect(
ledger
.list()
.map((record) => record.runId)
.sort()
).toEqual(['handed-back', 'on-its-way'])
})
it('keeps meaning to stop a run a stop for everything could not confirm', async () => {
const dir = ledgerDir()
createRunLedger(dir).record(run('unconfirmed', '%1', 'delivered'))
panes.set('unconfirmed', 'unknown')
const ledger = createRunLedger(dir)
// Sign-out could not confirm the run ended.
await new TerminalRegistry(undefined, undefined, ledger).stopAgentCommands()
panes.set('unconfirmed', 'running')
// A later launch for the same user, with the journal cleared, still stops it.
await new TerminalRegistry(undefined, undefined, ledger).stopUncollectableRuns(new Set())
expect(panes.get('unconfirmed')).toBe('stopped')
expect(ledger.list()).toEqual([])
})
it("stops only the run of a call whose result never reached the model, this process's or a previous one's", async () => {
const dir = ledgerDir()
createRunLedger(dir).record(run('previous', '%1'))
const ledger = createRunLedger(dir)
ledger.record(run('current', '%2'))
ledger.record(run('other', '%3'))
// Recorded as handed back: the model has its pane.
ledger.record(run('handed-back', '%4', 'delivered'))
const registry = new TerminalRegistry(undefined, undefined, ledger)
await registry.stopUndeliveredRun('call-current')
await registry.stopUndeliveredRun('call-previous')
await registry.stopUndeliveredRun('call-handed-back')
expect(panes.get('current')).toBe('stopped')
expect(panes.get('previous')).toBe('stopped')
expect(panes.get('other')).toBeUndefined()
expect(panes.get('handed-back')).toBeUndefined()
expect(
ledger
.list()
.map((record) => record.runId)
.sort()
).toEqual(['handed-back', 'other'])
})
it('notes the run of a call whose result reached the model as handed back', () => {
const dir = ledgerDir()
const ledger = createRunLedger(dir)
ledger.record(run('watched', '%1'))
ledger.record(run('other', '%2'))
new TerminalRegistry(undefined, undefined, ledger).markRunDelivered('call-watched')
expect(Object.fromEntries(ledger.list().map((record) => [record.runId, record.state]))).toEqual(
{ watched: 'delivered', other: 'started' }
)
})
it("at launch, stops the previous process's runs and none this one has started", async () => {
const dir = ledgerDir()
createRunLedger(dir).record(run('previous', '%1'))
const ledger = createRunLedger(dir)
ledger.record(run('current', '%2'))
await new TerminalRegistry(undefined, undefined, ledger).stopRecordedRuns({ excludeLive: true })
expect(ledger.list().map((record) => record.runId)).toEqual(['current'])
})
})
apps/desktop/src/main/terminal/registry.test.ts
+29
View File
@@ -1,4 +1,5 @@
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import type { TerminalCommandEvent } from '@sim/terminal-protocol'
import { beforeEach, describe, expect, it, vi } from 'vitest'
@@ -89,11 +90,13 @@ vi.mock('@/main/terminal/session', () => ({
},
}))
import { TerminalService, type TerminalServiceOptions } from '@/main/terminal'
import {
type ScopedTerminalSink,
TerminalRegistry,
type TerminalScopePersistence,
} from '@/main/terminal/registry'
import { createRunLedger } from '@/main/terminal/run-ledger'
function registry(): TerminalRegistry {
return new TerminalRegistry()
@@ -186,6 +189,32 @@ describe('TerminalRegistry', () => {
terminals.dispose()
})
it('gives a service rebuilt after a failed restore the run ledger too', () => {
const ledger = createRunLedger(join(tmpdir(), `sim-registry-ledger-${process.pid}`))
const built: Array<TerminalServiceOptions['runLedger']> = []
const persistence: TerminalScopePersistence = {
load: () => ({ v: 1 as const, tabs: [{ cwd: tmpdir() }, { cwd: tmpdir() }], activeIndex: 0 }),
save: () => true,
migrate: () => true,
disposeScope: () => {},
}
const terminals = new TerminalRegistry(
persistence,
(_scope, options) => {
built.push(options.runLedger)
return new TerminalService(options)
},
ledger
)
createControl.failAt = 2
expect(() => terminals.restoreScope('chat-A')).toThrow('PTY spawn failed')
// The service that failed to restore, and the one built in its place.
expect(built).toHaveLength(2)
expect(built.every((runLedger) => runLedger === ledger)).toBe(true)
})
it('rolls back a partial restore before retrying the complete descriptor', () => {
const persistedTabs = [{ cwd: tmpdir() }, { cwd: process.cwd() }, { cwd: tmpdir() }]
const persistence: TerminalScopePersistence = {
apps/desktop/src/main/terminal/registry.ts
+95 -1
View File
@@ -19,6 +19,11 @@ import {
type TerminalServiceOptions,
type TerminalSink,
} from '@/main/terminal'
import type { RunLedger, RunRecord } from '@/main/terminal/run-ledger'
import { recordedRunState, stopRecordedRun } from '@/main/terminal/tmux'
/** How long a recorded run gets to end on Ctrl-C before its pane is closed. */
const RECORDED_RUN_GRACE_MS = 2_000
/** Native PTYs and their headless xterm buffers are process-wide resources. */
export const MAX_TERMINALS_PER_PROCESS = 48
@@ -106,7 +111,8 @@ export class TerminalRegistry {
constructor(
private readonly persistence?: TerminalScopePersistence,
private readonly serviceFactory: TerminalServiceFactory = createTerminalService
private readonly serviceFactory: TerminalServiceFactory = createTerminalService,
private readonly runLedger?: RunLedger
) {}
setSink(sink: ScopedTerminalSink | null): void {
@@ -234,6 +240,11 @@ export class TerminalRegistry {
return entry.service.executeTool(toolCallId, operation, args)
}
/** Stops one in-flight tool call in a chat's terminals; false when none is running it. */
cancelTool(scope: string, toolCallId: string): Promise<boolean> {
return this.entries.get(scope)?.service.cancelTool(toolCallId) ?? Promise.resolve(false)
}
/**
* Moves a provisional chat's live service to its durable chat id.
*
@@ -364,6 +375,87 @@ export class TerminalRegistry {
return true
}
/**
* Stops every command the agent started in any chat's terminals; the user's own are untouched.
* That includes tmux runs no live terminal holds any more: a chat put away, or a previous
* process that quit or crashed while they ran.
*/
async stopAgentCommands(): Promise<void> {
await Promise.allSettled(
[...this.entries.values()].map((entry) => entry.service.stopAgentCommands())
)
await this.stopRecordedRuns()
}
/**
* At launch, for the same user: leaves a previous process's tmux run going only when the model
* has, or will get, the result that handed it back as still going: Sim acknowledged it
* (`delivered`), or the executor's journal holds it for recovery to send (`pendingResults`).
* Every other run is stopped, as is one a stop for everything could not confirm (`stop`).
*/
stopUncollectableRuns(pendingResults: ReadonlySet<string>): Promise<void> {
return this.stopRecordedRuns({
excludeLive: true,
keep: (run) =>
run.state === 'delivered' || (run.state === 'started' && pendingResults.has(run.callId)),
})
}
/**
* Notes that a call's result reached the model, so a tmux run it handed back as still going may
* be left to the model across a restart.
*/
markRunDelivered(callId: string): void {
const runId = this.runLedger?.runOf(callId)
if (runId) this.runLedger?.advance(runId, 'delivered')
}
/**
* Stops what a call's `run` handed back as still going once Sim is done with the call without
* its result reaching the model: a plain shell's command, or a tmux run, this process's or a
* previous one's. A run recorded as handed back is the model's, and is left going.
*/
async stopUndeliveredRun(callId: string): Promise<void> {
await Promise.allSettled(
[...this.entries.values()].map((entry) => entry.service.stopAgentCommand(callId))
)
await this.stopRecordedRuns({ callId, keep: (run) => run.state === 'delivered' })
}
/**
* Stops the recorded tmux runs, each only while its pane still carries its tag, and drops the
* records with nothing left to stop. `excludeLive` skips the runs this process has started;
* `callId` limits it to the run that call started; `keep` names runs to leave going, such as a
* previous process's runs whose results the model already has and may come back to.
*/
async stopRecordedRuns(
options: {
excludeLive?: boolean
callId?: string
/** Runs to leave going; their records are only dropped once their panes are gone. */
keep?: (run: RunRecord) => boolean
} = {}
): Promise<void> {
const ledger = this.runLedger
if (!ledger) return
const runs = ledger
.list({ excludeLive: options.excludeLive })
.filter((run) => options.callId === undefined || run.callId === options.callId)
await Promise.allSettled(
runs.map(async (run) => {
let keeping = options.keep?.(run) ?? false
let state = keeping ? await recordedRunState(run, process.env) : 'ours'
// A kept run whose command already ended leaves only its dead pane (`remain-on-exit`).
if (state === 'finished') keeping = false
if (!keeping) state = await stopRecordedRun(run, process.env, RECORDED_RUN_GRACE_MS)
if (state === 'gone') ledger.forget(run.runId)
// A run this sweep meant to stop but could not confirm stays meant to stop, so no later
// sweep keeps it.
else if (!keeping) ledger.advance(run.runId, 'stop')
})
)
}
/** Tears down every shell owned by every chat scope. */
dispose(): void {
const entries = [...this.entries.values()]
@@ -393,6 +485,7 @@ export class TerminalRegistry {
service: this.serviceFactory(scope, {
loadCwd: () => this.entries.get(scope)?.persisted?.tabs[0]?.cwd,
canSpawn: () => this.liveTerminalCount() < MAX_TERMINALS_PER_PROCESS,
runLedger: this.runLedger,
}),
persisted,
restoreApplied: false,
@@ -445,6 +538,7 @@ export class TerminalRegistry {
replacement = this.serviceFactory(entry.scope, {
loadCwd: () => entry.persisted?.tabs[0]?.cwd,
canSpawn: () => this.liveTerminalCount() < MAX_TERMINALS_PER_PROCESS,
runLedger: this.runLedger,
})
} catch {
this.entries.delete(entry.scope)
apps/desktop/src/main/terminal/run-ledger.test.ts
+165
View File
@@ -0,0 +1,165 @@
import { mkdirSync, mkdtempSync, readdirSync, rmSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { afterEach, describe, expect, it } from 'vitest'
import { createRunLedger } from '@/main/terminal/run-ledger'
const dirs: string[] = []
function scratch(): string {
const dir = mkdtempSync(join(tmpdir(), 'sim-run-ledger-'))
dirs.push(dir)
return join(dir, 'terminal-runs')
}
const RUN = {
runId: 'run-1',
pane: '%3',
socket: '/tmp/tmux-501/default',
callId: 'call-1',
state: 'started' as const,
}
afterEach(() => {
for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true })
})
describe('the tmux run ledger', () => {
it('keeps a run recorded until it is forgotten, for the next process too', () => {
const dir = scratch()
createRunLedger(dir).record(RUN)
const nextProcess = createRunLedger(dir)
expect(nextProcess.list()).toEqual([RUN])
nextProcess.forget(RUN.runId)
expect(createRunLedger(dir).list()).toEqual([])
})
it("leaves out this process's own runs when asked", () => {
const dir = scratch()
createRunLedger(dir).record(RUN)
const ledger = createRunLedger(dir)
ledger.record({ ...RUN, runId: 'run-2', pane: '%4' })
expect(ledger.list({ excludeLive: true })).toEqual([RUN])
expect(ledger.list()).toHaveLength(2)
})
it('drops files nothing could act on safely', () => {
const dir = scratch()
const ledger = createRunLedger(dir)
ledger.record(RUN)
writeFileSync(join(dir, 'garbled.json'), '{not json')
writeFileSync(join(dir, 'run-9.json'), JSON.stringify({ ...RUN, socket: 'relative.sock' }))
// A record under another run's name could stop the wrong run.
writeFileSync(join(dir, 'run-8.json'), JSON.stringify({ ...RUN, runId: 'run-7' }))
// A state no version wrote: nothing could tell what to do with the run.
writeFileSync(
join(dir, 'run-5.json'),
JSON.stringify({ ...RUN, runId: 'run-5', state: 'done' })
)
// Only a pane id names one pane; a target like this one names whatever pane is active there.
writeFileSync(
join(dir, 'run-6.json'),
JSON.stringify({ ...RUN, runId: 'run-6', pane: 'work:0.0' })
)
expect(ledger.list()).toEqual([RUN])
expect(readdirSync(dir)).toEqual(['run-1.json'])
})
it('cleans up after a write that never finished, keeping the saved record', () => {
const dir = scratch()
const ledger = createRunLedger(dir)
ledger.record(RUN)
writeFileSync(join(dir, 'run-2.json.4242.1.tmp'), '{"runId":"run-2","pa')
expect(ledger.list()).toEqual([RUN])
expect(readdirSync(dir)).toEqual(['run-1.json'])
})
it('keeps sweeping, and lets a call finish, when an entry cannot be removed', () => {
const dir = scratch()
const ledger = createRunLedger(dir)
ledger.record(RUN)
// Not a file, so removing it fails.
mkdirSync(join(dir, 'run-3.json'))
mkdirSync(join(dir, 'run-4.json'))
expect(ledger.list()).toEqual([RUN])
expect(() => ledger.forget('run-4')).not.toThrow()
})
it('reports a record it could not save, so the run is not started', () => {
const dir = scratch()
// A file where the directory should be: nothing can be saved under it.
writeFileSync(join(dir, '..', 'blocked'), '')
const ledger = createRunLedger(join(dir, '..', 'blocked'))
expect(ledger.record(RUN)).toBe(false)
})
it('moves a run forward only, for the next process too', () => {
const dir = scratch()
const ledger = createRunLedger(dir)
ledger.record(RUN)
ledger.advance(RUN.runId, 'delivered')
expect(createRunLedger(dir).list()).toEqual([{ ...RUN, state: 'delivered' }])
ledger.advance(RUN.runId, 'stop')
// A late acknowledgement never undoes a stop.
ledger.advance(RUN.runId, 'delivered')
expect(createRunLedger(dir).list()).toEqual([{ ...RUN, state: 'stop' }])
})
it('reads a record saved before runs had a state as the state it meant', () => {
const dir = scratch()
mkdirSync(dir, { recursive: true })
const { state: _state, ...saved } = RUN
const legacy = {
'run-started': { delivered: false },
'run-delivered': { delivered: true },
'run-stop': { delivered: false, mustStop: true },
// A stop wins over an acknowledgement.
'run-stop-delivered': { delivered: true, mustStop: true },
}
for (const [runId, fields] of Object.entries(legacy)) {
writeFileSync(join(dir, `${runId}.json`), JSON.stringify({ ...saved, runId, ...fields }))
}
const states = Object.fromEntries(
createRunLedger(dir)
.list()
.map((record) => [record.runId, record.state])
)
expect(states).toEqual({
'run-started': 'started',
'run-delivered': 'delivered',
'run-stop': 'stop',
'run-stop-delivered': 'stop',
})
})
it('finds the run a call started, a previous process recorded it or this one', () => {
const dir = scratch()
// Recovery may hand the model the result of a previous process's call.
createRunLedger(dir).record(RUN)
const ledger = createRunLedger(dir)
ledger.record({ ...RUN, runId: 'run-2', callId: 'call-2' })
expect(ledger.runOf(RUN.callId)).toBe(RUN.runId)
expect(ledger.runOf('call-2')).toBe('run-2')
ledger.forget(RUN.runId)
expect(ledger.runOf(RUN.callId)).toBeUndefined()
})
it('records nothing for a run tag that is not a plain id', () => {
const dir = scratch()
const ledger = createRunLedger(dir)
ledger.record({ ...RUN, runId: '../escape' })
expect(ledger.list()).toEqual([])
})
})
apps/desktop/src/main/terminal/run-ledger.ts
+192
View File
@@ -0,0 +1,192 @@
/**
* A durable record of the agent's tagged tmux runs, so a later process can still stop them.
*
* A tmux run outlives the app: quitting, crashing or signing out mid-teardown leaves its command
* going in the user's tmux server, and the next launch would otherwise know nothing about it.
* Each record names the run's tag, its pane and its tmux server's socket, and nothing else: no
* command line and no output, since it lives outside the account's encrypted data. A record is
* saved before the run's command may start, and removed once the run has ended, its pane is gone,
* or Sim has stopped it.
*/
import { readdirSync, readFileSync, rmSync } from 'node:fs'
import { join } from 'node:path'
import { createLogger } from '@sim/logger'
import { getErrorMessage } from '@sim/utils/errors'
import { writeJsonFileAtomicallySync } from '@/main/atomic-json-file'
import type { RecordedRun } from '@/main/terminal/tmux'
const logger = createLogger('DesktopTerminalRunLedger')
/**
* Where a recorded run stands, moving only forward:
* - `started`: its call has not handed its result to the model;
* - `delivered`: the model has the result that handed the run back as still going (with its
* pane), so a restart must leave the run be;
* - `stop`: a stop for everything (sign-out, Terminal off) could not confirm the run ended, so
* it is stopped whatever comes after, a late `delivered` included.
*/
export type RunState = 'started' | 'delivered' | 'stop'
const RUN_STATES: readonly RunState[] = ['started', 'delivered', 'stop']
/** A recorded run, with the call it belongs to and where it stands. */
export interface RunRecord extends RecordedRun {
/** The tool call that started the run, to match it against the executor's journal. */
callId: string
state: RunState
}
export interface RunLedger {
/** Saves a run's record; false when it could not be saved, so the run must not start. */
record(run: RunRecord): boolean
/** Moves a run forward to `state`; a run already there or past it stays as it is. */
advance(runId: string, state: RunState): void
/** The recorded run a tool call started, if the ledger still holds one. */
runOf(callId: string): string | undefined
forget(runId: string): void
/** Every recorded run; `excludeLive` leaves out runs this process recorded. */
list(options?: { excludeLive?: boolean }): RunRecord[]
}
/** Run tags are generated ids; anything else in the directory is not a record. */
const RUN_ID = /^[A-Za-z0-9_-]{1,128}$/
/** A record as saved: the current shape, or the earlier one with `delivered` and `mustStop`. */
type SavedRecord = Partial<RunRecord> & { delivered?: unknown; mustStop?: unknown }
/**
* The record's state. A record from before `state` existed (a released build wrote them) is read
* as the state it meant: `mustStop` as `stop`, which wins, then `delivered` as `delivered`.
*/
function stateOf(saved: SavedRecord): RunState | null {
if (RUN_STATES.includes(saved.state as RunState)) return saved.state as RunState
if (typeof saved.delivered !== 'boolean') return null
if (saved.mustStop === true) return 'stop'
return saved.delivered ? 'delivered' : 'started'
}
function parseRecord(text: string): RunRecord | null {
try {
const parsed = JSON.parse(text) as SavedRecord
const state = stateOf(parsed)
if (
typeof parsed.runId === 'string' &&
RUN_ID.test(parsed.runId) &&
typeof parsed.pane === 'string' &&
/^%\d+$/.test(parsed.pane) &&
typeof parsed.callId === 'string' &&
state &&
typeof parsed.socket === 'string' &&
parsed.socket.startsWith('/')
) {
return {
runId: parsed.runId,
pane: parsed.pane,
socket: parsed.socket,
callId: parsed.callId,
state,
}
}
} catch {
// Unreadable: treated as no record below.
}
return null
}
/** Removes a file the ledger no longer needs; a failure is logged, never thrown at a caller. */
function remove(path: string): void {
try {
rmSync(path, { force: true })
} catch (error) {
logger.warn('Could not remove a tmux run record', { error: getErrorMessage(error) })
}
}
export function createRunLedger(dir: string): RunLedger {
/** Runs recorded by this process, still going as far as it knows. */
const live = new Set<string>()
/** Each recorded run by the call that started it: read from the directory once, then kept. */
let byCall: Map<string, string> | null = null
const pathFor = (runId: string) => join(dir, `${runId}.json`)
/** Every saved record; anything else in the directory is removed. */
const readAll = (): RunRecord[] => {
let names: string[]
try {
names = readdirSync(dir)
} catch {
return []
}
const runs: RunRecord[] = []
for (const name of names) {
// A write that never finished leaves only its temporary file behind.
if (name.endsWith('.tmp')) {
remove(join(dir, name))
continue
}
if (!name.endsWith('.json')) continue
let record: RunRecord | null = null
try {
record = parseRecord(readFileSync(join(dir, name), 'utf8'))
} catch {
record = null
}
if (!record || `${record.runId}.json` !== name) {
// Nothing could act on it safely; it only takes up space.
remove(join(dir, name))
continue
}
runs.push(record)
}
return runs
}
const index = (): Map<string, string> => {
byCall ??= new Map(readAll().map((run) => [run.callId, run.runId]))
return byCall
}
return {
record(run) {
if (!RUN_ID.test(run.runId)) return false
try {
writeJsonFileAtomicallySync(pathFor(run.runId), run)
live.add(run.runId)
index().set(run.callId, run.runId)
return true
} catch (error) {
logger.warn('Could not record a tmux run', { error: getErrorMessage(error) })
return false
}
},
advance(runId, state) {
if (!RUN_ID.test(runId)) return
let record: RunRecord | null = null
try {
record = parseRecord(readFileSync(pathFor(runId), 'utf8'))
} catch {
record = null
}
if (!record || RUN_STATES.indexOf(state) <= RUN_STATES.indexOf(record.state)) return
try {
writeJsonFileAtomicallySync(pathFor(runId), { ...record, state })
} catch (error) {
// Left behind, a restart stops the run: the conservative side.
logger.warn('Could not update a tmux run record', { error: getErrorMessage(error) })
}
},
runOf(callId) {
return index().get(callId)
},
forget(runId) {
live.delete(runId)
for (const [callId, indexed] of byCall ?? []) if (indexed === runId) byCall?.delete(callId)
if (RUN_ID.test(runId)) remove(pathFor(runId))
},
list(options = {}) {
const runs = readAll()
return options.excludeLive ? runs.filter((run) => !live.has(run.runId)) : runs
},
}
}
apps/desktop/src/main/terminal/service.test.ts
+618 -2
View File
@@ -1,11 +1,116 @@
import { existsSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { sleep } from '@sim/utils/helpers'
import { describe, expect, it, vi } from 'vitest'
import { TerminalService } from '@/main/terminal'
import { createRunLedger } from '@/main/terminal/run-ledger'
/**
* A tmux attachment the service sees only when a test turns it on. Runs get real status files;
* stopping a run ends its command the way Ctrl-C in its pane would. Which panes tmux still holds
* as the run's, and the identity checks behind that, are covered against a fake tmux binary in
* `tmux.test.ts`; here the service's bookkeeping is what is under test.
*/
const tmuxFake = vi.hoisted(() => ({
on: false,
/** Panes a run was stopped in, in order. */
stopped: [] as string[],
/** Panes no longer the run's (closed by the user, or reused after a tmux restart). */
gone: new Set<string>(),
/** Runs start untracked, as on a tmux too old to tag their panes. */
untracked: false,
/** Run panes tmux still shows, kept open after their command ends (`remain-on-exit`). */
open: new Set<string>(),
/** tmux stops answering: no pane can be confirmed, so none is closed. */
unanswered: false,
statusPaths: new Map<string, string>(),
}))
vi.mock('@/main/terminal/tmux', async () => {
const actual =
await vi.importActual<typeof import('@/main/terminal/tmux')>('@/main/terminal/tmux')
let nextPane = 1
return {
...actual,
isTmuxUnavailable: () => (tmuxFake.on ? false : actual.isTmuxUnavailable()),
resolveAttachment: async (pid: number, env: NodeJS.ProcessEnv) =>
tmuxFake.on
? { session: 'agent', clientTty: '/dev/ttys001' }
: actual.resolveAttachment(pid, env),
startRun: async (...args: Parameters<typeof actual.startRun>) => {
if (!tmuxFake.on) return actual.startRun(...args)
const options = args[4]
const pane = `%${nextPane++}`
const runId = tmuxFake.untracked ? null : `run-${pane.slice(1)}`
const socket = tmuxFake.untracked ? null : '/tmp/tmux-fake/default'
// Like the real one, a tagged run starts only once its record is saved.
if (
runId &&
socket &&
options?.beforeStart &&
!options.beforeStart({ runId, pane, socket })
) {
return { error: 'The command could not be recorded for a later stop, so it was not run.' }
}
const dir = mkdtempSync(join(tmpdir(), 'sim-tmux-fake-'))
const statusPath = join(dir, 'status')
writeFileSync(join(dir, 'out'), 'partial output')
tmuxFake.statusPaths.set(pane, statusPath)
tmuxFake.open.add(pane)
return {
window: `@${pane.slice(1)}`,
pane,
runId,
socket,
outPath: join(dir, 'out'),
statusPath,
dispose: () => rmSync(dir, { recursive: true, force: true }),
}
},
runPaneState: async (...args: Parameters<typeof actual.runPaneState>) => {
if (!tmuxFake.on) return actual.runPaneState(...args)
if (tmuxFake.unanswered) return 'unknown'
if (tmuxFake.gone.has(args[0].pane)) return 'gone'
return args[0].runId === null ? 'unknown' : 'ours'
},
stopRun: async (...args: Parameters<typeof actual.stopRun>) => {
if (!tmuxFake.on) return actual.stopRun(...args)
const [handle] = args
if (tmuxFake.gone.has(handle.pane) || handle.runId === null) return
tmuxFake.stopped.push(handle.pane)
writeFileSync(handle.statusPath, '130')
},
closeRunPane: async (...args: Parameters<typeof actual.closeRunPane>) => {
if (!tmuxFake.on) return actual.closeRunPane(...args)
if (tmuxFake.unanswered) return
// An untracked run's pane is proven its own only from the run's files, read after tmux
// has answered, as the real check does.
await sleep(10)
if (args[0].runId === null && !existsSync(args[0].statusPath)) return
tmuxFake.open.delete(args[0].pane)
tmuxFake.gone.add(args[0].pane)
},
}
})
/** Stub sessions by terminal id, populated by the mock below. */
const { stubSessions } = vi.hoisted(() => ({
stubSessions: new Map<
string,
{ setBusy(busy: boolean): void; exit(): void; clearScrollback(): void }
{
setBusy(busy: boolean): void
exit(): void
clearScrollback(): void
/** Whether Ctrl-C ends the running command, as it does for most programs. */
setInterruptible(interruptible: boolean): void
/** Ends the running command the way its process exiting would. */
finishRun(exitCode: number): void
/** Returns the run as still going, its command left running, as a wait that ran out does. */
handBack(): void
kill: ReturnType<typeof vi.fn>
readonly runningToolCallId: string | null
}
>(),
}))
@@ -32,11 +137,50 @@ vi.mock('@/main/terminal/session', async () => {
terminalId: string
callbacks: { onExit(terminalId: string): void }
}) => {
const state = { cwd, disposed: false, busy: false }
const state = {
cwd,
disposed: false,
busy: false,
interruptible: true,
toolCallId: null as string | null,
resolveRun: null as ((result: Record<string, unknown>) => void) | null,
}
const finishRun = (exitCode: number) => {
const resolve = state.resolveRun
state.busy = false
state.toolCallId = null
state.resolveRun = null
resolve?.({ status: 'completed', exitCode, terminalId })
}
const stub = {
setBusy: (busy: boolean) => {
state.busy = busy
},
setInterruptible: (interruptible: boolean) => {
state.interruptible = interruptible
},
finishRun,
handBack: () => {
const resolve = state.resolveRun
state.resolveRun = null
resolve?.({ status: 'running', exitCode: null, terminalId })
},
runCommand: (_command: string, toolCallId: string) =>
new Promise((resolve) => {
state.busy = true
state.toolCallId = toolCallId
state.resolveRun = resolve
}),
get runningToolCallId() {
return state.toolCallId
},
get agentCommandToolCallId() {
return state.toolCallId
},
kill: vi.fn((signal: string) => {
if (signal === 'SIGINT' && state.interruptible) finishRun(130)
}),
waitForShellIntegration: async () => {},
/** Stands in for the user running `exit` or pressing Ctrl-D. */
exit: () => {
state.disposed = true
@@ -332,3 +476,475 @@ describe('a shell that ends by itself', () => {
expect(after.activeTerminalId).toBeNull()
})
})
describe('stopping a tool call', () => {
const COMMAND_GROUP = 4242
function cancellableService() {
/** The tty's foreground group as the OS would report it at each read. */
let foreground: number | null = COMMAND_GROUP
const processGroups = {
foreground: vi.fn(async (_shellPid: number) => foreground),
signal: vi.fn((_pgid: number, _signal: NodeJS.Signals) => {}),
}
const terminal = new TerminalService({ loadCwd: () => '/tmp', processGroups })
const { activeTerminalId } = terminal.start({ cols: 80, rows: 24 })
const session = stubSessions.get(activeTerminalId as string)
if (!session) throw new Error('No stub session')
return {
terminal,
session,
processGroups,
setForeground: (pgid: number | null) => {
foreground = pgid
},
}
}
/** Starts a run and waits until its command holds the terminal's foreground. */
async function startRun(
terminal: TerminalService,
session: { runningToolCallId: string | null },
toolCallId: string
) {
const running = terminal.executeTool(toolCallId, 'run', { command: 'sleep 60' })
await vi.waitFor(() => expect(session.runningToolCallId).toBe(toolCallId))
return { running }
}
it('interrupts the command its run started, and the run reports how it ended', async () => {
const { terminal, session, processGroups } = cancellableService()
const { running } = await startRun(terminal, session, 'call-run')
await expect(terminal.cancelTool('call-run')).resolves.toBe(true)
expect(session.kill).toHaveBeenCalledWith('SIGINT')
await expect(running).resolves.toMatchObject({ ok: true, result: { exitCode: 130 } })
expect(processGroups.signal).not.toHaveBeenCalled()
})
it("terminates the command's own process group when it ignores Ctrl-C", async () => {
const { terminal, session, processGroups } = cancellableService()
session.setInterruptible(false)
processGroups.signal.mockImplementation(() => session.finishRun(143))
const { running } = await startRun(terminal, session, 'call-stubborn')
await terminal.cancelTool('call-stubborn')
expect(processGroups.signal).toHaveBeenCalledWith(COMMAND_GROUP, 'SIGTERM')
await expect(running).resolves.toMatchObject({ ok: true, result: { exitCode: 143 } })
})
it('never signals a different command that took the foreground meanwhile', async () => {
const { terminal, session, processGroups, setForeground } = cancellableService()
session.setInterruptible(false)
const { running } = await startRun(terminal, session, 'call-replaced')
session.kill.mockImplementation(() => setForeground(9999))
await terminal.cancelTool('call-replaced')
expect(processGroups.signal).not.toHaveBeenCalled()
session.finishRun(0)
await running
})
it('runs nothing when the Stop arrives before the command starts', async () => {
const { terminal, session } = cancellableService()
const running = terminal.executeTool('call-early', 'run', { command: 'rm -rf build' })
await terminal.cancelTool('call-early')
await expect(running).resolves.toMatchObject({ ok: false, code: 'CANCELLED' })
expect(session.runningToolCallId).toBeNull()
})
it('sends no signal or keys for a kill or input stopped before it starts', async () => {
const { terminal, session } = cancellableService()
session.setBusy(true)
const killing = terminal.executeTool('call-kill', 'kill', { signal: 'SIGTERM' })
const typing = terminal.executeTool('call-input', 'input', { text: 'yes\n' })
await Promise.all([terminal.cancelTool('call-kill'), terminal.cancelTool('call-input')])
await expect(killing).resolves.toMatchObject({ ok: false, code: 'CANCELLED' })
await expect(typing).resolves.toMatchObject({ ok: false, code: 'CANCELLED' })
expect(session.kill).not.toHaveBeenCalled()
})
it("stops the agent's running command at sign-out", async () => {
const { terminal, session } = cancellableService()
const { running } = await startRun(terminal, session, 'call-left-running')
await terminal.stopAgentCommands()
expect(session.kill).toHaveBeenCalledWith('SIGINT')
await expect(running).resolves.toMatchObject({ ok: true, result: { exitCode: 130 } })
})
it('stops the command a run handed back as still going, once asked for that call', async () => {
const { terminal, session } = cancellableService()
const { running } = await startRun(terminal, session, 'call-handed-back')
session.handBack()
await expect(running).resolves.toMatchObject({ ok: true, result: { status: 'running' } })
// The call is over, so a Stop for it in flight finds nothing.
await expect(terminal.cancelTool('call-handed-back')).resolves.toBe(false)
await terminal.stopAgentCommand('call-other')
expect(session.kill).not.toHaveBeenCalled()
await terminal.stopAgentCommand('call-handed-back')
expect(session.kill).toHaveBeenCalledWith('SIGINT')
expect(session.runningToolCallId).toBeNull()
})
it('leaves a command the user started alone at sign-out', async () => {
const { terminal, session, processGroups } = cancellableService()
session.setBusy(true)
await terminal.stopAgentCommands()
expect(session.kill).not.toHaveBeenCalled()
expect(processGroups.signal).not.toHaveBeenCalled()
})
it('leaves the terminal alone for a call it is not running', async () => {
const { terminal, session, processGroups } = cancellableService()
const { running } = await startRun(terminal, session, 'call-other')
await expect(terminal.cancelTool('call-unknown')).resolves.toBe(false)
expect(session.kill).not.toHaveBeenCalled()
expect(processGroups.signal).not.toHaveBeenCalled()
session.finishRun(0)
await running
})
})
describe('agent commands in tmux', () => {
it('stops a tmux run at sign-out while its call still waits on it', async () => {
tmuxFake.on = true
try {
const terminal = new TerminalService({ loadCwd: () => '/tmp' })
terminal.start({ cols: 80, rows: 24 })
const running = terminal.executeTool('call-tmux', 'run', {
command: 'sleep 600',
waitSeconds: 60,
})
await vi.waitFor(() => expect(tmuxFake.statusPaths.size).toBe(1))
await terminal.stopAgentCommands()
expect(tmuxFake.stopped).toEqual(['%1'])
await expect(running).resolves.toMatchObject({
ok: true,
result: { status: 'completed', exitCode: 130 },
})
} finally {
tmuxFake.on = false
}
})
it('reports an untracked run it could not stop as still running, and keeps tracking it', async () => {
tmuxFake.on = true
tmuxFake.untracked = true
tmuxFake.statusPaths.clear()
try {
const terminal = new TerminalService({ loadCwd: () => '/tmp' })
terminal.start({ cols: 80, rows: 24 })
const running = terminal.executeTool('call-untracked', 'run', {
command: 'sleep 600',
waitSeconds: 60,
})
await vi.waitFor(() => expect(tmuxFake.statusPaths.size).toBe(1))
const [, statusPath = ''] = [...tmuxFake.statusPaths][0] ?? []
await terminal.cancelTool('call-untracked')
await expect(running).resolves.toMatchObject({ ok: true, result: { status: 'running' } })
expect(existsSync(join(statusPath, '..'))).toBe(true)
// Once it does finish, the next run's bookkeeping reaps it.
writeFileSync(statusPath, '0')
await terminal.executeTool('call-next', 'run', { command: 'ls', waitSeconds: 1 })
expect(existsSync(join(statusPath, '..'))).toBe(false)
} finally {
tmuxFake.on = false
tmuxFake.untracked = false
}
})
it('keeps a record of a tagged run exactly as long as the run goes on', async () => {
tmuxFake.on = true
tmuxFake.statusPaths.clear()
const scratch = mkdtempSync(join(tmpdir(), 'sim-ledger-'))
const ledgerDir = join(scratch, 'terminal-runs')
const ledger = createRunLedger(ledgerDir)
try {
const terminal = new TerminalService({ loadCwd: () => '/tmp', runLedger: ledger })
terminal.start({ cols: 80, rows: 24 })
await terminal.executeTool('call-long', 'run', { command: 'make build', waitSeconds: 1 })
const [[pane = '', statusPath = ''] = []] = [...tmuxFake.statusPaths]
// Still going after its call returned: a later process must be able to find it.
expect(createRunLedger(ledgerDir).list()).toEqual([
{
runId: `run-${pane.slice(1)}`,
pane,
socket: '/tmp/tmux-fake/default',
callId: 'call-long',
// Handed back, but the model has it only once Sim acknowledges the result.
state: 'started',
},
])
writeFileSync(statusPath, '0')
await terminal.executeTool('call-next', 'run', { command: 'ls', waitSeconds: 1 })
// The first finished and is forgotten; the one still going is recorded in its place.
expect(
createRunLedger(ledgerDir)
.list()
.map((run) => run.pane)
).toEqual([[...tmuxFake.statusPaths.keys()][1]])
} finally {
tmuxFake.on = false
rmSync(scratch, { recursive: true, force: true })
}
})
it('closes and then forgets a run that finished before its terminal closed', async () => {
tmuxFake.on = true
tmuxFake.statusPaths.clear()
tmuxFake.open.clear()
const scratch = mkdtempSync(join(tmpdir(), 'sim-ledger-'))
const ledgerDir = join(scratch, 'terminal-runs')
try {
const terminal = new TerminalService({
loadCwd: () => '/tmp',
runLedger: createRunLedger(ledgerDir),
})
const { activeTerminalId } = terminal.start({ cols: 80, rows: 24 })
await terminal.executeTool('call-long', 'run', { command: 'make build', waitSeconds: 1 })
const [[, statusPath = ''] = []] = [...tmuxFake.statusPaths]
writeFileSync(statusPath, '0')
const [pane = ''] = [...tmuxFake.statusPaths.keys()]
// Its dead pane is still open, as with `remain-on-exit`.
expect(tmuxFake.open.has(pane)).toBe(true)
terminal.closeTerminal(activeTerminalId as string)
await vi.waitFor(() => expect(createRunLedger(ledgerDir).list()).toEqual([]))
expect(tmuxFake.open.has(pane)).toBe(false)
} finally {
tmuxFake.on = false
rmSync(scratch, { recursive: true, force: true })
}
})
it("forgets a run that finishes within its call, and a closed tab's run once its pane is gone", async () => {
tmuxFake.on = true
tmuxFake.statusPaths.clear()
const scratch = mkdtempSync(join(tmpdir(), 'sim-ledger-'))
const ledgerDir = join(scratch, 'terminal-runs')
try {
const terminal = new TerminalService({
loadCwd: () => '/tmp',
runLedger: createRunLedger(ledgerDir),
})
const { activeTerminalId } = terminal.start({ cols: 80, rows: 24 })
// Finishes while its call still waits on it.
const quick = terminal.executeTool('call-quick', 'run', { command: 'ls', waitSeconds: 30 })
await vi.waitFor(() => expect(tmuxFake.statusPaths.size).toBe(1))
writeFileSync([...tmuxFake.statusPaths.values()][0] ?? '', '0')
await quick
expect(createRunLedger(ledgerDir).list()).toEqual([])
// Still going when its tab closes; its pane goes later, and the next run's bookkeeping sees.
await terminal.executeTool('call-long', 'run', { command: 'make build', waitSeconds: 1 })
const [, longPane = ''] = [...tmuxFake.statusPaths.keys()]
terminal.closeTerminal(activeTerminalId as string)
expect(
createRunLedger(ledgerDir)
.list()
.map((run) => run.pane)
).toEqual([longPane])
tmuxFake.gone.add(longPane)
await terminal.executeTool('call-new', 'new', {})
await terminal.executeTool('call-next', 'run', { command: 'pwd', waitSeconds: 1 })
expect(
createRunLedger(ledgerDir)
.list()
.map((run) => run.pane)
).not.toContain(longPane)
} finally {
tmuxFake.on = false
tmuxFake.gone.clear()
rmSync(scratch, { recursive: true, force: true })
}
})
it("closes a finished untracked run's pane when its terminal closes", async () => {
tmuxFake.on = true
tmuxFake.untracked = true
tmuxFake.statusPaths.clear()
tmuxFake.open.clear()
try {
const terminal = new TerminalService({ loadCwd: () => '/tmp' })
const { activeTerminalId } = terminal.start({ cols: 80, rows: 24 })
await terminal.executeTool('call-old-tmux', 'run', { command: 'make build', waitSeconds: 1 })
const [[pane = '', statusPath = ''] = []] = [...tmuxFake.statusPaths]
writeFileSync(statusPath, '0')
terminal.closeTerminal(activeTerminalId as string)
await vi.waitFor(() => expect(tmuxFake.open.has(pane)).toBe(false))
} finally {
tmuxFake.on = false
tmuxFake.untracked = false
}
})
it('keeps the record of a finished run whose pane tmux could not confirm closing', async () => {
tmuxFake.on = true
tmuxFake.statusPaths.clear()
const scratch = mkdtempSync(join(tmpdir(), 'sim-ledger-'))
const ledgerDir = join(scratch, 'terminal-runs')
try {
const terminal = new TerminalService({
loadCwd: () => '/tmp',
runLedger: createRunLedger(ledgerDir),
})
const { activeTerminalId } = terminal.start({ cols: 80, rows: 24 })
await terminal.executeTool('call-long', 'run', { command: 'make build', waitSeconds: 1 })
writeFileSync([...tmuxFake.statusPaths.values()][0] ?? '', '0')
tmuxFake.unanswered = true
terminal.closeTerminal(activeTerminalId as string)
await sleep(200)
// The next sweep will close its pane.
expect(createRunLedger(ledgerDir).list()).toHaveLength(1)
} finally {
tmuxFake.on = false
tmuxFake.unanswered = false
rmSync(scratch, { recursive: true, force: true })
}
})
it('never starts a tagged run it could not record', async () => {
tmuxFake.on = true
tmuxFake.statusPaths.clear()
const scratch = mkdtempSync(join(tmpdir(), 'sim-ledger-'))
// A file where the ledger's directory should be: no record can be saved.
writeFileSync(join(scratch, 'terminal-runs'), '')
try {
const terminal = new TerminalService({
loadCwd: () => '/tmp',
runLedger: createRunLedger(join(scratch, 'terminal-runs')),
})
terminal.start({ cols: 80, rows: 24 })
await expect(
terminal.executeTool('call-unrecorded', 'run', { command: 'make build', waitSeconds: 1 })
).resolves.toMatchObject({ ok: false, code: 'SPAWN_FAILED' })
expect(tmuxFake.statusPaths.size).toBe(0)
} finally {
tmuxFake.on = false
rmSync(scratch, { recursive: true, force: true })
}
})
it("closes a run's pane when a later run reaps it after it finished", async () => {
tmuxFake.on = true
tmuxFake.statusPaths.clear()
tmuxFake.open.clear()
try {
const terminal = new TerminalService({ loadCwd: () => '/tmp' })
terminal.start({ cols: 80, rows: 24 })
await terminal.executeTool('call-long', 'run', { command: 'make build', waitSeconds: 1 })
const [[pane = '', statusPath = ''] = []] = [...tmuxFake.statusPaths]
// It finishes after its call returned, and its dead pane stays open.
writeFileSync(statusPath, '0')
expect(tmuxFake.open.has(pane)).toBe(true)
await terminal.executeTool('call-next', 'run', { command: 'ls', waitSeconds: 1 })
expect(tmuxFake.open.has(pane)).toBe(false)
} finally {
tmuxFake.on = false
}
})
it("keeps a run's output readable for its call when the terminal goes away mid-wait", async () => {
tmuxFake.on = true
tmuxFake.statusPaths.clear()
try {
const terminal = new TerminalService({ loadCwd: () => '/tmp' })
terminal.start({ cols: 80, rows: 24 })
const running = terminal.executeTool('call-tmux-closed', 'run', {
command: 'make build',
waitSeconds: 1,
})
await vi.waitFor(() => expect(tmuxFake.statusPaths.size).toBe(1))
terminal.dispose()
await expect(running).resolves.toMatchObject({
ok: true,
result: { status: 'running', output: 'partial output' },
})
} finally {
tmuxFake.on = false
}
})
it('releases a run whose pane is gone instead of ever stopping it', async () => {
tmuxFake.on = true
tmuxFake.statusPaths.clear()
tmuxFake.stopped.length = 0
try {
const terminal = new TerminalService({ loadCwd: () => '/tmp' })
terminal.start({ cols: 80, rows: 24 })
const first = await terminal.executeTool('call-first', 'run', {
command: 'sleep 600',
waitSeconds: 1,
})
expect(first).toMatchObject({ ok: true, result: { status: 'running' } })
const [pane, statusPath] = [...tmuxFake.statusPaths][0] ?? []
tmuxFake.gone.add(pane ?? '')
await terminal.stopAgentCommands()
expect(tmuxFake.stopped).toEqual([])
// The next run's bookkeeping drops it for good.
void terminal.executeTool('call-next', 'run', { command: 'ls', waitSeconds: 1 })
await vi.waitFor(() => expect(existsSync(statusPath ?? '')).toBe(false))
await vi.waitFor(() => expect(tmuxFake.statusPaths.size).toBe(2))
expect(existsSync(join(statusPath ?? '', '..'))).toBe(false)
} finally {
tmuxFake.on = false
tmuxFake.gone.clear()
}
})
it('still stops at sign-out a run whose Sim terminal was closed', async () => {
tmuxFake.on = true
tmuxFake.statusPaths.clear()
tmuxFake.stopped.length = 0
try {
const terminal = new TerminalService({ loadCwd: () => '/tmp' })
const { activeTerminalId } = terminal.start({ cols: 80, rows: 24 })
const first = await terminal.executeTool('call-orphan', 'run', {
command: 'sleep 600',
waitSeconds: 1,
})
expect(first).toMatchObject({ ok: true, result: { status: 'running' } })
terminal.closeTerminal(activeTerminalId as string)
await terminal.stopAgentCommands()
expect(tmuxFake.stopped).toEqual([[...tmuxFake.statusPaths.keys()][0]])
} finally {
tmuxFake.on = false
}
})
})
apps/desktop/src/main/terminal/session.test.ts
+60
View File
@@ -137,6 +137,9 @@ describe('TerminalSession command lifecycle', () => {
const resultPromise = session.runCommand('vim', 'tool-call-1', 10_000)
ptyStub.dataHandler?.('\u001b]633;C;test-nonce\u0007\u001b[?1049h')
expect(await resultPromise).toMatchObject({ status: 'interactive' })
// Detached from the tool call, the command is still the agent's until it exits.
expect(session.runningToolCallId).toBeNull()
expect(session.agentCommandToolCallId).toBe('tool-call-1')
expect(commandEvents.at(-1)).toMatchObject({
terminalId: 'terminal-1',
@@ -153,6 +156,7 @@ describe('TerminalSession command lifecycle', () => {
command: 'vim',
exitCode: 0,
})
expect(session.agentCommandToolCallId).toBeNull()
expect(commandEvents.at(-1)?.toolCallId).toBeUndefined()
} finally {
session.dispose()
@@ -160,4 +164,60 @@ describe('TerminalSession command lifecycle', () => {
else process.env.SHELL = originalShell
}
})
it('stops pressing a batch of keys once the call is stopped', async () => {
vi.useFakeTimers()
const session = TerminalSession.create({
terminalId: 'terminal-keys',
cwd: '/tmp',
cols: 80,
rows: 24,
callbacks: { onData: () => {}, onState: () => {}, onCommand: () => {}, onExit: () => {} },
})
try {
const writesBefore = ptyStub.writes.length
const stop = new AbortController()
const pressing = session.pressKeys(['down', 'down', 'down', 'enter'], stop.signal)
expect(ptyStub.writes.length - writesBefore).toBe(1)
stop.abort()
await vi.runAllTimersAsync()
await pressing
expect(ptyStub.writes.length - writesBefore).toBe(1)
} finally {
session.dispose()
}
})
it('ends the wait between keystrokes on Stop while the program keeps redrawing', async () => {
vi.useFakeTimers()
const session = TerminalSession.create({
terminalId: 'terminal-redraw',
cwd: '/tmp',
cols: 80,
rows: 24,
callbacks: { onData: () => {}, onState: () => {}, onCommand: () => {}, onExit: () => {} },
})
const redraw = setInterval(() => ptyStub.dataHandler?.('frame'), 20)
try {
const writesBefore = ptyStub.writes.length
const stop = new AbortController()
let settled = false
const typing = session.type('first\nsecond', stop.signal).then(() => {
settled = true
})
await vi.advanceTimersByTimeAsync(300)
expect(settled).toBe(false)
stop.abort()
await vi.advanceTimersByTimeAsync(1)
expect(settled).toBe(true)
await typing
expect(ptyStub.writes.slice(writesBefore)).toEqual(['first'])
} finally {
clearInterval(redraw)
session.dispose()
}
})
})
apps/desktop/src/main/terminal/session.ts
+120 -36
View File
@@ -23,7 +23,7 @@ import {
type TerminalRunResult,
type TerminalTabState,
} from '@sim/terminal-protocol'
import { sleep } from '@sim/utils/helpers'
import { interruptibleSleep } from '@sim/utils/helpers'
import {
Terminal as HeadlessTerminal,
type IBuffer,
@@ -261,6 +261,21 @@ interface PendingCommand {
resolve(result: TerminalRunResult): void
}
/** How long a new shell gets to reach its first prompt. */
export interface ShellStartupBounds {
/** From spawn, to begin the startup files we generated; a shell that has not by then never will. */
unstartedMs: number
/** From the moment they began, for those startup files to reach a prompt. */
startingMs: number
}
/**
* How a wait for the shell's first prompt ended: `ready` with integration live; `not-instrumented`
* when our startup files never ran; `starting` when they are still running at the bound, often
* because something in them is waiting for input; `exited`; or `stopped` by the caller.
*/
export type ShellReadiness = 'ready' | 'not-instrumented' | 'starting' | 'exited' | 'stopped'
export interface TerminalSessionCallbacks {
onData(terminalId: string, data: string): void
onState(): void
@@ -313,13 +328,23 @@ export class TerminalSession {
private columns: number
private lines: number
private shellIntegration = false
/** When the shell began the startup files we generated, or null while it has not. */
private startupBegunAt: number | null = null
private readonly spawnedAt = Date.now()
private altScreen = false
private foregroundCommand: string | null = null
private foregroundToolCallId: string | null = null
/**
* The agent tool call whose command is still running, until that command really ends. Unlike
* {@link runningToolCallId} it survives an interactive command detaching from the tool call:
* the command is still the agent's, so Stop and sign-out can still end it.
*/
private agentToolCallId: string | null = null
private pendingCommand: PendingCommand | null = null
/** Command line reported by the shell but not yet bracketed by output-start. */
private announcedCommand: string | null = null
private integrationWaiters: Array<() => void> = []
/** Pending shell-readiness waits, rechecked whenever the shell's startup moves on. */
private readonly readinessWaiters = new Set<() => void>()
private constructor(
options: TerminalSessionOptions,
@@ -327,7 +352,8 @@ export class TerminalSession {
integrationDir: string,
nonce: string,
shellName: string,
shellEnv: NodeJS.ProcessEnv
shellEnv: NodeJS.ProcessEnv,
private readonly instrumented: boolean
) {
this.callbacks = options.callbacks
this.terminalId = options.terminalId
@@ -372,7 +398,15 @@ export class TerminalSession {
})
logger.info('Started terminal session', { shell: shellPath, instrumented: shell !== null })
return new TerminalSession(options, pty, integrationDir, nonce, shell ?? shellPath, shellEnv)
return new TerminalSession(
options,
pty,
integrationDir,
nonce,
shell ?? shellPath,
shellEnv,
shell !== null
)
}
/**
@@ -434,6 +468,16 @@ export class TerminalSession {
return this.foregroundCommand
}
/** The agent tool call whose command holds the foreground, if one does. */
get runningToolCallId(): string | null {
return this.foregroundToolCallId
}
/** The agent tool call whose command is still running in this shell, if one is. */
get agentCommandToolCallId(): string | null {
return this.agentToolCallId
}
/**
* Tab-strip view of this terminal. The label prefers the running command,
* which is what the user is actually waiting on, and falls back to the
@@ -470,27 +514,57 @@ export class TerminalSession {
}
/**
* Resolves once the shell has emitted its first integration marker, or when
* `timeoutMs` elapses. A shell takes a few hundred milliseconds to run its
* startup files, so a command issued immediately after spawn would otherwise
* be refused for having no integration when it is merely early.
* Resolves once the shell reaches its first prompt with integration live, or once it is clear
* it will not: it never began our startup files within `bounds.unstartedMs` of spawning, it is
* still running them `bounds.startingMs` after they began, it exited, or `signal` stopped the
* wait. A shell's startup files take a while, and longer on a busy machine, so a command issued
* soon after spawn would otherwise be refused when the shell is merely early.
*/
waitForShellIntegration(timeoutMs: number): Promise<boolean> {
if (this.shellIntegration) return Promise.resolve(true)
if (this.disposed) return Promise.resolve(false)
waitForShellIntegration(
bounds: ShellStartupBounds,
signal?: AbortSignal
): Promise<ShellReadiness> {
return new Promise((resolve) => {
const notify = () => {
clearTimeout(timer)
resolve(this.shellIntegration)
let timer: NodeJS.Timeout | null = null
const check = () => {
if (timer) clearTimeout(timer)
timer = null
const readiness = signal?.aborted ? 'stopped' : this.readiness(bounds)
if (readiness === null) {
const deadline =
this.startupBegunAt === null
? this.spawnedAt + bounds.unstartedMs
: this.startupBegunAt + bounds.startingMs
timer = setTimeout(check, Math.max(0, deadline - Date.now()))
return
}
this.readinessWaiters.delete(check)
signal?.removeEventListener('abort', check)
resolve(readiness)
}
const timer = setTimeout(() => {
this.integrationWaiters = this.integrationWaiters.filter((entry) => entry !== notify)
resolve(this.shellIntegration)
}, timeoutMs)
this.integrationWaiters.push(notify)
this.readinessWaiters.add(check)
signal?.addEventListener('abort', check)
check()
})
}
/** Where the shell's startup stands against `bounds`, or null while it may still get there. */
private readiness(bounds: ShellStartupBounds): ShellReadiness | null {
if (this.shellIntegration) return 'ready'
if (this.disposed) return 'exited'
const now = Date.now()
if (this.startupBegunAt === null) {
return !this.instrumented || now - this.spawnedAt >= bounds.unstartedMs
? 'not-instrumented'
: null
}
return now - this.startupBegunAt >= bounds.startingMs ? 'starting' : null
}
private notifyReadinessWaiters(): void {
for (const check of [...this.readinessWaiters]) check()
}
write(data: string): void {
if (this.disposed) return
this.pty.write(data)
@@ -508,10 +582,11 @@ export class TerminalSession {
* The pause also lets a menu redraw between presses, which is what makes a
* batch land on the row a person pressing the same keys would reach.
*/
async pressKeys(keys: TerminalControlKey[]): Promise<void> {
async pressKeys(keys: TerminalControlKey[], signal?: AbortSignal): Promise<void> {
for (let index = 0; index < keys.length; index += 1) {
if (this.disposed) return
if (index > 0) await this.settleBetweenKeystrokes()
if (this.disposed || signal?.aborted) return
if (index > 0) await this.settleBetweenKeystrokes(signal)
if (signal?.aborted) return
this.sendKey(keys[index])
}
}
@@ -522,24 +597,28 @@ export class TerminalSession {
* gets a chance to redraw between them. See {@link toInputChunks} for why
* sending it all at once leaves the text unsubmitted.
*/
async type(text: string): Promise<void> {
async type(text: string, signal?: AbortSignal): Promise<void> {
const chunks = toInputChunks(text)
for (let index = 0; index < chunks.length; index += 1) {
if (this.disposed) return
if (index > 0) await this.settleBetweenKeystrokes()
if (this.disposed || signal?.aborted) return
if (index > 0) await this.settleBetweenKeystrokes(signal)
if (signal?.aborted) return
this.write(chunks[index])
}
}
/** Holds a gap, then lets any resulting redraw finish before the next write. */
private async settleBetweenKeystrokes(): Promise<void> {
await sleep(KEYSTROKE_GAP_MS)
/**
* Holds a gap, then lets any resulting redraw finish before the next write.
* Ends early on Stop, so a program that redraws constantly cannot hold it.
*/
private async settleBetweenKeystrokes(signal?: AbortSignal): Promise<void> {
await interruptibleSleep(KEYSTROKE_GAP_MS, signal)
const deadline = Date.now() + KEYSTROKE_SETTLE_MAX_MS
while (!this.disposed) {
while (!this.disposed && !signal?.aborted) {
const quietFor = Date.now() - this.lastOutputAt
const remaining = Math.min(KEYSTROKE_GAP_MS - quietFor, deadline - Date.now())
if (remaining <= 0) return
await sleep(remaining)
await interruptibleSleep(remaining, signal)
}
}
@@ -597,6 +676,7 @@ export class TerminalSession {
}
this.foregroundCommand = command
this.foregroundToolCallId = toolCallId
this.agentToolCallId = toolCallId
this.emitState()
this.callbacks.onCommand({ terminalId: this.terminalId, phase: 'start', command, toolCallId })
@@ -717,6 +797,7 @@ export class TerminalSession {
if (this.flushTimer) clearTimeout(this.flushTimer)
this.flushTimer = null
this.finishCommand(null)
this.notifyReadinessWaiters()
try {
this.pty.kill()
} catch {
@@ -772,13 +853,17 @@ export class TerminalSession {
marker: ReturnType<ShellIntegrationParser['parse']>['markers'][number]
): void {
switch (marker.kind) {
case 'startup':
if (this.startupBegunAt === null) {
this.startupBegunAt = Date.now()
this.notifyReadinessWaiters()
}
break
case 'prompt-start':
if (!this.shellIntegration) {
this.shellIntegration = true
this.emitState()
const waiters = this.integrationWaiters
this.integrationWaiters = []
for (const notify of waiters) notify()
this.notifyReadinessWaiters()
}
break
case 'command-line':
@@ -976,6 +1061,7 @@ export class TerminalSession {
this.foregroundCommand = null
this.foregroundToolCallId = null
this.agentToolCallId = null
this.announcedCommand = null
this.altScreen = false
this.emitState()
@@ -1007,9 +1093,7 @@ export class TerminalSession {
private handleExit(): void {
if (this.disposed) return
this.disposed = true
const waiters = this.integrationWaiters
this.integrationWaiters = []
for (const notify of waiters) notify()
this.notifyReadinessWaiters()
if (this.flushTimer) clearTimeout(this.flushTimer)
this.flushTimer = null
this.flush()
apps/desktop/src/main/terminal/shell-integration.test.ts
+93 -1
View File
@@ -1,5 +1,13 @@
import { spawnSync } from 'node:child_process'
import { existsSync, mkdirSync, mkdtempSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { describe, expect, it } from 'vitest'
import { detectShell, ShellIntegrationParser } from '@/main/terminal/shell-integration'
import {
buildShellLaunch,
detectShell,
ShellIntegrationParser,
} from '@/main/terminal/shell-integration'
const NONCE = 'testnonce'
@@ -65,6 +73,14 @@ describe('ShellIntegrationParser', () => {
expect(markers).toEqual([{ kind: 'cwd', cwd: '/tmp/some dir' }])
})
it("recognises the startup marker sent before the user's startup files", () => {
const parser = new ShellIntegrationParser(NONCE)
const { text, markers } = parser.parse(`${osc(`SimStartup;${NONCE}`)}loading`)
expect(text).toBe('loading')
expect(markers).toEqual([{ kind: 'startup' }])
})
it('accepts ST as well as BEL as a terminator', () => {
const parser = new ShellIntegrationParser(NONCE)
const { text, markers } = parser.parse(`a\u001b]633;A;${NONCE}\u001b\\b`)
@@ -93,3 +109,79 @@ describe('detectShell', () => {
expect(detectShell('/bin/sh')).toBeNull()
})
})
/**
* The generated startup files run in a real shell, with the user's own files standing in as a
* temp home whose startup prints a line. The startup marker has to reach the terminal before that
* line, and only from the interactive shell the terminal runs.
*/
describe('startup marker', () => {
const STARTUP = `\u001b]633;SimStartup;${NONCE}\u0007`
function userHome(rcFile: string): string {
const home = mkdtempSync(join(tmpdir(), 'sim-shell-home-'))
writeFileSync(join(home, rcFile), 'echo user-startup\n')
return home
}
it.skipIf(!existsSync('/bin/zsh'))(
"zsh sends it before the user's files, interactively only",
() => {
const home = userHome('.zshrc')
const env = { PATH: process.env.PATH ?? '/usr/bin:/bin', HOME: home }
const launch = buildShellLaunch('zsh', mkdtempSync(join(tmpdir(), 'sim-zsh-')), NONCE, env)
const run = (args: string[]) =>
spawnSync('/bin/zsh', args, { env: { ...env, ...launch.env }, encoding: 'utf8' }).stdout
const interactive = run([...launch.args, '-i', '-c', 'true'])
expect(interactive.indexOf(STARTUP)).toBeGreaterThanOrEqual(0)
expect(interactive.indexOf(STARTUP)).toBeLessThan(interactive.indexOf('user-startup'))
expect(run(['-c', 'echo script'])).toBe('script\n')
}
)
it.skipIf(!existsSync('/bin/zsh'))(
"zsh keeps integrating when the user's .zshenv moves ZDOTDIR, and runs all their files there",
() => {
const home = mkdtempSync(join(tmpdir(), 'sim-shell-home-'))
const userDir = join(home, '.config', 'zsh')
mkdirSync(userDir, { recursive: true })
writeFileSync(join(home, '.zshenv'), 'export ZDOTDIR="$HOME/.config/zsh"\n')
writeFileSync(join(userDir, '.zprofile'), 'USER_PROFILE_RAN=1\n')
writeFileSync(join(userDir, 'plugins.zsh'), 'USER_PLUGIN_RAN=1\n')
writeFileSync(join(userDir, '.zshrc'), 'USER_RC_RAN=1\nsource "$ZDOTDIR/plugins.zsh"\n')
writeFileSync(join(userDir, '.zlogin'), 'echo "login=$USER_RC_RAN"\n')
const env = { PATH: process.env.PATH ?? '/usr/bin:/bin', HOME: home }
const launch = buildShellLaunch('zsh', mkdtempSync(join(tmpdir(), 'sim-zsh-')), NONCE, env)
const output = spawnSync(
'/bin/zsh',
[
...launch.args,
'-i',
'-c',
'echo "profile=$USER_PROFILE_RAN rc=$USER_RC_RAN plugin=$USER_PLUGIN_RAN zdotdir=$ZDOTDIR"; whence -w __sim_precmd',
],
{ env: { ...env, ...launch.env }, encoding: 'utf8' }
).stdout
expect(output).toContain(`profile=1 rc=1 plugin=1 zdotdir=${userDir}`)
expect(output).toContain('__sim_precmd: function')
expect(output).toContain('login=1')
}
)
it.skipIf(!existsSync('/bin/bash'))("bash sends it before the user's files", () => {
const home = userHome('.bashrc')
const env = { PATH: process.env.PATH ?? '/usr/bin:/bin', HOME: home }
const launch = buildShellLaunch('bash', mkdtempSync(join(tmpdir(), 'sim-bash-')), NONCE, env)
const output = spawnSync('/bin/bash', launch.args, {
env: { ...env, ...launch.env },
input: 'exit\n',
encoding: 'utf8',
}).stdout
expect(output.indexOf(STARTUP)).toBeGreaterThanOrEqual(0)
expect(output.indexOf(STARTUP)).toBeLessThan(output.indexOf('user-startup'))
})
})
apps/desktop/src/main/terminal/shell-integration.ts
+26 -5
View File
@@ -36,9 +36,13 @@ export function createNonce(): string {
/**
* Marker kinds we act on. `A` (prompt start) doubles as the "integration is
* live" signal; `C`/`D` bracket a command's output; `E` reports the exact
* command line; `P` tracks the working directory across `cd`.
* command line; `P` tracks the working directory across `cd`. `SimStartup` is
* ours, not VS Code's: the shell sends it before running the user's startup
* files, so a shell that is still starting can be told from one we never
* instrumented.
*/
export type ShellMarker =
| { kind: 'startup' }
| { kind: 'prompt-start' }
| { kind: 'command-line'; command: string }
| { kind: 'output-start' }
@@ -123,6 +127,8 @@ export class ShellIntegrationParser {
if (nonce !== this.nonce) return null
switch (kind) {
case 'SimStartup':
return { kind: 'startup' }
case 'A':
return { kind: 'prompt-start' }
case 'C':
@@ -160,12 +166,26 @@ function findTerminator(buffer: string, from: number): { index: number; length:
* prompt, aliases, and PATH win over ours.
*/
function writeZshFiles(dir: string, nonce: string, originalZdotdir: string): void {
const sourceOriginal = (file: string) =>
`[ -f "$SIM_ZDOTDIR_ORIG/${file}" ] && builtin source "$SIM_ZDOTDIR_ORIG/${file}"`
// The user's file runs with ZDOTDIR at their own directory, so it finds its siblings and
// plugins there. A file that moves ZDOTDIR (an XDG `.zshenv`) moves where the rest of theirs are
// read from; ZDOTDIR then points back here, so zsh reads our next file.
const sourceOriginal = (file: string) => `if [ -f "$SIM_ZDOTDIR_ORIG/${file}" ]; then
__sim_zdotdir="$ZDOTDIR"
ZDOTDIR="$SIM_ZDOTDIR_ORIG"
builtin source "$SIM_ZDOTDIR_ORIG/${file}"
SIM_ZDOTDIR_ORIG="\${ZDOTDIR:-$HOME}"
ZDOTDIR="$__sim_zdotdir"
builtin unset __sim_zdotdir
fi`
// `.zshenv` is the first file zsh reads, so the startup marker goes out before any of the user's
// files run. Only from an interactive shell: a script's output must not carry it.
writeFileSync(
join(dir, '.zshenv'),
`SIM_ZDOTDIR_ORIG="\${SIM_ZDOTDIR_ORIG:-${originalZdotdir}}"\n${sourceOriginal('.zshenv')}\n`
`[[ -o interactive ]] && builtin printf '\\e]633;SimStartup;%s\\a' '${nonce}'
SIM_ZDOTDIR_ORIG="\${SIM_ZDOTDIR_ORIG:-${originalZdotdir}}"
${sourceOriginal('.zshenv')}
`
)
writeFileSync(join(dir, '.zprofile'), `${sourceOriginal('.zprofile')}\n`)
writeFileSync(join(dir, '.zlogin'), `${sourceOriginal('.zlogin')}\n`)
@@ -225,7 +245,8 @@ function writeBashFile(dir: string, nonce: string): string {
const rcPath = join(dir, 'sim-bash-rc.sh')
writeFileSync(
rcPath,
`[ -f "$HOME/.bashrc" ] && builtin source "$HOME/.bashrc"
`builtin printf '\\e]633;SimStartup;%s\\a' '${nonce}'
[ -f "$HOME/.bashrc" ] && builtin source "$HOME/.bashrc"
__sim_nonce='${nonce}'
__sim_in_cmd=''
apps/desktop/src/main/terminal/shell-startup.test.ts
+173
View File
@@ -0,0 +1,173 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { TerminalService } from '@/main/terminal'
/**
* A new shell's first prompt, as the service waits for it before running the agent's command.
* node-pty is stubbed so each test plays the shell's side: the startup marker our generated files
* send before the user's run, whatever the user's files print, and the first prompt. The service
* and the session are real.
*/
const pty = vi.hoisted(() => ({
emit: null as ((data: string) => void) | null,
exit: null as (() => void) | null,
writes: [] as string[],
}))
vi.mock('@lydell/node-pty', () => ({
spawn: () => ({
pid: 4321,
onData: (handler: (data: string) => void) => {
pty.emit = handler
},
onExit: (handler: () => void) => {
pty.exit = handler
},
write: (data: string) => pty.writes.push(data),
resize: vi.fn(),
kill: vi.fn(),
pause: vi.fn(),
resume: vi.fn(),
}),
}))
vi.mock('@/main/terminal/shell-integration', async (importOriginal) => {
const actual = await importOriginal<typeof import('@/main/terminal/shell-integration')>()
return { ...actual, createNonce: () => 'nonce' }
})
vi.mock('@/main/terminal/tmux', async (importOriginal) => {
const actual = await importOriginal<typeof import('@/main/terminal/tmux')>()
return { ...actual, isTmuxUnavailable: () => true }
})
const marker = (body: string) => `\u001b]633;${body};nonce\u0007`
const STARTUP = marker('SimStartup')
const PROMPT = marker('A')
function shell(data: string): void {
pty.emit?.(data)
}
/** Plays a command the shell was asked to run through to its prompt, once it was typed. */
async function answerCommand(output: string): Promise<void> {
await vi.waitFor(() => expect(pty.writes.some((write) => write.endsWith('\r'))).toBe(true))
shell(`${marker('C')}${output}\r\n${marker('D;0')}${PROMPT}`)
}
/** Lets timers run until `promise` settles: the screen is read through an emulator that parses on them. */
async function settle<T>(promise: Promise<T>): Promise<T> {
let done = false
void promise.finally(() => {
done = true
})
for (let step = 0; step < 1_000 && !done; step++) await vi.advanceTimersByTimeAsync(10)
return promise
}
beforeEach(() => {
vi.stubEnv('SHELL', '/bin/zsh')
vi.useFakeTimers({
toFake: ['setTimeout', 'clearTimeout', 'setInterval', 'clearInterval', 'Date'],
})
})
afterEach(() => {
vi.useRealTimers()
pty.emit = null
pty.exit = null
pty.writes.length = 0
vi.unstubAllEnvs()
})
describe('a shell that is still starting', () => {
it('runs the command once slow startup files reach a prompt', async () => {
// A heavy .zshrc on a busy machine: our files began at once, the user's took 20 s.
const terminal = new TerminalService({ loadCwd: () => '/tmp' })
const running = terminal.executeTool('call-slow', 'run', {
command: 'echo hi',
waitSeconds: 30,
})
shell(STARTUP)
await vi.advanceTimersByTimeAsync(20_000)
expect(pty.writes).toEqual([])
shell(PROMPT)
await answerCommand('hi')
const response = await running
expect(response).toMatchObject({ ok: true, result: { exitCode: 0 } })
terminal.dispose()
})
it('gives startup files their full bound from when they began, however late that was', async () => {
// A machine so busy our files began 7 s after spawn, and the user's then took 25 s more.
const terminal = new TerminalService({ loadCwd: () => '/tmp' })
const running = terminal.executeTool('call-late', 'run', { command: 'echo hi' })
await vi.advanceTimersByTimeAsync(7_000)
shell(STARTUP)
await vi.advanceTimersByTimeAsync(25_000)
expect(pty.writes).toEqual([])
shell(PROMPT)
await answerCommand('hi')
const response = await running
expect(response).toMatchObject({ ok: true, result: { exitCode: 0 } })
terminal.dispose()
})
it('refuses with the screen when startup files stall on a question', async () => {
const terminal = new TerminalService({ loadCwd: () => '/tmp' })
const running = terminal.executeTool('call-stalled', 'run', { command: 'echo hi' })
shell(`${STARTUP}[oh-my-zsh] Would you like to update? [Y/n] `)
await vi.advanceTimersByTimeAsync(30_000)
const response = await settle(running)
expect(response).toMatchObject({ ok: false, code: 'NO_SHELL_INTEGRATION' })
expect(response.error).toContain('[oh-my-zsh] Would you like to update? [Y/n]')
expect(response.error).toContain('ask the user to answer it')
expect(response.error).toContain('a startup file replaced the shell')
expect(pty.writes).toEqual([])
terminal.dispose()
})
it('reports the session closed when the shell exits while its startup files run', async () => {
const terminal = new TerminalService({ loadCwd: () => '/tmp' })
const running = terminal.executeTool('call-exit', 'run', { command: 'echo hi' })
shell(STARTUP)
await vi.advanceTimersByTimeAsync(2_000)
pty.exit?.()
const response = await running
expect(response).toMatchObject({ ok: false, code: 'SESSION_CLOSED' })
expect(pty.writes).toEqual([])
terminal.dispose()
})
it('refuses a shell that never began our startup files at the short bound', async () => {
const terminal = new TerminalService({ loadCwd: () => '/tmp' })
const running = terminal.executeTool('call-plain', 'run', { command: 'echo hi' })
await vi.advanceTimersByTimeAsync(8_000)
const response = await running
expect(response).toMatchObject({ ok: false, code: 'NO_SHELL_INTEGRATION' })
expect(response.error).toContain('did not load Sim shell integration')
terminal.dispose()
})
it('ends the wait as soon as the call is stopped, without running anything', async () => {
const terminal = new TerminalService({ loadCwd: () => '/tmp' })
const running = terminal.executeTool('call-stop', 'run', { command: 'echo hi' })
shell(STARTUP)
await vi.advanceTimersByTimeAsync(1_000)
expect(await terminal.cancelTool('call-stop')).toBe(true)
const response = await running
expect(response).toMatchObject({ ok: false, code: 'CANCELLED' })
shell(PROMPT)
await vi.advanceTimersByTimeAsync(1_000)
expect(pty.writes).toEqual([])
terminal.dispose()
})
})
apps/desktop/src/main/terminal/tmux.test.ts
+792 -4
View File
@@ -1,21 +1,112 @@
import { mkdtempSync, rmSync, writeFileSync } from 'node:fs'
import { chmodSync, existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { sleep } from '@sim/utils/helpers'
import { afterEach, describe, expect, it } from 'vitest'
import {
awaitRun,
closeRunPane,
isDescendantOf,
listPanes,
parseFormatLines,
pollRun,
recordedRunState,
resolveAttachment,
runPaneState,
startRun,
stopRecordedRun,
stopRun,
type TmuxRunHandle,
} from '@/main/terminal/tmux'
/** The separator the format strings use; no tmux field can contain it. */
const F = '\u001f'
/** What real tmux 2.9a writes for `set-option -p`, captured from the binary. */
const TMUX_29_NO_PANE_OPTIONS =
'tmux: unknown option -- p\nusage: set-option [-aFgosquw] [-t target-window] option [value]\n'
/** The same refusal from a tmux built against BSD getopt, as on macOS. */
const TMUX_29_BSD_NO_PANE_OPTIONS =
'tmux: illegal option -- p\nusage: set-option [-aFgosquw] [-t target-window] option [value]\n'
/** The separator the format strings use. */
const F = '<~sim~>'
/** A frame as one call makes it; the real one is random per call. */
const FRAME = '<~0123456789abcdef~>'
/** One record as tmux prints it for a framed format. */
const framed = (record: string) => `${FRAME}${record}${FRAME}`
/**
* Real `list-panes -F` output, framed with {@link FRAME}, for a pane whose working directory is
* `…/a` + newline + `user:0.0<~sim~>forged<~sim~>rm -rf<~sim~>home`: its own record breaks in two,
* and the second half reads as a whole pane of its own. Captured verbatim from each binary.
*/
const FORGED_ROW = {
'tmux 2.9a':
'<~0123456789abcdef~>user:0.0<~sim~>sleep<~sim~>sleep<~sim~>/tmp/tmp.oI9xEVhWLA/a\nuser:0.0<~sim~>forged<~sim~>rm -rf<~sim~>home<~sim~>1<~0123456789abcdef~>\n',
'tmux 3.4':
'<~0123456789abcdef~>user:0.0<~sim~>bash<~sim~>sleep<~sim~>/tmp/tmp.2V5Ie57hMp/a\nuser:0.0<~sim~>forged<~sim~>rm -rf<~sim~>home<~sim~>1<~0123456789abcdef~>\n',
}
/**
* The same pane, captured verbatim from each binary with newlines neutralised in the fields others
* can set (`#{s/<newline>/<NL>/:…}`): one line, holding the would-be forged row as plain text.
*/
const NEUTRALISED_ROW = {
'tmux 2.9a':
'<~0123456789abcdef~>user:0.0<~sim~>sleep<~sim~>sleep<~sim~>/tmp/tmp.CXoBFxxZAf/a<NL>user:0.0<~sim~>forged<~sim~>rm -rf<~sim~>home<~sim~>1<~0123456789abcdef~>\n',
'tmux 3.4':
'<~0123456789abcdef~>user:0.0<~sim~>sleep<~sim~>sleep<~sim~>/tmp/tmp.Wz6L8cjmce/a<NL>user:0.0<~sim~>forged<~sim~>rm -rf<~sim~>home<~sim~>1<~0123456789abcdef~>\n',
}
describe('parseFormatLines', () => {
it('drops lines with the wrong field count rather than mis-assigning them', () => {
expect(parseFormatLines(`a${F}b\nonly-one\n`, 2)).toEqual([['a', 'b']])
expect(parseFormatLines(`${framed(`a${F}b`)}\n${framed('only-one')}\n`, 2, FRAME)).toEqual([
['a', 'b'],
])
})
it('reads a field that ends with part of the separator as it is', () => {
// A cwd or window name may end with any text, including all but the separator's last character.
const partial = F.slice(0, -1)
expect(parseFormatLines(`${framed(`/tmp/x/p${partial}${F}1`)}\n`, 2, FRAME)).toEqual([
[`/tmp/x/p${partial}`, '1'],
])
expect(parseFormatLines(`${framed(`tail${partial}${F}%3${F}zsh`)}\n`, 3, FRAME)).toEqual([
[`tail${partial}`, '%3', 'zsh'],
])
})
it('drops a line whose field holds the whole separator rather than misread it', () => {
expect(parseFormatLines(`${framed(`a${F}b${F}c`)}\n`, 2, FRAME)).toEqual([])
})
it.each(Object.entries(FORGED_ROW))(
'never reads a row a directory name forges with a newline (%s)',
(_version, stdout) => {
expect(parseFormatLines(stdout, 5, FRAME)).toEqual([])
}
)
it('drops a line framed at one end only, whatever its field count', () => {
const fields = `user:0.0${F}x${F}y${F}z${F}1`
// As long as a frame, so a check of one end alone would cut it off and find five fields.
const pad = 'J'.repeat(FRAME.length)
expect(parseFormatLines(`${pad}${fields}${FRAME}\n`, 5, FRAME)).toEqual([])
expect(parseFormatLines(`${FRAME}${fields}${pad}\n`, 5, FRAME)).toEqual([])
})
it.each(Object.entries(NEUTRALISED_ROW))(
'reads the forging pane as one line once newlines are neutralised (%s)',
(_version, stdout) => {
// One line: no forged row. The separator text in its path then drops it whole.
expect(stdout.trimEnd().split('\n')).toHaveLength(1)
expect(parseFormatLines(stdout, 5, FRAME)).toEqual([])
}
)
it('reads only records framed by this call', () => {
const other = '<~fedcba9876543210~>'
expect(parseFormatLines(`${other}a${F}b${other}\n${framed(`c${F}d`)}\n`, 2, FRAME)).toEqual([
['c', 'd'],
])
})
})
@@ -43,6 +134,9 @@ describe('run status files', () => {
const handleIn = (dir: string): TmuxRunHandle => ({
window: '@1',
pane: '%1',
runId: 'run-1',
socket: null,
outPath: join(dir, 'out'),
statusPath: join(dir, 'status'),
dispose: () => {},
@@ -84,3 +178,697 @@ describe('run status files', () => {
expect(outcome).toEqual({ output: 'still going\n', exitCode: null, done: false })
})
})
/**
* A stand-in `tmux` binary on PATH that keeps its panes in a JSON file, so the run helpers are
* exercised through the real `spawn` path. It knows the handful of commands runs use; a test
* reshapes the server directly (a split, a closed window, a restart) by rewriting that file.
*/
interface FakeTmuxState {
/** The server's socket; `-S` naming any other reaches no server. */
socket: string
nextWindow: number
nextPane: number
panes: Record<
string,
{
window: string
options: Record<string, string>
command?: string
/** Its command has ended, and `remain-on-exit` keeps the pane open. */
dead?: boolean
}
>
/** Every command that reached a pane: `send-keys %1 C-c`, `kill-pane %1`. */
log: string[]
/** Commands the fake fails, with the error tmux would print. */
fail?: Record<string, string>
/** Once a key is sent, tmux stops answering: every later command fails like a dying server. */
dieAfterKeys?: boolean
/** On this many-th display-message, the pane is retagged as another run's (a restart race). */
retagAtCheck?: number
/** display-message calls so far. */
checks?: number
/** tmux restarts and the user's pane takes the id just before the next guarded action. */
retagBeforeAction?: boolean
/** Attached clients, as `list-clients` reports them. */
clients?: Array<{ pid: string; tty: string; session: string }>
/** A session's panes as `list-panes` reports them, with fields others can set. */
listed?: Array<{ windowName: string; command: string; cwd: string }>
/** list-panes writes its output in two pieces, split inside a multi-byte character. */
splitWrites?: boolean
/** Every `-F` format the fake was asked for, in order. */
formats?: string[]
/** Commands the fake holds until the file named here exists, like a busy tmux server. */
hold?: Record<string, string>
/** Commands the fake is holding right now. */
held?: string[]
}
const FAKE_TMUX = `
const fs = require('node:fs')
const file = process.env.FAKE_TMUX_STATE
const state = JSON.parse(fs.readFileSync(file, 'utf8'))
let args = process.argv.slice(2)
const save = () => fs.writeFileSync(file, JSON.stringify(state))
// \`-S socket\` names the server; any server but this one does not exist.
if (args[0] === '-S') {
if (args[1] !== state.socket) {
process.stderr.write('no server running on ' + args[1])
process.exit(1)
}
args = args.slice(2)
}
const target = () => args[args.indexOf('-t') + 1]
const fail = (message) => { process.stderr.write(message); process.exit(1) }
// Prints a format's output as tmux 3.4 and 3.5 do: a backslash doubled, and every other control
// character as its octal escape, so a control-character separator would arrive as text.
const escaped = (text) =>
text
.replace(/\\\\/g, '\\\\\\\\')
.replace(/[\\x00-\\x1f]/g, (c) => '\\\\' + c.charCodeAt(0).toString(8).padStart(3, '0'))
if (state.hold && state.hold[args[0]]) {
state.held = [...(state.held ?? []), args[0]]
save()
while (!fs.existsSync(state.hold[args[0]])) {
Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 20)
}
state.held = state.held.filter((command) => command !== args[0])
}
// \`if-shell -F -t pane '#{==:#{option},value}' command\`: the check and the action in one command.
if (args[0] === 'if-shell') {
const pane = state.panes[target()]
if (pane && state.retagBeforeAction) {
pane.options['@sim-run-id'] = 'someone-else'
state.retagBeforeAction = false
save()
}
const check = /^#\\{==:#\\{([^}]+)\\},(.*)\\}$/.exec(args[args.length - 2])
if (!pane || !check || (pane.options[check[1]] ?? '') !== check[2]) process.exit(0)
args = args[args.length - 1].split(' ')
}
if (state.fail && state.fail[args[0]]) fail(state.fail[args[0]])
switch (args[0]) {
case 'new-window': {
const window = '@' + state.nextWindow++
const pane = '%' + state.nextPane++
state.panes[pane] = { window, options: {}, command: args[args.length - 1] }
save()
// Runs the pane's command for real, the way tmux would, when a test asks for it.
if (process.env.FAKE_TMUX_EXEC) {
require('node:child_process')
.spawn('sh', ['-c', args[args.length - 1]], { detached: true, stdio: 'ignore' })
.unref()
}
process.stdout.write(window + ' ' + pane + '\\n')
break
}
case 'set-option': {
const pane = state.panes[target()]
if (!pane) fail("can't find pane")
pane.options[args[args.length - 2]] = args[args.length - 1]
save()
break
}
case 'display-message': {
state.checks = (state.checks ?? 0) + 1
if (state.retagAtCheck === state.checks && state.panes[target()]) {
state.panes[target()].options['@sim-run-id'] = 'someone-else'
}
save()
// Like tmux 3.x, a pane that is gone answers with an empty line rather than an error.
const pane = state.panes[target()]
const field = (name) =>
name === 'pane_id'
? target()
: name === 'socket_path'
? state.socket
: name === 'pane_start_command'
? (pane.command ?? '')
: name === 'pane_dead'
? (pane.dead ? '1' : '0')
: (pane.options[name] ?? '')
const value = pane ? args[args.length - 1].replace(/#\\{([^}]+)\\}/g, (_, name) => field(name)) : ''
process.stdout.write(value + '\\n')
break
}
case 'list-panes': {
// Formats as tmux evaluates them: \`#{s/pattern/replacement/:field}\` substitutes, and every
// other field prints as is, a newline included.
const format = args[args.indexOf('-F') + 1]
state.formats = [...(state.formats ?? []), format]
save()
let output = ''
;(state.listed ?? []).forEach((pane, index) => {
const fields = {
session_name: 'work',
window_index: '0',
pane_index: String(index),
window_name: pane.windowName,
pane_current_command: pane.command,
pane_current_path: pane.cwd,
pane_active: index === 0 ? '1' : '0',
}
const line = format
.replace(/#\\{s\\/([^/]*)\\/([^/]*)\\/:([a-z_]+)\\}/g, (_, from, to, name) =>
fields[name].split(from).join(to)
)
.replace(/#\\{([a-z_]+)\\}/g, (_, name) => fields[name])
output += line + '\\n'
})
const bytes = Buffer.from(output)
if (!state.splitWrites) {
process.stdout.write(bytes)
break
}
// The first piece ends inside a character, and arrives well before the rest.
const cut = bytes.findIndex((byte) => byte >= 0x80) + 1
fs.writeSync(1, bytes.subarray(0, cut))
Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 100)
fs.writeSync(1, bytes.subarray(cut))
break
}
case 'list-clients': {
const format = args[args.indexOf('-F') + 1]
for (const client of state.clients ?? []) {
const line = format
.replace('#{client_pid}', client.pid)
.replace('#{client_tty}', client.tty)
.replace('#{client_session}', client.session)
process.stdout.write(escaped(line) + '\\n')
}
break
}
case 'send-keys':
case 'kill-pane': {
if (!state.panes[target()]) fail("can't find pane")
state.log.push(args[0] + ' ' + target() + (args[0] === 'send-keys' ? ' ' + args[args.length - 1] : ''))
if (args[0] === 'send-keys' && state.dieAfterKeys) {
state.fail = { 'display-message': 'server exited unexpectedly', 'kill-pane': 'server exited unexpectedly' }
}
if (args[0] === 'kill-pane') delete state.panes[target()]
save()
break
}
default:
fail('unsupported: ' + args[0])
}
`
function fakeTmux(options: { exec?: boolean } = {}) {
const dir = mkdtempSync(join(tmpdir(), 'fake-tmux-'))
const stateFile = join(dir, 'state.json')
const binary = join(dir, 'tmux')
writeFileSync(binary, `#!${process.execPath}\n${FAKE_TMUX}`)
chmodSync(binary, 0o755)
const write = (state: FakeTmuxState) => writeFileSync(stateFile, JSON.stringify(state))
write({ nextWindow: 0, nextPane: 0, panes: {}, log: [], socket: join(dir, 'server.sock') })
const read = (): FakeTmuxState => JSON.parse(readFileSync(stateFile, 'utf8'))
return {
dir,
env: {
...process.env,
PATH: `${dir}:${process.env.PATH ?? ''}`,
FAKE_TMUX_STATE: stateFile,
...(options.exec ? { FAKE_TMUX_EXEC: '1' } : {}),
},
read,
write,
/** The user splits a run's window: a new pane of theirs beside the run's. */
split(window: string): string {
const state = read()
const pane = `%${state.nextPane++}`
state.panes[pane] = { window, options: {} }
write(state)
return pane
},
/** tmux restarts: every pane is gone and ids start over. */
restart() {
write({ ...read(), nextWindow: 0, nextPane: 0, panes: {} })
},
}
}
describe("listing a session's panes", () => {
const dirs: string[] = []
afterEach(() => {
for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true })
})
it('lists a pane whose directory, command or title holds a newline, as one pane', async () => {
const tmux = fakeTmux()
dirs.push(tmux.dir)
tmux.write({
...tmux.read(),
listed: [
{ windowName: 'build\nlogs', command: 'make\nuser:0.9', cwd: '/tmp/a\nuser:0.0' },
{ windowName: 'zsh', command: 'zsh', cwd: '/tmp' },
],
})
expect(await listPanes('work', tmux.env)).toEqual([
{
target: 'work:0.0',
windowName: 'build<NL>logs',
command: 'make<NL>user:0.9',
cwd: '/tmp/a<NL>user:0.0',
active: true,
},
{ target: 'work:0.1', windowName: 'zsh', command: 'zsh', cwd: '/tmp', active: false },
])
})
it('reads a character tmux wrote in two pieces as one', async () => {
const tmux = fakeTmux()
dirs.push(tmux.dir)
tmux.write({
...tmux.read(),
splitWrites: true,
listed: [{ windowName: 'café ✓', command: 'zsh', cwd: '/tmp' }],
})
expect((await listPanes('work', tmux.env))[0]?.windowName).toBe('café ✓')
})
it('frames each call with a marker of its own', async () => {
const tmux = fakeTmux()
dirs.push(tmux.dir)
await listPanes('work', tmux.env)
await listPanes('work', tmux.env)
const frames = (tmux.read().formats ?? []).map((format) => /^<~([0-9a-f]+)~>/.exec(format)?.[1])
expect(frames).toHaveLength(2)
expect(frames[0]).toMatch(/^[0-9a-f]{16}$/)
expect(frames[1]).not.toBe(frames[0])
})
})
describe('finding the tmux session a shell runs', () => {
const dirs: string[] = []
afterEach(() => {
for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true })
})
it('reads the clients tmux 3.4 and 3.5 list, which escape control characters', async () => {
const tmux = fakeTmux()
dirs.push(tmux.dir)
// A client of this very process: its parent stands in for the shell the client runs in.
tmux.write({
...tmux.read(),
clients: [{ pid: String(process.pid), tty: '/dev/pts/3', session: 'work' }],
})
expect(await resolveAttachment(process.ppid, tmux.env)).toEqual({
session: 'work',
clientTty: '/dev/pts/3',
})
})
})
describe('stopping a run another process started, from its record', () => {
const dirs: string[] = []
afterEach(() => {
for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true })
})
async function recorded(tmux: ReturnType<typeof fakeTmux>) {
dirs.push(tmux.dir)
const run = await startRun('agent', 'sleep 600', null, tmux.env)
if ('error' in run) throw new Error(run.error)
return { run, record: { runId: run.runId ?? '', pane: run.pane, socket: run.socket ?? '' } }
}
it('records the server a tagged run runs on', async () => {
const tmux = fakeTmux()
const { run } = await recorded(tmux)
expect(run.socket).toBe(tmux.read().socket)
})
it('interrupts and then closes the pane while it still carries the run tag', async () => {
const tmux = fakeTmux()
const { record } = await recorded(tmux)
expect(await stopRecordedRun(record, tmux.env, 0)).toBe('gone')
expect(tmux.read().log).toEqual([`send-keys ${record.pane} C-c`, `kill-pane ${record.pane}`])
expect(tmux.read().panes).toEqual({})
})
it('reads a pane kept open after its command ended as finished, and closes it without keys', async () => {
const tmux = fakeTmux()
const { record } = await recorded(tmux)
const state = tmux.read()
const pane = state.panes[record.pane]
if (pane) pane.dead = true
tmux.write(state)
expect(await recordedRunState(record, tmux.env)).toBe('finished')
expect(await stopRecordedRun(record, tmux.env, 0)).toBe('gone')
expect(tmux.read().log).toEqual([`kill-pane ${record.pane}`])
})
it('never touches a pane that took the recorded id after tmux restarted', async () => {
const tmux = fakeTmux()
const { run, record } = await recorded(tmux)
tmux.restart()
const state = tmux.read()
state.panes[run.pane] = { window: run.window, options: {}, command: 'zsh' }
tmux.write(state)
expect(await stopRecordedRun(record, tmux.env, 0)).toBe('gone')
expect(tmux.read().log).toEqual([])
expect(Object.keys(tmux.read().panes)).toEqual([run.pane])
})
it('never asks a different tmux server about the pane', async () => {
const tmux = fakeTmux()
const { record } = await recorded(tmux)
const elsewhere = { ...record, socket: join(tmux.dir, 'another.sock') }
expect(await stopRecordedRun(elsewhere, tmux.env, 0)).toBe('gone')
expect(tmux.read().log).toEqual([])
expect(Object.keys(tmux.read().panes)).toEqual([record.pane])
})
it('checks the pane is still the run once more right before closing it', async () => {
const tmux = fakeTmux()
const { record } = await recorded(tmux)
// The first check (before Ctrl-C) sees the run; the next, right before closing, does not.
tmux.write({ ...tmux.read(), checks: 0, retagAtCheck: 2 })
expect(await stopRecordedRun(record, tmux.env, 0)).toBe('gone')
expect(tmux.read().log).toEqual([`send-keys ${record.pane} C-c`])
expect(Object.keys(tmux.read().panes)).toEqual([record.pane])
})
it('never starts a tagged run whose server it cannot name for a record', async () => {
const tmux = fakeTmux({ exec: true })
dirs.push(tmux.dir)
tmux.write({ ...tmux.read(), socket: '' })
const marker = join(tmux.dir, 'ran')
const result = await startRun('agent', `touch ${JSON.stringify(marker)}`, null, tmux.env, {
beforeStart: () => true,
})
expect(result).toMatchObject({ error: expect.stringContaining('was not run') })
// The pane it opened and tagged is closed with it.
expect(tmux.read().panes).toEqual({})
await sleep(1_500)
expect(existsSync(marker)).toBe(false)
})
it('closes the tagged pane of a run that then could not start', async () => {
const tmux = fakeTmux()
dirs.push(tmux.dir)
let runDir = ''
const result = await startRun('agent', 'sleep 600', null, tmux.env, {
beforeStart: (run) => {
// The run's directory turns read-only, so its go file cannot be written.
const command = tmux.read().panes[run.pane]?.command ?? ''
runDir = /"([^"]+)\/run\.sh"/.exec(command)?.[1] ?? ''
chmodSync(runDir, 0o500)
return true
},
})
if (runDir) chmodSync(runDir, 0o700)
if (runDir) dirs.push(runDir)
expect(result).toMatchObject({ error: expect.stringContaining('could not be started') })
expect(tmux.read().panes).toEqual({})
})
it('acts on no pane that took the recorded id between the last check and the action', async () => {
const tmux = fakeTmux()
const { record } = await recorded(tmux)
tmux.write({ ...tmux.read(), retagBeforeAction: true })
await stopRecordedRun(record, tmux.env, 0)
expect(tmux.read().log).toEqual([])
expect(Object.keys(tmux.read().panes)).toEqual([record.pane])
})
it('keeps the record when tmux stops answering part-way through the stop', async () => {
const tmux = fakeTmux()
const { record } = await recorded(tmux)
tmux.write({ ...tmux.read(), dieAfterKeys: true })
expect(await stopRecordedRun(record, tmux.env, 0)).toBe('unknown')
// Its pane could not be confirmed as the run's, so it was not closed.
expect(tmux.read().log).toEqual([`send-keys ${record.pane} C-c`])
})
it('saves the record before the command may start, and never starts one it could not save', async () => {
const tmux = fakeTmux({ exec: true })
dirs.push(tmux.dir)
const marker = join(tmux.dir, 'ran')
let ranBeforeRecord = true
const run = await startRun('agent', `touch ${JSON.stringify(marker)}`, null, tmux.env, {
beforeStart: () => {
// Time enough for a command already released to have run.
Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 1_000)
ranBeforeRecord = existsSync(marker)
return true
},
})
if ('error' in run) throw new Error(run.error)
expect(ranBeforeRecord).toBe(false)
await expect.poll(() => existsSync(marker), { timeout: 10_000 }).toBe(true)
const unsaved = fakeTmux({ exec: true })
dirs.push(unsaved.dir)
const unsavedMarker = join(unsaved.dir, 'ran')
const refused = await startRun(
'agent',
`touch ${JSON.stringify(unsavedMarker)}`,
null,
unsaved.env,
{
beforeStart: () => false,
}
)
expect(refused).toMatchObject({ error: expect.stringContaining('was not run') })
await sleep(1_500)
expect(existsSync(unsavedMarker)).toBe(false)
}, 20_000)
it('leaves the record for later while tmux cannot be asked', async () => {
const tmux = fakeTmux()
const { record } = await recorded(tmux)
tmux.write({ ...tmux.read(), fail: { 'display-message': 'server exited unexpectedly' } })
expect(await stopRecordedRun(record, tmux.env, 0)).toBe('unknown')
expect(tmux.read().log).toEqual([])
})
})
describe('stopping a tmux run touches only its own pane', () => {
const dirs: string[] = []
afterEach(() => {
for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true })
})
async function started(tmux: ReturnType<typeof fakeTmux>): Promise<TmuxRunHandle> {
dirs.push(tmux.dir)
const handle = await startRun('agent', 'sleep 600', null, tmux.env)
if ('error' in handle) throw new Error(handle.error)
return handle
}
it('interrupts and then closes only the run pane, leaving a pane the user split beside it', async () => {
const tmux = fakeTmux()
const run = await started(tmux)
const users = tmux.split(run.window)
await stopRun(run, tmux.env, 0)
expect(tmux.read().log).toEqual([`send-keys ${run.pane} C-c`, `kill-pane ${run.pane}`])
expect(Object.keys(tmux.read().panes)).toEqual([users])
})
it('sends a live run nothing once its pane is retagged between the check and the action', async () => {
const tmux = fakeTmux()
const run = await started(tmux)
tmux.write({ ...tmux.read(), retagBeforeAction: true })
await stopRun(run, tmux.env, 0)
expect(tmux.read().log).toEqual([])
expect(Object.keys(tmux.read().panes)).toEqual([run.pane])
})
it('sends nothing to a pane that reused the run pane id after tmux restarted', async () => {
const tmux = fakeTmux()
const run = await started(tmux)
tmux.restart()
// The user's own new window gets the ids the run's pane had.
const state = tmux.read()
state.panes[run.pane] = { window: run.window, options: {} }
tmux.write(state)
expect(await runPaneState(run, tmux.env)).toBe('gone')
await stopRun(run, tmux.env, 0)
expect(tmux.read().log).toEqual([])
expect(Object.keys(tmux.read().panes)).toEqual([run.pane])
})
it('treats a run pane the user closed as gone', async () => {
const tmux = fakeTmux()
const run = await started(tmux)
const state = tmux.read()
delete state.panes[run.pane]
tmux.write(state)
expect(await runPaneState(run, tmux.env)).toBe('gone')
await stopRun(run, tmux.env, 0)
expect(tmux.read().log).toEqual([])
})
it.each([
['tmux 2.9a', TMUX_29_NO_PANE_OPTIONS],
['BSD getopt', TMUX_29_BSD_NO_PANE_OPTIONS],
])(
'starts a run untracked on a tmux without pane options (%s), and never stops it by a pane id',
async (_build, refusal) => {
const tmux = fakeTmux()
dirs.push(tmux.dir)
// tmux before 3.0 has no pane options.
tmux.write({ ...tmux.read(), fail: { 'set-option': refusal } })
const run = await startRun('agent', 'sleep 600', null, tmux.env)
if ('error' in run) throw new Error(run.error)
expect(run.runId).toBeNull()
expect(existsSync(join(run.statusPath, '..', 'go'))).toBe(true)
expect(await runPaneState(run, tmux.env)).toBe('unknown')
await stopRun(run, tmux.env, 0)
// No pane is touched by an id that a restarted server might have handed to the user.
expect(tmux.read().log).toEqual([])
expect(Object.keys(tmux.read().panes)).toEqual([run.pane])
// Once its pane is gone, it can be let go.
const state = tmux.read()
delete state.panes[run.pane]
tmux.write(state)
expect(await runPaneState(run, tmux.env)).toBe('gone')
}
)
it.each(['tmux did not respond', 'server exited unexpectedly'])(
'refuses a run that a tmux able to tag panes did not tag (%s)',
async (failure) => {
const tmux = fakeTmux()
dirs.push(tmux.dir)
tmux.write({ ...tmux.read(), fail: { 'set-option': failure } })
const result = await startRun('agent', 'sleep 600', null, tmux.env)
// Untagged on a tmux that tags, nothing could stop it later, so it never starts.
expect(result).toMatchObject({ error: expect.stringContaining('was not run') })
expect(tmux.read().log).toEqual([])
}
)
it("closes a finished untracked run's pane, and only once it has finished", async () => {
const tmux = fakeTmux()
dirs.push(tmux.dir)
tmux.write({ ...tmux.read(), fail: { 'set-option': TMUX_29_NO_PANE_OPTIONS } })
const run = await startRun('agent', 'make build', null, tmux.env)
if ('error' in run) throw new Error(run.error)
await closeRunPane(run, tmux.env)
expect(Object.keys(tmux.read().panes)).toEqual([run.pane])
// Its command ended; with `remain-on-exit` its dead pane would otherwise stay open.
writeFileSync(run.statusPath, '0')
await closeRunPane(run, tmux.env)
expect(Object.keys(tmux.read().panes)).toEqual([])
})
it("never closes a pane that took a finished untracked run's id after tmux restarted", async () => {
const tmux = fakeTmux()
dirs.push(tmux.dir)
tmux.write({ ...tmux.read(), fail: { 'set-option': TMUX_29_NO_PANE_OPTIONS } })
const run = await startRun('agent', 'make build', null, tmux.env)
if ('error' in run) throw new Error(run.error)
writeFileSync(run.statusPath, '0')
tmux.restart()
// The user's own shell gets the ids the run's pane had.
const state = tmux.read()
state.panes[run.pane] = { window: run.window, options: {}, command: 'zsh' }
tmux.write(state)
await closeRunPane(run, tmux.env)
expect(tmux.read().log).toEqual([])
expect(Object.keys(tmux.read().panes)).toEqual([run.pane])
})
it('lets a tagged run start only once its pane is tagged', async () => {
const tmux = fakeTmux()
const run = await started(tmux)
expect(tmux.read().panes[run.pane]?.options['@sim-run-id']).toBe(run.runId)
expect(existsSync(join(run.statusPath, '..', 'go'))).toBe(true)
})
it('neither stops nor gives up on a run while tmux cannot be asked', async () => {
const tmux = fakeTmux()
const run = await started(tmux)
tmux.write({ ...tmux.read(), fail: { 'display-message': 'server exited unexpectedly' } })
expect(await runPaneState(run, tmux.env)).toBe('unknown')
await stopRun(run, tmux.env, 0)
expect(tmux.read().log).toEqual([])
})
it('runs a command for real whether or not tmux could tag its pane', async () => {
const tagged = fakeTmux({ exec: true })
dirs.push(tagged.dir)
const run = await startRun('agent', 'echo ran', null, tagged.env)
if ('error' in run) throw new Error(run.error)
await expect
.poll(() => pollRun(run), { timeout: 10_000 })
.toMatchObject({
done: true,
exitCode: 0,
})
const untagged = fakeTmux({ exec: true })
dirs.push(untagged.dir)
untagged.write({ ...untagged.read(), fail: { 'set-option': TMUX_29_NO_PANE_OPTIONS } })
const untracked = await startRun('agent', 'echo ran', null, untagged.env)
if ('error' in untracked) throw new Error(untracked.error)
await expect
.poll(() => pollRun(untracked), { timeout: 10_000 })
.toMatchObject({ done: true, exitCode: 0, output: 'ran\n' })
}, 20_000)
it('holds a command until tmux has finished tagging its pane', async () => {
const tmux = fakeTmux({ exec: true })
dirs.push(tmux.dir)
const release = join(tmux.dir, 'release')
tmux.write({ ...tmux.read(), hold: { 'set-option': release } })
const marker = join(tmux.dir, 'ran')
const starting = startRun('agent', `touch ${JSON.stringify(marker)}`, null, tmux.env)
// The pane is open and the tagging call is in flight, held by tmux.
await expect.poll(() => tmux.read().held ?? [], { timeout: 10_000 }).toEqual(['set-option'])
// Time enough for an ungated command to have run.
await sleep(1_000)
expect(existsSync(marker)).toBe(false)
writeFileSync(release, '')
const run = await starting
if ('error' in run) throw new Error(run.error)
await expect.poll(() => existsSync(marker), { timeout: 10_000 }).toBe(true)
}, 20_000)
})
apps/desktop/src/main/terminal/tmux.ts
+330 -38
View File
@@ -17,12 +17,15 @@
* of the shell that launched it.
*/
import { spawn } from 'node:child_process'
import { mkdtempSync, readFileSync, rmSync } from 'node:fs'
import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { dirname, join } from 'node:path'
import { createLogger } from '@sim/logger'
import type { TerminalPaneState } from '@sim/terminal-protocol'
import { getErrorMessage } from '@sim/utils/errors'
import { sleep } from '@sim/utils/helpers'
import { generateId } from '@sim/utils/id'
import { generateRandomHex } from '@sim/utils/random'
const logger = createLogger('DesktopTmux')
@@ -36,8 +39,36 @@ const TMUX_TIMEOUT_MS = 5_000
/** How often the status file is checked while a tmux-run command is going. */
const RUN_POLL_INTERVAL_MS = 250
/** Field separator for `-F` output. Chosen because no tmux field contains it. */
const FIELD = '\u001f'
/**
* Field separator for `-F` output. Printable on purpose: tmux 3.4 and 3.5 print a control
* character as its octal escape, so a control-character separator arrived as the text `\037` and
* no line split. No tmux escapes these characters. Fields are untrusted text (a directory or
* window name can hold the separator, or a newline), so records are also framed per call: see
* {@link framedFormat}.
*/
const FIELD = '<~sim~>'
/**
* A field someone other than the user can set: a directory name (`pane_current_path`), a program's
* name (`pane_current_command`), or a window title a program sets (`window_name`). tmux prints a
* newline inside a field as is, and a newline would end the record early and let the rest read as
* a record of its own, so tmux replaces each newline with `<NL>` before printing.
*/
function untrusted(field: string): string {
return `#{s/\n/<NL>/:${field}}`
}
/**
* A `-F` format whose every record starts and ends with a marker made fresh for this call; only
* lines framed whole by it are read, so a line that is not a whole record is dropped rather than
* misread. The marker is a second line of defence, not a secret (another user can read a process's
* arguments on many systems), which is why newlines are neutralised at the source too
* ({@link untrusted}).
*/
function framedFormat(fields: string[]): { format: string; frame: string } {
const frame = `<~${generateRandomHex(16)}~>`
return { format: `${frame}${fields.join(FIELD)}${frame}`, frame }
}
export interface TmuxCommandResult {
ok: boolean
@@ -104,11 +135,14 @@ export function runTmux(args: string[], env: NodeJS.ProcessEnv): Promise<TmuxCom
finish({ ok: false, stdout, stderr: 'tmux did not respond' })
}, TMUX_TIMEOUT_MS)
child.stdout?.on('data', (chunk: Buffer) => {
stdout += chunk.toString()
// Decoded as streams, so a character split across two chunks arrives whole.
child.stdout?.setEncoding('utf8')
child.stderr?.setEncoding('utf8')
child.stdout?.on('data', (chunk: string) => {
stdout += chunk
})
child.stderr?.on('data', (chunk: Buffer) => {
stderr += chunk.toString()
child.stderr?.on('data', (chunk: string) => {
stderr += chunk
})
child.on('error', (error) => {
if ((error as NodeJS.ErrnoException).code === 'ENOENT') tmuxBinaryMissing = true
@@ -121,18 +155,20 @@ export function runTmux(args: string[], env: NodeJS.ProcessEnv): Promise<TmuxCom
}
/**
* Parses `list-clients`/`list-panes` output into records.
* Parses `list-clients`/`list-panes` output into records: only lines framed whole by this call's
* marker, each with exactly the fields asked for.
*
* Split on a control character rather than whitespace: window names and
* Split on a dedicated separator rather than whitespace: window names and
* working directories contain spaces, and a path with a space would otherwise
* shift every later field by one.
*/
export function parseFormatLines(stdout: string, fields: number): string[][] {
export function parseFormatLines(stdout: string, fields: number, frame: string): string[][] {
return stdout
.split('\n')
.map((line) => line.trimEnd())
.filter((line) => line.length > 0)
.map((line) => line.split(FIELD))
.filter(
(line) => line.length >= frame.length * 2 && line.startsWith(frame) && line.endsWith(frame)
)
.map((line) => line.slice(frame.length, line.length - frame.length).split(FIELD))
.filter((parts) => parts.length === fields)
}
@@ -198,11 +234,11 @@ export async function resolveAttachment(
shellPid: number,
env: NodeJS.ProcessEnv
): Promise<TmuxAttachment | null> {
const format = ['#{client_pid}', '#{client_tty}', '#{client_session}'].join(FIELD)
const { format, frame } = framedFormat(['#{client_pid}', '#{client_tty}', '#{client_session}'])
const listed = await runTmux(['list-clients', '-F', format], env)
if (!listed.ok) return null
const clients = parseFormatLines(listed.stdout, 3)
const clients = parseFormatLines(listed.stdout, 3, frame)
if (clients.length === 0) return null
const parents = await listProcessParents()
@@ -218,28 +254,27 @@ export async function resolveAttachment(
/** The active pane of a session, as a target usable by every other call. */
export async function activePane(session: string, env: NodeJS.ProcessEnv): Promise<string | null> {
const result = await runTmux(
['display-message', '-p', '-t', session, '#{session_name}:#{window_index}.#{pane_index}'],
env
)
const target = result.stdout.trim()
return result.ok && target ? target : null
const { format, frame } = framedFormat(['#{session_name}:#{window_index}.#{pane_index}'])
const result = await runTmux(['display-message', '-p', '-t', session, format], env)
if (!result.ok) return null
const [target] = parseFormatLines(result.stdout, 1, frame)[0] ?? []
return target || null
}
export async function listPanes(
session: string,
env: NodeJS.ProcessEnv
): Promise<TerminalPaneState[]> {
const format = [
const { format, frame } = framedFormat([
'#{session_name}:#{window_index}.#{pane_index}',
'#{window_name}',
'#{pane_current_command}',
'#{pane_current_path}',
untrusted('window_name'),
untrusted('pane_current_command'),
untrusted('pane_current_path'),
'#{pane_active}',
].join(FIELD)
])
const result = await runTmux(['list-panes', '-s', '-t', session, '-F', format], env)
if (!result.ok) return []
return parseFormatLines(result.stdout, 5).map(
return parseFormatLines(result.stdout, 5, frame).map(
([target, windowName, command, cwd, active]): TerminalPaneState => ({
target,
windowName,
@@ -300,11 +335,39 @@ export const TMUX_KEY_NAMES: Record<string, string> = {
*/
export interface TmuxRunHandle {
window: string
/** The run's own pane: input and stops go here, never to whatever pane is active. */
pane: string
/**
* Tagged on the pane as the `@sim-run-id` user option. Window and pane ids restart from zero
* with the tmux server, so only the tag proves a pane is still this run's. Null when tmux could
* not tag the pane (tmux before 3.0 has no pane options): the run goes ahead untracked, and
* nothing ever stops it, since nothing could tell its pane from one of the user's.
*/
runId: string | null
/** The tmux server's socket, for a record a later process can act on; null if unknown. */
socket: string | null
outPath: string
statusPath: string
dispose(): void
}
/**
* How many 50 ms polls a run's command waits for its go file: longer than the tagging call can
* take before it times out, so a tag that succeeds always lands inside the window.
*/
const RUN_GATE_POLLS = Math.ceil((2 * TMUX_TIMEOUT_MS + 5_000) / 50)
/** The tmux user option that marks a pane as one run's own. */
const RUN_ID_OPTION = '@sim-run-id'
/**
* How tmux before 3.0, which has no pane options, refuses `set-option -p`. Its own getopt prints
* `unknown option -- p` (BSD getopt on macOS: `illegal option -- p`) followed by set-option's usage
* line; later wordings are kept for any build that phrases it so.
*/
const NO_PANE_OPTIONS =
/unknown option -- p|illegal option -- p|usage: set-option|unknown flag|invalid option/i
/**
* Starts a command in a dedicated tmux window.
*
@@ -321,11 +384,19 @@ export async function startRun(
session: string,
command: string,
cwd: string | null,
env: NodeJS.ProcessEnv
env: NodeJS.ProcessEnv,
options: {
/**
* Records a tagged run before its command may start; false keeps the command from starting,
* since a run no later process could find must not outlive this one.
*/
beforeStart?: (run: RecordedRun) => boolean
} = {}
): Promise<TmuxRunHandle | { error: string }> {
const dir = mkdtempSync(join(tmpdir(), 'sim-tmux-run-'))
const outPath = join(dir, 'out')
const statusPath = join(dir, 'status')
const goPath = join(dir, 'go')
const dispose = () => {
try {
rmSync(dir, { recursive: true, force: true })
@@ -343,10 +414,40 @@ export async function startRun(
// writing to the unlinked inode — but an unredirected `printf` would fail
// into the pipeline and print `No such file or directory` into the user's own
// tmux window, minutes after they closed the tab.
const script = `${command}\nprintf %s "\${PIPESTATUS[0]}" > ${JSON.stringify(statusPath)} 2>/dev/null`
const wrapper = `bash -lc ${JSON.stringify(`{ ${script}; } 2>&1 | tee ${JSON.stringify(outPath)}`)}`
//
// The command waits for the tagging call to finish (the go file), so a tagged run's command
// never runs before a later stop could recognize its pane. If Sim never releases it (it quit
// mid-start), the script gives up once the tagging call has surely ended and its pane closes on
// its own; no one has to close a pane whose id might no longer be its own.
//
// The script is a file rather than a `bash -c` string: tmux hands its command to `sh -c`, which
// would expand `$` references meant for bash (the gate's counter, PIPESTATUS) before bash ran.
const scriptPath = join(dir, 'run.sh')
writeFileSync(
scriptPath,
[
'i=0',
`while [ ! -e ${JSON.stringify(goPath)} ] && [ "$i" -lt ${RUN_GATE_POLLS} ]; do sleep 0.05; i=$((i + 1)); done`,
`[ -e ${JSON.stringify(goPath)} ] || exit 0`,
`{ ${command}`,
`printf %s "\${PIPESTATUS[0]}" > ${JSON.stringify(statusPath)} 2>/dev/null; } 2>&1 | tee ${JSON.stringify(outPath)}`,
'',
].join('\n'),
{ mode: 0o600 }
)
const wrapper = `bash -l ${JSON.stringify(scriptPath)}`
const args = ['new-window', '-d', '-P', '-F', '#{window_id}', '-t', session, '-n', 'sim-run']
const args = [
'new-window',
'-d',
'-P',
'-F',
'#{window_id} #{pane_id}',
'-t',
session,
'-n',
'sim-run',
]
if (cwd) args.push('-c', cwd)
args.push(wrapper)
@@ -355,8 +456,170 @@ export async function startRun(
dispose()
return { error: created.stderr.trim() || 'tmux could not open a window for the command.' }
}
const [window = '', pane = ''] = created.stdout.trim().split(' ')
const tag = generateId()
// An untagged pane is never treated as the run's: without the tag a stop could not tell it from
// a pane the user opened later under the same id.
const tagged = await runTmux(['set-option', '-p', '-t', pane, RUN_ID_OPTION, tag], env)
if (!tagged.ok && !NO_PANE_OPTIONS.test(tagged.stderr)) {
// A tmux that can tag panes but did not (it timed out, or failed otherwise) gets no command
// that nothing could stop: without the go file the wrapper exits by itself.
dispose()
return {
error: `tmux could not mark the command's pane (${tagged.stderr.trim() || 'no detail'}), so the command was not run.`,
}
}
if (!tagged.ok) {
logger.warn('This tmux cannot tag a run pane; the run goes ahead untracked', {
error: tagged.stderr.trim(),
})
}
const runId = tagged.ok ? tag : null
// Which server the pane lives on, so a later process stops it there and nowhere else.
const shown = runId
? await runTmux(['display-message', '-p', '-t', pane, '#{socket_path}'], env)
: null
const socket = shown?.ok && shown.stdout.trim().startsWith('/') ? shown.stdout.trim() : null
if (runId && options.beforeStart && !(socket && options.beforeStart({ runId, pane, socket }))) {
// The pane was tagged a moment ago, so it is closed as the run's; without the go file its
// command never starts in any case.
await runTmux(ifTagged(pane, runId, `kill-pane -t ${pane}`), env)
dispose()
return { error: 'The command could not be recorded for a later stop, so it was not run.' }
}
try {
writeFileSync(goPath, '')
} catch (error) {
// Like a refused run: its tagged pane closes now, and the next sweep drops a saved record.
if (runId) await runTmux(ifTagged(pane, runId, `kill-pane -t ${pane}`), env)
dispose()
return { error: `The command could not be started: ${getErrorMessage(error)}` }
}
return { window: created.stdout.trim(), outPath, statusPath, dispose }
return { window, pane, runId, socket, outPath, statusPath, dispose }
}
/**
* Whether the run's pane is still the run's: `ours`, or `gone` when tmux has no such pane or the
* pane under that id is not tagged as this run's (the user closed it, or a restarted tmux server
* handed the id to one of the user's own panes). `unknown` when tmux could not be asked, or an
* untracked run's pane still exists, since nothing proves whose it is: such a pane is neither
* touched nor given up on.
*/
export async function runPaneState(
handle: TmuxRunHandle,
env: NodeJS.ProcessEnv
): Promise<'ours' | 'gone' | 'unknown'> {
if (!handle.pane) return 'gone'
// An untracked run's pane can still be found missing, with a format every tmux knows.
const format = handle.runId === null ? '#{pane_id}' : `#{${RUN_ID_OPTION}}`
const shown = await runTmux(['display-message', '-p', '-t', handle.pane, format], env)
// tmux 3.x answers for a missing pane with an empty line rather than an error.
if (shown.ok && handle.runId === null) {
return shown.stdout.trim() === handle.pane ? 'unknown' : 'gone'
}
if (shown.ok) return shown.stdout.trim() === handle.runId ? 'ours' : 'gone'
return /can't find|no server running/i.test(shown.stderr) ? 'gone' : 'unknown'
}
/**
* Arguments for one tmux command that runs `command` on a pane only while the pane carries the
* run's tag. tmux checks the tag and acts within the one command, so a restart between a check
* and an action can never hand the action to a pane that took the id.
*/
function ifTagged(pane: string, runId: string, command: string, socket?: string): string[] {
return [
...(socket ? ['-S', socket] : []),
'if-shell',
'-F',
'-t',
pane,
`#{==:#{${RUN_ID_OPTION}},${runId}}`,
command,
]
}
/**
* Stops a run: Ctrl-C in its own pane, then closing that pane if the command ignored it. Every
* step first checks the pane is still the run's, and only that pane is ever closed, so a pane the
* user split off beside it, or a window that reused its ids, is never touched.
*/
export async function stopRun(
handle: TmuxRunHandle,
env: NodeJS.ProcessEnv,
graceMs: number
): Promise<void> {
if ((await runPaneState(handle, env)) !== 'ours' || !handle.runId) return
await runTmux(ifTagged(handle.pane, handle.runId, `send-keys -t ${handle.pane} C-c`), env)
const deadline = Date.now() + graceMs
while (!isRunComplete(handle) && Date.now() < deadline) await sleep(100)
if (!isRunComplete(handle)) await closeRunPane(handle, env)
}
/** A recorded run as a later process finds it: what it takes to find its pane again. */
export interface RecordedRun {
/** The tag on the run's pane; only a pane carrying it is ever acted on. */
runId: string
pane: string
/** The tmux server's socket, so a different server is never asked about this pane. */
socket: string
}
/**
* Whether a recorded run's pane still carries its tag, asked of the run's own server only: `gone`
* when that server or pane no longer exists or the pane is not tagged as this run's, `unknown`
* when tmux could not be asked.
*/
export async function recordedRunState(
run: RecordedRun,
env: NodeJS.ProcessEnv
): Promise<'ours' | 'finished' | 'gone' | 'unknown'> {
const shown = await runTmux(
['-S', run.socket, 'display-message', '-p', '-t', run.pane, `#{${RUN_ID_OPTION}} #{pane_dead}`],
env
)
if (shown.ok) {
const [tag, dead] = shown.stdout.trim().split(' ')
if (tag !== run.runId) return 'gone'
// Kept open after its command ended (`remain-on-exit`): nothing in it is left to run.
return dead === '1' ? 'finished' : 'ours'
}
return /can't find|no server running|error connecting|no such file/i.test(shown.stderr)
? 'gone'
: 'unknown'
}
/**
* Stops a run another process started, from its record: Ctrl-C in its pane, then closing that pane
* if the command ignored it. Every step first checks, on the run's own server, that the pane still
* carries the run's tag, so a pane that took its id after a tmux restart is never touched.
* Resolves `gone` once nothing of the run is left to stop, and `unknown` when tmux could not say.
*/
export async function stopRecordedRun(
run: RecordedRun,
env: NodeJS.ProcessEnv,
graceMs: number
): Promise<'gone' | 'unknown'> {
const before = await recordedRunState(run, env)
if (before === 'gone' || before === 'unknown') return before
let state: 'ours' | 'finished' | 'gone' | 'unknown' = before
if (before === 'ours') {
await runTmux(ifTagged(run.pane, run.runId, `send-keys -t ${run.pane} C-c`, run.socket), env)
const deadline = Date.now() + graceMs
while (Date.now() < deadline) {
await sleep(100)
state = await recordedRunState(run, env)
if (state !== 'ours') break
}
}
// The pane is closed only while it is confirmed the run's; a pane tmux could not answer for is
// left alone, and its record kept for the next sweep.
if (state === 'ours' || state === 'finished') state = await recordedRunState(run, env)
if (state === 'ours' || state === 'finished') {
await runTmux(ifTagged(run.pane, run.runId, `kill-pane -t ${run.pane}`, run.socket), env)
state = await recordedRunState(run, env)
}
return state === 'gone' ? 'gone' : 'unknown'
}
function readIfPresent(path: string): string | null {
@@ -419,11 +682,40 @@ export async function killPane(target: string, env: NodeJS.ProcessEnv): Promise<
return runTmux(['kill-pane', '-t', target], env)
}
/** Closes a window opened by {@link startRun}. */
export async function closeRunWindow(handle: TmuxRunHandle, env: NodeJS.ProcessEnv): Promise<void> {
if (!handle.window) return
const killed = await runTmux(['kill-window', '-t', handle.window], env)
/**
* Whether the pane under an untracked run's id was started with that run's own script, whose path
* is unique to the run. Every tmux reports a pane's start command, so this holds where tags do not.
*/
async function startedByRun(handle: TmuxRunHandle, env: NodeJS.ProcessEnv): Promise<boolean> {
const script = join(dirname(handle.statusPath), 'run.sh')
const shown = await runTmux(
['display-message', '-p', '-t', handle.pane, '#{pane_start_command}'],
env
)
return shown.ok && shown.stdout.includes(script)
}
/**
* Closes the pane opened by {@link startRun}, and with it the window once that pane is the last
* one in it. Only the run's own pane, and only while it is still the run's. An untracked run's
* pane is closed only once the run has written its exit status, and only if the pane was started
* by the run's own script: a restarted tmux may have handed the id to one of the user's panes.
*/
export async function closeRunPane(handle: TmuxRunHandle, env: NodeJS.ProcessEnv): Promise<void> {
const state = await runPaneState(handle, env)
const finishedUntracked =
handle.runId === null &&
state === 'unknown' &&
isRunComplete(handle) &&
(await startedByRun(handle, env))
if (state !== 'ours' && !finishedUntracked) return
const killed = await runTmux(
handle.runId
? ifTagged(handle.pane, handle.runId, `kill-pane -t ${handle.pane}`)
: ['kill-pane', '-t', handle.pane],
env
)
if (!killed.ok) {
logger.warn('Could not close the tmux run window', { error: killed.stderr.trim() })
logger.warn('Could not close the tmux run pane', { error: killed.stderr.trim() })
}
}
apps/desktop/src/preload/index.ts
+8
View File
@@ -28,6 +28,7 @@ import type {
BrowserToolbarCommand,
DesktopAppearanceTheme,
DesktopCommand,
DesktopExecutorDevice,
DesktopLocalFileRequest,
DesktopLocalFileResponse,
DesktopNotificationPayload,
@@ -156,6 +157,7 @@ const api: SimDesktopApi = {
ipcRenderer.invoke('desktop:local-filesystem', request),
localFiles: (request: DesktopLocalFileRequest): Promise<DesktopLocalFileResponse> =>
ipcRenderer.invoke('desktop:local-files', request),
localReadClaims: true,
onCommand: (callback: (command: DesktopCommand) => void): (() => void) => {
const listener = (_event: unknown, command: DesktopCommand) => callback(command)
ipcRenderer.on('desktop:command', listener)
@@ -177,6 +179,8 @@ const api: SimDesktopApi = {
getPreferences: (): Promise<DesktopPreferences> => ipcRenderer.invoke('desktop:settings:get'),
setPreference: (key: DesktopPreferenceKey, value: boolean): Promise<DesktopPreferences> =>
ipcRenderer.invoke('desktop:settings:set', key, value),
setPreventSleepWhileRunning: (enabled: boolean): Promise<DesktopPreferences> =>
ipcRenderer.invoke('desktop:settings:set-prevent-sleep', enabled),
setBrowserSearchSuggestionsEnabled: (enabled: boolean): Promise<DesktopPreferences> =>
ipcRenderer.invoke('desktop:settings:set-browser-search-suggestions', enabled),
notify: (payload: DesktopNotificationPayload): Promise<boolean> =>
@@ -192,6 +196,10 @@ const api: SimDesktopApi = {
setTerminalDefaultZoom: (zoom: DesktopZoomPercent): Promise<DesktopPreferences> =>
ipcRenderer.invoke('desktop:settings:set-terminal-default-zoom', zoom),
},
desktopExecutor: {
getDevice: (): Promise<DesktopExecutorDevice | null> =>
ipcRenderer.invoke('desktop-executor:get-device'),
},
updates: {
getState: (): Promise<DesktopUpdateState> => ipcRenderer.invoke('desktop:updates:get-state'),
check: (): void => {
apps/desktop/src/test/electron-mock.ts
+5
View File
@@ -99,6 +99,10 @@ export const net = {
fetch: vi.fn(),
}
export const powerMonitor = {
on: vi.fn(),
}
export const session = {
fromPartition: vi.fn(),
}
@@ -393,6 +397,7 @@ export const electronMock = {
screen,
Menu,
net,
powerMonitor,
session,
protocol,
ipcMain,
apps/docs/.gitignore
+4
View File
@@ -41,3 +41,7 @@ next-env.d.ts
# fumadocs generates .source dirs anywhere a source.config sits
**/.source/
# Build-generated documentation asset
/public/llms-full.txt
/public/.llms-full-*/
apps/docs/app/api/og/route.tsx
+2 -6
View File
@@ -2,8 +2,6 @@ import type { CSSProperties } from 'react'
import { ImageResponse } from 'next/og'
import type { NextRequest } from 'next/server'
export const runtime = 'edge'
const TITLE_FONT_SIZE = {
large: 110,
medium: 96,
@@ -136,10 +134,8 @@ function wrapTitleLines(title: string, fontSize: number): string[] {
* Loads Söhne Kräftig (weight 500), the typeface used on the reference cover
* template this OG image matches. Converted to a plain TTF from the
* last-shipped `soehne-kraftig.woff2` since Satori (`next/og`'s renderer)
* can't parse WOFF2 or variable fonts. Fetched over HTTP since the edge
* runtime has no filesystem access — served from `/static/fonts/` (not
* `/fonts/`) so it isn't intercepted by the site's proxy (`proxy.ts`),
* whose matcher excludes `static` but not `fonts`.
* can't parse WOFF2 or variable fonts. Fetched over HTTP for Next's revalidation
* cache, from `/static/fonts/` so the site's proxy matcher excludes it.
*/
async function loadTitleFont(baseUrl: string): Promise<ArrayBuffer> {
const response = await fetch(new URL('/static/fonts/Soehne-Kraftig.ttf', baseUrl), {
apps/docs/app/llms-full.txt/route.ts
-27
View File
@@ -1,27 +0,0 @@
import { createLogger } from '@sim/logger'
import { getLLMText } from '@/lib/llms'
import { source } from '@/lib/source'
const logger = createLogger('DocsLlmsFullText')
export const revalidate = false
export async function GET() {
try {
const pages = source.getPages().filter((page) => Boolean(page?.data && page.url))
const scan = pages.map((page) => getLLMText(page))
const scanned = await Promise.all(scan)
const filtered = scanned.filter((text) => text && text.length > 0)
return new Response(filtered.join('\n\n---\n\n'), {
headers: {
'Content-Type': 'text/plain; charset=utf-8',
},
})
} catch (error) {
logger.error('Error generating LLM full text:', error)
return new Response('Error generating full documentation text', { status: 500 })
}
}
apps/docs/components/icons.tsx
+34 -2
View File
@@ -2811,7 +2811,7 @@ export function BufferIcon(props: SVGProps<SVGSVGElement>) {
return (
<svg
{...props}
viewBox='0 0 86.7 97.9'
viewBox='0 0 67 67'
fill='currentColor'
role='img'
xmlns='http://www.w3.org/2000/svg'
@@ -2819,7 +2819,7 @@ export function BufferIcon(props: SVGProps<SVGSVGElement>) {
<path
fillRule='evenodd'
clipRule='evenodd'
d='M0,22.3L43.1,0l43.6,22.3L43.1,44.8L0,22.3z M43.1,83.1l-29.4-16L0,74.5l43.1,23.4l43.6-23.4l-13.9-7.4 L43.1,83.1z M13.7,42l29.4,14.5L72.9,42l13.9,6.8L43.1,69.9L0,48.7L13.7,42z'
d='M18.009,43.225l15.178,7.817l15.378,-7.817l7.164,3.639l-22.542,11.469l-22.25,-11.469l7.072,-3.639Z M18.017,29.688l15.17,7.812l15.37,-7.812l7.172,3.645l-22.542,11.459l-22.25,-11.459l7.08,-3.645Z M33.187,8.333l-22.25,11.459l22.25,11.458l22.542,-11.458l-22.542,-11.459Z'
/>
</svg>
)
@@ -3560,6 +3560,30 @@ export function MicrosoftIcon(props: SVGProps<SVGSVGElement>) {
)
}
export function IntuneIcon(props: SVGProps<SVGSVGElement>) {
return (
<svg {...props} viewBox='0 0 24 24' fill='none' xmlns='http://www.w3.org/2000/svg'>
<rect x='3' y='3' width='18' height='13' rx='2' stroke='currentColor' strokeWidth='2' />
<path
d='M8 21h8M12 16v5m-4-12 3 3 5-5'
stroke='currentColor'
strokeWidth='2'
strokeLinecap='round'
strokeLinejoin='round'
/>
</svg>
)
}
export function RampIcon(props: SVGProps<SVGSVGElement>) {
return (
<svg {...props} viewBox='0 0 24 24' fill='none' xmlns='http://www.w3.org/2000/svg'>
<rect x='2' y='4' width='20' height='16' rx='3' stroke='currentColor' strokeWidth='2' />
<path d='M2 9h20M6 15h4' stroke='currentColor' strokeWidth='2' strokeLinecap='round' />
</svg>
)
}
export function MicrosoftCopilotIcon(props: SVGProps<SVGSVGElement>) {
const id = useId()
const gid = (name: string) => `mscopilot_${name}_${id}`
@@ -10124,3 +10148,11 @@ export function YouComIcon(props: SVGProps<SVGSVGElement>) {
</svg>
)
}
export function PlaneIcon(props: SVGProps<SVGSVGElement>) {
return (
<svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 85 52' fill='currentColor' {...props}>
<path d='M44.322 2.926C44.322 0.755 46.608 -0.658 48.551 0.313L80.455 16.265C82.929 17.502 84.492 20.032 84.492 22.799V48.249C84.492 50.42 82.207 51.833 80.265 50.862L62.328 41.893V22.797C62.328 20.031 60.765 17.502 58.291 16.264L44.322 9.28V2.926ZM0 2.925C0 0.754 2.286 -0.659 4.229 0.312L22.158 9.277V28.377C22.158 31.143 23.721 33.672 26.195 34.909L40.17 41.895V48.249C40.17 50.42 37.885 51.832 35.942 50.861L4.037 34.909C1.563 33.672 0 31.143 0 28.377V2.925ZM22.158 2.925C22.158 0.754 24.444 -0.659 26.387 0.312L44.322 9.28V28.378C44.322 31.144 45.886 33.673 48.36 34.91L62.328 41.893V48.249C62.328 50.42 60.043 51.832 58.101 50.861L40.17 41.895V22.797C40.17 20.031 38.607 17.502 36.133 16.264L22.158 9.277V2.925Z' />
</svg>
)
}
apps/docs/components/ui/icon-mapping.ts
+6
View File
@@ -148,6 +148,7 @@ import {
InfisicalIcon,
InstantlyIcon,
IntercomIcon,
IntuneIcon,
JinaAIIcon,
JiraIcon,
JiraServiceManagementIcon,
@@ -212,6 +213,7 @@ import {
PineconeIcon,
PipedriveIcon,
PitchBookIcon,
PlaneIcon,
PlanetScaleIcon,
PolymarketIcon,
PostgresIcon,
@@ -226,6 +228,7 @@ import {
QuiverIcon,
RabbitmqIcon,
RailwayIcon,
RampIcon,
RB2BIcon,
RDSIcon,
RedditIcon,
@@ -478,6 +481,7 @@ export const blockTypeToIconMap: Record<string, IconComponent> = {
instantly: InstantlyIcon,
intercom: IntercomIcon,
intercom_v2: IntercomIcon,
intune: IntuneIcon,
jina: JinaAIIcon,
jira: JiraIcon,
jira_service_management: JiraServiceManagementIcon,
@@ -558,6 +562,7 @@ export const blockTypeToIconMap: Record<string, IconComponent> = {
pinecone: PineconeIcon,
pipedrive: PipedriveIcon,
pitchbook: PitchBookIcon,
plane: PlaneIcon,
planetscale: PlanetScaleIcon,
polymarket: PolymarketIcon,
postgresql: PostgresIcon,
@@ -574,6 +579,7 @@ export const blockTypeToIconMap: Record<string, IconComponent> = {
quiver_v2: QuiverIcon,
rabbitmq: RabbitmqIcon,
railway: RailwayIcon,
ramp: RampIcon,
rb2b: RB2BIcon,
rds: RDSIcon,
reddit: RedditIcon,
apps/docs/content/docs/cli/credentials.mdx
+88
View File
@@ -37,6 +37,94 @@ Disconnect Credential (OAuth login or personal API key required)
</CommandTable>
## List credential members
```bash
sim credentials members list <credentialId> [options]
```
List Credential Members (OAuth login or personal API key required)
**Arguments**
<CommandTable>
| Argument | Required | Description |
| --- | --- | --- |
| `credentialId` | Yes | Credential whose sharing grants are managed. |
</CommandTable>
**Options**
<CommandTable>
| Option | Required | Description |
| --- | --- | --- |
| `--limit <n>` | No | Maximum items to return (0 for everything). Defaults to `0`. |
| `--sort-by <value>` | No | Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order. Accepted values: `email`, `name`. |
| `--sort-order <value>` | No | Sort direction. Accepted values: `asc`, `desc`. |
</CommandTable>
## Remove credential member
```bash
sim credentials members remove <credentialId> <userId> [options]
```
Remove Credential Member (OAuth login or personal API key required)
**Arguments**
<CommandTable>
| Argument | Required | Description |
| --- | --- | --- |
| `credentialId` | Yes | Credential whose sharing grants are managed. |
| `userId` | Yes | User whose explicit grant will be revoked. |
</CommandTable>
**Options**
<CommandTable>
| Option | Required | Description |
| --- | --- | --- |
| `-y, --yes` | Yes | Confirm this operation. |
</CommandTable>
## Upsert credential member
```bash
sim credentials members upsert <credentialId> [options]
```
Upsert Credential Member (OAuth login or personal API key required)
**Arguments**
<CommandTable>
| Argument | Required | Description |
| --- | --- | --- |
| `credentialId` | Yes | Credential whose sharing grants are managed. |
</CommandTable>
**Options**
<CommandTable>
| Option | Required | Description |
| --- | --- | --- |
| `--user <value>` | Yes | Existing workspace member to grant or change access for. |
| `--role <value>` | Yes | Credential role to grant; workspace administrators cannot be demoted. Accepted values: `admin`, `member`. |
</CommandTable>
## List credential providers
```bash
apps/docs/content/docs/cli/logs.mdx
+1 -1
View File
@@ -31,7 +31,7 @@ sim logs get <runId> [options]
| Option | Required | Description |
| --- | --- | --- |
| `--include-workflow-state` | No | Include the saved workflow snapshot (default: true). Set false to omit block configuration from a log read. Other run fields are unchanged. |
| `--include-workflow-state` | No | Include the saved workflow snapshot (default: true; Sim’s in-app agent leaves it out unless this flag is passed). Set false to omit block configuration from a log read. Other run fields are unchanged. |
| `--no-include-workflow-state` | No | Send --include-workflow-state as false. |
| `--trace` | No | Show expanded trace spans with inputs, outputs, errors, timing, and cost. |
apps/docs/content/docs/cli/organizations.mdx
+286
View File
@@ -7,6 +7,103 @@ import { CommandTable } from '@/components/ui/command-table'
Every command below also accepts the [global options](/cli/commands#global-options).
## Add organization domain
```bash
sim organizations domains add [options]
```
Add Organization Domain (OAuth login or personal API key required)
**Options**
<CommandTable>
| Option | Required | Description |
| --- | --- | --- |
| `--organization <value>` | Yes | Organization identifier. |
| `--domain <value>` | Yes | Domain to claim and verify through a DNS TXT record. |
</CommandTable>
## List organization domains
```bash
sim organizations domains list [options]
```
List Organization Domains (OAuth login or personal API key required)
**Options**
<CommandTable>
| Option | Required | Description |
| --- | --- | --- |
| `--organization <value>` | Yes | Organization identifier. |
| `--limit <n>` | No | Maximum items to return (0 for everything). Defaults to `0`. |
| `--sort-by <value>` | No | Field used to sort the result. Accepted values: `domain`. |
| `--sort-order <value>` | No | Sort direction. Accepted values: `asc`, `desc`. |
</CommandTable>
## Remove organization domain
```bash
sim organizations domains remove <domainId> [options]
```
Remove Organization Domain (OAuth login or personal API key required)
**Arguments**
<CommandTable>
| Argument | Required | Description |
| --- | --- | --- |
| `domainId` | Yes | Domain claim owned by this organization. |
</CommandTable>
**Options**
<CommandTable>
| Option | Required | Description |
| --- | --- | --- |
| `--organization <value>` | Yes | Organization identifier. |
| `-y, --yes` | Yes | Confirm this operation. |
</CommandTable>
## Verify organization domain
```bash
sim organizations domains verify <domainId> [options]
```
Verify Organization Domain (OAuth login or personal API key required)
**Arguments**
<CommandTable>
| Argument | Required | Description |
| --- | --- | --- |
| `domainId` | Yes | Domain claim owned by this organization. |
</CommandTable>
**Options**
<CommandTable>
| Option | Required | Description |
| --- | --- | --- |
| `--organization <value>` | Yes | Organization identifier. |
</CommandTable>
## Cancel organization access request
```bash
@@ -386,6 +483,195 @@ Revoke Organization Invitation (OAuth login or personal API key required)
</CommandTable>
## Delete SSO provider
```bash
sim organizations sso providers delete <providerId> [options]
```
Delete SSO Provider (OAuth login or personal API key required)
**Arguments**
<CommandTable>
| Argument | Required | Description |
| --- | --- | --- |
| `providerId` | Yes | Identity provider identifier. |
</CommandTable>
**Options**
<CommandTable>
| Option | Required | Description |
| --- | --- | --- |
| `--organization <value>` | Yes | Organization identifier. |
| `-y, --yes` | Yes | Confirm this operation. |
</CommandTable>
## Get SSO provider
```bash
sim organizations sso providers get <providerId> [options]
```
Get SSO Provider (OAuth login or personal API key required)
**Arguments**
<CommandTable>
| Argument | Required | Description |
| --- | --- | --- |
| `providerId` | Yes | Identity provider identifier. |
</CommandTable>
**Options**
<CommandTable>
| Option | Required | Description |
| --- | --- | --- |
| `--organization <value>` | Yes | Organization identifier. |
</CommandTable>
## List SSO providers
```bash
sim organizations sso providers list [options]
```
List SSO Providers (OAuth login or personal API key required)
**Options**
<CommandTable>
| Option | Required | Description |
| --- | --- | --- |
| `--organization <value>` | Yes | Organization identifier. |
| `--limit <n>` | No | Maximum items to return (0 for everything). Defaults to `0`. |
| `--sort-by <value>` | No | Field used to sort the result. Accepted values: `providerId`, `domain`. |
| `--sort-order <value>` | No | Sort direction. Accepted values: `asc`, `desc`. |
</CommandTable>
## Save SSO provider
```bash
sim organizations sso providers save [options]
```
Save SSO Provider (OAuth login or personal API key required)
**Options**
<CommandTable>
| Option | Required | Description |
| --- | --- | --- |
| `--organization <value>` | Yes | Organization identifier. |
| `--provider-type <value>` | Yes | oidc: Configure an OpenID Connect identity provider. saml: Configure a SAML identity provider. Accepted values: `oidc`, `saml`. |
| `--provider-id <value>` | Yes | Globally unique provider ID; saving an existing provider replaces its supplied configuration. |
| `--issuer <value>` | Yes | Identity provider issuer URL. |
| `--domain <value>` | Yes | Email domain already verified by this organization. |
| `--jit-provisioning-enabled` | No | Allow SSO sign-in to provision organization membership, subject to eligibility and available seats. |
| `--no-jit-provisioning-enabled` | No | Send --jit-provisioning-enabled as false. |
| `--mapping <json\|@file>` | No | Identity-provider claims mapped to user fields. (JSON, or @path / @- to read a file or stdin). |
| `--client-id <value>` | No | Identity provider client identifier. Available when providerType is oidc. Required when providerType is oidc. |
| `--client-secret <value\|@file>` | No | Write-only OIDC client secret; the redacted marker from providers get preserves an existing secret. Passing it inline exposes it to shell history and process listings. Required when --provider-type is oidc (@path / @- reads a file or stdin verbatim, including trailing newlines; @@value for a literal leading @). |
| `--scopes <json\|@file>` | No | OIDC scopes; offline_access is omitted. Available when providerType is oidc. (JSON, or @path / @- to read a file or stdin). |
| `--pkce` | No | Use PKCE for the authorization flow. Available when providerType is oidc. |
| `--no-pkce` | No | Send --pkce as false. |
| `--authorization-endpoint <value>` | No | Optional authorization endpoint; otherwise resolved through issuer discovery. Available when providerType is oidc. |
| `--token-endpoint <value>` | No | Optional token endpoint; otherwise resolved through issuer discovery. Available when providerType is oidc. |
| `--user-info-endpoint <value>` | No | Optional UserInfo endpoint. Available when providerType is oidc. |
| `--skip-user-info-endpoint` | No | Read identity claims from the ID token instead of calling UserInfo. Available when providerType is oidc. |
| `--no-skip-user-info-endpoint` | No | Send --skip-user-info-endpoint as false. |
| `--jwks-endpoint <value>` | No | Optional signing-key endpoint; otherwise resolved through issuer discovery. Available when providerType is oidc. |
| `--entry-point <value>` | No | Identity provider SAML sign-in endpoint. Available when providerType is saml. Required when providerType is saml. |
| `--cert <value>` | No | Identity provider signing certificate. Available when providerType is saml. Required when providerType is saml. |
| `--callback-url <value>` | No | SAML callback URL; defaults to this provider’s Sim callback. Available when providerType is saml. |
| `--audience <value>` | No | SAML audience; omission preserves the saved value. Available when providerType is saml. |
| `--want-assertions-signed` | No | Require signed assertions; omission preserves the saved value. Available when providerType is saml. |
| `--no-want-assertions-signed` | No | Send --want-assertions-signed as false. |
| `--signature-algorithm <value>` | No | Signature algorithm accepted by the SAML configuration validator; omission preserves the saved value. Available when providerType is saml. |
| `--digest-algorithm <value>` | No | Digest algorithm accepted by the SAML configuration validator; omission preserves the saved value. Available when providerType is saml. |
| `--identifier-format <value>` | No | SAML NameID format; omission clears the saved value. Available when providerType is saml. |
| `--idp-metadata <value>` | No | Identity provider metadata XML; omission clears the saved document. Available when providerType is saml. |
</CommandTable>
## Set primary SSO provider
```bash
sim organizations sso providers primary <providerId> [options]
```
Set Primary SSO Provider (OAuth login or personal API key required)
**Arguments**
<CommandTable>
| Argument | Required | Description |
| --- | --- | --- |
| `providerId` | Yes | Identity provider identifier. |
</CommandTable>
**Options**
<CommandTable>
| Option | Required | Description |
| --- | --- | --- |
| `--organization <value>` | Yes | Organization identifier. |
</CommandTable>
## Get SSO policy
```bash
sim organizations sso policy get [options]
```
Get SSO Policy (OAuth login or personal API key required)
**Options**
<CommandTable>
| Option | Required | Description |
| --- | --- | --- |
| `--organization <value>` | Yes | Organization identifier. |
</CommandTable>
## Update SSO policy
```bash
sim organizations sso policy update [options]
```
Update SSO Policy (OAuth login or personal API key required)
**Options**
<CommandTable>
| Option | Required | Description |
| --- | --- | --- |
| `--organization <value>` | Yes | Organization identifier. |
| `--require-sso <true\|false>` | Yes | Require organization SSO on future sign-ins; existing sessions remain active. Accepted values: `true`, `false`. |
</CommandTable>
## Get organization
```bash

Some files were not shown because too many files have changed in this diff Show More