Files
sim/scripts/setup/wizard.ts
T
Theodore Li 19c3b6f47d feat(setup): setup wizard with browser-based Chat key handoff (#5911)
* feat(setup): setup wizard with browser-based Chat key handoff

Adds `bun run setup` and `bun run doctor` for local installs, and replaces
the wizard's paste-your-Chat-key step with a browser handoff that never puts
the key in a URL.

* improvement(setup): drop the paste-a-key fallback, simplify consent copy

The browser handoff is now the only path — the wizard waits on a spinner
instead of racing a paste prompt. Consent card leads with "Connect your
terminal" and moves the match-the-code disclaimer into the description.

* fix(setup): pin kube context, keep secrets out of argv, validate reused keys

Review findings from #5911:
- helm/kubectl now run against the validated context instead of the ambient one
- helm values are piped on stdin rather than passed as --set arguments
- ENCRYPTION_KEY/API_ENCRYPTION_KEY are checked for the 64-hex format the app
  requires, not just length, so an unusable key is replaced rather than kept
- the managed Redis container's published port is read back instead of assumed

* refactor(copilot): one module for Chat API key operations

list/generate/delete each repeated the same /api/validate-key envelope in
their route. They now share callValidateKey in lib/copilot/server/api-keys.ts,
which also keeps the display masking server-side so the full key can only ever
leave at creation.

* improvement(setup): reuse shared helpers, parallelize probes, drop dead code

- PKCE verifier/state/pairing code now use generateSecureToken, generateRandomHex
  and generateShortId instead of hand-rolled randomBytes; the pairing loop's
  modulo was unbiased only because 256 % 32 == 0
- new sha256Base64Url in @sim/security/hash so both sides of the PKCE exchange
  derive the challenge from one implementation
- isUsableSecret moved beside SECRET_KEYS so setup and doctor apply the same
  rule; doctor previously passed a key setup would replace
- isTruthy narrowed to true/1, matching the app it claims to mirror — it accepted
  yes/on, so a flag could read on in doctor and off in the app
- checkLive runs its five probes concurrently (~17s serial worst case)
- detection overlaps the banner animation instead of queueing behind it
- glyph.fail/glyph.warn at 13 sites that bypassed the constant; removed unused
  prompter exports, a dead ENV_PATHS re-export, and an unused export keyword

* fix(setup): make doctor understand the compose env layout

Compose writes a single root .env (what docker-compose reads via env_file) but
the checks required the three per-app files, so a successful compose install was
followed by doctor printing three failures and exiting 1 — and the whole
coherence catalog was skipped because it keyed off apps/sim/.env existing.

Layout is now derived from what's on disk and every check consults it: file and
schema checks iterate the layout's targets, consistency reports skip when
there's only one file to mirror, and coherence/live read the layout's primary
file. The wizard's existing-config detection counts root for the same reason —
a compose install used to read as unconfigured and re-run from scratch.

* feat(cli-auth): device-authorization poll flow, drop the loopback listener

The CLI no longer binds a local port. It generates a request id + poll secret,
opens /cli/auth, and polls /api/cli/auth/poll over TLS while the user approves
in the browser — so the flow works over SSH and inside containers, where the
browser and terminal don't share a machine.

- approve stores the approval keyed by request id (session-authed, userId from
  the session only); poll verifies the secret before an atomic claim, so an
  observer of the semi-public request id can neither mint nor cancel it
- pairing code stays as the anti-phishing compare; no key ever crosses the
  browser; done page just confirms
- removes the loopback listener, /token exchange, buildCliHandoffUrl, and
  validateCliCallbackUrl (+ its tests) — nothing hands a key to a URL anymore

* fix(setup): reuse an existing managed Postgres container instead of colliding

A running sim-postgres fell through to `docker run --name sim-postgres` and died
on the name conflict; a stopped one failed with "no DATABASE_URL to reach it"
because the generated password only lived in the env files a fresh clone lacks.

Both facts are recoverable from Docker: the ladder now reads the published port
and password back via `docker inspect` and reuses the container (starting it if
stopped). A container that won't answer prompts before recreating, and never
drops the data volume silently.

* improvement(setup): audience-first run-mode hints

Each run mode now names who it's for — compose for self-hosting/evaluating, dev
for contributing to Sim, k8s for rehearsing a production deploy — with the live
detection state (Docker/kube/VM) appended.

* fix(cli-auth): retry a failed mint, port container/port fixes to Redis + k8s

Review findings from #5911:
- poll now reserves the mint with an atomic NX lock instead of deleting the
  approval up front, so a failed mint (e.g. mothership blip) is retried by the
  next poll instead of forcing a fresh browser approval; the lock still prevents
  a double-mint and its TTL frees the slot if the caller dies
- setup reuses/recreates an unhealthy managed sim-redis instead of colliding on
  the name (Redis has no data volume, so it removes and recreates without a prompt)
- k8s failure-path hints carry --context, matching the success-path hints, so a
  changed ambient context can't send diagnostics to the wrong cluster
- compose port-free waits for a killed port to actually release before
  re-checking; SIGKILL is async, so the immediate re-check re-saw the port

* fix(setup): harden mint cleanup, Windows browser, container detection, helm cwd

Review findings from #5911:
- a post-mint completeApproval failure no longer routes into releaseMint — the
  mint lock now outlives the approval (shared TTL), so a cleanup blip can't leave
  a re-mintable window and orphan a key; cleanup is best-effort after the key ships
- compose doctor --fix writes the feature-flag twin to the layout's primary env
  (root .env on a compose install), not always apps/sim/.env
- Windows opens the browser via `cmd /c start "" <url>` — `start` is a shell
  builtin, so spawning it directly ENOENT'd and the handoff never opened
- managed-container detection filters loosely and pins the exact name in code;
  Docker's `name=^x$` anchor matches the internal `/x` form and often missed,
  skipping the reuse branch
- the shared helm/kind run helper pins cwd to the repo root, matching helm test,
  so `helm upgrade --install ./helm/sim` works from any working directory

* feat(chat-keys): standalone manage page, drop from settings nav, refresh README

- Add /account/settings/chat-keys — a linkable page to view, create, and revoke Chat API keys
- Remove Chat keys from the settings sidebar (account + unified nav) and its render branches
- README: replace Docker Compose + Manual Setup with the bun run setup wizard; drop the manual COPILOT_API_KEY step, point to the manage page

* fix(setup): per-key reason in the secret-replacement warning

Cursor: the warn hardcoded '64-character hex key', but only ENCRYPTION_KEY/API_ENCRYPTION_KEY require that — BETTER_AUTH_SECRET/INTERNAL_API_SECRET only need length >= 32. Use the existing secretRequirement(key) helper so each replaced key reports its actual requirement.

* fix(setup): compose doctor schema, cross-platform binary detection, quoted context hints

- Doctor: for the compose (root) env layout, require only the secrets compose has no interpolation default for (BETTER_AUTH_SECRET/ENCRYPTION_KEY/INTERNAL_API_SECRET). DATABASE_URL/BETTER_AUTH_URL/NEXT_PUBLIC_APP_URL come from docker-compose ${VAR:-default}, so a healthy compose install no longer fails doctor.
- Binary detection: use Bun.which instead of which (which is absent on Windows), so kubectl/helm/kind/docker resolve cross-platform.
- k8s diagnostic hints: POSIX-quote the kube-context so a context with whitespace/metacharacters can't break or inject into a copied command.

* fix(setup): quote kube-context in the helm uninstall tear-down hint too

The tear-down hint used --kube-context ${context} raw while the sibling kubectl hints already used shq(); a context with whitespace/metacharacters could break or inject into the copied command. All copyable k8s hints now go through shq(context).

* feat(setup): sim lifecycle CLI — start/stop/status/logs/down/reset

Turn the setup entry into a 'sim' command umbrella so there's one place to run everything, not scattered docker/bun commands. Adds a global bin (bun link) + a bun run sim fallback.

- Detects how you're running (compose file / managed dev containers / helm release) from disk + docker/helm state — no persisted mode. Ambiguous installs prompt.
- start/stop/restart/logs work per mode; down removes containers (volumes kept); reset archives .env + wipes managed data; both destructive verbs confirm first.
- status shows detected mode, container states, and app/realtime health.
- Wizard outro + README now point at the sim commands and the one-time bun link.

* feat(setup): 'bun run sim' is the primary entry; bare invocation prints help

- Lead usage/wizard-outro/README with 'bun run sim <cmd>' (works with zero PATH setup); global bare 'sim' via bun link is an optional upgrade, with the ~/.bun/bin PATH caveat spelled out (Homebrew's bun omits it).
- Bare 'sim' now prints help instead of launching the wizard; the wizard is 'sim setup'. The 'setup' npm script passes the keyword so 'bun run setup' is unchanged.

* fix(setup): quote the auth URL for cmd /c start on Windows

Cursor (High): cmd re-parses the command line and treats & in the query string as a command separator, so cmd /c start opened a URL truncated at the first &, breaking the key flow on win32 (the handoff URL always has request/challenge/pairing). Quote the URL and pass args verbatim so & stays literal.

* fix(setup): verify kube-context is really local; lengthen CLI handoff wait

- k8s: a context named like a local cluster (kind-*, docker-desktop) can actually point at a remote API server. Verify the server host is loopback/docker-internal before defaulting the 'use this context?' confirm to yes; otherwise warn and default to no, so generated secrets can't ship to a remote cluster on a blind Enter.
- cli-auth: bump the device-flow wait from 3 to 15 minutes so first-time users have time to sign up, wait for the email OTP, and approve before the terminal stops polling. The server-side approval record keeps its own short TTL, so a longer client wait only costs cheap rate-limited polls.

* fix(setup): only manage k8s lifecycle on a verified-local context

Greptile: sim down/reset used the ambient kube-context, so switching context after setup could uninstall a same-named sim-dev release from the wrong cluster. Gate k8sInstall on the same locality check the wizard uses (API server is loopback/docker-internal) via a shared isLocalKubeContext helper — the wizard only ever deploys locally, so a remote current-context is never treated as a Sim install.

* fix(setup): doctor skips placeholder secrets when seeding; reset names its target

- checks: the missing-file autofix copied shared keys from apps/sim/.env whenever truthy, including .env.example placeholders — doctor --fix could seed unusable secrets into realtime/db env files. Skip placeholders, matching autofixForMissing.
- lifecycle: reset now names the exact install (k8s context / compose file / dev containers) in its confirm, so a destructive reset can't silently hit the wrong same-named install after a context switch (down already names the context).

* fix(cli-auth): size the poll rate limit to the poll cadence; honor Retry-After

The poll route used the default public-IP bucket (10 burst, 5/min) but the CLI polls every 2s (30/min), so it 429'd within ~20s — worse behind a slow dev cold-compile. Give the endpoint a bucket matched to its cadence (60 burst, 60/min); it's not a brute-force surface (unknown request id returns pending, minting needs the 256-bit verifier). Also make the CLI honor Retry-After and back off on 429 so a shared-NAT per-IP limit degrades gracefully instead of hammering.

* fix(setup): check ports before starting the dev server, not just compose

Local dev auto-start spawned bun run dev:full with no port check, so it silently started a server that couldn't bind when 3000/3002 were already taken (e.g. another worktree's dev server). Extract compose's port-conflict resolver into a shared ensurePortsFree(ports) and run it before the dev start too — kill/recheck/leave, same as compose. Leaving the ports skips the auto-start with guidance instead of failing; compose still treats it as fatal.

* fix(setup): verify the kube cluster is reachable, not just local

A kubeconfig context can outlive its cluster — a kind cluster gets deleted or its Docker container stops (Docker/machine restart), but the context entry remains, pointing at a dead API-server port. The wizard checked the context looked local and handed it to helm, which failed with 'cluster unreachable'.

Add a clusterReachable() liveness probe: only offer the current context when it actually answers; if a local context is dead, fall through to the kind path. There, if kind still knows 'sim' but it's stopped, start its node containers and wait for the API; if it's gone, create fresh. Either way the user gets a working cluster instead of a cryptic helm failure.

* fix(helm): point appVersion at published image tags (v-prefixed, current)

The chart's appVersion was "0.6.73", but CI publishes GHCR tags with a v prefix (its release-commit regex captures v0.7.45). Since sim.image defaults every image tag to Chart.AppVersion, a default helm install requested ghcr.io/simstudioai/{simstudio,realtime,migrations}:0.6.73 — a tag that has never existed — so app and realtime sat in ImagePullBackOff and helm --wait failed with 'progress deadline exceeded'. Any self-hoster installing with default values hit this, not just the setup wizard.

Set appVersion to v0.7.45 (latest release on main; all three images verified present on ghcr) and bump the chart version to 1.1.1. Verified with helm lint, helm template (all images render as v0.7.45), and a live helm upgrade on a kind cluster where the new pods pull successfully while the old 0.6.73 pods remain in ImagePullBackOff.

* Revert "fix(helm): point appVersion at published image tags (v-prefixed, current)"

This reverts commit 28b6047d1d.

* chore(api-validation): rebaseline route count to 977 after staging merge

Staging moved the baseline to 975; this branch's two CLI-auth routes (approve, poll) make 977. The clean merge absorbed the earlier +2 adjustment.

* fix(settings): don't highlight a sibling nav item on nested settings pages

/account/settings/chat-keys is a real page but deliberately not a nav item, so the sidebar's parseSettingsPathSection fell through to defaultSection ('general') and highlighted General — the page read as though it lived inside General.

Resolve the sidebar's active item with a null default so an unmatched nested route highlights nothing, and widen SettingsSidebar's activeSection to string | null. The section feeding the title/description provider keeps its default (pages override title/description anyway), and /account/settings/billing/credit-usage still correctly highlights Billing.

* fix(setup,auth): manage explicitly-confirmed k8s contexts, fail loudly on reset, clear stale post-auth redirect

- lifecycle: detection is now factual — a sim-dev release either exists on the current context or it doesn't. Gating on locality stranded a release the user explicitly confirmed during setup (status/start/stop/down/reset all claimed no k8s install). Locality is recorded instead and surfaced through describeInstall, which every destructive confirm renders, so acting on a non-local cluster is named and defaulted to no rather than silently blocked or silently allowed.
- lifecycle: reset no longer discards helm uninstall's exit status. Env files are archived by that point, so claiming 'Reset complete' while the release still runs is the worst outcome — it now throws with retry/inspect commands.
- auth: signup clears POST_AUTH_REDIRECT_STORAGE_KEY when it has no callbackUrl, and the verification-disabled path consumes it, so a stale CLI/invite destination can't leak into a later flow in the same tab.
2026-07-25 04:24:36 -04:00

194 lines
7.1 KiB
TypeScript

import { spawnSync } from 'node:child_process'
import { showBanner } from './banner.ts'
import { loadCheckContext, runChecks } from './checks.ts'
import { type Detection, runDetection } from './detect.ts'
import { archiveEnvFile, ROOT } from './env-files.ts'
import { runComposeMode } from './modes/compose.ts'
import { runDevMode } from './modes/dev.ts'
import { runK8sMode } from './modes/k8s.ts'
import { ensurePortsFree } from './ports.ts'
import * as p from './prompter.ts'
import { glyph, theme } from './theme.ts'
export type WizardMode = 'compose' | 'dev' | 'k8s'
export interface WizardFlags {
quick: boolean
mode?: WizardMode
}
async function handleExistingConfig(detection: Detection): Promise<'continue' | 'doctor'> {
// Root counts: a compose install writes only `.env`, so excluding it made a
// configured machine look unconfigured and silently re-run from scratch.
const present = Object.entries(detection.envFiles)
.filter(([, exists]) => exists)
.map(([target]) => (target === 'root' ? '.env' : `${target}/.env`))
if (present.length === 0) return 'continue'
const choice = await p.select({
message: `Found existing config (${present.join(', ')}) — what should we do?`,
options: [
{ value: 'keep', label: 'Keep it', hint: 'run doctor against the current setup and exit' },
{
value: 'review',
label: 'Review and update',
hint: 'walk the wizard with current values prefilled',
},
{ value: 'reset', label: 'Reset', hint: 'archive current .env files and start fresh' },
],
initialValue: 'keep',
})
if (choice === 'keep') return 'doctor'
if (choice === 'reset') {
for (const target of ['sim', 'realtime', 'db', 'root'] as const) {
const backup = archiveEnvFile(target)
if (backup) p.log.step(`Archived ${backup}`)
}
}
return 'continue'
}
const LOW_DOCKER_MEM_GB = 6
const LOW_DISK_GB = 15
async function selectMode(detection: Detection, flags: WizardFlags): Promise<WizardMode> {
if (flags.mode) return flags.mode
const { dockerMemGb } = detection.specs
const vm = dockerMemGb !== null ? ` · VM ${dockerMemGb}GB` : ''
const composeState = detection.dockerRunning ? `Docker ready${vm}` : 'Docker is NOT running'
const k8sState = detection.kubeContext
? `context: ${detection.kubeContext}${vm}`
: detection.binaries.kind
? `kind available${vm}`
: 'needs kind or Docker Desktop k8s'
return p.select({
message: 'How do you want to run Sim?',
options: [
{
value: 'compose',
label: 'Docker Compose',
// Self-host or just try Sim without touching the code.
hint: `Run a bundled Sim — self-hosting or evaluating · ${composeState}`,
},
{
value: 'dev',
label: 'Local dev (bun run dev:full)',
// Iterate on the source with hot reload.
hint: 'Work on Sim itself — contributing · app :3000 + realtime :3002',
},
{
value: 'k8s',
label: 'Kubernetes (helm)',
// Rehearse a real cluster deploy on kind / Docker Desktop.
hint: `Test a production-style deploy — self-hosting on k8s · ${k8sState}`,
},
],
initialValue: detection.dockerRunning ? 'compose' : 'dev',
})
}
/** Spec warnings before committing to a mode — informed choice, no silent degradation. */
function warnLowSpecs(detection: Detection, mode: WizardMode): void {
const { dockerMemGb, freeDiskGb } = detection.specs
if (
(mode === 'compose' || mode === 'k8s') &&
dockerMemGb !== null &&
dockerMemGb < LOW_DOCKER_MEM_GB
) {
p.log.warn(
`Docker's VM has only ${dockerMemGb}GB of memory${mode === 'k8s' ? ' — pods will likely sit Pending or OOM' : ' — containers may OOM'}. Raise it in Docker Desktop → Settings → Resources (8GB+ recommended).`
)
}
if ((mode === 'compose' || mode === 'k8s') && freeDiskGb !== null && freeDiskGb < LOW_DISK_GB) {
p.log.warn(`Only ${freeDiskGb}GB of free disk — image pulls need 5-10GB.`)
}
}
async function finalVerify(): Promise<void> {
const findings = await runChecks(loadCheckContext(true), ['live'])
for (const finding of findings) {
console.log(` ${glyph[finding.status]} ${finding.message}`)
}
}
export async function runWizard(flags: WizardFlags): Promise<void> {
// Detection is ~600ms of subprocess work and the banner is ~600ms of animation
// with nothing to do — overlap them so the spinner below usually resolves at once.
const detecting = runDetection()
await showBanner()
const spin = p.spinner()
spin.start('Looking at what you already have…')
const detection = await detecting
spin.stop(
`Detected: docker ${detection.dockerRunning ? '✓' : '✗'} · postgres ${detection.postgresPortOpen ? '✓' : '✗'} · ` +
`${detection.shellLlmKeys.length} shell LLM key${detection.shellLlmKeys.length === 1 ? '' : 's'}` +
(detection.kubeContext ? ` · kube: ${detection.kubeContext}` : '')
)
if ((await handleExistingConfig(detection)) === 'doctor') {
const { runDoctor } = await import('./doctor.ts')
process.exitCode = await runDoctor({ fix: false, json: false })
return
}
const quick =
flags.quick ||
(await p.select({
message: 'Setup style?',
options: [
{ value: 'quick', label: 'Quick', hint: 'sensible defaults, minimal questions' },
{
value: 'custom',
label: 'Custom',
hint: 'every option: redis, trigger.dev, image variants',
},
],
initialValue: 'quick',
})) === 'quick'
const mode = await selectMode(detection, flags)
warnLowSpecs(detection, mode)
let startDevNow = false
let devScript = 'dev:full'
if (mode === 'compose') await runComposeMode(detection, quick)
else if (mode === 'dev') {
const dev = await runDevMode(detection, quick)
startDevNow = dev.startNow
devScript = dev.script
} else await runK8sMode(detection)
if (mode !== 'k8s' && !startDevNow) await finalVerify()
const url = 'http://localhost:3000'
p.note(
[
mode === 'k8s' ? `port-forward, then open ${url}` : `open ${url}`,
'manage it: bun run sim start · stop · status · logs',
'check your setup: bun run sim doctor',
mode === 'dev' && !startDevNow ? `start Sim: bun run ${devScript}` : null,
`prefer a bare "sim"? ${theme.command('bun link')} once (needs ~/.bun/bin on PATH)`,
]
.filter(Boolean)
.join('\n'),
'Next steps'
)
p.outro(theme.accent('Sim is ready.'))
if (mode === 'compose' && process.platform === 'darwin') {
spawnSync('open', [url], { stdio: 'ignore' })
}
if (startDevNow) {
// dev:full binds 3000 (app) and 3002 (realtime) — resolve any conflict
// (e.g. another worktree's dev server) before starting, instead of spawning
// a server that fails to bind. If the user leaves the ports, skip the start.
if (await ensurePortsFree([3000, 3002])) {
const child = spawnSync('bun', ['run', devScript], { cwd: ROOT, stdio: 'inherit' })
process.exitCode = child.status ?? 0
} else {
p.log.warn(
`Ports still in use — Sim wasn't started. Free them, then run ${theme.command(`bun run ${devScript}`)}.`
)
}
}
}