mirror of
https://github.com/simstudioai/sim.git
synced 2026-09-01 14:59:19 +08:00
Bill Leoutsakos
3b4d9e9149
* fix schedule execution recovery * simplify schedule recovery provider handling * fix(schedules): stop reconciliation rewriting healthy carriers and index its scan Follow-up hardening on the schedule recovery reconciliation. Reconciliation settled every carrier it examined, including ones that had already reached a terminal status and recorded their own outcome. Because the normal completion path never stamps the reconciled marker, each successful run's carrier was picked up on the next tick and rewritten: `completedAt` bumped, `error` nulled and `output` replaced with a recovery stub, all of which `GET /api/jobs/[jobId]` surfaces. A completed carrier whose execution log had aged out was additionally flipped to failed. Settle only carriers still in flight; a terminal one is owed schedule accounting and the marker, nothing more. The recovery scan matched no index. There was none on `async_jobs.updated_at`, and the unreconciled-terminal branch tested a jsonb extraction, so the whole OR fell back to a sequential scan and sort of `async_jobs` on every tick. Add the partial index, and spell the branch's status list and metadata key as SQL literals: Postgres cannot prove a parameterised predicate implies a literal index predicate, so a bound key would have left the new index unused. Irrecoverable carrier tombstones were exempted from retention with no secondary expiry, so the one class of row that can never reconcile grew without bound. Give them a longer bounded window instead. `WORKFLOW_EXECUTION_MAX_ROWS_PER_RUN` had become a per-status budget when the stale-execution sweep gained its `redacting` pass, silently doubling the per-run cap. Share the budget across both passes, and add the matching partial indexes so the second pass keeps an index rather than seq-scanning. Also consolidates the carrier metadata keys, their predicates and the jsonb merge into one module -- the two routes were the writer and reader of the same keys with no shared symbol, so a rename type-checked clean while silently breaking retention. * fix(schedules): rotate deferred carriers and stop the redacting sweep failing live runs Addresses the review findings on the reconciliation hardening. A carrier whose accounting was deferred got no write at all, so it kept its `updatedAt` and stayed at the head of the `updatedAt`-ordered recovery batch on every tick, starving every other claimed carrier and never becoming eligible for retention. This was a regression from the previous commit: settling used to bump the timestamp for every examined row, and skipping the settle for already-terminal carriers removed the bump with it. A payload with no `scheduledFor` can never reconcile, so such a row pinned a batch slot permanently. Bump `updatedAt` unconditionally and keep only the reconciled marker conditional. The stale-execution sweep terminalized `redacting` logs on the execution deadline. That deadline bounds execution, while `redacting` covers payload masking after the run already finished -- so a run that used most of its budget entered redaction with the deadline due, and the sweep failed it five minutes later while the worker was still masking. Schedule recovery then read the log as a failed occurrence and counted a failure that never happened, even though the worker's terminal write later restored `completed`. Sweep `redacting` on the generic stale window only. `getScheduleNextRunAt` falls back to a daily cadence when a schedule has no cron expression. Deployment cannot persist such a schedule, so the branch is unreachable, but this change widened its use from failure recovery to every outcome -- log a warning when it fires rather than silently guessing a cadence. `executionDeadlineAt` was missing from the shared `workflowExecutionLogs` schema mock, so it read as `undefined` and assertions comparing against that column were trivially true. Add it. * refactor(schedules): drop vestigial recovery code and close a builder gap Follow-ups from a full re-read of the change. No behavior change except the removed dead code paths. The metadata merge stripped a `scheduleRecoveryBlocked` key on every write. That key has never been written by any shipped code -- it appears nowhere in staging and nowhere in history outside this branch -- so the strip guarded against a state that cannot exist, at the cost of an extra jsonb operation and a bind parameter on every reconciliation write. `processScheduleItem` set `carrierObservedOrLookupUncertain` immediately before returning on an ambiguous enqueue. The flag is only read from the surrounding catch block, which a normal return skips, so the assignment was dead and read as though it were load-bearing. Replaced with a comment stating why the occurrence is preserved. The stale-execution sweep carried two near-identical `jsonb_set` templates that differed only in their error expression, kept flat because the test mock renders nested SQL fragments as placeholders. The suite now has a recursive renderer, so the error expression is a named per-status value and there is one `jsonb_set`. Success was the only schedule outcome without a named update builder, which left `executeScheduleJob` using two idioms for the same guarded write and left the update shape untested. Add `buildScheduleSuccessUpdate` beside its cancellation and failure siblings, use it from both call sites, and cover it the way `buildScheduleCancellationUpdate` is covered -- a mutation of its `failedCount` reset previously passed every suite. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com>
A workspace to build, deploy and manage AI agents and workflows.
Quickstart
Cloud-hosted: sim.ai
Self-hosted
npx sim-setup
Capabilities
- Connect 1,000+ integrations and every major LLM
- Add Slack, Notion, HubSpot, Salesforce, databases, and more
- Build agents visually, conversationally, or with code
- Ingest files, knowledge bases, and structured table data
- Monitor runs, logs, schedules, and workflow activity
One workspace, every surface
Chat and workflows are just the start — tables, files, and knowledge all live in the same workspace.
Tables — a database, built in |
Files — one store for your team and every agent |
Knowledge — your agents' memory |
Self-hosting
Requirements: Node.js 20+ and Docker.
npx sim-setup is an interactive wizard that creates a small sim/ deployment directory, provisions the database, generates secrets, writes .env, connects a Chat API key, and starts the published Sim images with Docker Compose. It does not clone the repository.
When it finishes, open http://localhost:3000.
Inside a cloned Sim repository, run bun run sim-setup to unlock the source-only local development and Kubernetes modes.
Reconfigure an optional capability without rerunning the full wizard:
npx sim-setup config
npx sim-setup add email
npx sim-setup add storage
npx sim-setup add sandbox
npx sim-setup add jobs
npx sim-setup add cache
npx sim-setup add knowledge
npx sim-setup add llm
npx sim-setup add integration slack
npx sim-setup config detects the effective local-dev, Docker Compose, or current-context
Helm configuration and reports configured, missing, or invalid capabilities and OAuth
integrations without printing credential values. This is separate from npx sim-setup status,
which reports whether installed services are running and healthy.
Manage your install from its directory:
npx sim-setup start | stop | restart # bring your install up / down / cycle
npx sim-setup update # pull and apply Compose images
npx sim-setup status # what's installed and healthy
npx sim-setup logs # follow logs
npx sim-setup doctor # diagnose configuration problems
npx sim-setup down # remove containers (data kept)
npx sim-setup reset # archive .env and wipe managed data
The setup package detects how you're running and acts accordingly. Use --dir <path> to create or manage a deployment somewhere other than ./sim.
Sim also supports local models via Ollama and vLLM. See the self-hosting docs for details.
Chat API Keys
Chat is a Sim-managed service. npx sim-setup connects a Chat API key for you — sign in when it opens your browser and the key is stored automatically. To view, create, or revoke keys later, go to sim.ai/selfhost/settings/chat-keys.
Environment Variables
See the environment variables reference for the full list, or apps/sim/.env.example for defaults.
Tech Stack
Next.js · Bun · PostgreSQL · Drizzle · Better Auth · Tailwind — and the rest of the stack
- Framework: Next.js (App Router)
- Runtime: Bun
- Database: PostgreSQL with Drizzle ORM
- Authentication: Better Auth
- Schema Validation: Zod
- UI: Shadcn, Tailwind CSS
- Streaming Markdown: Streamdown
- State Management: Zustand, TanStack Query
- Flow Editor: ReactFlow
- Docs: Fumadocs
- Monorepo: Turborepo
- Realtime: Socket.io
- Background Jobs: Trigger.dev
- Remote Code Execution: E2B
- Isolated Code Execution: isolated-vm
Contributing
We welcome contributions! Please see our Contributing Guide for details.
License
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
