mirror of
https://github.com/simstudioai/sim.git
synced 2026-09-24 15:45:35 +08:00
Theodore Li
24a6086d95
* feat(tables): add order_key column, fractional-indexing util, and ordering flag (off)
* feat(tables): write order_key on insert, flag-gate delete reindex + query ordering, add backfill
Flag off (default) = identical behavior. Single-insert assigns a fractional
order_key; queryRows orders by order_key when the flag is on; deletes skip the
O(N) reindex when on. Per-table-atomic backfill script populates existing rows.
* feat(tables): write order_key on all insert paths (batch, upsert, replace, import, create, copilot)
Completes the always-write-keys prerequisite: every row insert now assigns a
fractional order_key consistent with position order, so the flag can be flipped
safely after backfill. Flag off (default) still = identical behavior.
* feat(tables): insert-by-neighbor-id + orderKey on wire + client order-by-key
Inserts express intent as afterRowId/beforeRowId (O(1) key mint via the
(table_id,order_key,id) index); orderKey is returned on every row; client
reconcile/undo place by orderKey (no neighbor bump) with position fallback.
Flag off = unchanged. 205 table tests pass.
* feat(tables): resolve position-based inserts by key ordinal under the flag
Position-based callers (mothership tool, v1 API, undo fallback, transient old
clients) resolve their insert neighbor by order_key ordinal (OFFSET) when the
flag is on — positions are gappy then, so WHERE position=N would miss. Flag off
keeps the indexed position lookup. The mothership tool itself is unchanged.
* test(tables): flag-on coverage — delete skips reindex, insert mints key + no shift
* fix lint
* chore(db): regenerate order_key migration with default drizzle name
* fix(tables): address review — guard neighbor insert + mutual-exclusion + safe reconcile
- resolveInsertByNeighbor throws when the anchor row is missing (was silently
inserting at the front) and when its order_key is null under the flag.
- insert contract: afterRowId/beforeRowId are mutually exclusive (refine).
- reconcileCreatedRow only key-sorts when every cached row is keyed, so mid-
backfill un-keyed rows aren't yanked to the front.
* fix(kb): restore non-null guard in storage-key filter (unsafe-lint regression)
* refactor(tables): extract maxOrderKey + thread import append key
- Extract maxOrderKey(executor, tableId) helper; replaces three identical
max(order_key) selects (single/batch insert append + import).
- Import: read the append anchor once up front and thread each batch's last
key forward (nextImportStartOrderKey + afterOrderKey) instead of re-scanning
max(order_key) per batch over a growing table — one scan per import, not
one per 1k-row batch.
* fix(tables): keep insert body base omittable for v1 contract
The afterRowId/beforeRowId mutual-exclusion .refine() turned the schema into a
ZodEffects, which Zod forbids .omit() on — v1's insertTableRowBodySchema.omit({
position }) threw at module load (runtime-only; tsc misses it). Split the plain
object base out, apply the shared refine on top, and have v1 omit from the base
then re-apply it.
* fix(tables): chunk backfill order-key writes
A single UPDATE … FROM (VALUES …) over a whole large table overflows the JS call
stack while drizzle assembles the VALUES list (and would blow past Postgres's
65535 bound-param limit at ~32k rows) — large tables failed with 'Maximum call
stack size exceeded'. Write in 1000-row chunks inside the same per-table
transaction so keying stays atomic.
* fix(tables): emit orderKey in insert responses
The single-row and batch insert handlers dropped orderKey from the JSON
response even though the service returns it, so reconcileCreatedRow always fell
back to position-sorting and could place neighbor inserts wrong under the
fractional-ordering flag. Serialize orderKey alongside position.
* fix(tables): restore by orderKey, not position, under fractional flag
A saved position is the gappy column value, but under the flag insert reads
position as a visual rank (OFFSET) — so position-based restore misplaces rows.
- create-row redo now goes through the batch path carrying the saved orderKey
(the single-insert API has no orderKey field); drop the now-unused single
create mutation.
- resolveBatchInsertOrderKeys appends under the flag instead of feeding gappy
positions to resolveInsertOrderKey; positions remain the flag-off path.
* perf(tables): backfill writes 5000 rows/chunk (was 1000)
5x fewer round-trips per table; ~10k bound params stays well under Postgres's
65535 ceiling and far below the single-statement size that overflows the stack.
* fix(tables): drop rowNumber from table trigger payload
position is gappy under the fractional-ordering flag, so rowNumber (= row.position)
no longer reflects a contiguous visual rank. Rather than compute-on-read, remove
it from the trigger payload, output schema, and column-execution input.
Also pin isTablesFractionalOrderingEnabled=false in update-row.test.ts so its
flag-off position-shift assertions are deterministic regardless of local env.
* chore(db): format generated 0226 migration metadata
biome check . flagged the drizzle-generated _journal.json and 0226_snapshot.json;
apply the formatter so packages/db lint:check passes in CI.
* docs(triggers): drop rowNumber from table trigger outputs
rowNumber was removed from the table trigger payload; remove it from the
documented output fields to match.
* test(tables): remove flag-on fractional-ordering unit suite
Flag-on behavior is covered by manual large-table verification; the heavily-
mocked DB-chain suite added little signal.
The open-source AI workspace where teams build, deploy, and manage AI agents. Build conversationally, visually, or with code. Connect 1,000+ integrations and every major LLM to automate real work.
Build everything in Mothership
Your AI command center. Describe what you want in plain language. Mothership knows your entire workspace and takes action: building agents, running them, querying data, and more.
Create files and documents
Generate documents, reports, and presentations from a single prompt, grounded in your workspace data.
Ground agents in your knowledge
Upload documents to a knowledge base and let agents answer questions from your own content.
Structured data with Tables
A database, built in. Store, query, and wire structured data into agent runs.
Build visually with Workflows
Prefer a canvas? Design agents block by block in the visual builder, and let Copilot generate blocks, wire variables, and fix errors from natural language.
Quickstart
Cloud-hosted: sim.ai
Self-hosted: NPM Package
npx simstudio
Note
Docker must be installed and running on your machine.
Options
| Flag | Description |
|---|---|
-p, --port <port> |
Port to run Sim on (default 3000) |
--no-pull |
Skip pulling latest Docker images |
Self-hosted: Docker Compose
git clone https://github.com/simstudioai/sim.git && cd sim
docker compose -f docker-compose.prod.yml up -d
Sim also supports local models via Ollama and vLLM. See the Docker self-hosting docs for setup details.
Self-hosted: Manual Setup
Requirements: Bun, Node.js v20+, PostgreSQL 12+ with pgvector
- Clone and install:
git clone https://github.com/simstudioai/sim.git
cd sim
bun install
bun run prepare # Set up pre-commit hooks
- Set up PostgreSQL with pgvector:
docker run --name simstudio-db -e POSTGRES_PASSWORD=your_password -e POSTGRES_DB=simstudio -p 5432:5432 -d pgvector/pgvector:pg17
Or install manually via the pgvector guide.
- Configure environment:
cp apps/sim/.env.example apps/sim/.env
# Create your secrets
perl -i -pe "s/your_encryption_key/$(openssl rand -hex 32)/" apps/sim/.env
perl -i -pe "s/your_internal_api_secret/$(openssl rand -hex 32)/" apps/sim/.env
perl -i -pe "s/your_api_encryption_key/$(openssl rand -hex 32)/" apps/sim/.env
# DB configs for migration
cp packages/db/.env.example packages/db/.env
# Edit both .env files to set DATABASE_URL="postgresql://postgres:your_password@localhost:5432/simstudio"
- Run migrations:
cd packages/db && bun run db:migrate
- Start development servers:
bun run dev:full # Starts Next.js app and realtime socket server
Or run separately: bun run dev (Next.js) and cd apps/sim && bun run dev:sockets (realtime).
Copilot API Keys
Copilot is a Sim-managed service. To use Copilot on a self-hosted instance:
- Go to https://sim.ai → Settings → Copilot and generate a Copilot API key
- Set
COPILOT_API_KEYenvironment variable in your self-hosted apps/sim/.env file to that value
Environment Variables
See the environment variables reference for the full list, or apps/sim/.env.example for defaults.
Tech Stack
- Framework: Next.js (App Router)
- Runtime: Bun
- Database: PostgreSQL with Drizzle ORM
- Authentication: Better Auth
- Schema Validation: Zod
- UI: Shadcn, Tailwind CSS
- Streaming Markdown: Streamdown
- State Management: Zustand, TanStack Query
- Flow Editor: ReactFlow
- Docs: Fumadocs
- Monorepo: Turborepo
- Realtime: Socket.io
- Background Jobs: Trigger.dev
- Remote Code Execution: E2B
- Isolated Code Execution: isolated-vm
Contributing
We welcome contributions! Please see our Contributing Guide for details.
License
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
Made with ❤️ by the Sim Team