mirror of
https://github.com/simstudioai/sim.git
synced 2026-09-24 15:45:35 +08:00
Waleed
cc56408be3
* perf(execution): parallelize preflight gates, cache deployed state, memoize Anthropic client
- Memoize Anthropic + Azure-Anthropic SDK clients (new client-cache.ts) keyed
by apiKey (+beta header; +baseURL/version/pinnedIP for Azure) so HTTP
keep-alive connections are reused instead of a fresh TLS handshake per call.
apiKey is the tenant boundary.
- Parallelize the read-only preflight gates in preprocessing.ts (ban +
subscription, then usage + org-member + rate-limit) while preserving exact
error precedence (ban 403 -> usage 402 -> rate 429) and keeping the sole
write (admission reservation) last.
- Parallelize the independent workflow-state and env-var loads in execution-core.
- Cache deployed workflow state by immutable deploymentVersionId with
deep-clone-on-read, oldest-first eviction, and a 5-min TTL bounding the
credential-mapping edge across ECS tasks.
- Parallelize the independent personal-subscription + membership queries in
getHighestPrioritySubscription.
- BYOK: drop the redundant getWorkspaceById existence check (auth already
validates the workspace); read the key list fresh every call for zero
cross-instance staleness.
Billing/usage/ban/permission reads stay fresh on the primary (no cache, no
replica). Adds tests for every new mechanism and fixes a pre-existing vitest
class-mock incompatibility that had execution-core.test.ts fully red on staging.
* fix(execution): run rate-limit gate only after ban/usage pass
The rate-limit gate is not read-only — checkRateLimitWithSubscription consumes
a token — so running it in parallel with the read-only gates debited rate-limit
quota for requests that the ban (403) or usage (402) gates reject, which the
original sequential flow never did.
Move the rate-limit gate to run sequentially after the ban and usage gates pass,
preserving the read-only gates' parallelism (ban + subscription + usage) and the
exact ban -> usage -> rate precedence. Add regression tests asserting the rate
limiter is not consumed when an earlier gate rejects, and is consumed once when
they pass.
Caught by Cursor Bugbot review.
* chore(execution): trim redundant preflight comments
Tighten the gate overview to match the sequential rate-limit gate and drop
inline notes that duplicated it or the runRateLimitGate doc.
* refactor(cache): address review — idle TTL for client cache, LRUCache for deployed state
- client-cache: add updateAgeOnGet so the TTL is genuinely idle-based (active
clients keep their warm keep-alive connections; the JSDoc now matches behavior).
- deployed-state: replace the hand-rolled Map + manual FIFO eviction/TTL with
LRUCache (real LRU eviction, built-in TTL), matching the effectiveDecryptedEnv
and integration-tool-schema caches. TTL stays absolute (not reset on read) so
the credential-migration remap still propagates across ECS tasks.
Both per review feedback from Greptile.
* test(execution): isolate rate-limit gate test from STEP 7 reservation
The 'consumes the rate-limit gate once' test reached the STEP 7 admission
reservation, which depends on Redis — it passed locally (reserve throws and is
swallowed) but failed in CI (reserve returns not-reserved -> 429). Pass
skipConcurrencyReservation so the test isolates the rate gate deterministically.
* perf(providers): memoize SDK clients where the pool is per-client (bedrock, vllm)
Generalize the Anthropic client cache into one shared memoizer
(providers/client-cache.ts) and apply it only where each new client owns its own
connection pool — so reuse actually keeps connections warm:
- bedrock: AWS SDK clients hold a per-client connection pool (reuse is the AWS
best practice). Keyed by region + credential identity.
- vllm: a pinned endpoint creates its own undici Agent per call; key by the
resolved IP so DNS re-validation still runs each request.
- anthropic + azure-anthropic: migrated onto the shared memoizer.
Deliberately NOT applied to the OpenAI-compatible providers, groq, cerebras, or
google: their SDKs share a process-global keep-alive pool (Node openai-sdk module
singleton agent; anthropic/global undici), so a fresh client per request already
reuses connections and memoization would add complexity with ~no benefit. litellm
uses a plain shared-agent client (no pinning) and is likewise skipped.
Bounded LRU (max 1000, 30m idle TTL) with no close-on-eviction, avoiding the
unbounded-growth and eviction-closes-in-use-client failure modes seen in similar
client caches.
* chore(perf): trim verbose comments to terse why-notes
* chore(perf): drop obvious inline comments, keep nuance as TSDoc
* fix(bedrock): key client cache on full credential, not just access key id
A corrected secret under the same access key id would otherwise keep serving the
stale cached client until TTL/eviction. Caught by Cursor Bugbot.
* test(execution,providers): fix preflight mock reset + isolate provider client cache in tests
- preprocessing.test: re-establish the checkOrgMemberUsageLimit mock in beforeEach
(the only gate mock not re-set). In the full suite its implementation was reset
so the success-path test got undefined -> threw -> 500 -> success:false. Mirrors
how checkServerSideUsageLimits is handled.
- client-cache: add clearProviderClientCacheForTests; call it in the bedrock and
vllm test beforeEach so construction assertions always start from a cache miss
now that those providers memoize their client.
* test(execution): make RateLimiter mock constructable under vitest 4.x
The RateLimiter mock used an arrow factory (vi.fn(() => ({...}))). vitest 4.x
(CI) rejects `new` on an arrow-implemented mock ("not a constructor"); 3.2.4
allowed it. The new rate-gate test is the first to actually `new RateLimiter()`,
so it surfaced the failure only in CI. Switch the mock to a regular function and
drop the speculative beforeEach re-establishments that didn't address it.
The open-source AI workspace where teams build, deploy, and manage AI agents. Build conversationally, visually, or with code. Connect 1,000+ integrations and every major LLM to automate real work.
Build everything in Chat
Your AI command center. Describe what you want in plain language. Sim knows your entire workspace and takes action: building agents, running them, querying data, and more.
Create files and documents
Generate documents, reports, and presentations from a single prompt, grounded in your workspace data.
Ground agents in your knowledge
Upload documents to a knowledge base and let agents answer questions from your own content.
Structured data with Tables
A database, built in. Store, query, and wire structured data into agent runs.
Build visually with Workflows
Prefer a canvas? Design agents block by block in the visual builder, and let Sim generate blocks, wire variables, and fix errors from natural language.
Quickstart
Cloud-hosted: sim.ai
Self-hosted: NPM Package
npx simstudio
Note
Docker must be installed and running on your machine.
Options
| Flag | Description |
|---|---|
-p, --port <port> |
Port to run Sim on (default 3000) |
--no-pull |
Skip pulling latest Docker images |
Self-hosted: Docker Compose
git clone https://github.com/simstudioai/sim.git && cd sim
docker compose -f docker-compose.prod.yml up -d
Sim also supports local models via Ollama and vLLM. See the Docker self-hosting docs for setup details.
Self-hosted: Manual Setup
Requirements: Bun, Node.js v20+, PostgreSQL 12+ with pgvector
- Clone and install:
git clone https://github.com/simstudioai/sim.git
cd sim
bun install
bun run prepare # Set up pre-commit hooks
- Set up PostgreSQL with pgvector:
docker run --name simstudio-db -e POSTGRES_PASSWORD=your_password -e POSTGRES_DB=simstudio -p 5432:5432 -d pgvector/pgvector:pg17
Or install manually via the pgvector guide.
- Configure environment:
cp apps/sim/.env.example apps/sim/.env
# Create your secrets
perl -i -pe "s/your_encryption_key/$(openssl rand -hex 32)/" apps/sim/.env
perl -i -pe "s/your_internal_api_secret/$(openssl rand -hex 32)/" apps/sim/.env
perl -i -pe "s/your_api_encryption_key/$(openssl rand -hex 32)/" apps/sim/.env
# DB configs for migration
cp packages/db/.env.example packages/db/.env
# Edit both .env files to set DATABASE_URL="postgresql://postgres:your_password@localhost:5432/simstudio"
- Run migrations:
cd packages/db && bun run db:migrate
- Start development servers:
bun run dev:full # Starts Next.js app and realtime socket server
Or run separately: bun run dev (Next.js) and cd apps/sim && bun run dev:sockets (realtime).
Chat API Keys
Chat is a Sim-managed service. To use Chat on a self-hosted instance:
- Go to https://sim.ai → Settings → Chat keys and generate a Chat API key
- Set
COPILOT_API_KEYenvironment variable in your self-hosted apps/sim/.env file to that value
Environment Variables
See the environment variables reference for the full list, or apps/sim/.env.example for defaults.
Tech Stack
- Framework: Next.js (App Router)
- Runtime: Bun
- Database: PostgreSQL with Drizzle ORM
- Authentication: Better Auth
- Schema Validation: Zod
- UI: Shadcn, Tailwind CSS
- Streaming Markdown: Streamdown
- State Management: Zustand, TanStack Query
- Flow Editor: ReactFlow
- Docs: Fumadocs
- Monorepo: Turborepo
- Realtime: Socket.io
- Background Jobs: Trigger.dev
- Remote Code Execution: E2B
- Isolated Code Execution: isolated-vm
Contributing
We welcome contributions! Please see our Contributing Guide for details.
License
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
Made with ❤️ by the Sim Team