Theodore LiandClaude Opus 4.7 4fa7e74c41 feat(tools): queue hosted-key tool calls instead of failing with 429 (#4416)
* Add queueing for hosted keys

* feat(rate-limiter): FIFO queue for hosted-key per-workspace fairness

Replace the per-call distributed lock with a Redis-backed FIFO queue so
callers within a workspace get strict ordering instead of racing the
bucket. Adds heartbeat-based crash recovery and dead-head reaping in a
single Lua script. Bumps Exa search hosted RPM from 5 to 60.

* fix(rate-limiter): bound hosted-key queue wait to execution budget; fix heartbeat + telemetry

Tie the per-workspace hosted-key queue wait to the surrounding execution
budget instead of a flat 5-minute cap. acquireKey now accepts the execution
AbortSignal (threaded from ExecutionContext): when present, the wait is
bounded by the run's actual plan timeout / cancellation, with the enterprise
async ceiling as a backstop; when absent it falls back to MAX_QUEUE_WAIT_MS.
This lets long-running async (Trigger.dev) runs use their full budget while
no longer letting a single queued call burn a short sync run's entire budget.

Also addresses Greptile review:
- P1: share one lastHeartbeatAt across all wait phases and cap every sleep to
  HEARTBEAT_REFRESH_INTERVAL_MS so a long low-RPM retryAfterMs can no longer
  let the head's heartbeat lapse mid-wait and break FIFO ordering.
- P2: derive hostedKeyQueueWaited telemetry reason from the actual bottleneck
  (queue_position / dimension / actor_requests) instead of hardcoding it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(rate-limiter): make hosted-key queue waits abort-interruptible

Replace the plain capped sleeps in the queue-head and bucket-capacity wait
loops with an interruptibleSleep that resolves early when the execution
AbortSignal fires (timeout or cancellation), cleaning up its own timer and
listener. Previously a cancelled/timed-out run could overshoot by up to the
heartbeat cap (~10s) before the loop re-checked its budget; now it wakes
within a tick. The cap remains for heartbeat renewal.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 19:52:05 -04:00

Sim Logo

The open-source platform to build AI agents and run your agentic workforce. Connect 1,000+ integrations and LLMs to orchestrate agentic workflows.

Sim.ai Discord Twitter Documentation

Ask DeepWiki Set Up with Cursor

Build Workflows with Ease

Design agent workflows visually on a canvas—connect agents, tools, and blocks, then run them instantly.

Workflow Builder Demo

Supercharge with Copilot

Leverage Copilot to generate nodes, fix errors, and iterate on flows directly from natural language.

Copilot Demo

Integrate Vector Databases

Upload documents to a vector store and let agents answer questions grounded in your specific content.

Knowledge Uploads and Retrieval Demo

Quickstart

Cloud-hosted: sim.ai

Sim.ai

Self-hosted: NPM Package

npx simstudio

→ http://localhost:3000

Note

Docker must be installed and running on your machine.

Options

Flag Description
-p, --port <port> Port to run Sim on (default 3000)
--no-pull Skip pulling latest Docker images

Self-hosted: Docker Compose

git clone https://github.com/simstudioai/sim.git && cd sim
docker compose -f docker-compose.prod.yml up -d

Open http://localhost:3000

Sim also supports local models via Ollama and vLLM — see the Docker self-hosting docs for setup details.

Self-hosted: Manual Setup

Requirements: Bun, Node.js v20+, PostgreSQL 12+ with pgvector

  1. Clone and install:
git clone https://github.com/simstudioai/sim.git
cd sim
bun install
bun run prepare  # Set up pre-commit hooks
  1. Set up PostgreSQL with pgvector:
docker run --name simstudio-db -e POSTGRES_PASSWORD=your_password -e POSTGRES_DB=simstudio -p 5432:5432 -d pgvector/pgvector:pg17

Or install manually via the pgvector guide.

  1. Configure environment:
cp apps/sim/.env.example apps/sim/.env
# Create your secrets
perl -i -pe "s/your_encryption_key/$(openssl rand -hex 32)/" apps/sim/.env
perl -i -pe "s/your_internal_api_secret/$(openssl rand -hex 32)/" apps/sim/.env
perl -i -pe "s/your_api_encryption_key/$(openssl rand -hex 32)/" apps/sim/.env
# DB configs for migration
cp packages/db/.env.example packages/db/.env
# Edit both .env files to set DATABASE_URL="postgresql://postgres:your_password@localhost:5432/simstudio"
  1. Run migrations:
cd packages/db && bun run db:migrate
  1. Start development servers:
bun run dev:full  # Starts Next.js app and realtime socket server

Or run separately: bun run dev (Next.js) and cd apps/sim && bun run dev:sockets (realtime).

Copilot API Keys

Copilot is a Sim-managed service. To use Copilot on a self-hosted instance:

  • Go to https://sim.ai → Settings → Copilot and generate a Copilot API key
  • Set COPILOT_API_KEY environment variable in your self-hosted apps/sim/.env file to that value

Environment Variables

See the environment variables reference for the full list, or apps/sim/.env.example for defaults.

Tech Stack

Contributing

We welcome contributions! Please see our Contributing Guide for details.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Made with ❤️ by the Sim Team

Languages
TypeScript 77%
MDX 20.8%
JavaScript 1.9%
CSS 0.1%