opensource/sim
3
0
Fork 0
mirror of https://github.com/simstudioai/sim.git synced 2026-09-24 15:45:35 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat(self-host): align Docker Compose with Helm and overhaul self-hosting docs (#6225)

Browse Source 
* feat(self-host): align Docker Compose with Helm and overhaul self-hosting docs

Docker Compose shipped no scheduler, so scheduled workflows, every polling
trigger, connector syncs, the outbox, and data drains silently never ran.
Adds a cron service running the same 18 jobs the Helm chart schedules as
CronJobs, and closes the remaining behavioral gaps between the two paths:
bundled Redis in the chart, no hosted plan caps in chart defaults, pinned
image tags, and fail-fast secrets. A CI check keeps the schedulers in sync.

Also rewrites the self-hosting docs: 14 new pages, 8 updated, reorganized
into Install / Configure / Operate.

* fix(self-host): drop bun install from chart CI, remove air-gapped and backup docs

The scheduler-parity check pulled a full dependency install into the
chart-validation job, which fails building isolated-vm on that runner.
Rewritten to use only node builtins so the job installs nothing.

Also removes the air-gapped and backup/restore pages, and stops pinning a
concrete release in the docs so the examples do not go stale each release.

* fix(helm): bundle Redis in secret-manager modes unless the URL is supplied

Suppressing Redis whenever a secret mode was active left those deployments
with no Redis at all — REDIS_URL is optional there and both shipped examples
omit it. The chart now steps aside only on a detectable signal: an explicit
app.env.REDIS_URL, an ESO remoteRefs.app.REDIS_URL mapping, or the new
redis.provideUrl=false opt-out for a pre-created Secret it cannot read.

* fix(compose): derive realtime BETTER_AUTH_URL from NEXT_PUBLIC_APP_URL

realtime read BETTER_AUTH_URL directly and fell back to localhost while
simstudio derived it from NEXT_PUBLIC_APP_URL, so setting only the public
origin left realtime authenticating against http://localhost:3000.

* fix(helm): deliver bundled REDIS_URL via ConfigMap so an operator value always wins

Injecting REDIS_URL as an inline container env made it beat every envFrom
source, so a REDIS_URL held in a pre-created Secret or synced by External
Secrets was silently shadowed and traffic moved to a fresh in-cluster Redis.

Kubernetes resolves duplicate envFrom keys by letting the last source win, so
the bundled URL now ships as a ConfigMap listed before the app Secret. Any
operator-supplied value overrides it without the chart needing to read it,
which also removes the redis.provideUrl flag the previous attempt required.

* docs(helm): spell out the egress rule external datastores need

The default NetworkPolicy allows 443 plus the bundled Postgres and Redis by
pod selector. Anything you run outside the chart on another port needs its own
rule, which is easiest to miss when REDIS_URL arrives via a Secret the chart
cannot inspect. Adds a copyable example to the production checklist and the
security guide.

* feat(helm): add networkPolicy.allowExternalEgress for managed datastores

The default policy allows 443 plus the bundled Postgres and Redis by pod
selector, so a managed datastore on another port needs a hand-written CIDR
rule — awkward when REDIS_URL arrives via a Secret the chart cannot inspect.

Adds an opt-in switch that drops the port restriction while still blocking the
cloud metadata endpoints. Defaults to false, keeping this chart stricter than
the common chart default of unrestricted egress.
This commit is contained in:
Waleed
2026-08-03 14:52:48 -07:00
committed by GitHub
parent 1242301bd3
commit 3de63c94e3
53 changed files with 3561 additions and 342 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/docs/content/docs/en/platform/self-hosting/architecture.mdx
+112
View File
@@ -0,0 +1,112 @@
---
title: Architecture
description: Every service Sim runs, what it depends on, and where state lives
---
import { Callout } from 'fumadocs-ui/components/callout'
import { FAQ } from '@/components/ui/faq'
Understanding what runs where makes every other operational decision — scaling, backup, network policy, upgrades — straightforward.
## Services
| Service | Image | Port | Stateless | Required |
|---|---|---|---|---|
| **app** | `ghcr.io/simstudioai/simstudio` | 3000 | Only with object storage configured | Yes |
| **realtime** | `ghcr.io/simstudioai/realtime` | 3002 | Yes | Yes |
| **migrations** | `ghcr.io/simstudioai/migrations` | — | Yes (runs once) | Yes |
| **postgresql** | `pgvector/pgvector:pg17` | 5432 | **No** | Yes |
| **redis** | `redis:7-alpine` | 6379 | Mostly | Bundled by both; swap for a managed instance in production |
| **cron** | `ghcr.io/simstudioai/cron` (Compose) / `curlimages/curl` (CronJobs) | — | Yes | Yes |
| **pii** | `ghcr.io/simstudioai/pii` | 5001 | Yes | Optional |
| **ollama** | `ollama/ollama` | 11434 | **No** (model cache) | Optional |
| **telemetry** | `otel/opentelemetry-collector-contrib` | 4317/4318 | Yes | Optional |
### app
The Next.js application: the editor UI, every API route, and the workflow execution engine. Workflow runs happen **inside the app process** by default, using an isolated-vm sandbox, which is why memory rather than CPU is the constraining resource. Both the chart and the compose file request 4 Gi and cap the app at 8 Gi. Configuring a remote sandbox provider (E2B or Daytona) moves code execution out of the process; see [Security](/platform/self-hosting/security).
Any replica can serve any request **once object storage is configured**. Until then the app writes uploads to its own container filesystem, which makes it stateful — see [Where state lives](#where-state-lives). Scale it horizontally only after reading [Scaling & HA](/platform/self-hosting/scaling) for the Redis, storage, and connection-pool prerequisites.
### realtime
A Bun Socket.IO server handling collaborative editing, live execution updates, and collaborative documents. Clients connect at `/socket.io`.
It shares the database and `BETTER_AUTH_SECRET` with the app (Better Auth's shared-database-session pattern), so it authenticates the same users without a separate login.
<Callout type="warn">
Scaling realtime past one replica **requires** `REDIS_URL` — the Socket.IO Redis adapter is what carries events between pods. Without it, two users on different pods silently stop seeing each other's edits.
</Callout>
### migrations
Applies Drizzle schema migrations, then exits. In Docker Compose it is a one-shot service; in Kubernetes it is an **init container on the app Deployment**, so migrations run before any app pod becomes ready and re-run (as a no-op) on every rollout.
Migrations are forward-only. See [Upgrades](/platform/self-hosting/upgrades).
### postgresql
PostgreSQL 17 with the **pgvector** extension, which is required — knowledge base embeddings are stored and searched as vectors. The `pgvector/pgvector:pg17` image ships it; a managed instance needs the extension enabled (Sim's migrations issue `CREATE EXTENSION` automatically where permissions allow).
This holds essentially all durable state: workflows, runs, logs, users, organizations, credentials, knowledge base chunks and embeddings, and table data.
### redis
Backs pub/sub, the Socket.IO adapter, the idempotency store, execution progress markers, distributed execution limits, and the CLI-auth approval store. The storage-like uses fall back to Postgres or in-process state. Pub/sub falls back to a **process-local** emitter, which is fine on one replica and drops every cross-pod event on more than one. See [Redis](/platform/self-hosting/redis).
### cron
Eighteen scheduled jobs that call internal endpoints — schedule execution, polling triggers, webhook-subscription renewal, connector syncs, outbox processing, data drains, and sandbox-image cleanup. Kubernetes runs them as CronJobs; Docker Compose runs them from a single supercronic service. Same paths, same schedules. See [Background Jobs](/platform/self-hosting/background-jobs).
## Where state lives
Three places once the deployment is configured for production. Everything else is disposable.
| Store | Contents | Backup |
|---|---|---|
| **PostgreSQL** | All application data | `pg_dump` / managed snapshots + PITR |
| **Object storage** | Uploaded files, KB documents, execution outputs, avatars, logos | Bucket versioning + lifecycle |
| **Secrets** | `ENCRYPTION_KEY`, `API_ENCRYPTION_KEY`, `BETTER_AUTH_SECRET`, `INTERNAL_API_SECRET`, `CRON_SECRET` | Secret manager |
<Callout type="error">
**Object storage is not configured by default, and the fallback is not durable.** Sim only uses S3, Azure Blob, or GCS when the corresponding variables are set (`S3_BUCKET_NAME` + `AWS_REGION`, `AZURE_STORAGE_CONTAINER_NAME` + credentials, or `GCS_BUCKET_NAME`). With none set it writes uploads to a directory inside the app container — and neither `docker-compose.prod.yml` nor the Helm chart mounts a volume there. Files are lost when the container is recreated and are invisible to other replicas. Configure [object storage](/platform/self-hosting/object-storage) before storing anything you care about, and before scaling past one replica.
</Callout>
<Callout type="error">
`ENCRYPTION_KEY` is not recoverable and not derivable. It encrypts workspace and personal environment variables, stored provider API keys, MCP OAuth credentials, and deployment/chat secrets at rest — a database restore paired with a *different* key yields a working app in which none of that can be decrypted. Back it up separately from the database, and never rotate it casually.
</Callout>
Redis is a cache and message bus. Losing it drops in-flight live updates; it does not lose committed data.
## Request paths
**Editor / API** — browser → ingress/reverse proxy → app:3000 → Postgres, Redis, object storage.
**Collaboration** — browser → ingress → realtime:3002 (`/socket.io`, WebSocket upgrade) → Redis pub/sub → other realtime pods. The proxy must pass upgrade headers and allow long-lived idle connections; see [Networking](/platform/self-hosting/networking).
**File upload (object storage configured)** — browser asks app for a presigned URL → browser `PUT`s **directly to object storage** → app records metadata. This is why buckets need a CORS policy naming your Sim origin. Downloads are proxied back through the app.
**File upload (local disk)** — the presigned endpoint reports `directUploadSupported: false` and the browser uploads through the app instead. No CORS configuration is involved, and no bucket is used.
**Workflow execution** — trigger (manual, API, webhook, or schedule) → app enqueues or runs inline → isolated-vm sandbox → results and logs to Postgres, progress markers to Redis.
**Background work** — CronJob → `Authorization: Bearer $CRON_SECRET` → app endpoint → same execution path.
## Network boundaries
| From | To | Purpose |
|---|---|---|
| Internet | app:3000, realtime:3002 | Users |
| app, realtime | postgresql:5432 | Data |
| app, realtime | redis:6379 | Pub/sub, cache |
| app | Object storage endpoint | Files (server side) |
| **Browser** | **Object storage endpoint** | Presigned uploads — must be publicly reachable |
| app | Model provider APIs, integration APIs, SMTP/email provider | Outbound |
| cron | app:3000 (internal Service / compose network) | Scheduled triggers |
The chart's optional NetworkPolicy blocks cloud metadata endpoints (`169.254.169.254`) by default but allows ingress from any pod in the cluster unless you scope `networkPolicy.ingressFrom`. See [Security](/platform/self-hosting/security).
<FAQ items={[
{ question: "Can I run the app without the realtime service?", answer: "A realtime server must exist — it carries collaborative editing and live execution updates, and the editor degrades badly without one. You can set realtime.enabled=false in the chart, but only when pointing app.env.SOCKET_SERVER_URL at a realtime instance you run yourself."},
{ question: "Does workflow execution run in a separate worker process?", answer: "By default, no — executions run inside the app process using an isolated-vm sandbox, which is why the app has an 8 Gi memory limit. Setting E2B_ENABLED or SANDBOX_PROVIDER=daytona moves user code to a remote sandbox, and TRIGGER_DEV_ENABLED routes async jobs to Trigger.dev; otherwise the database-backed job queue is used."},
]} />
apps/docs/content/docs/en/platform/self-hosting/authentication.mdx
+147
View File
@@ -0,0 +1,147 @@
---
title: Authentication
description: Login providers, SSO, and controlling who can sign up
---
import { Callout } from 'fumadocs-ui/components/callout'
import { FAQ } from '@/components/ui/faq'
## Required configuration
```bash
BETTER_AUTH_SECRET=<openssl rand -hex 32>
BETTER_AUTH_URL=https://sim.yourdomain.com
NEXT_PUBLIC_APP_URL=https://sim.yourdomain.com
```
<Callout type="warn">
`BETTER_AUTH_URL` and `NEXT_PUBLIC_APP_URL` must be your exact public origin — correct scheme, no trailing slash. Leaving either as `localhost` in a deployed instance breaks sign-in, and the failure looks like a redirect loop rather than a configuration error.
`BETTER_AUTH_SECRET` must be **identical** on the app and realtime services. They share sessions through the database; a mismatch means realtime rejects every authenticated socket connection.
</Callout>
If users reach Sim from more than one origin — an apex and `www`, or an alias domain — list the extras:
```bash
TRUSTED_ORIGINS=https://www.example.com,https://app.example.com
```
## Email and password
Enabled by default. Users sign up with an email address and password.
```bash
EMAIL_VERIFICATION_ENABLED=true
```
Requires a configured email provider — see [Email](/platform/self-hosting/email). Without one the mailer no-ops silently, so users can never verify and never sign in. Do not enable this before email works.
## Social login
Three providers are supported for signing in to Sim itself.
| Provider | Variables | Callback URL |
|---|---|---|
| Google | `GOOGLE_CLIENT_ID` / `GOOGLE_CLIENT_SECRET` | `https://<host>/api/auth/callback/google` |
| GitHub | `GITHUB_CLIENT_ID` / `GITHUB_CLIENT_SECRET` | `https://<host>/api/auth/callback/github` |
| Microsoft | `MICROSOFT_CLIENT_ID` / `MICROSOFT_CLIENT_SECRET` | `https://<host>/api/auth/callback/microsoft` |
A provider appears on the login page once its credentials are set. Microsoft additionally requires both variables to be present before it is registered at all.
Turn one off without removing its credentials — useful when the same Google or Microsoft app powers integrations but you do not want it as a login method:
```bash
DISABLE_GOOGLE_AUTH=true
DISABLE_GITHUB_AUTH=true
DISABLE_MICROSOFT_AUTH=true
```
<Callout type="info">
`GOOGLE_CLIENT_ID` and `MICROSOFT_CLIENT_ID` are shared with the integration connectors. One app registration can serve both login and integrations — just register both sets of redirect URIs. See [Integrations & OAuth](/platform/self-hosting/integrations-oauth).
</Callout>
## SSO (SAML and OIDC)
SAML and OIDC single sign-on is an enterprise feature, available on self-hosted deployments through configuration rather than billing:
```bash
ENTERPRISE_ENABLED=true
NEXT_PUBLIC_ENTERPRISE_ENABLED=true
```
Or enable just SSO:
```bash
SSO_ENABLED=true
NEXT_PUBLIC_SSO_ENABLED=true
```
Providers are then registered in the app under **Settings → Enterprise → Single Sign-On**. A provider can be scoped to an organization or registered without one. Most other enterprise features do read their settings from the organization that owns a workspace, so a deployment using them still needs an organization model — set `INSTANCE_ORG_NAME` to place every user in one shared organization, or provision organizations through the Admin API.
See the [SSO guide](/platform/enterprise/sso) for identity-provider setup and the [self-hosted enterprise guide](/platform/enterprise/self-hosted) for the organization patterns.
## Controlling who can sign up
| Variable | Effect |
|---|---|
| `DISABLE_REGISTRATION=true` | Blocks email/password registration |
| `DISABLE_EMAIL_SIGNUP=true` | Blocks new email/password registrations; existing email login keeps working |
| `ALLOWED_LOGIN_DOMAINS` | Comma-separated domain allowlist, e.g. `acme.com,acme.co.uk`. Gates email sign-**in** as well as signup |
| `ALLOWED_LOGIN_EMAILS` | Comma-separated address allowlist, applied the same way |
| `BLOCKED_SIGNUP_DOMAINS` | Comma-separated domain blocklist |
| `SIGNUP_MX_VALIDATION_ENABLED=true` | Reject domains with no MX record or a denylisted mail backend |
| `BLOCKED_EMAIL_MX_HOSTS` | MX-host substrings to block; used only with the above |
<Callout type="warn">
These controls gate the **email/password** path. A first-time sign-in through Google, GitHub, or Microsoft creates an account through the social provider and is not filtered by them. If you need a hard boundary, disable the social providers you have not vetted (`DISABLE_GOOGLE_AUTH`, `DISABLE_GITHUB_AUTH`, `DISABLE_MICROSOFT_AUTH`) or restrict membership at the identity provider and use SSO.
</Callout>
For a company deployment, the usual pairing is domain-restricted signup plus SSO:
```bash
ALLOWED_LOGIN_DOMAINS=acme.com
DISABLE_EMAIL_SIGNUP=true
SSO_ENABLED=true
NEXT_PUBLIC_SSO_ENABLED=true
```
Both SSO flags are needed: the server-side one grants access, and the `NEXT_PUBLIC_` one makes the login page render the SSO entry point.
## Behind a load balancer
Tell Better Auth which forwarding hops to trust when resolving the client IP:
```bash
AUTH_TRUSTED_PROXIES=10.0.0.0/24,192.0.2.10
```
Better Auth walks `X-Forwarded-For` right to left, skips these hops, and uses the first untrusted address as the client IP for session records and its own IP-based checks. Use your proxies' actual addresses — a broad private range that also covers client traffic defeats the purpose. See [Security](/platform/self-hosting/security).
## Disabling authentication entirely
```bash
DISABLE_AUTH=true
```
Bypasses authentication and creates an anonymous session for every request.
<Callout type="error">
Everyone who can reach the instance becomes a fully privileged user — including anything that can reach it through an SSRF bug elsewhere on your network. Use this only for a single-user instance on a private network, never behind an internet-facing ingress.
</Callout>
## Other controls
| Variable | Effect |
|---|---|
| `DISABLE_INVITATIONS=true` / `NEXT_PUBLIC_DISABLE_INVITATIONS=true` | Disable workspace invitations globally |
| `DISABLE_PUBLIC_API=true` / `NEXT_PUBLIC_DISABLE_PUBLIC_API=true` | Disable the public API globally |
| `ADMIN_API_KEY` | Enables the Admin API for GitOps operations and organization provisioning |
The `NEXT_PUBLIC_` twin controls what the UI shows; the server-side variable enforces it. Set both.
<FAQ items={[
{ question: "Sign-in redirects in a loop. What is wrong?", answer: "BETTER_AUTH_URL or NEXT_PUBLIC_APP_URL does not match the origin users are browsing. Both must be the exact public URL including scheme, with no trailing slash. This is the single most common authentication misconfiguration."},
{ question: "Realtime rejects my session even though I am logged in.", answer: "BETTER_AUTH_SECRET differs between the app and realtime services. They share sessions through the database, so the secret must be byte-identical on both."},
{ question: "Can I use one Google app for both login and the Gmail integration?", answer: "Yes. GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRET power both. Register the login callback (/api/auth/callback/google) alongside the connector callbacks (/api/auth/oauth2/callback/google-email and friends) on the same OAuth client."},
{ question: "Why do some settings have a NEXT_PUBLIC_ twin?", answer: "The server-side variable enforces the behavior; the NEXT_PUBLIC_ variable tells the browser UI what to render. Setting only one produces a UI that disagrees with the server, so set both." },
]} />
apps/docs/content/docs/en/platform/self-hosting/background-jobs.mdx
+149
View File
@@ -0,0 +1,149 @@
---
title: Background Jobs
description: Scheduled workflows, polling triggers, and the cron endpoints that drive them
---
import { Tab, Tabs } from 'fumadocs-ui/components/tabs'
import { Callout } from 'fumadocs-ui/components/callout'
import { FAQ } from '@/components/ui/faq'
A large part of Sim runs on a schedule rather than in response to a user request: scheduled workflows, every polling trigger, connector syncs, the outbox, data drains, and retention. All of it is driven by **HTTP endpoints that something external must call on a timer**.
Both deployments ship a scheduler and enable it by default: Kubernetes as CronJobs, Docker Compose as a `cron` service. Both authenticate with `CRON_SECRET`, and both use the same schedules.
## Authentication
Every endpoint is protected by `CRON_SECRET` and expects it as a bearer token:
```bash
curl -f -s -S --max-time 60 \
-H "Authorization: Bearer $CRON_SECRET" \
https://sim.yourdomain.com/api/schedules/execute
```
Generate it like the other secrets:
```bash
openssl rand -hex 32
```
<Callout type="warn">
`CRON_SECRET` is **required** whenever background jobs are enabled — which is the Helm chart's default. The chart refuses to render without it. If it is unset, the endpoints reject every call and all scheduled work silently stops.
</Callout>
Point cron at an **internal** address where possible (the in-cluster Service, or `localhost` on a single node). These endpoints should not be reachable from the internet; if they are, `CRON_SECRET` is the only thing protecting them.
## The jobs
| Job | Endpoint | Schedule | Drives |
|---|---|---|---|
| Schedule execution | `/api/schedules/execute` | `*/1 * * * *` | **Scheduled workflows** |
| Gmail poll | `/api/webhooks/poll/gmail` | `*/1 * * * *` | Gmail trigger |
| Outlook poll | `/api/webhooks/poll/outlook` | `*/1 * * * *` | Outlook trigger |
| IMAP poll | `/api/webhooks/poll/imap` | `*/1 * * * *` | IMAP trigger |
| RSS poll | `/api/webhooks/poll/rss` | `*/1 * * * *` | RSS trigger |
| Google Sheets poll | `/api/webhooks/poll/google-sheets` | `*/1 * * * *` | Sheets trigger |
| Google Drive poll | `/api/webhooks/poll/google-drive` | `*/1 * * * *` | Drive trigger |
| Google Calendar poll | `/api/webhooks/poll/google-calendar` | `*/1 * * * *` | Calendar trigger |
| HubSpot poll | `/api/webhooks/poll/hubspot` | `*/1 * * * *` | HubSpot trigger |
| Time pause/resume | `/api/resume/poll` | `*/1 * * * *` | Workflows paused on a timer |
| Outbox processing | `/api/webhooks/outbox/process` | `*/1 * * * *` | Transactional-outbox retries for billing, membership, enterprise issuance, and workflow-deployment side effects |
| Connector sync | `/api/knowledge/connectors/sync` | `*/5 * * * *` | Knowledge base connector syncs |
| Workspace events poll | `/api/workspace-events/poll` | `*/15 * * * *` | Workspace event triggers |
| Data drains | `/api/cron/run-data-drains` | `0 * * * *` | Enterprise data drains |
| Renew subscriptions | `/api/cron/renew-subscriptions` | `0 */12 * * *` | Renews Microsoft Teams chat subscriptions (Graph caps them at ~3 days) |
| Reconcile billing seats | `/api/cron/reconcile-billing-seats` | `0 * * * *` | Billing only — safe to disable when self-hosted |
| Reconcile inbox entitlement | `/api/cron/reconcile-inbox-entitlement` | `0 3 * * *` | Inbox access reconciliation |
| Cleanup sandbox images | `/api/cron/cleanup-sandbox-images` | `30 4 * * *` | Reclaims sandbox images |
<Callout type="info">
**Subscription renewal** covers Microsoft Teams chat triggers, whose Microsoft Graph subscriptions are hard-capped at about three days. Without it, Teams triggers work for a couple of days and then quietly stop. Gmail, Outlook, Drive, Calendar, and Sheets triggers are **polled** instead — they depend on the per-minute poll jobs above, not on this one.
</Callout>
## Kubernetes
Enabled by default. Nothing to do beyond setting `CRON_SECRET`.
```yaml
cronjobs:
enabled: true
```
Each job runs a small `curlimages/curl` pod that calls the app's **in-cluster Service** (not the ingress), with `concurrencyPolicy: Forbid` so a slow run never overlaps itself, and up to three retries.
Disable individual jobs you do not need — billing reconciliation is the obvious one on a self-hosted install:
```yaml
cronjobs:
jobs:
reconcileBillingSeats:
enabled: false
```
Check they are running:
```bash
kubectl get cronjobs -n simstudio
kubectl get jobs -n simstudio --sort-by=.metadata.creationTimestamp | tail
kubectl logs -n simstudio job/<job-name>
```
A CronJob whose `LAST SCHEDULE` is stale, or whose jobs are failing, means the corresponding feature is dead. Alert on it — see [Observability](/platform/self-hosting/observability).
## Docker Compose
The `cron` service runs the same jobs on the same schedules, so nothing to configure beyond `CRON_SECRET`:
```bash
CRON_SECRET=$(openssl rand -hex 32)
```
Without it the `cron` service logs exactly what to set — including a freshly generated value — and exits, leaving the rest of the stack running. Schedules live in `docker/crontab` and mirror `helm/sim/values.yaml` `cronjobs.jobs` one-for-one.
<Callout type="info">
Upgrading a deployment created before the scheduler existed? Your `.env` has no `CRON_SECRET`, so the stack comes up as before and `cron` exits with instructions. Add the value and re-run `up -d` to turn background jobs on.
</Callout>
The service runs [supercronic](https://github.com/aptible/supercronic) rather than the app image: it logs each job's output to the container log, forwards `SIGTERM` so `docker compose stop` is graceful, and will not start an iteration while the previous one is still running.
```bash
docker compose -f docker-compose.prod.yml logs -f cron
```
A healthy log line looks like:
```
level=info msg=starting iteration=0 job.schedule="*/1 * * * *"
level=info msg="job succeeded" iteration=0
```
To drop a job you do not need, comment out its line in `docker/crontab` and restart the service.
## Verifying
Create a workflow with a Schedule trigger set to every minute, deploy it, and watch the Logs view. An execution should appear within ~2 minutes. If nothing appears:
1. Check the scheduler's own logs — `docker compose logs cron`, or `kubectl get cronjobs -n simstudio` for a recent `LAST SCHEDULE`.
2. Confirm the app and the scheduler share the same `CRON_SECRET`. A mismatch shows up as `401` in the scheduler log.
3. A `202` means the endpoint accepted the run; it does not confirm a schedule was due, so check the Logs view.
## Concurrency
Scheduled execution volume is bounded per app instance by:
| Variable | Default | Applies to |
|---|---|---|
| `SCHEDULE_EXECUTION_CONCURRENCY_LIMIT` | `30` | Scheduled workflows in flight, on every install |
<Callout type="info">
`WORKFLOW_EXECUTION_CONCURRENCY_LIMIT`, `WEBHOOK_EXECUTION_CONCURRENCY_LIMIT`, and `RESUME_EXECUTION_CONCURRENCY_LIMIT` are `concurrencyLimit` settings on Trigger.dev task definitions. They have **no effect** unless `TRIGGER_DEV_ENABLED` is set, and neither the Helm chart nor Docker Compose configures Trigger.dev — so on a default self-host they are inert.
</Callout>
Raise the schedule limit only alongside memory headroom: concurrent executions run in the app process, so throughput is bounded by the pod's memory before it is bounded by this number.
<FAQ items={[
{ question: "Do scheduled workflows work on Docker Compose?", answer: "Yes. The cron service runs the same 18 jobs on the same schedules the Helm chart uses, from docker/crontab. It needs CRON_SECRET; without it the service prints what to set and exits while the rest of the stack keeps running."},
{ question: "Should the cron endpoints be reachable from the internet?", answer: "No. Point cron at the in-cluster Service or localhost. If the endpoints are internet-reachable, CRON_SECRET is the only control protecting them."},
{ question: "Can I disable jobs I do not need?", answer: "Yes — set cronjobs.jobs.<name>.enabled: false in Helm, or omit the crontab line. Billing-seat reconciliation is the usual candidate on a self-hosted install. Do not disable schedule execution, outbox processing, or subscription renewal unless you know you do not use the corresponding features."},
{ question: "Do the jobs run against every replica?", answer: "No. Each CronJob makes one HTTP call to the app Service, which load-balances to a single replica. concurrencyPolicy: Forbid prevents a slow run from overlapping the next tick." },
]} />
apps/docs/content/docs/en/platform/self-hosting/docker.mdx
+67 -66
View File
@@ -10,8 +10,15 @@ import { FAQ } from '@/components/ui/faq'
## Quick Start
```bash
# Clone and start
git clone https://github.com/simstudioai/sim.git && cd sim
cat > .env << EOF
BETTER_AUTH_SECRET=$(openssl rand -hex 32)
ENCRYPTION_KEY=$(openssl rand -hex 32)
INTERNAL_API_SECRET=$(openssl rand -hex 32)
CRON_SECRET=$(openssl rand -hex 32)
EOF
docker compose -f docker-compose.prod.yml up -d
```
@@ -22,85 +29,81 @@ Open [http://localhost:3000](http://localhost:3000)
### 1. Configure Environment
```bash
# Generate secrets
cat > .env << EOF
DATABASE_URL=postgresql://postgres:postgres@db:5432/simstudio
BETTER_AUTH_SECRET=$(openssl rand -hex 32)
ENCRYPTION_KEY=$(openssl rand -hex 32)
INTERNAL_API_SECRET=$(openssl rand -hex 32)
API_ENCRYPTION_KEY=$(openssl rand -hex 32)
CRON_SECRET=$(openssl rand -hex 32)
# Your public origin. BETTER_AUTH_URL is derived from this automatically.
NEXT_PUBLIC_APP_URL=https://sim.yourdomain.com
BETTER_AUTH_URL=https://sim.yourdomain.com
# Database credentials. DATABASE_URL is composed from these by the compose file.
POSTGRES_USER=postgres
POSTGRES_PASSWORD=$(openssl rand -hex 24)
POSTGRES_DB=simstudio
EOF
```
<Callout type="info">
Do not set `DATABASE_URL` or `BETTER_AUTH_URL` in `.env` — `docker-compose.prod.yml` composes both on the service definition, and a value set here is ignored. Change `POSTGRES_*` and `NEXT_PUBLIC_APP_URL` instead.
</Callout>
<Callout type="error">
Save `ENCRYPTION_KEY` and `API_ENCRYPTION_KEY` somewhere outside this server. `ENCRYPTION_KEY` encrypts workspace and personal environment variables, stored provider API keys, MCP OAuth credentials, and deployment/chat secrets; `API_ENCRYPTION_KEY` encrypts user-generated Sim API keys. Neither can be regenerated — a database restore paired with a different key leaves the data it protected permanently unreadable.
</Callout>
The compose file refuses to start if `BETTER_AUTH_SECRET`, `ENCRYPTION_KEY`, or `INTERNAL_API_SECRET` is missing, rather than booting with empty values. `CRON_SECRET` is treated more gently: without it the `cron` service prints what to set and exits, leaving the rest of the stack running — so upgrading from a compose file that predates the scheduler still works.
Images track `latest` unless you pin them. For production, see [Upgrades](/platform/self-hosting/upgrades).
### 2. Start Services
```bash
docker compose -f docker-compose.prod.yml up -d
```
### 3. Set Up SSL
Six services start:
<Tabs items={['Caddy (Recommended)', 'Nginx + Certbot']}>
<Tab value="Caddy (Recommended)">
Caddy automatically handles SSL certificates.
| Service | Port | Purpose |
|---|---|---|
| `simstudio` | 3000 | Main application (8 GB memory limit) |
| `realtime` | 3002 | WebSocket server (1 GB memory limit) |
| `db` | 5432 | PostgreSQL 17 with pgvector |
| `redis` | internal | Pub/sub and shared cache — not published to the host |
| `cron` | — | Runs the [background jobs](/platform/self-hosting/background-jobs) on a schedule |
| `migrations` | — | Applies schema migrations once, then exits |
Confirm the five long-running services are up and that `migrations` has exited cleanly (it is a one-shot job with no healthcheck):
```bash
# Install Caddy
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update && sudo apt install caddy
docker compose -f docker-compose.prod.yml ps
```
Create `/etc/caddy/Caddyfile`:
### 3. Put it behind TLS
Caddy is the least-effort option — it obtains and renews certificates automatically.
```
sim.yourdomain.com {
reverse_proxy localhost:3000
request_body {
max_size 250MB
}
handle /socket.io/* {
reverse_proxy localhost:3002
}
}
```
```bash
sudo systemctl restart caddy
```
</Tab>
<Tab value="Nginx + Certbot">
```bash
# Install
sudo apt install nginx certbot python3-certbot-nginx -y
# Create /etc/nginx/sites-available/sim
server {
listen 80;
server_name sim.yourdomain.com;
location / {
proxy_pass http://127.0.0.1:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
}
location /socket.io/ {
proxy_pass http://127.0.0.1:3002;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
reverse_proxy localhost:3000 {
flush_interval -1
}
}
# Enable and get certificate
sudo ln -s /etc/nginx/sites-available/sim /etc/nginx/sites-enabled/
sudo certbot --nginx -d sim.yourdomain.com
```
</Tab>
</Tabs>
Three things in that config are Sim-specific and easy to get wrong: `/socket.io` must reach the realtime service on 3002, `flush_interval -1` stops Caddy buffering streamed agent output into one delayed block, and `max_size` has to clear the chat endpoint's 220 MB limit.
For nginx, Traefik, or a cloud load balancer — and for the GKE websocket timeout — see [Networking](/platform/self-hosting/networking).
## Ollama
@@ -112,9 +115,13 @@ docker compose -f docker-compose.ollama.yml --profile gpu --profile setup up -d
docker compose -f docker-compose.ollama.yml --profile cpu --profile setup up -d
```
Pull additional models:
Pull additional models — the service name differs by profile:
```bash
# GPU profile
docker compose -f docker-compose.ollama.yml exec ollama ollama pull llama3.2
# CPU profile
docker compose -f docker-compose.ollama.yml exec ollama-cpu ollama pull llama3.2
```
### External Ollama
@@ -136,26 +143,20 @@ OLLAMA_URL=http://192.168.1.100:11434 docker compose -f docker-compose.prod.yml
## Commands
```bash
# View logs
docker compose -f docker-compose.prod.yml logs -f simstudio
# Did migrations succeed?
docker compose -f docker-compose.prod.yml logs migrations
# Stop
docker compose -f docker-compose.prod.yml down
# Is the scheduler firing?
docker compose -f docker-compose.prod.yml logs -f cron
# Update
# Upgrade: bump SIM_VERSION in .env, then
docker compose -f docker-compose.prod.yml pull && docker compose -f docker-compose.prod.yml up -d
# Backup database
docker compose -f docker-compose.prod.yml exec db pg_dump -U postgres simstudio > backup.sql
```
<FAQ items={[
{ question: "What containers are started by docker-compose.prod.yml?", answer: "Four services are started: simstudio (main app on port 3000, 8 GB memory limit), realtime (WebSocket server on port 3002, 1 GB memory limit), db (PostgreSQL 17 with pgvector on port 5432), and migrations (runs once to apply database schema changes, then exits)." },
{ question: "How do I configure SSL for production?", answer: "You can use either Caddy (recommended, handles certificates automatically) or Nginx with Certbot. Both need to reverse-proxy port 3000 for the main app and port 3002 for WebSocket connections at the /socket.io/ path." },
{ question: "Why can I not connect to Ollama running on my host from inside Docker?", answer: "Inside a Docker container, localhost refers to the container itself, not your host machine. On macOS and Windows, use http://host.docker.internal:11434. On Linux, use your host machine's actual IP address (e.g., http://192.168.1.100:11434)." },
{ question: "What is the difference between the GPU and CPU Ollama profiles?", answer: "The GPU profile (--profile gpu) configures NVIDIA driver capabilities and reserves GPU devices for accelerated inference. The CPU profile (--profile cpu) runs Ollama without GPU acceleration. Both use the --profile setup flag to automatically pull the gemma3:4b starter model." },
{ question: "How do I update Sim to the latest version?", answer: "Run docker compose -f docker-compose.prod.yml pull to fetch the latest images, then docker compose -f docker-compose.prod.yml up -d to restart with the new versions. The migrations container will automatically apply any new database schema changes on startup." },
{ question: "How do I back up and restore the database?", answer: "Back up with: docker compose -f docker-compose.prod.yml exec db pg_dump -U postgres simstudio > backup.sql. Restore with: docker compose -f docker-compose.prod.yml exec -T db psql -U postgres simstudio < backup.sql. The database data is persisted in a Docker volume named postgres_data." },
{ question: "Do scheduled workflows work on Docker Compose?", answer: "Yes. The cron service runs the same jobs the Helm chart schedules as Kubernetes CronJobs, using the schedules in docker/crontab. It needs CRON_SECRET — without it the service prints what to set and exits, and the rest of the stack keeps running."},
{ question: "Why is there a Redis container?", answer: "Redis backs pub/sub for live Chat task status and table events, plus shared caches. Pub/sub has no fallback that works across processes, so live status would not stream without it. The port is deliberately not published so it cannot collide with a local Redis."},
{ question: "How do I back up and restore the database?", answer: "Back up with: docker compose -f docker-compose.prod.yml exec db pg_dump -U postgres simstudio > backup.sql. Restore with: docker compose -f docker-compose.prod.yml exec -T db psql -U postgres simstudio < backup.sql. The database data is persisted in a Docker volume named postgres_data."},
{ question: "Can I customize the PostgreSQL credentials?", answer: "Yes. The docker-compose.prod.yml uses environment variable defaults: POSTGRES_USER (default: postgres), POSTGRES_PASSWORD (default: postgres), POSTGRES_DB (default: simstudio), and POSTGRES_PORT (default: 5432). Set these in your .env file to override them." },
]} />
apps/docs/content/docs/en/platform/self-hosting/email.mdx
+174
View File
@@ -0,0 +1,174 @@
---
title: Email
description: Configure transactional email for invitations, verification, and notifications
---
import { Tabs, Tab } from 'fumadocs-ui/components/tabs'
import { Callout } from 'fumadocs-ui/components/callout'
import { FAQ } from '@/components/ui/faq'
Sim sends workspace invitations, email verification, password resets, and notifications. Configure at least one provider.
<Callout type="warn">
With no provider configured, nothing is sent and nothing errors — the mailer logs a single line with the recipient, subject, and sender, then reports success. On a production deployment that means workspace invitations silently never arrive. The message body is **not** logged, so an email missed this way cannot be recovered from the logs.
</Callout>
## Provider selection
There is no provider flag. Every provider whose variables are set becomes active, and the mailer tries them in this fixed order, falling through to the next one only when the previous **fails**:
**Resend → AWS SES → SMTP → Azure Communication Services → Gmail**
So the earliest configured provider handles normal traffic, and any others act as automatic failover. Configuring one is the simple case; configuring two gives you a fallback path at the cost of mail occasionally leaving from a different sender.
## Shared settings
| Variable | Description |
|---|---|
| `FROM_EMAIL_ADDRESS` | Sender address, e.g. `Sim <noreply@example.com>` |
| `EMAIL_DOMAIN` | Fallback domain when `FROM_EMAIL_ADDRESS` is unset — sends as `noreply@EMAIL_DOMAIN` |
| `EMAIL_VERIFICATION_ENABLED` | Set `true` to require email verification at signup |
The sender address must be one your provider is authorized to send as. A mismatch is the most common cause of mail that is accepted by the provider and then silently dropped or spam-filtered downstream.
## Providers
<Tabs items={['Resend', 'AWS SES', 'SMTP', 'Azure ACS', 'Gmail']}>
<Tab value="Resend">
Simplest option if you have no existing mail infrastructure.
```bash
RESEND_API_KEY=re_...
FROM_EMAIL_ADDRESS="Sim <noreply@yourdomain.com>"
```
Verify your sending domain in the Resend dashboard and add the DNS records it gives you before sending in production.
</Tab>
<Tab value="AWS SES">
```bash
AWS_SES_REGION=us-east-1
FROM_EMAIL_ADDRESS="Sim <noreply@yourdomain.com>"
```
Credentials resolve through the standard AWS provider chain — environment variables, shared config, ECS/EKS task role (IRSA), EC2 instance profile, or SSO. On EKS, attach an IRSA role with `ses:SendEmail` and `ses:SendRawEmail` and set no keys at all.
<Callout type="warn">
New SES accounts are in the **sandbox**, which only permits sending to verified addresses. Invitations to your team will fail until you request production access. Verify the sending domain and configure DKIM as well.
</Callout>
</Tab>
<Tab value="SMTP">
Works with any relay — Postfix, SendGrid, Mailgun, Google Workspace SMTP relay, or MailHog for local testing.
```bash
SMTP_HOST=smtp.example.com
SMTP_PORT=587 # 465 implicit TLS, 587 STARTTLS, 25 plain
SMTP_USER=apikey # omit for unauthenticated relays
SMTP_PASS=... # omit for unauthenticated relays
# SMTP_SECURE=true # only for implicit TLS. Leave unset on 587 — it is
# automatic on 465, and forcing it on a STARTTLS port fails to connect
FROM_EMAIL_ADDRESS="Sim <noreply@yourdomain.com>"
```
For Google Workspace without a service account:
```bash
SMTP_HOST=smtp-relay.gmail.com
SMTP_PORT=587
```
The relay must be configured to accept mail from your deployment's egress IP in the Workspace admin console.
</Tab>
<Tab value="Azure ACS">
```bash
AZURE_ACS_CONNECTION_STRING=endpoint=https://...;accesskey=...
FROM_EMAIL_ADDRESS="Sim <noreply@yourdomain.com>"
```
Provision an Email Communication Service, connect a verified domain, then link it to the Communication Service resource. The sender address must belong to the linked domain.
</Tab>
<Tab value="Gmail">
GCP has no first-party transactional email service, so the Google-native path is the Gmail API with a Google Workspace sender.
```bash
GMAIL_CREDENTIALS_JSON='{"type":"service_account",...}'
GMAIL_SENDER=noreply@yourdomain.com
FROM_EMAIL_ADDRESS="Sim <noreply@yourdomain.com>"
```
Setup:
1. Create a service account and download its JSON key.
2. In the Workspace admin console → **Security → Access and data control → API controls → Domain-wide delegation → Add new**, add the service account's `client_id` with the scope `https://www.googleapis.com/auth/gmail.send`.
3. Set `GMAIL_SENDER` to the Workspace user the service account impersonates.
<Callout type="warn">
`FROM_EMAIL_ADDRESS` must match `GMAIL_SENDER` or one of its registered aliases. Gmail rewrites unrecognized From addresses, so a mismatch produces mail that sends successfully but arrives from the wrong address.
</Callout>
<Callout type="info">
Gmail caps sending at roughly 2,000 messages per day per user — ample for invitations and verification on most deployments. If you need more, switch to the Workspace SMTP relay or Resend; both are configuration-only changes.
</Callout>
Keep the JSON on one line when pasting it into a values file:
```bash
jq -c . service-account-key.json
```
</Tab>
</Tabs>
## Kubernetes
Email credentials are secrets. Supply them through your secret store rather than plain values:
```yaml
app:
env:
FROM_EMAIL_ADDRESS: "Sim <noreply@yourdomain.com>"
RESEND_API_KEY: "re_..." # via External Secrets or an existing Secret
```
In the default and External Secrets modes, `app.env` keys are written into a chart-managed Secret and mounted via `envFrom`, so values do not appear in pod specs. A secret committed to `values.yaml` is still a secret in your git history — pass it through External Secrets or a pre-created Secret instead.
## Verifying
Invite a user to a workspace from workspace settings and watch the app logs:
```bash
kubectl logs -n simstudio -l app.kubernetes.io/component=app --tail=100 | grep -i mail
```
| What you see | Meaning |
|---|---|
| A single log line with recipient, subject, and sender, and no delivery | No provider is configured — the mailer no-opped |
| A provider API error | Credentials or sender address problem; the error names which |
| Success, but nothing arrives | Delivered to the provider — check the provider's dashboard, then spam filtering, SPF, and DKIM |
## Troubleshooting
**"Delegation denied" / `unauthorized_client` (Gmail)** — the domain-wide delegation entry is missing or has the wrong client ID or scope. Re-check the admin console entry against the `client_id` in the service-account JSON, and confirm the scope is exactly `https://www.googleapis.com/auth/gmail.send`.
**Mail rejected with a From-address error** — the sender is not authorized for the provider's verified domain. Align `FROM_EMAIL_ADDRESS` with the verified domain (and with `GMAIL_SENDER` on the Gmail path).
**SES rejects recipients** — the account is still in the SES sandbox. Request production access.
**Mail lands in spam** — configure SPF, DKIM, and DMARC for your sending domain. This is on your DNS, not on Sim.
**Nothing at all happens and no error appears** — no provider is configured. The logs will show one line naming the recipient and subject.
<FAQ items={[
{ question: "How does Sim pick a provider?", answer: "Every configured provider is active, tried in a fixed order — Resend, AWS SES, SMTP, Azure Communication Services, Gmail — with the next one used only if the previous fails. The earliest configured provider handles normal traffic; the rest act as automatic failover."},
{ question: "Why does my Gmail-sent mail come from the wrong address?", answer: "Gmail rewrites From addresses it does not recognize. FROM_EMAIL_ADDRESS must match GMAIL_SENDER or one of that user's registered aliases."},
{ question: "Can I use SES without access keys?", answer: "Yes. Credentials resolve through the standard AWS provider chain, so an IRSA role on EKS or an instance profile on EC2 works — set only AWS_SES_REGION and grant ses:SendEmail and ses:SendRawEmail."},
{ question: "Is the Google Workspace SMTP relay easier than the Gmail API?", answer: "Usually yes — it needs no service account or domain-wide delegation, just SMTP_HOST=smtp-relay.gmail.com on port 587 and an allowlist entry for your egress IP in the Workspace admin console. Use the Gmail API path when you cannot allowlist a stable egress IP." },
]} />
apps/docs/content/docs/en/platform/self-hosting/environment-variables.mdx
+160 -65
View File
@@ -10,31 +10,65 @@ import { Callout } from 'fumadocs-ui/components/callout'
| Variable | Description |
|----------|-------------|
| `DATABASE_URL` | PostgreSQL connection string |
| `BETTER_AUTH_SECRET` | Auth secret (32 hex chars): `openssl rand -hex 32` |
| `BETTER_AUTH_URL` | Your app URL |
| `BETTER_AUTH_SECRET` | Auth secret (32 hex chars): `openssl rand -hex 32`. **Must be identical on the app and realtime services** |
| `BETTER_AUTH_URL` | Your app URL — must be the real public origin, not `localhost` |
| `ENCRYPTION_KEY` | Encryption key (32 hex chars): `openssl rand -hex 32` |
| `INTERNAL_API_SECRET` | Internal API secret (32 hex chars): `openssl rand -hex 32` |
| `NEXT_PUBLIC_APP_URL` | Public app URL |
| `NEXT_PUBLIC_SOCKET_URL` | Optional. WebSocket URL — defaults to the page origin; set only if realtime is on a separate host. |
| `CRON_SECRET` | Bearer token for the background job endpoints (32 hex chars). **Required whenever background jobs are enabled** — the Helm chart's default, and the chart will not render without it. See [Background Jobs](/platform/self-hosting/background-jobs) |
| `API_ENCRYPTION_KEY` | Encrypts user-generated API keys at rest (32 hex chars). Required to create API keys |
<Callout type="error">
`ENCRYPTION_KEY` and `API_ENCRYPTION_KEY` cannot be rotated or recovered. Losing either makes the data it protects permanently unreadable — workspace and personal environment variables, stored provider API keys, MCP OAuth credentials, and deployment/chat secrets in the first case, user-generated Sim API keys in the second. Back them up separately from the database.
</Callout>
## Strongly recommended
| Variable | Description |
|----------|-------------|
| `REDIS_URL` | Redis connection string. Optional on a single replica; **required** past one app or realtime replica — see [Redis](/platform/self-hosting/redis) |
| `REDIS_TLS_SERVERNAME` | TLS SNI override. Required when `REDIS_URL` uses `rediss://` with a bare IP, or the app throws at startup |
| `NEXT_PUBLIC_SOCKET_URL` | WebSocket URL — defaults to the page origin; set only if realtime is on a separate host |
| `TRUSTED_ORIGINS` | Comma-separated additional origins to trust for auth (apex + `www`, alias domains) |
| `AUTH_TRUSTED_PROXIES` | Comma-separated reverse-proxy IPs/CIDRs so the client IP cannot be forged through `X-Forwarded-For` |
| `INTERNAL_API_BASE_URL` | Internal URL for server-side self-calls, e.g. `http://sim-app.simstudio.svc.cluster.local:3000`. Required for PII log redaction; defaults to `NEXT_PUBLIC_APP_URL` |
| `DATABASE_REPLICA_URL` | Read-replica connection string for log listing, audit logs, and dashboard aggregations. Falls back to the primary when unset |
## AI Providers
| Variable | Provider |
|----------|----------|
| `OPENAI_API_KEY` | OpenAI |
| `OPENAI_API_KEY` | OpenAI — also the default knowledge base embedding provider |
| `ANTHROPIC_API_KEY_1` | Anthropic Claude |
| `GEMINI_API_KEY_1` | Google Gemini |
| `GEMINI_API_KEY` / `GEMINI_API_KEY_1` | Google Gemini |
| `MISTRAL_API_KEY` | Mistral |
| `XAI_API_KEY_1` | xAI |
| `KIMI_API_KEY_1` | Moonshot Kimi |
| `ZAI_API_KEY_1` | Z.ai |
| `TOGETHER_API_KEY` | Together AI |
| `FIREWORKS_API_KEY` | Fireworks AI |
| `BASETEN_API_KEY` | Baseten |
| `COHERE_API_KEY` | Cohere — required for the Knowledge block reranker |
| `OLLAMA_URL` | Ollama (default: `http://localhost:11434`) |
<Callout type="warn">
**Knowledge bases require a hosted embedding provider.** Three are supported, selected with `KB_EMBEDDING_MODEL`: `text-embedding-3-small` (default) and `text-embedding-3-large` on OpenAI or Azure OpenAI, and `gemini-embedding-001` on Gemini. There is no local embedding backend — configuring Ollama or vLLM does not substitute, because embeddings do not route through the configured chat model.
</Callout>
<Callout type="info">
For load balancing, add multiple keys with `_1`, `_2`, `_3` suffixes (e.g., `OPENAI_API_KEY_1`, `OPENAI_API_KEY_2`). Works with OpenAI, Anthropic, and Gemini.
For load balancing, add multiple keys with `_1`, `_2`, `_3` suffixes (e.g., `OPENAI_API_KEY_1`, `OPENAI_API_KEY_2`). Works with OpenAI, Anthropic, Gemini, xAI, Kimi, Z.ai, Cohere, and Fireworks.
</Callout>
<Callout type="info">
In Docker, use `OLLAMA_URL=http://host.docker.internal:11434` for host-machine Ollama.
</Callout>
### AWS Bedrock
| Variable | Description |
|----------|-------------|
| `NEXT_PUBLIC_BEDROCK_DEFAULT_CREDENTIALS` | Set `true` when using the AWS default credential chain (IAM roles, ECS task roles, IRSA). Hides credential fields in the Agent block UI |
### Azure OpenAI
| Variable | Description |
@@ -43,33 +77,111 @@ import { Callout } from 'fumadocs-ui/components/callout'
| `AZURE_OPENAI_ENDPOINT` | Azure OpenAI endpoint URL |
| `AZURE_OPENAI_API_VERSION` | API version (e.g., `2024-02-15-preview`) |
### vLLM (Self-Hosted)
### Self-hosted OpenAI-compatible endpoints
| Variable | Description |
|----------|-------------|
| `VLLM_BASE_URL` | vLLM server URL (e.g., `http://localhost:8000/v1`) |
| `VLLM_BASE_URL` | vLLM server URL, **without** a `/v1` suffix (e.g. `http://localhost:8000`) — Sim appends `/v1` itself |
| `VLLM_API_KEY` | Optional bearer token for vLLM |
| `LITELLM_BASE_URL` | LiteLLM proxy base URL |
| `LITELLM_API_KEY` | Optional bearer token for LiteLLM |
## OAuth Providers
## Login Providers
| Variable | Description |
|----------|-------------|
| `GOOGLE_CLIENT_ID` | Google OAuth client ID |
| `GOOGLE_CLIENT_SECRET` | Google OAuth client secret |
| `GITHUB_CLIENT_ID` | GitHub OAuth client ID |
| `GITHUB_CLIENT_SECRET` | GitHub OAuth client secret |
| `GOOGLE_CLIENT_ID` / `GOOGLE_CLIENT_SECRET` | Google — also powers all Google integrations |
| `GITHUB_CLIENT_ID` / `GITHUB_CLIENT_SECRET` | GitHub |
| `MICROSOFT_CLIENT_ID` / `MICROSOFT_CLIENT_SECRET` | Microsoft — also powers all Microsoft integrations |
| `DISABLE_GOOGLE_AUTH` / `DISABLE_GITHUB_AUTH` / `DISABLE_MICROSOFT_AUTH` | Hide a provider from the login page without removing its credentials |
## Optional
See [Authentication](/platform/self-hosting/authentication).
## Integration Credentials
<Callout type="warn">
Integrations do not work on a self-hosted deployment until you register your own OAuth app with each service and set its `*_CLIENT_ID` / `*_CLIENT_SECRET`. There are around 27 of them covering 50 connectors. The full table, redirect-URI format, and setup steps are in [Integrations & OAuth](/platform/self-hosting/integrations-oauth).
</Callout>
## Access Control
| Variable | Description |
|----------|-------------|
| `API_ENCRYPTION_KEY` | Encrypts stored API keys (32 hex chars): `openssl rand -hex 32` |
| `COPILOT_API_KEY` | API key for Chat. Without it the Sim Chat block, scheduled prompt jobs, and Inbox cannot run |
| `NEXT_PUBLIC_CHAT_DISABLED` | Set to `true` to hide the Chat module: the workspace lands on your first workflow, with no chats list, scheduled tasks, or editor Chat panel. Chat is shown when unset; `bun run setup` sets it for you if you skip the chat key |
| `ADMIN_API_KEY` | Admin API key for GitOps operations |
| `DISABLE_REGISTRATION` | Set `true` to disable new user signups entirely |
| `DISABLE_EMAIL_SIGNUP` | Block new email/password registrations; existing email login keeps working |
| `ALLOWED_LOGIN_DOMAINS` | Restrict signups to domains (comma-separated) |
| `ALLOWED_LOGIN_EMAILS` | Restrict signups to specific emails (comma-separated) |
| `DISABLE_REGISTRATION` | Set to `true` to disable new user signups |
| `BLOCKED_SIGNUP_DOMAINS` | Block specific domains from signing up (comma-separated) |
| `SIGNUP_MX_VALIDATION_ENABLED` | Reject domains with no MX record or a denylisted mail backend |
| `BLOCKED_EMAIL_MX_HOSTS` | MX-host substrings to block; used only with the above |
| `DISABLE_INVITATIONS` / `NEXT_PUBLIC_DISABLE_INVITATIONS` | Disable workspace invitations globally |
| `DISABLE_PUBLIC_API` / `NEXT_PUBLIC_DISABLE_PUBLIC_API` | Disable the public API globally |
| `DISABLE_AUTH` | Bypass authentication entirely, creating an anonymous session for every request |
<Callout type="error">
`DISABLE_AUTH=true` makes everyone who can reach the instance a fully privileged user. Use it only for a single-user instance on a private network, never behind an internet-facing ingress.
</Callout>
## Code Execution
| Variable | Description |
|----------|-------------|
| `SANDBOX_PROVIDER` | Remote sandbox provider: `e2b` (default) or `daytona` |
| `E2B_ENABLED` / `E2B_API_KEY` | Enable E2B remote execution |
| `DAYTONA_API_KEY` | Daytona API key (used when `SANDBOX_PROVIDER=daytona`) |
| `IVM_MAX_EXECUTIONS_PER_WORKER` | Executions before an isolated-vm worker is recycled |
| `IVM_MAX_BROKERS_PER_EXECUTION` | Host-call brokers per execution |
| `IVM_MAX_BROKER_ARGS_JSON_CHARS` | Max argument payload size |
| `IVM_MAX_BROKER_RESULT_JSON_CHARS` | Max result payload size |
Without a remote provider, user code runs in an in-process V8 isolate inside the app container. See [Security](/platform/self-hosting/security).
## Networking & Limits
| Variable | Default | Description |
|----------|---------|-------------|
| `API_MAX_JSON_BODY_BYTES` | 50 MB | Max JSON body on contract-validated API routes |
| `CHAT_MAX_REQUEST_BYTES` | 220 MB | Max body on the public deployed-chat endpoint |
| `WEBHOOK_MAX_REQUEST_BYTES` | 10 MB | Max body on public webhook receiver endpoints |
| `WORKFLOW_EXECUTION_CONCURRENCY_LIMIT` | `75` | Workflow executions in parallel |
| `WEBHOOK_EXECUTION_CONCURRENCY_LIMIT` | `75` | Webhook-triggered executions in parallel |
| `SCHEDULE_EXECUTION_CONCURRENCY_LIMIT` | `30` | Scheduled executions in parallel |
| `RESUME_EXECUTION_CONCURRENCY_LIMIT` | `50` | Resumed executions in parallel |
| `ALLOW_PRIVATE_DATABASE_HOSTS` | unset | Let database/connector tools reach private, reserved, and loopback hosts. Loosens the SSRF boundary |
Your reverse proxy's body-size limit must be at least as large as the app limits above. See [Networking](/platform/self-hosting/networking).
## Observability
| Variable | Description |
|----------|-------------|
| `OTEL_EXPORTER_OTLP_ENDPOINT` | OTLP collector endpoint |
| `OTEL_EXPORTER_OTLP_HEADERS` | Auth headers, `key=value` comma-separated |
| `OTEL_TRACES_SAMPLER_ARG` | Trace sampling ratio |
| `OTEL_DEPLOYMENT_ENVIRONMENT` | Environment label on emitted spans |
| `TELEMETRY_SAMPLING_RATIO` | Application-level sampling ratio |
| `TELEMETRY_ENDPOINT` | Where anonymous telemetry is sent. Defaults to `https://telemetry.simstudio.ai/v1/traces` — point it at your own collector to keep traces internal |
| `NEXT_TELEMETRY_DISABLED` | Set to `1` to disable anonymous telemetry entirely |
| `GRAFANA_OTLP_ENDPOINT` / `GRAFANA_OTLP_HEADERS` / `GRAFANA_DEPLOYMENT_ENVIRONMENT` | Grafana Cloud OTLP export |
See [Observability](/platform/self-hosting/observability).
## Knowledge Bases
| Variable | Description |
|----------|-------------|
| `KB_EMBEDDING_MODEL` | Embedding model for new knowledge bases. Defaults to `text-embedding-3-small`; an unsupported value falls back to the default |
| `COHERE_API_KEY` | Enables the Knowledge block reranker |
## Chat & PII
| Variable | Description |
|----------|-------------|
| `COPILOT_API_KEY` | API key for Chat. Without it the Sim Chat block, scheduled prompt jobs, and Inbox cannot run |
| `NEXT_PUBLIC_CHAT_DISABLED` | Set to `true` to hide the Chat module: the workspace lands on your first workflow, with no chats list, scheduled tasks, or editor Chat panel. Chat is shown when unset; `bun run setup` sets it for you if you skip the chat key |
| `PII_REDACTION` | Redact PII from workflow logs via Data Retention rules; requires the PII service and a cluster-reachable `INTERNAL_API_BASE_URL` |
| `PII_GRANULAR_REDACTION` | Additionally expose the execution-altering redaction stages |
| `ADMIN_API_KEY` | Admin API key for GitOps operations and organization provisioning |
## Enterprise Features
@@ -102,50 +214,16 @@ By default Sim writes uploads to local disk. For production, point it at AWS S3,
## Email Providers
Configure one provider — the mailer auto-detects in priority order: **Resend → AWS SES → SMTP → Azure Communication Services → Gmail**. If none are configured, emails are logged to the console instead.
Configure at least one. Every configured provider stays active and is tried in order — **Resend → AWS SES → SMTP → Azure Communication Services → Gmail** — falling through only on failure. With none configured, mail is silently not sent. Setup, verification, and troubleshooting are in [Email](/platform/self-hosting/email).
| Variable | Description |
|----------|-------------|
| `FROM_EMAIL_ADDRESS` | Sender address (e.g. `Sim <noreply@example.com>`). Falls back to `noreply@EMAIL_DOMAIN`. |
| `EMAIL_DOMAIN` | Default domain when `FROM_EMAIL_ADDRESS` is unset |
| `EMAIL_VERIFICATION_ENABLED` | Set to `true` to require email verification on signup |
**Resend**
| Variable | Description |
|----------|-------------|
| `RESEND_API_KEY` | API key from [resend.com](https://resend.com) |
**AWS SES**
| Variable | Description |
|----------|-------------|
| `AWS_SES_REGION` | AWS region for SES (e.g. `us-east-1`). Credentials are resolved through the standard AWS SDK provider chain (env vars, IRSA, ECS/EC2 instance role, SSO). |
**SMTP** (works with MailHog, Postfix, SendGrid SMTP, etc.)
| Variable | Description |
|----------|-------------|
| `SMTP_HOST` | SMTP server hostname |
| `SMTP_PORT` | `465` for implicit TLS, `587` for STARTTLS, `25` for plain |
| `SMTP_USER` | Optional — omit for unauthenticated relays |
| `SMTP_PASS` | Optional — omit for unauthenticated relays |
| `SMTP_SECURE` | Set to `true` to force TLS on connect; auto-true on port 465 |
**Azure Communication Services**
| Variable | Description |
|----------|-------------|
| `AZURE_ACS_CONNECTION_STRING` | Azure Communication Services connection string |
**Gmail** (Google-native — GCP has no first-party transactional email service, so the native path is the Gmail API with a Google Workspace sender)
| Variable | Description |
|----------|-------------|
| `GMAIL_CREDENTIALS_JSON` | Inline service-account JSON. The service account needs [domain-wide delegation](https://developers.google.com/workspace/guides/create-credentials#delegate_domain-wide_authority_to_a_service_account) granted for the `https://www.googleapis.com/auth/gmail.send` scope in the Workspace admin console |
| `GMAIL_SENDER` | The Workspace user the service account impersonates when sending (e.g. `noreply@yourdomain.com`). `FROM_EMAIL_ADDRESS` should match this user or one of its registered aliases — Gmail rewrites unrecognized From addresses |
Alternatively, the [Google Workspace SMTP relay](https://support.google.com/a/answer/2956491) works through the generic SMTP provider (`SMTP_HOST=smtp-relay.gmail.com`, port `587`) with no service account required.
| Provider | Variables |
|----------|-----------|
| Shared | `FROM_EMAIL_ADDRESS`, `EMAIL_DOMAIN`, `EMAIL_VERIFICATION_ENABLED` |
| Resend | `RESEND_API_KEY` |
| AWS SES | `AWS_SES_REGION` (credentials via the AWS provider chain) |
| SMTP | `SMTP_HOST`, `SMTP_PORT`, `SMTP_USER`, `SMTP_PASS`, `SMTP_SECURE` |
| Azure ACS | `AZURE_ACS_CONNECTION_STRING` |
| Gmail | `GMAIL_CREDENTIALS_JSON`, `GMAIL_SENDER` |
## Limits
@@ -167,19 +245,36 @@ Self-hosted deployments (billing disabled) run without plan limits: no rate limi
</Callout>
<Callout type="warn">
Deployments installed with the Helm chart ship these variables preset in `app.envDefaults`, so chart-based installs keep enforcement unless those keys are removed or overridden.
Neither deployment presets these. The Helm chart previously did, which enforced hosted-plan caps on self-hosted installs; chart 1.5.0 removed the presets so Compose and Kubernetes behave identically.
</Callout>
## Example .env
```bash
# Core
DATABASE_URL=postgresql://postgres:postgres@db:5432/simstudio
BETTER_AUTH_SECRET=<openssl rand -hex 32>
NEXT_PUBLIC_APP_URL=https://sim.yourdomain.com
BETTER_AUTH_URL=https://sim.yourdomain.com
# Secrets — generate each with `openssl rand -hex 32`
BETTER_AUTH_SECRET=<openssl rand -hex 32>
ENCRYPTION_KEY=<openssl rand -hex 32>
INTERNAL_API_SECRET=<openssl rand -hex 32>
NEXT_PUBLIC_APP_URL=https://sim.yourdomain.com
API_ENCRYPTION_KEY=<openssl rand -hex 32>
CRON_SECRET=<openssl rand -hex 32>
# Coordination (required past one replica)
REDIS_URL=redis://redis:6379
# Models — OPENAI_API_KEY also powers knowledge base embeddings
OPENAI_API_KEY=sk-...
# Email
RESEND_API_KEY=re_...
FROM_EMAIL_ADDRESS="Sim <noreply@yourdomain.com>"
# Access control
ALLOWED_LOGIN_DOMAINS=yourdomain.com
```
See `apps/sim/.env.example` for all options.
apps/docs/content/docs/en/platform/self-hosting/index.mdx
+58 -17
View File
@@ -10,7 +10,7 @@ import { FAQ } from '@/components/ui/faq'
Deploy Sim on your own infrastructure with Docker or Kubernetes.
<div className="flex gap-2 my-4">
<a href="https://cursor.com/link/prompt?text=Help%20me%20set%20up%20Sim%20locally.%20Follow%20these%20steps%3A%0A%0A1.%20First%2C%20verify%20Docker%20is%20installed%20and%20running%3A%0A%20%20%20docker%20--version%0A%20%20%20docker%20info%0A%0A2.%20Clone%20the%20repository%3A%0A%20%20%20git%20clone%20https%3A%2F%2Fgithub.com%2Fsimstudioai%2Fsim.git%0A%20%20%20cd%20sim%0A%0A3.%20Start%20the%20services%20with%20Docker%20Compose%3A%0A%20%20%20docker%20compose%20-f%20docker-compose.prod.yml%20up%20-d%0A%0A4.%20Wait%20for%20all%20containers%20to%20be%20healthy%20(this%20may%20take%201-2%20minutes)%3A%0A%20%20%20docker%20compose%20-f%20docker-compose.prod.yml%20ps%0A%0A5.%20Verify%20the%20app%20is%20accessible%20at%20http%3A%2F%2Flocalhost%3A3000%0A%0AIf%20there%20are%20any%20errors%2C%20help%20me%20troubleshoot%20them.%20Common%20issues%3A%0A-%20Port%203000%2C%203002%2C%20or%205432%20already%20in%20use%0A-%20Docker%20not%20running%0A-%20Insufficient%20memory%20(needs%2012GB%2B%20RAM)%0A%0AFor%20local%20AI%20models%20with%20Ollama%2C%20use%20this%20instead%20of%20step%203%3A%0A%20%20%20docker%20compose%20-f%20docker-compose.ollama.yml%20--profile%20setup%20up%20-d">
<a href="https://cursor.com/link/prompt?text=Help%20me%20set%20up%20Sim%20locally.%20Follow%20these%20steps%3A%0A%0A1.%20First%2C%20verify%20Docker%20is%20installed%20and%20running%3A%0A%20%20%20docker%20--version%0A%20%20%20docker%20info%0A%0A2.%20Clone%20the%20repository%3A%0A%20%20%20git%20clone%20https%3A%2F%2Fgithub.com%2Fsimstudioai%2Fsim.git%0A%20%20%20cd%20sim%0A%0A3.%20Generate%20required%20secrets%20%28the%20stack%20will%20not%20start%20without%20them%29%3A%0A%20%20%20cat%20%3E%20.env%20%3C%3C%20EOF%0A%20%20%20BETTER_AUTH_SECRET%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20ENCRYPTION_KEY%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20INTERNAL_API_SECRET%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20CRON_SECRET%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20EOF%0A%0A4.%20Start%20the%20services%20with%20Docker%20Compose%3A%0A%20%20%20docker%20compose%20-f%20docker-compose.prod.yml%20up%20-d%0A%0A4.%20Wait%20for%20all%20containers%20to%20be%20healthy%20(this%20may%20take%201-2%20minutes)%3A%0A%20%20%20docker%20compose%20-f%20docker-compose.prod.yml%20ps%0A%0A5.%20Verify%20the%20app%20is%20accessible%20at%20http%3A%2F%2Flocalhost%3A3000%0A%0AIf%20there%20are%20any%20errors%2C%20help%20me%20troubleshoot%20them.%20Common%20issues%3A%0A-%20Port%203000%2C%203002%2C%20or%205432%20already%20in%20use%0A-%20Docker%20not%20running%0A-%20Insufficient%20memory%20(needs%2012GB%2B%20RAM)%0A%0AFor%20local%20AI%20models%20with%20Ollama%2C%20use%20this%20instead%20of%20step%203%3A%0A%20%20%20docker%20compose%20-f%20docker-compose.ollama.yml%20--profile%20setup%20up%20-d">
<img src="https://img.shields.io/badge/Set%20Up%20with-Cursor-000000?logo=cursor&logoColor=white" alt="Set Up with Cursor" />
</a>
</div>
@@ -29,18 +29,30 @@ Deploy Sim on your own infrastructure with Docker or Kubernetes.
**Production**: Large teams (50+ users), high availability, heavy workflow execution
<Callout type="info">
Resource requirements are driven by workflow execution (isolated-vm sandboxing), file processing (in-memory document parsing), and vector operations (pgvector). Memory is typically the constraining factor rather than CPU. Production telemetry shows the main app uses 4-8 GB average with peaks up to 12 GB under heavy load.
Resource requirements are driven by workflow execution (isolated-vm sandboxing), file processing (in-memory document parsing), and vector operations (pgvector). Memory is typically the constraining factor rather than CPU — both the Helm chart and the compose file request 4 Gi for the app and cap it at 8 Gi.
</Callout>
## Quick Start
```bash
git clone https://github.com/simstudioai/sim.git && cd sim
cat > .env << EOF
BETTER_AUTH_SECRET=$(openssl rand -hex 32)
ENCRYPTION_KEY=$(openssl rand -hex 32)
INTERNAL_API_SECRET=$(openssl rand -hex 32)
CRON_SECRET=$(openssl rand -hex 32)
EOF
docker compose -f docker-compose.prod.yml up -d
```
Open [http://localhost:3000](http://localhost:3000)
<Callout type="info">
The `.env` step is not optional — the compose file refuses to start without the first three rather than booting with empty values. `CRON_SECRET` is what the scheduler uses; without it the `cron` service exits with instructions and everything else still runs. See [Docker](/platform/self-hosting/docker) for the full production setup.
</Callout>
## Deployment Options
<Cards>
@@ -55,34 +67,63 @@ Open [http://localhost:3000](http://localhost:3000)
</Card>
</Cards>
### Which one to pick
Both deploy the same feature set. Docker Compose is the fastest way to evaluate Sim and is fine for a single-node team install; Kubernetes is the path for high availability and managed secrets.
| Capability | Docker Compose | Kubernetes (Helm) |
|---|---|---|
| App, realtime, migrations, Postgres, Redis | Yes | Yes |
| Scheduled workflows and polling triggers | Yes — `cron` service | Yes — 18 CronJobs |
| Horizontal scaling / HA | No (single node) | Yes (`replicaCount`, HPA, PDB) |
| Managed secrets (Vault, ESO, cloud KMS) | Manual `.env` | Yes |
| Network policy, Pod Security Standards | Host-level only | Yes |
| PII redaction, OpenTelemetry collector | Not bundled | Optional components |
The remaining differences are inherent to the platform — Compose has no analogue of a HorizontalPodAutoscaler or a PodDisruptionBudget. Application behavior is the same on both.
## Where to go next
<Cards>
<Card title="Integrations & OAuth" href="/platform/self-hosting/integrations-oauth">
Required before any integration works — start here
</Card>
<Card title="Object Storage" href="/platform/self-hosting/object-storage">
Required before storing anything you care about
</Card>
<Card title="Verify Your Install" href="/platform/self-hosting/verify">
Ten-minute smoke test across every subsystem
</Card>
</Cards>
The sidebar covers the rest: architecture, email, Redis, authentication, background jobs, networking, security, scaling, observability, and upgrades.
## Enterprise Features
Organizations, SSO, permission groups, audit logs, whitelabeling, session policies, data retention, and data drains all run on a self-hosted deployment — no billing or subscription required. One switch turns on the set:
Organizations, SSO, permission groups, audit logs, whitelabeling, session policies, data retention, and data drains all run on a self-hosted deployment with no billing or subscription. Turn the set on with:
```bash
ENTERPRISE_ENABLED=true
NEXT_PUBLIC_ENTERPRISE_ENABLED=true
```
Most of these features read their settings from the organization that owns a workspace, so enabling the flags is only half of it — your deployment also needs an organization model. Set `INSTANCE_ORG_NAME` to put every user in one shared organization automatically, or provision organizations yourself through the Admin API.
Your deployment also needs an organization model for most of them to apply. The [self-hosted enterprise guide](/platform/enterprise/self-hosted) covers both patterns, the per-feature flags, and troubleshooting.
See the [self-hosted enterprise guide](/platform/enterprise/self-hosted) for both patterns, the per-feature flags, and troubleshooting.
## External dependencies to plan for
## Architecture
Sim is self-contained for the core editor and execution engine. A few features reach outside the deployment:
| Component | Port | Description |
|-----------|------|-------------|
| simstudio | 3000 | Main application |
| realtime | 3002 | WebSocket server |
| db | 5432 | PostgreSQL with pgvector |
| migrations | - | Database migrations (runs once) |
| Feature | Requires | Notes |
|---|---|---|
| **Knowledge bases** | An OpenAI, Azure OpenAI, or Gemini API key | Embeddings are generated by a hosted provider, selected with `KB_EMBEDDING_MODEL` (`text-embedding-3-small` by default). There is no local embedding backend — knowledge bases are unavailable without one of these keys. |
| **Agent blocks** | An API key for at least one model provider | Or a self-hosted OpenAI-compatible endpoint: Ollama, vLLM, or LiteLLM. |
| **Chat module** | `COPILOT_API_KEY` from sim.ai | Set `NEXT_PUBLIC_CHAT_DISABLED=true` to hide the module instead. |
| **Integrations** | Your own OAuth app per service | See [Integrations & OAuth](/platform/self-hosting/integrations-oauth). |
| **Function / Pi blocks at scale** | Optional E2B or Daytona key | Without one, code runs in the in-process isolated-vm sandbox. See [Security](/platform/self-hosting/security). |
<FAQ items={[
{ question: "What are the minimum requirements to self-host Sim?", answer: "At minimum you need 2 CPU cores, 12 GB RAM, 20 GB SSD storage, and Docker 20.10 or later. Memory is typically the constraining factor due to workflow execution (isolated-vm sandboxing), file processing, and vector operations (pgvector)." },
{ question: "Which database does Sim use?", answer: "Sim uses PostgreSQL 17 with the pgvector extension for vector similarity search. The Docker setup uses the pgvector/pgvector:pg17 image, which comes with the vector extension pre-installed." },
{ question: "What are the required environment variables for production?", answer: "Three secrets are required: BETTER_AUTH_SECRET (authentication), ENCRYPTION_KEY (data encryption), and INTERNAL_API_SECRET (service-to-service auth). Generate each with openssl rand -hex 32. You also need to set NEXT_PUBLIC_APP_URL and BETTER_AUTH_URL to your domain." },
{ question: "What does the realtime service do?", answer: "The realtime service is a dedicated WebSocket server that runs on port 3002. It handles real-time communication for features like live workflow execution updates. It has a 1 GB memory limit and requires its own DATABASE_URL and BETTER_AUTH_SECRET configuration." },
{ question: "How does the migrations service work?", answer: "The migrations container runs once at startup (restart: no) and executes bun run db:migrate to apply database schema changes. It depends on the database being healthy before running and must complete successfully before the main application starts." },
{ question: "What are the minimum requirements to self-host Sim?", answer: "At minimum you need 2 CPU cores, 12 GB RAM, 20 GB SSD storage, and Docker 20.10 or later. Memory is typically the constraining factor due to workflow execution (isolated-vm sandboxing), file processing, and vector operations (pgvector)."},
{ question: "What are the required environment variables for production?", answer: "Three secrets are required: BETTER_AUTH_SECRET (authentication), ENCRYPTION_KEY (data encryption), and INTERNAL_API_SECRET (service-to-service auth). Generate each with openssl rand -hex 32. You also need to set NEXT_PUBLIC_APP_URL and BETTER_AUTH_URL to your domain."},
{ question: "Can I use Sim with local AI models?", answer: "Yes. Sim supports Ollama for local model inference. Use docker-compose.ollama.yml instead of docker-compose.prod.yml. It offers both GPU (with NVIDIA support) and CPU-only profiles, and automatically pulls gemma3:4b as a starter model." },
]} />
apps/docs/content/docs/en/platform/self-hosting/integrations-oauth.mdx
+197
View File
@@ -0,0 +1,197 @@
---
title: Integrations & OAuth
description: Register OAuth apps so your users can connect Slack, Google, Jira, and every other integration
---
import { Callout } from 'fumadocs-ui/components/callout'
import { Step, Steps } from 'fumadocs-ui/components/steps'
import { FAQ } from '@/components/ui/faq'
<Callout type="warn">
**On a self-hosted deployment, integrations do not work until you register your own OAuth application with each service.** Sim's hosted platform ships credentials for every integration; a self-hosted instance ships none. Users will see the connector in the UI, click "Connect", and get an error from the provider until the corresponding `*_CLIENT_ID` and `*_CLIENT_SECRET` are set.
</Callout>
You only need to register the services your team actually uses. One OAuth app covers every Sim connector that shares its credential — a single Google app serves Gmail, Drive, Sheets, Calendar, Docs, Forms, BigQuery, and more.
## How it works
Each connector has a **provider ID**. When a user connects an account, Sim redirects them to the provider, and the provider redirects back to:
```
https://<your-sim-domain>/api/auth/oauth2/callback/<provider-id>
```
That URL is derived from `NEXT_PUBLIC_APP_URL`, so set it correctly before registering anything — the redirect URI you register with the provider must match byte for byte, including scheme and the absence of a trailing slash.
<Callout type="info">
Most providers let you register several redirect URIs on one app. Register your production URL and any staging URL together so one OAuth app serves both environments.
</Callout>
## Setup
<Steps>
<Step>
### Confirm your public URL
```bash
NEXT_PUBLIC_APP_URL=https://sim.yourdomain.com
BETTER_AUTH_URL=https://sim.yourdomain.com
```
Both must be your real public origin. If these are wrong, every OAuth round-trip fails with a redirect-URI mismatch.
</Step>
<Step>
### Register an app with the provider
In the provider's developer console, create an OAuth 2.0 application. Register the redirect URI(s) for every Sim connector you want from that provider — one line per provider ID from the tables below.
For a Google app covering Gmail and Drive, for example, you register both:
```
https://sim.yourdomain.com/api/auth/oauth2/callback/google-email
https://sim.yourdomain.com/api/auth/oauth2/callback/google-drive
```
Scopes are requested by Sim at authorization time; you generally do not need to pre-declare them, but Google and Microsoft require you to enable the corresponding APIs on the project/app first (for example Gmail API, Drive API, Calendar API).
</Step>
<Step>
### Set the credentials
Add the client ID and secret to the app's environment. In Kubernetes they go under `app.env` — the chart writes every key there into a chart-managed Secret — but supply the values through External Secrets or a pre-created Secret rather than committing them to a values file:
```yaml
app:
env:
GOOGLE_CLIENT_ID: "..."
GOOGLE_CLIENT_SECRET: "..."
SLACK_CLIENT_ID: "..."
SLACK_CLIENT_SECRET: "..."
```
Restart the app. Credentials are read at startup — a running pod will not pick up new ones.
</Step>
<Step>
### Verify
Open a workflow, add the integration's block, and connect an account. A successful round-trip returns you to Sim with the account listed. A redirect-URI mismatch is the failure you will hit most; compare the registered URI against `NEXT_PUBLIC_APP_URL` character by character.
</Step>
</Steps>
## Provider reference
Every provider ID below maps to the redirect URI `https://<your-domain>/api/auth/oauth2/callback/<provider-id>`.
### Google
One OAuth client in [Google Cloud Console](https://console.cloud.google.com/apis/credentials) covers all of these. Enable the matching API for each connector you use.
| Environment variables | Provider IDs |
|---|---|
| `GOOGLE_CLIENT_ID`<br />`GOOGLE_CLIENT_SECRET` | `google-email`, `google-drive`, `google-sheets`, `google-docs`, `google-calendar`, `google-contacts`, `google-forms`, `google-tasks`, `google-meet`, `google-groups`, `google-ads`, `google-bigquery`, `google-vault`, `vertex-ai` |
The same variables also power "Sign in with Google". See [Authentication](/platform/self-hosting/authentication).
### Microsoft
One app registration in [Entra ID](https://entra.microsoft.com) covers all of these.
| Environment variables | Provider IDs |
|---|---|
| `MICROSOFT_CLIENT_ID`<br />`MICROSOFT_CLIENT_SECRET` | `outlook`, `onedrive`, `sharepoint`, `microsoft-teams`, `microsoft-excel`, `microsoft-planner`, `microsoft-dataverse`, `microsoft-ad` |
The same variables also power "Sign in with Microsoft".
### Everything else
| Service | Environment variables | Provider ID |
|---|---|---|
| Slack | `SLACK_CLIENT_ID` / `SLACK_CLIENT_SECRET` | `slack` |
| Notion | `NOTION_CLIENT_ID` / `NOTION_CLIENT_SECRET` | `notion` |
| Jira | `JIRA_CLIENT_ID` / `JIRA_CLIENT_SECRET` | `jira` |
| Confluence | `CONFLUENCE_CLIENT_ID` / `CONFLUENCE_CLIENT_SECRET` | `confluence` |
| Linear | `LINEAR_CLIENT_ID` / `LINEAR_CLIENT_SECRET` | `linear` |
| Asana | `ASANA_CLIENT_ID` / `ASANA_CLIENT_SECRET` | `asana` |
| ClickUp | `CLICKUP_CLIENT_ID` / `CLICKUP_CLIENT_SECRET` | `clickup` |
| Monday | `MONDAY_CLIENT_ID` / `MONDAY_CLIENT_SECRET` | `monday` |
| Airtable | `AIRTABLE_CLIENT_ID` / `AIRTABLE_CLIENT_SECRET` | `airtable` |
| HubSpot | `HUBSPOT_CLIENT_ID` / `HUBSPOT_CLIENT_SECRET` | `hubspot` |
| Salesforce | `SALESFORCE_CLIENT_ID` / `SALESFORCE_CLIENT_SECRET` | `salesforce` |
| Pipedrive | `PIPEDRIVE_CLIENT_ID` / `PIPEDRIVE_CLIENT_SECRET` | `pipedrive` |
| Attio | `ATTIO_CLIENT_ID` / `ATTIO_CLIENT_SECRET` | `attio` |
| Zoho Desk | `ZOHO_CLIENT_ID` / `ZOHO_CLIENT_SECRET` | `zoho-desk` |
| Wealthbox | `WEALTHBOX_CLIENT_ID` / `WEALTHBOX_CLIENT_SECRET` | `wealthbox` |
| Box | `BOX_CLIENT_ID` / `BOX_CLIENT_SECRET` | `box` |
| Dropbox | `DROPBOX_CLIENT_ID` / `DROPBOX_CLIENT_SECRET` | `dropbox` |
| DocuSign | `DOCUSIGN_CLIENT_ID` / `DOCUSIGN_CLIENT_SECRET` | `docusign` |
| Zoom | `ZOOM_CLIENT_ID` / `ZOOM_CLIENT_SECRET` | `zoom` |
| Cal.com | `CALCOM_CLIENT_ID` only — PKCE public client, no secret | `calcom` |
| Webflow | `WEBFLOW_CLIENT_ID` / `WEBFLOW_CLIENT_SECRET` | `webflow` |
| WordPress | `WORDPRESS_CLIENT_ID` / `WORDPRESS_CLIENT_SECRET` | `wordpress` |
| LinkedIn | `LINKEDIN_CLIENT_ID` / `LINKEDIN_CLIENT_SECRET` | `linkedin` |
| X | `X_CLIENT_ID` / `X_CLIENT_SECRET` | `x` |
| Reddit | `REDDIT_CLIENT_ID` / `REDDIT_CLIENT_SECRET` | `reddit` |
| Spotify | `SPOTIFY_CLIENT_ID` / `SPOTIFY_CLIENT_SECRET` | `spotify` |
| TikTok | `TIKTOK_CLIENT_ID` / `TIKTOK_CLIENT_SECRET` | `tiktok` |
### Services with a different flow
| Service | Configuration | Notes |
|---|---|---|
| **Instagram** | `INSTAGRAM_CLIENT_ID` / `INSTAGRAM_CLIENT_SECRET` | Instagram App ID/Secret from the Meta App Dashboard (Instagram → API setup with Instagram login). Redirect URI: `/api/auth/oauth2/callback/instagram`. **Publishing requires cloud object storage** — Meta fetches a public HTTPS URL, so local-disk storage will not work. |
| **Shopify** | `SHOPIFY_CLIENT_ID` / `SHOPIFY_CLIENT_SECRET` | Redirect URI: `/api/auth/oauth2/callback/shopify`. Per-shop install flow. |
| **Trello** | `TRELLO_API_KEY` | API-key based, not OAuth 2.0. Callback: `/api/auth/trello/callback`. |
## Non-OAuth integration credentials
Many blocks authenticate with an API key the user pastes into the block, and need nothing from you.
<Callout type="warn">
Sim also has a "hosted key" mechanism, configured with the `{PREFIX}_API_KEY_COUNT` + `{PREFIX}_API_KEY_1..N` variables below, that lets the platform supply a key so users do not have to. **The injection path is gated on the deployment being Sim's hosted platform** (`isHosted`, derived from the app hostname), so on a self-hosted instance these variables do not remove the need for users to bring their own key. Set them only if you are running a fork that has adapted that gate.
</Callout>
The variables, for reference:
| Variable | Service |
|---|---|
| `EXA_API_KEY` (or `EXA_API_KEY_COUNT` + `EXA_API_KEY_1..N`) | Exa search |
| `SERPER_API_KEY` | Serper search |
| `BROWSERBASE_API_KEY` / `BROWSERBASE_PROJECT_ID` | Browserbase |
| `HUNTER_API_KEY_COUNT` + `HUNTER_API_KEY_1..N` | Hunter.io |
| `PEOPLEDATALABS_API_KEY_COUNT` + `PEOPLEDATALABS_API_KEY_1..N` | People Data Labs |
| `CONTEXT_DEV_API_KEY_COUNT` + `CONTEXT_DEV_API_KEY_1..N` | Context.dev |
| `FALAI_API_KEY` | fal.ai |
| `TWILIO_ACCOUNT_SID` / `TWILIO_AUTH_TOKEN` / `TWILIO_PHONE_NUMBER` | Twilio |
| `AGENTMAIL_API_KEY` / `AGENTMAIL_DOMAIN` | AgentMail |
Providers that take a `_COUNT` plus numbered keys distribute requests round-robin across them.
## Triggers that need extra configuration
Webhook triggers receive callbacks from the provider and must be able to verify them:
| Variable | Needed for |
|---|---|
| `SLACK_SIGNING_SECRET` | Verifying Slack event and slash-command signatures |
| `SLACK_EXTENDED_SCOPES` / `NEXT_PUBLIC_SLACK_EXTENDED_SCOPES` | Requesting the broader Slack scope set |
Your deployment must also be reachable from the provider's servers for webhook triggers to fire — a Sim instance on a private network can use polling triggers but not webhook triggers. Polling triggers additionally require the scheduler; see [Background Jobs](/platform/self-hosting/background-jobs).
<FAQ items={[
{ question: "Do I have to register an OAuth app for every integration?", answer: "Only for the integrations your team uses. Nothing breaks if a provider's credentials are unset — that connector simply cannot be connected. One Google app covers 14 connectors and one Microsoft app covers 8, so most deployments need only a handful of registrations."},
{ question: "Users get a redirect_uri_mismatch error. What is wrong?", answer: "Almost always NEXT_PUBLIC_APP_URL does not match the URI registered with the provider. Check for http vs https, a trailing slash, an apex-vs-www difference, or a port. The URI Sim sends is built from NEXT_PUBLIC_APP_URL at request time."},
{ question: "Can I use the same OAuth app for staging and production?", answer: "Yes, if the provider allows multiple redirect URIs on one app — register both hosts. Providers that allow only one URI need a separate app per environment."},
{ question: "Why does the Instagram integration need cloud storage?", answer: "Meta fetches media from a public HTTPS URL rather than accepting an upload, so published files must live in S3, Azure Blob, or GCS. Local-disk storage cannot serve them. Gmail attachments and other integrations do not have this constraint." },
]} />
apps/docs/content/docs/en/platform/self-hosting/kubernetes.mdx
+73 -9
View File
@@ -7,11 +7,18 @@ import { Tab, Tabs } from 'fumadocs-ui/components/tabs'
import { Callout } from 'fumadocs-ui/components/callout'
import { FAQ } from '@/components/ui/faq'
<Callout type="info">
The chart's own [`helm/sim/README.md`](https://github.com/simstudioai/sim/tree/main/helm/sim) is the reference for every value, and goes deeper than this page on secret strategies, network policy, PII redaction, and per-error troubleshooting. This page covers the deployment path; read that alongside it.
</Callout>
## Prerequisites
- Kubernetes 1.25+
- Helm 3.8+
- PV provisioner support
- PV provisioner support (a default StorageClass supporting `ReadWriteOnce`)
- An ingress controller, if `ingress.enabled=true`
- `metrics-server`, if you enable autoscaling
- **Redis**, if you plan to run more than one replica — see [Redis](/platform/self-hosting/redis)
## Installation
@@ -36,9 +43,19 @@ helm install sim ./helm/sim \
--namespace simstudio --create-namespace
```
<Callout type="warn">
Save all five values somewhere durable before moving on. `ENCRYPTION_KEY` in particular cannot be regenerated — losing it makes workspace environment variables and stored provider keys permanently unreadable.
`CRON_SECRET` is required, not optional: the chart enables background jobs by default and will not render without it.
</Callout>
<Callout type="info">
This installs the chart's default image tag. For production, **pin `app`, `realtime`, and `migrations` to the same explicit release tag** — see [Upgrades](/platform/self-hosting/upgrades).
</Callout>
## Cloud-Specific Values
These are cloud-tuned **alternatives** to the generic install above — pick one path, don't run both. The commands reuse the `$BETTER_AUTH_SECRET`, `$ENCRYPTION_KEY`, `$INTERNAL_API_SECRET`, `$CRON_SECRET`, and `$POSTGRES_PASSWORD` variables generated in [Installation](#installation) above, so run that block's `openssl` lines first in the same shell. They use `helm upgrade --install`, so they work whether or not a release exists yet. Two caveats when converting an existing generic install rather than starting fresh: (1) **reuse the original secret values** — recover them with `helm get values sim -n simstudio` if your shell no longer has them; supplying a newly generated `ENCRYPTION_KEY` makes every previously encrypted credential (OAuth tokens, provider keys, environment variables) undecryptable. (2) The cloud values rename the bundled PostgreSQL database to `simstudio`, but Postgres only applies that setting on first initialization — add `--set postgresql.auth.database=sim` to keep your existing database. If you'd rather start clean, `helm uninstall sim -n simstudio`, delete its PVCs, and run the cloud command fresh.
These are cloud-tuned **alternatives** to the generic install above — pick one path, don't run both. The commands reuse the `$BETTER_AUTH_SECRET`, `$ENCRYPTION_KEY`, `$INTERNAL_API_SECRET`, `$CRON_SECRET`, and `$POSTGRES_PASSWORD` variables generated in [Installation](#installation) above, so run that block's `openssl` lines first in the same shell. They use `helm upgrade --install`, so they work whether or not a release exists yet. Two caveats when converting an existing generic install rather than starting fresh: (1) **reuse the original secret values** — recover them with `helm get values sim -n simstudio` if your shell no longer has them; supplying a newly generated `ENCRYPTION_KEY` makes every previously encrypted value (workspace environment variables, stored provider keys, MCP OAuth credentials) undecryptable. (2) The cloud values rename the bundled PostgreSQL database to `simstudio`, but Postgres only applies that setting on first initialization — add `--set postgresql.auth.database=sim` to keep your existing database. If you'd rather start clean, `helm uninstall sim -n simstudio`, delete its PVCs, and run the cloud command fresh.
<Tabs items={['AWS EKS', 'Azure AKS', 'GCP GKE']}>
<Tab value="AWS EKS">
@@ -88,9 +105,22 @@ helm upgrade --install sim ./helm/sim \
# Custom values.yaml
app:
replicaCount: 2
image:
tag: "v1.2.3" # a tag from the releases page
env:
NEXT_PUBLIC_APP_URL: "https://sim.yourdomain.com"
BETTER_AUTH_URL: "https://sim.yourdomain.com"
OPENAI_API_KEY: "sk-..."
# Required once replicaCount > 1
REDIS_URL: "redis://:<password>@redis.internal:6379"
realtime:
image:
tag: "v1.2.3" # a tag from the releases page
migrations:
image:
tag: "v1.2.3" # a tag from the releases page
postgresql:
persistence:
@@ -105,7 +135,43 @@ ingress:
host: sim.yourdomain.com
```
See `helm/sim/values.yaml` for all options.
<Callout type="error">
`NEXT_PUBLIC_APP_URL` and `BETTER_AUTH_URL` must both be your real public origin. Leaving either at `localhost` breaks sign-in.
</Callout>
<Callout type="info">
Keys set under `app.env` land on the realtime pod too — the chart writes them into one Secret that both Deployments consume via `envFrom`. Use `realtime.env` only for keys that must differ between the two, such as `ALLOWED_ORIGINS`.
</Callout>
<Callout type="error">
Setting `replicaCount: 2` **without** `REDIS_URL` silently breaks live collaboration and status updates — cross-pod events are dropped with no error anywhere. See [Redis](/platform/self-hosting/redis) and [Scaling & HA](/platform/self-hosting/scaling).
</Callout>
See `helm/sim/values.yaml` for all options, and the chart's [README](https://github.com/simstudioai/sim/tree/main/helm/sim) for the production checklist.
## Background jobs
The chart deploys 18 CronJobs by default, driving scheduled workflows, polling triggers, connector syncs, data drains, and outbox processing. They require `CRON_SECRET`.
```bash
kubectl get cronjobs -n simstudio
```
A CronJob with a stale `LAST SCHEDULE` means the corresponding feature has stopped working. See [Background Jobs](/platform/self-hosting/background-jobs).
## Ingress and TLS
For nginx-style ingress controllers, raise the body-size and timeout limits — Sim's defaults allow large chat attachments and long-running executions:
```yaml
ingress:
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: "250m"
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-buffering: "off"
```
On GKE, the load balancer's 30-second default backend timeout closes websockets and needs a `BackendConfig`, and the `ManagedCertificate` the chart references must be created by you. Both are covered in [Networking](/platform/self-hosting/networking).
## External Database
@@ -140,11 +206,9 @@ helm uninstall sim --namespace simstudio
```
<FAQ items={[
{ question: "What are the default resource limits for the Sim app pod?", answer: "The Helm chart defaults to 8 Gi memory limit / 4 Gi request and 2000m CPU limit / 1000m request per app pod. The pod runs as non-root (UID 1001) with fsGroup 1001 for security." },
{ question: "Can I use an external database instead of the bundled PostgreSQL?", answer: "Yes. Set postgresql.enabled to false and configure the externalDatabase section with your host, port, username, password, database name, and sslMode. The external database must have the pgvector extension installed." },
{ question: "How do I manage secrets securely in Kubernetes?", answer: "The Helm chart supports pre-existing Kubernetes secrets via app.secrets.existingSecret. Set enabled to true and provide the secret name. This integrates with External Secrets Operator, HashiCorp Vault, Azure Key Vault, and similar tools. The secret must use the standard key names (BETTER_AUTH_SECRET, ENCRYPTION_KEY, INTERNAL_API_SECRET, CRON_SECRET, ...) — it is consumed wholesale, so key remapping is not supported." },
{ question: "Can I scale the app to multiple replicas?", answer: "Yes. Set app.replicaCount in your values.yaml. The chart supports standard Kubernetes scaling. Ensure your database can handle the additional connections from multiple replicas." },
{ question: "Are there cloud-specific example configurations?", answer: "Yes. The Helm chart includes example values files for AWS EKS (values-aws.yaml), Azure AKS (values-azure.yaml), GCP GKE (values-gcp.yaml), as well as files for production, development, external databases, external secrets, and whitelabeled deployments." },
{ question: "What ingress options are supported?", answer: "The chart supports ingress with configurable className (e.g., nginx), TLS termination, and separate host configuration for the app. Enable it with ingress.enabled: true and set your host under ingress.app.host." },
{ question: "How do I manage secrets securely in Kubernetes?", answer: "The Helm chart supports pre-existing Kubernetes secrets via app.secrets.existingSecret. Set enabled to true and provide the secret name. This integrates with External Secrets Operator, HashiCorp Vault, Azure Key Vault, and similar tools. The secret must use the standard key names (BETTER_AUTH_SECRET, ENCRYPTION_KEY, INTERNAL_API_SECRET, CRON_SECRET, ...) — it is consumed wholesale, so key remapping is not supported."},
{ question: "Can I scale the app to multiple replicas?", answer: "Yes, but set REDIS_URL first. Without Redis, pub/sub and the Socket.IO adapter have no cross-pod transport and live collaboration breaks silently. Also make sure your database can handle the additional connections — see the Scaling & HA guide."},
{ question: "Is CRON_SECRET required?", answer: "Yes, whenever background jobs are enabled — which is the chart's default. The chart refuses to render without it, and the 18 CronJobs that drive scheduled workflows, polling triggers, and connector syncs use it as a bearer token."},
{ question: "Which image tags should I pin?", answer: "All three of app, realtime, and migrations, to the same explicit release tag. They share a database schema, so a mismatch runs the app against a schema it does not match. Do not rely on the chart's default tag in production."},
]} />
apps/docs/content/docs/en/platform/self-hosting/meta.json
+16 -1
View File
@@ -2,11 +2,26 @@
"title": "Self-Hosting",
"pages": [
"index",
"architecture",
"---Install---",
"docker",
"kubernetes",
"platforms",
"object-storage",
"---Configure---",
"environment-variables",
"object-storage",
"email",
"redis",
"integrations-oauth",
"authentication",
"background-jobs",
"networking",
"security",
"verify",
"---Operate---",
"observability",
"scaling",
"upgrades",
"troubleshooting"
],
"defaultOpen": false
apps/docs/content/docs/en/platform/self-hosting/networking.mdx
+273
View File
@@ -0,0 +1,273 @@
---
title: Networking
description: Reverse proxies, TLS, websockets, timeouts, and request size limits
---
import { Tab, Tabs } from 'fumadocs-ui/components/tabs'
import { Callout } from 'fumadocs-ui/components/callout'
import { FAQ } from '@/components/ui/faq'
Sim has three traffic patterns that trip up default proxy configurations: **long-lived websockets**, **server-sent event streams**, and **large uploads**. Most "it works locally but not in production" reports come from one of the three.
## Topology
Two services need to be reachable. You can put them on one hostname or two.
<Tabs items={['Single hostname', 'Two hostnames']}>
<Tab value="Single hostname">
Simplest. Route `/socket.io` to realtime and everything else to the app.
```
sim.yourdomain.com/ → app:3000
sim.yourdomain.com/socket.io → realtime:3002
```
`NEXT_PUBLIC_SOCKET_URL` can be left unset — the client defaults to the page origin.
</Tab>
<Tab value="Two hostnames">
Required by ingress controllers that cannot cleanly split paths across backends, and preferred on GKE's built-in load balancer.
```
sim.yourdomain.com → app:3000
sim-ws.yourdomain.com → realtime:3002
```
Then tell the client where realtime lives, and tell realtime which origins to accept:
```yaml
app:
env:
NEXT_PUBLIC_APP_URL: "https://sim.yourdomain.com"
BETTER_AUTH_URL: "https://sim.yourdomain.com"
NEXT_PUBLIC_SOCKET_URL: "https://sim-ws.yourdomain.com"
realtime:
env:
ALLOWED_ORIGINS: "https://sim.yourdomain.com"
```
`ALLOWED_ORIGINS` is the CORS allowlist realtime enforces on socket connections; it must contain the app's origin. The URL keys only need to be set once under `app.env` — the chart writes them into a Secret both Deployments consume.
Both hostnames need DNS records and TLS certificates.
</Tab>
</Tabs>
## Reverse proxy configuration
<Tabs items={['Caddy', 'Nginx', 'Traefik']}>
<Tab value="Caddy">
Caddy handles certificates, websockets, and streaming correctly by default.
```
sim.yourdomain.com {
request_body {
max_size 250MB
}
handle /socket.io/* {
reverse_proxy localhost:3002
}
reverse_proxy localhost:3000 {
flush_interval -1
}
}
```
`flush_interval -1` disables response buffering, which keeps streamed agent output flowing token by token instead of arriving in one block at the end.
</Tab>
<Tab value="Nginx">
Nginx buffers responses and times out idle connections by default. Both need overriding.
```nginx
server {
listen 443 ssl http2;
server_name sim.yourdomain.com;
# Large file uploads (chat attachments can reach ~220 MB)
client_max_body_size 250M;
location / {
proxy_pass http://127.0.0.1:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Streamed responses must not be buffered
proxy_buffering off;
proxy_cache off;
# Long-running workflow executions
proxy_read_timeout 3600s;
proxy_send_timeout 3600s;
}
location /socket.io/ {
proxy_pass http://127.0.0.1:3002;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 3600s;
proxy_send_timeout 3600s;
}
}
```
For ingress-nginx, the equivalents are annotations:
```yaml
ingress:
annotations:
nginx.ingress.kubernetes.io/proxy-body-size: "250m"
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-buffering: "off"
```
</Tab>
<Tab value="Traefik">
```yaml
ingress:
className: traefik
annotations:
traefik.ingress.kubernetes.io/router.entrypoints: websecure
traefik.ingress.kubernetes.io/router.tls: "true"
```
Set the read/idle timeouts on the entrypoint, since they are static configuration rather than per-ingress:
```yaml
entryPoints:
websecure:
address: ":443"
transport:
respondingTimeouts:
readTimeout: 3600s
idleTimeout: 3600s
```
Traefik streams responses by default and needs no buffering override.
</Tab>
</Tabs>
## Cloud load balancers
### GKE (GCE ingress)
<Callout type="warn">
The GCE load balancer defaults to a **30-second backend timeout**, which closes every websocket every 30 seconds. Clients reconnect, so this degrades rather than breaks — but collaboration feels unreliable and reconnect storms add load. Fix it with a `BackendConfig` on the realtime Service.
</Callout>
```yaml
apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
name: sim-realtime-backendconfig
namespace: simstudio
spec:
timeoutSec: 3600
connectionDraining:
drainingTimeoutSec: 60
```
Then annotate the realtime Service so the load balancer picks it up:
```yaml
realtime:
service:
annotations:
cloud.google.com/backend-config: '{"default": "sim-realtime-backendconfig"}'
```
Confirm your chart version renders `realtime.service.annotations` onto the Service (`helm template ./helm/sim --values my-values.yaml | grep -A5 'kind: Service'`). If it does not, annotate the Service directly with `kubectl annotate`.
TLS on GKE typically uses a **ManagedCertificate**, which the chart references by annotation but **does not create** — create it yourself before the first deploy:
```yaml
apiVersion: networking.gke.io/v1
kind: ManagedCertificate
metadata:
name: sim-ssl-cert
namespace: simstudio
spec:
domains:
- sim.yourdomain.com
- sim-ws.yourdomain.com
```
```yaml
ingress:
className: gce
annotations:
kubernetes.io/ingress.global-static-ip-name: "sim-ip"
networking.gke.io/managed-certificates: "sim-ssl-cert"
kubernetes.io/ingress.allow-http: "false"
# TLS comes from the ManagedCertificate — leaving the chart's secret-based
# TLS on makes the ingress reference a Secret that does not exist.
tls:
enabled: false
```
The certificate provisions once DNS resolves, typically 15–30 minutes after the first deploy.
### AWS (ALB ingress)
```yaml
ingress:
className: alb
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
alb.ingress.kubernetes.io/certificate-arn: "arn:aws:acm:..."
alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=3600
```
The ALB's default 60-second idle timeout also closes websockets. Raise it as shown.
### Azure (Application Gateway / NGINX)
Application Gateway's default request timeout is 30 seconds; raise it in the backend HTTP setting. Many AKS deployments use ingress-nginx instead — see the Nginx tab above.
## Request size limits
Sim enforces its own limits in addition to whatever your proxy allows. The proxy limit must be **at least** as large as the app limit, or the proxy rejects the request before Sim ever sees it.
| Variable | Default | Applies to |
|---|---|---|
| `API_MAX_JSON_BODY_BYTES` | 50 MB | Contract-validated API routes |
| `CHAT_MAX_REQUEST_BYTES` | 220 MB | The public deployed-chat endpoint (covers ~15 base64 file attachments) |
| `WEBHOOK_MAX_REQUEST_BYTES` | 10 MB | Public webhook receiver endpoints |
A proxy body limit of 250 MB accommodates all three defaults. If you lower the app limits, you can lower the proxy limit to match.
<Callout type="info">
**With object storage configured**, regular file uploads do not flow through the proxy — the browser `PUT`s them directly to the bucket using a presigned URL, so the proxy limits matter only for chat attachments, API payloads, and webhook bodies. On the default local-disk storage there is no presigned path and every upload goes through the proxy, so its body limit applies to all of them.
</Callout>
## Outbound connectivity
The app makes outbound calls to model providers, integration APIs, your email provider, and object storage. There is no global forward-proxy setting. Sim does not read `HTTP_PROXY` / `HTTPS_PROXY`, so model-provider calls, integration calls, and email delivery cannot be routed through a forward proxy. (The HTTP Request block accepts a per-request `proxyUrl`, but that covers only that one block, not the platform's own outbound traffic.) Environments with a mandatory egress proxy need a transparent proxy or NAT-based egress instead.
<FAQ items={[
{ question: "Do I need two hostnames?", answer: "No. A single hostname works if your proxy can route /socket.io to the realtime service and everything else to the app. Two hostnames are useful when path-based splitting is awkward, which is common on GKE's built-in load balancer."},
{ question: "Websockets disconnect every 30 seconds on GKE. How do I fix it?", answer: "The GCE load balancer defaults to a 30-second backend timeout. Create a BackendConfig with timeoutSec: 3600 and attach it to the realtime Service with the cloud.google.com/backend-config annotation. On AWS, raise the ALB idle_timeout instead."},
{ question: "Does Sim support an outbound HTTP proxy?", answer: "Not globally. Sim does not read HTTP_PROXY or HTTPS_PROXY, so platform traffic cannot be routed through a forward proxy. The HTTP Request block accepts a per-request proxyUrl for its own calls only. Use a transparent proxy or NAT-based egress if your network requires one."},
]} />
apps/docs/content/docs/en/platform/self-hosting/object-storage.mdx
+41 -11
View File
@@ -40,7 +40,7 @@ If more than one backend is configured, the first match in that order wins. Set
### Create the buckets
Sim separates files into purpose-specific buckets. At minimum you need the general workspace bucket; the rest are created on demand based on which env vars you set. A bucket that isn't configured falls back to the general bucket where the code allows it, but the recommended setup is one bucket per purpose.
Sim separates files into purpose-specific buckets. **Sim never creates buckets** — create each one yourself before configuring it. Only `S3_OG_IMAGES_BUCKET_NAME` and `S3_WORKSPACE_LOGOS_BUCKET_NAME` fall back to the general bucket; the others resolve to their own literal defaults, so set every bucket you intend to use.
```bash
# Set your region once
@@ -66,6 +66,41 @@ Keep all buckets **private** (block public access). Sim serves files through sho
<Step>
### Configure CORS on every bucket
Uploads are sent **directly from the browser** to S3 via presigned `PUT` requests, so each bucket needs a CORS policy that allows your Sim origin. Without this, every upload fails with a CORS error in the browser console even though the server-side configuration is correct.
```bash
cat > /tmp/cors.json <<'EOF'
{
"CORSRules": [
{
"AllowedOrigins": ["https://sim.yourdomain.com"],
"AllowedMethods": ["GET", "PUT"],
"AllowedHeaders": ["*"],
"ExposeHeaders": ["ETag"],
"MaxAgeSeconds": 3600
}
]
}
EOF
for name in workspace-files knowledge-base execution-files chat-files \
copilot-files profile-pictures og-images workspace-logos; do
aws s3api put-bucket-cors --bucket "myorg-sim-$name" --cors-configuration file:///tmp/cors.json
done
```
<Callout type="warn">
`ExposeHeaders` **must** include `ETag`. Files larger than 50 MB use multipart uploads, and the browser reads each part's `ETag` to complete the upload — CORS hides the header otherwise and large uploads fail at the final step.
</Callout>
Set `AllowedOrigins` to your exact Sim origin (scheme + host, no trailing slash). Add every origin users reach Sim from, including an apex/`www` pair if both are live.
</Step>
<Step>
### Grant access with an IAM policy
Create an IAM policy scoped to your buckets and attach it to the user (or role) Sim runs as:
@@ -80,7 +115,9 @@ Create an IAM policy scoped to your buckets and attach it to the user (or role)
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject",
"s3:ListBucket"
"s3:ListBucket",
"s3:AbortMultipartUpload",
"s3:ListMultipartUploadParts"
],
"Resource": [
"arn:aws:s3:::myorg-sim-*",
@@ -136,13 +173,12 @@ Only `AWS_REGION` and `S3_BUCKET_NAME` are strictly required to switch Sim into
| `AWS_SECRET_ACCESS_KEY` | Secret key | No (uses credential chain if unset) |
| `S3_BUCKET_NAME` | General workspace files | **Yes** (enables S3) |
| `S3_KB_BUCKET_NAME` | Knowledge base documents | Recommended |
| `S3_EXECUTION_FILES_BUCKET_NAME` | Workflow execution files (default: `sim-execution-files`) | Recommended |
| `S3_EXECUTION_FILES_BUCKET_NAME` | Workflow execution files. Falls back to the literal name `sim-execution-files`, which you almost certainly do not own — always set this explicitly | **Yes** |
| `S3_CHAT_BUCKET_NAME` | Deployed chat assets | Recommended |
| `S3_COPILOT_BUCKET_NAME` | Copilot attachments | Recommended |
| `S3_PROFILE_PICTURES_BUCKET_NAME` | User avatars | Recommended |
| `S3_OG_IMAGES_BUCKET_NAME` | OpenGraph preview images (falls back to `S3_BUCKET_NAME`) | Optional |
| `S3_WORKSPACE_LOGOS_BUCKET_NAME` | Workspace logos (falls back to `S3_BUCKET_NAME`) | Optional |
| `S3_LOGS_BUCKET_NAME` | Stored logs | Optional |
| `S3_ENDPOINT` | Custom endpoint for S3-compatible storage (R2, MinIO, B2) | Optional (AWS S3 if unset) |
| `S3_FORCE_PATH_STYLE` | `true` for path-style addressing (MinIO/Ceph) | Optional (defaults `false`) |
@@ -420,11 +456,5 @@ After restarting with the new configuration:
If uploads fail, check the app logs for credential or permission errors (see [Troubleshooting](/platform/self-hosting/troubleshooting)).
<FAQ items={[
{ question: "What happens if I do not configure any storage variables?", answer: "Sim falls back to local disk, writing files to the /uploads directory inside the app container. This is fine for evaluation but not durable across container recreation and not shared across replicas — use S3, Azure Blob, or Google Cloud Storage for production." },
{ question: "Do I have to create all eight S3 buckets?", answer: "No. Only AWS_REGION and S3_BUCKET_NAME are required to enable S3 mode. The purpose-specific buckets are recommended so each file type is isolated; og-images and workspace-logos fall back to the general bucket if their variables are unset." },
{ question: "How do I avoid storing AWS keys in plaintext?", answer: "On EC2/ECS/EKS, attach the IAM policy to the instance role, task role, or IRSA service-account role and leave AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY unset. Sim resolves credentials through the default AWS SDK provider chain automatically." },
{ question: "Can I configure more than one cloud backend at the same time?", answer: "No. Sim selects a single backend, in the order Azure Blob, then S3, then Google Cloud Storage. Set only the variables for the backend you want." },
{ question: "How do I use GCS without storing a service-account key?", answer: "Run Sim with GKE Workload Identity (or an attached GCE service account) and leave GCS_CREDENTIALS_JSON unset — credentials resolve through Application Default Credentials. Grant the service account roles/iam.serviceAccountTokenCreator on itself so signed URLs can be generated via the IAM signBlob API." },
{ question: "Are the buckets exposed publicly?", answer: "No, and they should not be. Keep them private with public access blocked. Sim serves files to users through short-lived presigned URLs, so the buckets never need public read permissions." },
{ question: "Can I use MinIO or Cloudflare R2?", answer: "Yes. Configure it like AWS S3, then set S3_ENDPOINT to your provider's endpoint. For R2, set AWS_REGION=auto and leave S3_FORCE_PATH_STYLE unset. For MinIO/Ceph, set S3_FORCE_PATH_STYLE=true. See the S3-compatible provider section above." },
{ question: "How do I use GCS without storing a service-account key?", answer: "Run Sim with GKE Workload Identity (or an attached GCE service account) and leave GCS_CREDENTIALS_JSON unset — credentials resolve through Application Default Credentials. Grant the service account roles/iam.serviceAccountTokenCreator on itself so signed URLs can be generated via the IAM signBlob API."},
]} />
apps/docs/content/docs/en/platform/self-hosting/observability.mdx
+188
View File
@@ -0,0 +1,188 @@
---
title: Observability
description: Health checks, logs, tracing, and what to alert on
---
import { Callout } from 'fumadocs-ui/components/callout'
import { FAQ } from '@/components/ui/faq'
## Health endpoints
| Service | Endpoint | Returns |
|---|---|---|
| app | `GET /api/health` | `{"status":"ok","timestamp":"..."}` |
| realtime | `GET /health` on port 3002 | `{"status":"ok","timestamp":"...","connections":0}` |
<Callout type="warn">
`/api/health` is a **liveness** signal only. It returns `200` as long as the process is serving HTTP — it does not check the database, Redis, or object storage. A healthy response does not mean the app can serve traffic successfully, so do not treat it as a dependency check. Verify dependencies with the [smoke test](/platform/self-hosting/verify) instead.
</Callout>
## Kubernetes probes
The chart ships probes tuned for a Next.js cold start. Defaults for the app:
| Probe | Path | Budget |
|---|---|---|
| `startupProbe` | `/` | 60 × 5s = **5 minutes** to become ready |
| `livenessProbe` | `/` | 6 × 30s = 180s of failure before restart |
| `readinessProbe` | `/` | 3 × 10s = ~30s to shift traffic |
Realtime uses `/health` on port 3002 with a 150-second startup budget.
The generous startup budget matters: a cold Next.js start on a large bundle can take minutes, and a tighter liveness probe will restart the pod mid-boot in a loop. If you customize probes, keep the startup budget well above your observed cold-start time.
```yaml
app:
startupProbe:
httpGet:
path: /
port: 3000
periodSeconds: 5
failureThreshold: 60
```
## Logs
Both services log structured JSON to stdout. Collect them with whatever you already run — Fluent Bit, Vector, Datadog Agent, Loki.
<Callout type="warn">
In production builds the logger defaults to `ERROR`, and the Helm chart does not set `LOG_LEVEL` for the app or realtime. Until you raise it, the only thing in the logs is errors — which is why a healthy-looking deployment can appear to log nothing at all. Set `LOG_LEVEL: "info"` while commissioning a deployment or debugging.
</Callout>
```bash
kubectl logs -n simstudio -l app.kubernetes.io/component=app --tail=200 -f
kubectl logs -n simstudio -l app.kubernetes.io/component=realtime --tail=200 -f
kubectl logs -n simstudio deploy/sim-app -c migrations --tail=100
```
```bash
docker compose -f docker-compose.prod.yml logs -f simstudio
```
Every API request carries a request ID that appears in all log lines for that request — the fastest way to reconstruct a failing call.
<Callout type="info">
Workflow execution logs are a separate, product-level surface stored in the database and visible in the Logs view of the app. They are not the same as container logs: use container logs for infrastructure problems and the Logs view for workflow behavior.
</Callout>
### Redacting PII from logs
Enable the PII service and log redaction if execution logs may contain sensitive data:
```yaml
pii:
enabled: true
app:
env:
PII_REDACTION: "true"
INTERNAL_API_BASE_URL: "http://sim-app.simstudio.svc.cluster.local:3000"
```
See [Security](/platform/self-hosting/security) for the `INTERNAL_API_BASE_URL` requirement — the path fails closed without a cluster-reachable value.
## Anonymous telemetry
<Callout type="warn">
**Sim sends anonymous usage telemetry by default.** OpenTelemetry traces are exported to `https://telemetry.simstudio.ai/v1/traces` unless you turn it off. Self-hosted deployments with an egress policy should decide about this explicitly.
</Callout>
What is collected, per `apps/sim/telemetry.config.ts`: feature-usage statistics, error rates, performance metrics (sampled at 10%), and AI/LLM operation traces. What is **not** collected: personal information, workflow content or outputs, API keys or tokens, and IP addresses or geolocation.
Three ways to change it:
```bash
# Disable entirely
NEXT_TELEMETRY_DISABLED=1
# Or redirect to your own OTLP collector instead of Sim's
TELEMETRY_ENDPOINT=http://otel-collector.observability.svc.cluster.local:4318/v1/traces
```
Users can also toggle it off individually under **Settings → Privacy → Allow anonymous telemetry**.
## Tracing
Sim emits OpenTelemetry traces. The chart can also deploy a collector for you:
```yaml
telemetry:
enabled: true
```
Or point the app at a collector you already run:
| Variable | Purpose |
|---|---|
| `OTEL_EXPORTER_OTLP_ENDPOINT` | Collector endpoint (OTLP) |
| `OTEL_EXPORTER_OTLP_HEADERS` | Auth headers, `key=value` comma-separated |
| `OTEL_TRACES_SAMPLER_ARG` | Sampling ratio |
| `OTEL_DEPLOYMENT_ENVIRONMENT` | Environment label on emitted spans |
| `TELEMETRY_SAMPLING_RATIO` | Application-level sampling ratio |
| `TELEMETRY_ENDPOINT` | Custom telemetry endpoint |
For Grafana Cloud specifically:
| Variable | Purpose |
|---|---|
| `GRAFANA_OTLP_ENDPOINT` | Grafana OTLP endpoint |
| `GRAFANA_OTLP_HEADERS` | e.g. `Authorization=Basic <base64(instanceId:token)>` |
| `GRAFANA_DEPLOYMENT_ENVIRONMENT` | Deployment tier label |
If you enable the chart's Jaeger export, point `telemetry.jaeger.endpoint` at Jaeger's **OTLP gRPC port (4317)** — the collector exports over OTLP.
## Metrics
<Callout type="warn">
The default app and realtime images **do not expose a `/metrics` endpoint**. The chart's `monitoring.serviceMonitor` option exists for builds that do — enabling it against the stock images produces a ServiceMonitor that scrapes nothing.
</Callout>
Until an application metrics endpoint ships, build alerting from the signals that do exist:
- **Kubernetes state** — pod restarts, `CrashLoopBackOff`, OOMKills, replica count vs desired, PVC utilization (kube-state-metrics).
- **Ingress/load balancer** — request rate, 5xx rate, p99 latency, websocket connection count.
- **PostgreSQL** — connection count vs `max_connections`, replication lag, disk usage, long-running queries.
- **Redis** — memory usage, evictions, connected clients.
- **CronJobs** — last successful completion per job.
## What to alert on
| Alert | Why it matters |
|---|---|
| App pod restart loop / OOMKilled | Memory is the constraining resource; OOMKills mean executions are dying mid-run |
| A CronJob has not succeeded within ~3× its own schedule interval | Scheduled workflows and polling triggers are silently dead. Threshold per job — the per-minute jobs justify ~15 minutes; the hourly, twice-daily, and daily jobs need proportionally longer windows |
| Ingress 5xx rate above baseline | Broad user impact |
| Postgres connections above 80% of `max_connections` | Next replica or traffic spike will start failing |
| Postgres disk above 80% | Knowledge base embeddings grow steadily |
| Redis unreachable | Live collaboration and status updates stop, without app errors |
| Certificate expiry within 14 days | Especially with manually managed certs |
| Object storage 4xx/5xx rate | Broken uploads usually show here first |
<Callout type="warn">
The CronJob alert is the one most deployments lack and most need. Background job failures produce no user-visible error — schedules simply stop firing. Alert on `kube_cronjob_status_last_successful_time` lagging, with a per-job threshold derived from that job's schedule.
</Callout>
## Quick diagnosis
```bash
# Overall state
kubectl get pods,cronjobs -n simstudio
# Why is a pod unhealthy
kubectl describe pod -n simstudio <pod>
# Did migrations succeed
kubectl logs -n simstudio deploy/sim-app -c migrations --tail=100
# Are background jobs running
kubectl get jobs -n simstudio --sort-by=.metadata.creationTimestamp | tail
# Resource pressure
kubectl top pods -n simstudio
```
<FAQ items={[
{ question: "Does Sim send telemetry to Sim by default?", answer: "Yes. Anonymous OpenTelemetry traces go to https://telemetry.simstudio.ai/v1/traces unless you set NEXT_TELEMETRY_DISABLED=1 or point TELEMETRY_ENDPOINT at your own collector. It excludes personal information, workflow content, API keys, and IP addresses, but it is outbound traffic you should decide about deliberately on a self-hosted deployment."},
{ question: "Does Sim expose Prometheus metrics?", answer: "The default images do not expose a /metrics endpoint. The chart's monitoring.serviceMonitor option exists for builds that do; against stock images it scrapes nothing. Build alerting from Kubernetes, ingress, Postgres, and Redis signals instead."},
{ question: "Why is the startup probe so lenient?", answer: "A cold Next.js start on a large bundle can take minutes. The 5-minute startup budget prevents the liveness probe from restarting the pod mid-boot, which would otherwise loop forever. Keep the budget above your observed cold-start time if you customize probes."},
{ question: "How do I trace a single failing API request?", answer: "Every request carries a request ID that appears in all log lines for that request. Find it in the response or the first log line and grep your log aggregator for it." },
]} />
apps/docs/content/docs/en/platform/self-hosting/platforms.mdx
+32 -95
View File
@@ -1,115 +1,52 @@
---
title: Cloud Platforms
description: Deploy Sim on cloud platforms
description: Provider-specific notes for running Sim on Railway, a VPS, or managed Kubernetes
---
import { Tab, Tabs } from 'fumadocs-ui/components/tabs'
import { Callout } from 'fumadocs-ui/components/callout'
This page covers what differs per provider. The deployment itself is the same everywhere — follow [Docker](/platform/self-hosting/docker) for a single node or [Kubernetes](/platform/self-hosting/kubernetes) for a cluster.
## Railway
One-click deployment with automatic PostgreSQL provisioning.
[![Deploy on Railway](https://railway.app/button.svg)](https://railway.com/new/template/sim-studio)
After deployment, add environment variables in Railway dashboard:
- `BETTER_AUTH_SECRET`, `ENCRYPTION_KEY`, `INTERNAL_API_SECRET` (auto-generated)
- `OPENAI_API_KEY` or other AI provider keys
- Custom domain in Settings → Networking
After deployment, set in the Railway dashboard:
## VPS Deployment
- `BETTER_AUTH_SECRET`, `ENCRYPTION_KEY`, `INTERNAL_API_SECRET` (auto-generated by the template)
- An AI provider key such as `OPENAI_API_KEY`
- Your custom domain under **Settings → Networking**, then `NEXT_PUBLIC_APP_URL` to match
For DigitalOcean, AWS EC2, Azure VMs, or any Linux server:
<Callout type="warn">
The Railway template deploys the app services but not the `cron` service, so scheduled workflows and polling triggers stay idle. Add a Railway cron service calling the endpoints in [Background Jobs](/platform/self-hosting/background-jobs), or deploy with Docker Compose instead.
</Callout>
<Tabs items={['DigitalOcean', 'AWS EC2', 'Azure VM']}>
<Tab value="DigitalOcean">
**Recommended:** 16 GB RAM Droplet, Ubuntu 24.04
## VPS
DigitalOcean, EC2, Azure VM, Hetzner, or any Linux box. Size it from the [requirements table](/platform/self-hosting) — 16 GB RAM is the practical floor for a team install, because memory rather than CPU is what bounds concurrent workflow executions.
Install Docker via [get.docker.com](https://get.docker.com), then follow the [Docker guide](/platform/self-hosting/docker), which covers secrets, the compose stack, and TLS.
## Managed Kubernetes
EKS, AKS, and GKE each have a tuned example values file in the chart. See [Kubernetes](/platform/self-hosting/kubernetes) for the install and [Networking](/platform/self-hosting/networking) for the load-balancer specifics — notably GKE's 30-second websocket timeout and its `ManagedCertificate` requirement.
## Managed PostgreSQL
Recommended for any production deployment. The requirement is **pgvector**.
| Service | Notes |
|---|---|
| AWS RDS / Aurora | Enable the `vector` extension |
| GCP Cloud SQL | Enable the `vector` extension |
| Azure Database for PostgreSQL | Enable the `vector` extension |
| Supabase / Neon | pgvector available by default |
```bash
# Create Droplet via console, then SSH in
ssh root@your-droplet-ip
```
</Tab>
<Tab value="AWS EC2">
**Recommended:** t3.xlarge (16 GB RAM), Ubuntu 24.04
```bash
ssh -i your-key.pem ubuntu@your-ec2-ip
```
</Tab>
<Tab value="Azure VM">
**Recommended:** Standard_D4s_v3 (16 GB RAM), Ubuntu 24.04
```bash
ssh azureuser@your-vm-ip
```
</Tab>
</Tabs>
### Install Docker
```bash
# Install Docker (official method)
curl -fsSL https://get.docker.com | sudo sh
sudo usermod -aG docker $USER
# Logout and reconnect, then verify
docker --version
DATABASE_URL="postgresql://user:pass@host:5432/simstudio?sslmode=require"
```
### Deploy Sim
```bash
git clone https://github.com/simstudioai/sim.git && cd sim
# Create .env with secrets
cat > .env << EOF
DATABASE_URL=postgresql://postgres:postgres@db:5432/simstudio
BETTER_AUTH_SECRET=$(openssl rand -hex 32)
ENCRYPTION_KEY=$(openssl rand -hex 32)
INTERNAL_API_SECRET=$(openssl rand -hex 32)
NEXT_PUBLIC_APP_URL=https://sim.yourdomain.com
BETTER_AUTH_URL=https://sim.yourdomain.com
EOF
# Start
docker compose -f docker-compose.prod.yml up -d
```
### SSL with Caddy
```bash
# Install Caddy
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update && sudo apt install caddy
# Configure (replace domain)
echo 'sim.yourdomain.com {
reverse_proxy localhost:3000
handle /socket.io/* {
reverse_proxy localhost:3002
}
}' | sudo tee /etc/caddy/Caddyfile
sudo systemctl restart caddy
```
Point your domain's DNS A record to your server IP.
## Kubernetes (EKS, AKS, GKE)
See the [Kubernetes guide](/platform/self-hosting/kubernetes) for Helm deployment on managed Kubernetes.
## Managed Database (Optional)
For production, use a managed PostgreSQL service:
- **AWS RDS** / **Azure Database** / **Cloud SQL** - Enable pgvector extension
- **Supabase** / **Neon** - pgvector included
Set `DATABASE_URL` in your environment:
```bash
DATABASE_URL="postgresql://user:pass@host:5432/db?sslmode=require"
```
For the Helm chart, disable the bundled Postgres and use `externalDatabase` — see [Kubernetes](/platform/self-hosting/kubernetes#external-database).
apps/docs/content/docs/en/platform/self-hosting/redis.mdx
+115
View File
@@ -0,0 +1,115 @@
---
title: Redis
description: When Redis is optional, when it is required, and how to configure it
---
import { Tab, Tabs } from 'fumadocs-ui/components/tabs'
import { Callout } from 'fumadocs-ui/components/callout'
import { FAQ } from '@/components/ui/faq'
Sim uses Redis as a message bus and shared cache. Both deployments ship it by default — Docker Compose as a `redis` service, Helm as a `redis` Deployment — so this page is mostly about when to replace the bundled instance with a managed one, and what breaks if Redis is absent entirely.
## What it backs
| Use | Without Redis |
|---|---|
| **Pub/sub** — live Chat task status, table events, execution cancellation, MCP tool-change notifications, copilot tool confirmations | Falls back to a **process-local** emitter: events never leave the pod that produced them |
| **Socket.IO adapter** (realtime) | Collaboration events are not delivered across realtime pods |
| Idempotency store | Falls back to PostgreSQL |
| Execution progress markers | Falls back to PostgreSQL |
| Distributed execution limits | Enforced per-pod instead of per-deployment |
| CLI auth approval store | **No fallback** — CLI authentication requires Redis regardless of replica count |
| Collaborative document store (realtime) | Falls back to in-process state |
<Callout type="error">
With more than one app or realtime replica and no `REDIS_URL`, users on different pods stop seeing each other's edits and live status updates. Beyond one startup log line noting single-pod mode, nothing is logged — the app looks healthy and quietly loses events. Treat Redis as mandatory the moment `replicaCount` exceeds 1.
</Callout>
## Configuration
```bash
REDIS_URL=redis://:password@redis-host:6379
# or, with TLS
REDIS_URL=rediss://:password@redis-host:6380
```
Both the app and the realtime service need it — they use it for different things.
<Tabs items={['Docker Compose', 'Kubernetes (Helm)', 'Managed Redis']}>
<Tab value="Docker Compose">
`docker-compose.prod.yml` already includes a `redis:7-alpine` service and wires `REDIS_URL=redis://redis:6379` into both the app and realtime containers. Nothing to configure.
The port is deliberately not published to the host, so a Redis already running locally will not collide. Override `REDIS_URL` in `.env` to point at an external instance instead.
</Tab>
<Tab value="Kubernetes (Helm)">
The chart deploys Redis by default, matching the Compose stack. Nothing to configure.
For production, prefer a managed instance — disable the bundled one and supply a URL:
```yaml
redis:
enabled: false
app:
env:
REDIS_URL: "rediss://:<password>@my-cache.internal:6380"
```
`app.env.REDIS_URL` takes over whenever it is set, and the chart skips the bundled Deployment so you do not get a stray pod.
If the URL lives in a secret store instead — a pre-created Secret or one synced by External Secrets — it also wins, and there is nothing extra to configure. The bundled URL is delivered as a ConfigMap listed before the app Secret in `envFrom`, and Kubernetes lets the last source win for duplicate keys, so your value overrides it without the chart ever reading it.
The bundled Redis is deliberately non-persistent (`--save ""`, `--appendonly no`) with a 512 MB cap: Sim stores coordination state and short-lived keys in it, so a restart costs in-flight live updates rather than committed data.
If `networkPolicy.enabled=true`, egress to the bundled Redis is allowed automatically. An **external** Redis needs its own rule under `networkPolicy.egress` — the chart cannot know your host and port at render time.
</Tab>
<Tab value="Managed Redis">
Managed services work and are the recommended production choice:
- **AWS** — ElastiCache for Redis or MemoryDB
- **GCP** — Memorystore for Redis
- **Azure** — Azure Cache for Redis
Place the instance in the same VPC/VNet as the cluster and use its private endpoint. Enable TLS (`rediss://`) and auth.
Sizing is modest: Sim uses Redis for coordination, not bulk storage. A 1–2 GB instance covers most deployments. Prefer a replicated/HA tier so a failover does not interrupt live collaboration.
</Tab>
</Tabs>
## TLS to an IP address
If `REDIS_URL` uses `rediss://` and the host is a **bare IP** — common with AWS PrivateLink endpoints — TLS hostname verification cannot match an IP against the certificate. Sim throws rather than connecting insecurely, the first time it opens a Redis connection. Set the SNI override to the DNS name the certificate was issued for:
```bash
REDIS_URL=rediss://:password@10.0.12.34:6379
REDIS_TLS_SERVERNAME=my-cluster.abc123.ng.0001.use1.cache.amazonaws.com
```
With a DNS hostname in `REDIS_URL`, default verification works and no override is needed.
## Verifying
```bash
# Kubernetes
kubectl exec -n simstudio deploy/sim-app -- printenv REDIS_URL
# Docker Compose
docker compose -f docker-compose.prod.yml exec redis redis-cli ping # PONG
```
The functional test: open the same workflow in two browser windows served by different replicas and confirm edits appear in both. With a single replica this always passes, so scale to two before testing.
Watch the app logs at startup for `Redis` connection errors — a wrong password or unreachable host is logged there.
<FAQ items={[
{ question: "Is Redis required?", answer: "It is required for any deployment with more than one app or realtime replica, because pub/sub and the Socket.IO adapter have no cross-pod fallback. On a single replica most subsystems fall back to Postgres or in-process state — with one exception: CLI authentication requires Redis at any replica count."},
{ question: "Does the Helm chart install Redis?", answer: "Yes, by default — the same redis:7-alpine image Docker Compose uses. Set redis.enabled: false and put a connection string in app.env.REDIS_URL to use a managed instance instead, which is the better production choice."},
{ question: "How much Redis do I need?", answer: "Little. Sim uses it for coordination and short-lived keys, not bulk data — 1–2 GB suits most deployments. Prioritize an HA/replicated tier over raw size."},
{ question: "Can I lose Redis data?", answer: "Yes, without data loss to the application. Redis holds caches, coordination keys, and in-flight events. Losing it drops live updates in progress; committed data lives in PostgreSQL and object storage. Persistence is not required." },
]} />
apps/docs/content/docs/en/platform/self-hosting/scaling.mdx
+169
View File
@@ -0,0 +1,169 @@
---
title: Scaling & High Availability
description: Replicas, connection pooling, autoscaling, and where the limits actually are
---
import { Callout } from 'fumadocs-ui/components/callout'
import { FAQ } from '@/components/ui/faq'
## What scales how
| Component | Scaling | Notes |
|---|---|---|
| **app** | Horizontal | Stateless. **Requires Redis** past one replica |
| **realtime** | Horizontal | **Requires Redis** past one replica (Socket.IO adapter) |
| **postgresql** | Vertical + read replicas | The eventual bottleneck |
| **redis** | Vertical / HA pair | Coordination only; small |
| **cronjobs** | Fixed | One call per tick regardless of replica count |
## Prerequisites before scaling past one replica
<Callout type="error">
Redis must be reachable before you raise `replicaCount`. Both deployments ship it by default, so this is already satisfied unless you set `redis.enabled: false` (Helm) or removed the `redis` service (Compose) without supplying `REDIS_URL`. Without Redis, pub/sub falls back to a process-local emitter and the Socket.IO adapter has no cross-pod transport — realtime logs one line at startup noting single-pod mode, then drops cross-pod events silently. See [Redis](/platform/self-hosting/redis).
</Callout>
You also need shared object storage — local-disk storage is per-pod, so a file uploaded through one replica is invisible to the others. See [Object Storage](/platform/self-hosting/object-storage).
## Scaling the app
```yaml
app:
replicaCount: 3
resources:
limits:
memory: 8Gi
cpu: 2000m
requests:
memory: 4Gi
cpu: 1000m
```
**Memory is the constraint, not CPU.** Workflow executions run inside the app process in isolated-vm sandboxes, and file parsing happens in memory. Production telemetry shows 4–8 GB steady with peaks to 12 GB under heavy execution load. Under-provision memory and you get OOMKills that terminate in-flight workflow runs.
A PodDisruptionBudget is created automatically once `replicaCount > 1` (`maxUnavailable: 25%`). Tighten it with `podDisruptionBudget.minAvailable` if you need to.
### Autoscaling
```yaml
autoscaling:
enabled: true
minReplicas: 2
maxReplicas: 10
targetCPUUtilizationPercentage: 70
targetMemoryUtilizationPercentage: 80
```
Requires `metrics-server`. When enabled, the chart omits `spec.replicas` so the HPA owns replica count.
<Callout type="warn">
Scale-down terminates pods that may be running workflows. Set a conservative `minReplicas`, and consider a `behavior` block with a long `stabilizationWindowSeconds` on scale-down so long executions are not repeatedly interrupted.
</Callout>
Realtime gets the same HPA unless you disable it — and again, only scale it past one replica with Redis configured:
```yaml
autoscaling:
realtime:
enabled: false
```
## Database
Postgres is where scaling eventually stops being about replicas.
### Connections
Each app replica opens a pool. Total connections grow with replica count, and Postgres has a hard `max_connections`. A deployment that works at 2 replicas can exhaust connections at 6.
Budget it: `replicas × pool size + realtime + cronjobs + migrations + headroom` must stay under `max_connections`.
For anything beyond a handful of replicas, put **PgBouncer** in transaction pooling mode in front of the database and point `DATABASE_URL` at it. This is the single highest-leverage change for a large deployment — it decouples app replica count from database connection count.
### Read replicas
Heavy read paths — log listing, audit logs, dashboard aggregations — can be offloaded:
```bash
DATABASE_REPLICA_URL=postgresql://user:pass@replica-host:5432/simstudio
```
Reads fall back to the primary when unset. Per-role overrides exist if different components should use different replicas:
| Variable | Applies to |
|---|---|
| `DATABASE_REPLICA_URL` | Default for all roles |
| `DATABASE_REPLICA_URL_WEB` | The web app |
| `DATABASE_REPLICA_URL_REALTIME` | The realtime service |
| `DATABASE_REPLICA_URL_TRIGGER` | Trigger.dev workers |
<Callout type="info">
Replicas lag. Sim routes only latency-tolerant reads to them, but if your replica lags badly, recently written logs may briefly not appear. Monitor replication lag.
</Callout>
### Sizing
| Deployment | Instance | Storage |
|---|---|---|
| Small (1–5 users) | 2 vCPU / 8 GB | 50 GB |
| Standard (5–50 users) | 4 vCPU / 16 GB | 100 GB+ |
| Large (50+ users) | 8+ vCPU / 32 GB+ | 250 GB+, auto-grow |
Knowledge base embeddings are the main growth driver — vector storage scales with document volume, not user count. Enable storage auto-increase.
## Execution concurrency
`SCHEDULE_EXECUTION_CONCURRENCY_LIMIT` (default `30`) bounds scheduled executions per app instance. The other three `*_EXECUTION_CONCURRENCY_LIMIT` variables apply only to Trigger.dev and are inert on a default self-host — see [Background Jobs](/platform/self-hosting/background-jobs#concurrency).
When executions queue but memory is fine, raise the limit; when memory is the ceiling, add replicas instead.
## Rate limits and quotas
Self-hosted deployments run **without plan limits by default** — no rate limits, execution timeouts, or table and storage caps. Each can be opted back in individually; the variable list and suggested values are in [Environment Variables](/platform/self-hosting/environment-variables#limits).
An execution timeout is worth setting even on an otherwise unlimited deployment — it is what stops a runaway workflow from holding a sandbox indefinitely.
## Reference topology
A production deployment serving ~100 active users:
```yaml
app:
replicaCount: 3
resources:
limits: { memory: 8Gi, cpu: 2000m }
requests: { memory: 4Gi, cpu: 1000m }
env:
REDIS_URL: "rediss://:<password>@redis.internal:6380"
realtime:
replicaCount: 2
env:
REDIS_URL: "rediss://:<password>@redis.internal:6380"
postgresql:
enabled: false
externalDatabase:
enabled: true
host: "pgbouncer.internal"
port: 6432
database: simstudio
sslMode: require
autoscaling:
enabled: true
minReplicas: 3
maxReplicas: 10
podDisruptionBudget:
minAvailable: 2
```
Plus: managed Postgres with PITR, managed Redis in an HA tier, object storage with versioning, and images pinned to an explicit tag.
<FAQ items={[
{ question: "Should I scale up or scale out?", answer: "Scale out for concurrent users and workflow throughput. Scale up when individual executions are memory-hungry — a single large document parse or execution has to fit in one pod. Memory is almost always the binding constraint."},
{ question: "When do I need PgBouncer?", answer: "Once replica count times pool size approaches Postgres max_connections — typically past a handful of replicas. Transaction pooling decouples app replicas from database connections and is the highest-leverage change for a large deployment."},
{ question: "Does the HPA interrupt running workflows?", answer: "Scale-down can terminate a pod mid-execution. Use a conservative minReplicas and a long scale-down stabilization window so long-running executions are not repeatedly interrupted."},
{ question: "Can I use a read replica?", answer: "Yes. Set DATABASE_REPLICA_URL and latency-tolerant reads — log listing, audit logs, dashboard aggregations — are routed to it. Per-role overrides exist for the web, realtime, and trigger components." },
]} />
apps/docs/content/docs/en/platform/self-hosting/security.mdx
+195
View File
@@ -0,0 +1,195 @@
---
title: Security & Hardening
description: Secrets, network boundaries, code sandboxing, and the settings to review before going live
---
import { Callout } from 'fumadocs-ui/components/callout'
import { FAQ } from '@/components/ui/faq'
## Secrets
Five secrets drive the security of a deployment. Generate each with `openssl rand -hex 32`.
| Secret | Protects | Rotatable |
|---|---|---|
| `BETTER_AUTH_SECRET` | Session tokens | Yes — invalidates all sessions |
| `ENCRYPTION_KEY` | Workspace env vars, stored provider keys, MCP OAuth credentials, deployment/chat secrets | **No** — see below |
| `API_ENCRYPTION_KEY` | Reversible stored copy of user-generated API keys | **No** — existing keys keep authenticating, but their stored copy can no longer be displayed |
| `INTERNAL_API_SECRET` | Service-to-service calls | Yes — roll app and realtime together |
| `CRON_SECRET` | Background job endpoints | Yes — roll app and cron together |
<Callout type="error">
`ENCRYPTION_KEY` cannot be rotated without re-encrypting the data it protects, and cannot be recovered if lost. Changing it renders all of that data permanently unreadable. Back it up independently of the database.
</Callout>
`BETTER_AUTH_SECRET` must be **identical** on the app and realtime services — they share sessions through the database, and a mismatch means realtime rejects every authenticated socket.
### Storing them
In increasing order of production-readiness:
1. **`--set` on the command line** — dev only. Values appear in `helm get values` output and shell history.
2. **A pre-created Kubernetes Secret** — set `app.secrets.existingSecret.enabled: true` and the secret name. Works with Sealed Secrets and SOPS. The secret is consumed wholesale and must use the standard key names.
3. **External Secrets Operator** — sync from Vault, AWS Secrets Manager, Azure Key Vault, or GCP Secret Manager. Recommended.
In the default and External Secrets modes, the chart writes every key under `app.env` and `realtime.env` into a chart-managed Secret mounted via `envFrom`, so no value is inlined into a pod spec. (In `existingSecret` mode the pre-created Secret is the source of truth and any `app.env` values you still pass are rendered inline — supply everything through the Secret in that mode.) Either way, a secret committed to `values.yaml` is a secret in your git history.
## Network boundaries
### Ingress
Expose only the app (3000) and realtime (3002). Everything else — Postgres, Redis, the PII service, the cron endpoints — should be reachable only from inside the deployment.
The background job endpoints under `/api/cron/*`, `/api/webhooks/poll/*`, and `/api/schedules/execute` are authenticated by `CRON_SECRET`, but there is no reason to expose them publicly. Point cron at the in-cluster Service.
### NetworkPolicy
The chart ships an optional policy that isolates east-west traffic and blocks cloud metadata endpoints (`169.254.169.254/32`, `169.254.170.2/32`) on egress — worth enabling, because those endpoints are the standard SSRF escalation target.
```yaml
networkPolicy:
enabled: true
```
<Callout type="warn">
`networkPolicy.ingressFrom` defaults to `[{}]` — an empty peer selector that allows ingress from **any pod in the cluster**. On a shared or multi-tenant cluster, scope it to your ingress controller:
```yaml
networkPolicy:
ingressFrom:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: ingress-nginx
```
</Callout>
The policy already allows HTTPS (443) egress to everything except the metadata CIDRs, which covers model provider APIs, integration APIs, and cloud object-storage endpoints. It also allows the bundled Postgres and Redis by pod selector.
What it does **not** cover is any datastore you run outside the chart — a managed Postgres or Redis on a non-443 port. Add a rule for each:
```yaml
networkPolicy:
enabled: true
egress:
- to:
- ipBlock:
cidr: 10.0.0.0/16 # your VPC / managed-service subnet
ports:
- protocol: TCP
port: 6379 # managed Redis
- protocol: TCP
port: 5432 # managed Postgres
```
<Callout type="warn">
This applies even when `REDIS_URL` reaches the pod through a Secret rather than `values.yaml` — the chart cannot see the host, so it cannot generate the rule. A deployment that accepts the URL but has no matching egress rule will fail to reach Redis with `networkPolicy.enabled: true`.
</Callout>
If maintaining CIDR lists is not worth it, drop the port restriction instead:
```yaml
networkPolicy:
enabled: true
allowExternalEgress: true
```
Cloud metadata endpoints stay blocked either way. This defaults to `false` because Sim's chart is deliberately stricter than the common chart default, which permits unrestricted egress.
### Pod Security Standards
All workloads set `runAsNonRoot`, drop all Linux capabilities, disable privilege escalation, and use `seccompProfile: RuntimeDefault` — the four controls the `restricted` profile requires. Label the namespace to enforce it:
```bash
kubectl label namespace simstudio pod-security.kubernetes.io/enforce=restricted
```
`readOnlyRootFilesystem` is not set by default: Postgres and Ollama need a writable root, and the app container writes to Next.js's `.next/cache`. It is viable on the genuinely stateless services (`realtime`, `pii`, `copilot`) — set `<component>.securityContext.readOnlyRootFilesystem: true` and mount an `emptyDir` at `/tmp` via `extraVolumes` / `extraVolumeMounts`.
## Where user code runs
Workflows can execute user-authored JavaScript and Python. Know which sandbox you are running before you expose Sim to untrusted authors.
| Mode | Configuration | Isolation |
|---|---|---|
| **isolated-vm** (default) | none | In-process V8 isolate inside the app container. No network namespace or filesystem separation from the app process — isolation is at the JS-engine level. JavaScript only. |
| **E2B** | `E2B_ENABLED=true`, `E2B_API_KEY` | Remote sandbox per execution. Strongest isolation; requires outbound access to E2B. |
| **Daytona** | `SANDBOX_PROVIDER=daytona`, `DAYTONA_API_KEY` | Remote sandbox per execution. |
Python execution and the tooling-dependent blocks require a remote sandbox provider — the in-process isolate runs JavaScript only.
<Callout type="warn">
With the default in-process sandbox, treat everyone who can author a workflow as someone running code in your app container's security context. If your Sim instance is open to a wide or partly-trusted audience, use a remote sandbox provider and enable the NetworkPolicy egress restrictions.
</Callout>
Resource ceilings for the in-process path:
| Variable | Controls |
|---|---|
| `IVM_MAX_EXECUTIONS_PER_WORKER` | Executions before a worker is recycled |
| `IVM_MAX_BROKERS_PER_EXECUTION` | Host-call brokers per execution |
| `IVM_MAX_BROKER_ARGS_JSON_CHARS` | Max argument payload size |
| `IVM_MAX_BROKER_RESULT_JSON_CHARS` | Max result payload size |
## The SSRF boundary
Sim blocks outbound requests from database and connector tools to private, reserved, and loopback addresses. This stops a workflow from being used to scan your internal network.
Self-hosted deployments often legitimately need to reach an internal database by service name. That is opt-in:
```bash
ALLOW_PRIVATE_DATABASE_HOSTS=true
```
<Callout type="warn">
This loosens the SSRF boundary for every workflow author on the instance. Enable it only on a trusted private network, and prefer pairing it with a NetworkPolicy that constrains what the app can actually reach.
</Callout>
## Client IP and forwarded headers
Behind a load balancer, `X-Forwarded-For` is client-controllable. Set `AUTH_TRUSTED_PROXIES` to your proxies' actual addresses so Better Auth resolves the real client IP, and `TRUSTED_ORIGINS` if users reach Sim from more than one origin. Both are covered in [Authentication](/platform/self-hosting/authentication#behind-a-load-balancer).
## Restricting who can use the instance
Signup allowlists and blocklists, social-login toggles, SSO, and the `DISABLE_AUTH` escape hatch are all covered in [Authentication](/platform/self-hosting/authentication). The security-relevant summary: restrict signup before exposing the instance, and never set `DISABLE_AUTH=true` behind an internet-facing ingress.
## PII redaction
The optional Presidio-based service supports the Guardrails PII block and, when enabled, automatic redaction of PII from workflow logs:
```yaml
pii:
enabled: true
app:
env:
PII_REDACTION: "true"
INTERNAL_API_BASE_URL: "http://sim-app.simstudio.svc.cluster.local:3000"
```
`INTERNAL_API_BASE_URL` must be the **in-cluster** Service URL. The redaction path calls the app's own API, and a public ingress URL is usually not hairpin-reachable from inside the cluster. Without a reachable value the path fails closed — affected fields are scrubbed to `[REDACTION_FAILED]` rather than leaking, but redaction does not actually run.
The service bundles ~2.2 GB of spaCy models, so first start takes around three minutes and it needs at least 4 GB of memory.
## Pre-launch checklist
- [ ] All five secrets generated fresh, stored in a secret manager, and **`ENCRYPTION_KEY` backed up separately**
- [ ] `BETTER_AUTH_SECRET` identical on app and realtime
- [ ] Images pinned to an explicit tag or digest on app, realtime, and migrations
- [ ] TLS terminating at the ingress; HTTP redirected or disabled
- [ ] `NEXT_PUBLIC_APP_URL` and `BETTER_AUTH_URL` set to the real public origin
- [ ] `AUTH_TRUSTED_PROXIES` set if behind a load balancer
- [ ] Signup restricted (`DISABLE_REGISTRATION` or `ALLOWED_LOGIN_DOMAINS`)
- [ ] `DISABLE_AUTH` **not** set
- [ ] NetworkPolicy enabled and `ingressFrom` scoped to the ingress controller
- [ ] Namespace labelled `pod-security.kubernetes.io/enforce=restricted`
- [ ] Object storage buckets private, with CORS limited to your Sim origin
- [ ] Database reachable only from the deployment; TLS enforced (`sslMode: require`)
- [ ] Backups configured **and a restore rehearsed**
- [ ] Sandbox strategy decided for user code
<FAQ items={[
{ question: "Can I rotate ENCRYPTION_KEY?", answer: "Not without re-encrypting everything it protects. Changing it makes workspace environment variables, stored provider API keys, MCP OAuth credentials, and deployment secrets permanently unreadable. Treat it as a permanent, backed-up value rather than a rotating secret."},
{ question: "Where does user-authored code run?", answer: "By default in an in-process V8 isolate inside the app container, which isolates at the JS-engine level but shares the container's network and filesystem context. For untrusted authors, or to run Python at all, use E2B or Daytona so each execution runs in a remote sandbox."},
{ question: "Why does the chart's NetworkPolicy allow traffic from any pod?", answer: "networkPolicy.ingressFrom defaults to an empty peer selector as a simple default that works on any cluster. On a shared cluster you should scope it to your ingress controller's namespace."},
{ question: "What does ALLOW_PRIVATE_DATABASE_HOSTS change?", answer: "It lets database and connector tools reach private, reserved, and loopback addresses — needed to connect to an internal database by Kubernetes service name. It also widens the SSRF boundary for every workflow author, so enable it only on a trusted network."},
]} />
apps/docs/content/docs/en/platform/self-hosting/troubleshooting.mdx
+144 -14
View File
@@ -43,12 +43,20 @@ docker compose logs simstudio
## Migration Errors
Migrations run in their own `migrations` service and image — the app image does not contain the migration tooling.
```bash
# View migration logs
docker compose logs migrations
docker compose -f docker-compose.prod.yml logs migrations
# Run manually
docker compose exec simstudio bun run db:migrate
# Re-run them
docker compose -f docker-compose.prod.yml up --force-recreate migrations
```
On Kubernetes, migrations are an init container on the app pod:
```bash
kubectl logs -n simstudio deploy/sim-app -c migrations --tail=200
```
## pgvector Not Found
@@ -62,14 +70,37 @@ image: pgvector/pgvector:pg17 # NOT postgres:17
If you see SSL certificate errors when calling external APIs:
```bash
# Update CA certificates in container
docker compose exec simstudio apt-get update && apt-get install -y ca-certificates
The image already ships current CA certificates and runs as a non-root user, so installing packages inside it is not the fix. This almost always means the endpoint presents a certificate signed by a private CA — a corporate TLS-inspecting proxy, or an internal service.
# Or set in environment (not recommended for production)
NODE_TLS_REJECT_UNAUTHORIZED=0
Mount your CA bundle and point Node at it:
```yaml
# docker-compose.prod.yml
services:
simstudio:
volumes:
- /etc/ssl/certs/corporate-ca.crt:/certs/corporate-ca.crt:ro
environment:
- NODE_EXTRA_CA_CERTS=/certs/corporate-ca.crt
```
```yaml
# Helm — mount a ConfigMap holding the CA
app:
env:
NODE_EXTRA_CA_CERTS: /certs/corporate-ca.crt
extraVolumes:
- name: corporate-ca
configMap:
name: corporate-ca
extraVolumeMounts:
- name: corporate-ca
mountPath: /certs
readOnly: true
```
`NODE_TLS_REJECT_UNAUTHORIZED=0` disables certificate verification entirely and should never be used outside a throwaway test.
## Blank Page After Login
1. Check browser console for errors
@@ -79,13 +110,12 @@ NODE_TLS_REJECT_UNAUTHORIZED=0
## Windows-Specific Issues
**Turbopack errors on Windows:**
```bash
# Use WSL2 for better compatibility
wsl --install
These apply to running Sim **from source** for development, not to the Docker or Kubernetes deployments, which are unaffected by the host OS.
# Or disable Turbopack in package.json
# Change "next dev --turbopack" to "next dev"
**Turbopack errors on Windows:** use WSL2.
```bash
wsl --install
```
**Line ending issues:**
@@ -104,6 +134,106 @@ docker compose logs -f
docker compose logs -f simstudio
```
## Scheduled Workflows Never Run
The most common self-hosting surprise.
**Docker Compose** — check the `cron` service is running and read its logs:
```bash
docker compose -f docker-compose.prod.yml logs --tail=50 cron
```
A `401` there means the app and the scheduler disagree on `CRON_SECRET`.
**Kubernetes** — check the CronJobs are present and firing:
```bash
kubectl get cronjobs -n simstudio
kubectl get jobs -n simstudio --sort-by=.metadata.creationTimestamp | tail
```
A stale `LAST SCHEDULE` or failing jobs usually means `CRON_SECRET` is missing or does not match between the cron pods and the app. Call the endpoint by hand to see the status code:
```bash
kubectl exec -n simstudio deploy/sim-app -- sh -c \
'curl -s -o /dev/null -w "%{http_code}\n" \
-H "Authorization: Bearer $CRON_SECRET" \
http://localhost:3000/api/schedules/execute'
```
Wrap it in `sh -c` with single quotes so `$CRON_SECRET` expands **inside the pod** — otherwise your local shell substitutes an empty value and you get a misleading `401`.
`401` means the secret does not match. `202` means the endpoint accepted the run; it does not tell you whether a schedule was actually due, so confirm in the Logs view.
## Gmail / Drive / Outlook Triggers Never Fire
These are **polling** triggers, driven by the per-minute `/api/webhooks/poll/*` jobs. Check the scheduler is running them — `docker compose logs cron`, or `kubectl get cronjobs -n simstudio` for a recent `LAST SCHEDULE`.
Microsoft Teams chat triggers are the different case: they use a Microsoft Graph subscription capped at about three days, renewed by the twice-daily `renew-subscriptions` job. If Teams triggers work for a couple of days and then stop, that job is not running. See [Background Jobs](/platform/self-hosting/background-jobs).
## Collaboration Breaks With Multiple Replicas
Two users editing the same workflow stop seeing each other, or live status never updates — with no error anywhere.
This is Redis. Pub/sub and the Socket.IO adapter have no cross-pod fallback:
```bash
kubectl exec -n simstudio deploy/sim-app -- printenv REDIS_URL
kubectl exec -n simstudio deploy/sim-realtime -- printenv REDIS_URL
```
Both pods must have `REDIS_URL`. On Helm they share one Secret, so setting it under `app.env` covers both. See [Redis](/platform/self-hosting/redis).
## App Crashes at Startup With a REDIS_TLS_SERVERNAME Error
`REDIS_URL` uses `rediss://` pointed at a bare IP address. TLS certificates cannot be verified against an IP, so set `REDIS_TLS_SERVERNAME` to the DNS name the certificate was issued for — or use a DNS hostname in the URL instead.
## File Uploads Fail With a CORS Error
The bucket's CORS policy does not allow your Sim origin. Uploads go directly from the browser to object storage via presigned `PUT`, so server-side configuration being correct is not enough.
If small uploads succeed but files over 50 MB fail at the last step, `ETag` is missing from the CORS exposed headers — multipart uploads read it from the browser. See [Object Storage](/platform/self-hosting/object-storage).
## Agent Output Arrives All at Once
Your reverse proxy is buffering the response stream. Set `proxy_buffering off` (Nginx) or `flush_interval -1` (Caddy). See [Networking](/platform/self-hosting/networking).
## Websockets Disconnect Every 30 Seconds
The load balancer's backend timeout is closing them. On GKE, attach a `BackendConfig` with `timeoutSec: 3600` to the realtime Service; on AWS, raise the ALB `idle_timeout`. Clients reconnect, so this degrades rather than breaks. See [Networking](/platform/self-hosting/networking).
## Knowledge Base Upload Fails
Embeddings need a hosted provider — set `OPENAI_API_KEY`, configure Azure OpenAI, or set `KB_EMBEDDING_MODEL=gemini-embedding-001` with a Gemini key. There is no local embedding backend, so Ollama or vLLM does not substitute. If a key is set, verify pgvector is installed on the database.
## Credentials Unreadable After a Restore
Integrations show as connected but fail, or provider keys error on decrypt. `ENCRYPTION_KEY` does not match the value in use when the backup was taken. There is no recovery — the original key must be restored.
## Kubernetes: App Pods Never Become Ready
Check the migrations init container first — a failed migration deliberately blocks the rollout:
```bash
kubectl logs -n simstudio deploy/sim-app -c migrations --tail=200
kubectl describe pod -n simstudio <pod>
```
Common causes: `DATABASE_URL` unreachable, the database user lacking rights to create the `vector` extension, or an OOMKill from insufficient memory. See [Upgrades](/platform/self-hosting/upgrades) for migration-failure recovery.
## Kubernetes: ImagePullBackOff
Either the tag does not exist in the registry (`helm get values sim` and check), or you are pulling from a private registry without `global.imagePullSecrets`. When mirroring into a private registry, set `global.useRegistryForAllImages: true` — otherwise third-party images still point at Docker Hub.
## Kubernetes: Postgres Pod Pending
```bash
kubectl describe pvc -n simstudio
```
Almost always no default StorageClass, no PV provisioner installed, or a StorageClass that does not support `ReadWriteOnce`. Set `global.storageClass` to pick a specific one.
## Getting Help
- [GitHub Issues](https://github.com/simstudioai/sim/issues)
apps/docs/content/docs/en/platform/self-hosting/upgrades.mdx
+216
View File
@@ -0,0 +1,216 @@
---
title: Upgrades
description: Pinning versions, running migrations safely, and rolling back
---
import { Tab, Tabs } from 'fumadocs-ui/components/tabs'
import { Callout } from 'fumadocs-ui/components/callout'
import { Step, Steps } from 'fumadocs-ui/components/steps'
import { FAQ } from '@/components/ui/faq'
## Pin your version
<Callout type="warn">
Never run `:latest` in production. An unpinned tag means an unplanned restart can pull a new version with new migrations at an arbitrary time.
</Callout>
Sim publishes images to GHCR, tagged by release alongside `latest`:
```
ghcr.io/simstudioai/simstudio
ghcr.io/simstudioai/realtime
ghcr.io/simstudioai/migrations
```
<Callout type="error">
**Pin `app`, `realtime`, and `migrations` to the same tag.** They share a database schema. An app newer than its migrations runs against a schema missing columns it expects; an app older than its migrations runs against a schema it does not understand. Mismatched tags is the most common self-inflicted upgrade failure.
</Callout>
<Tabs items={['Kubernetes (Helm)', 'Docker Compose']}>
<Tab value="Kubernetes (Helm)">
```yaml
app:
image:
tag: "v1.2.3" # a tag from the releases page
realtime:
image:
tag: "v1.2.3" # a tag from the releases page
migrations:
image:
tag: "v1.2.3" # a tag from the releases page
```
When `image.tag` is unset it defaults to the chart's `appVersion`, which moves when you upgrade the chart. Setting it explicitly decouples the two. For maximum determinism, pin `image.digest` instead:
```yaml
app:
image:
digest: "sha256:..."
```
</Tab>
<Tab value="Docker Compose">
Images track `latest` by default. To pin, set `SIM_VERSION` in `.env` to a tag from the [releases page](https://github.com/simstudioai/sim/releases):
```bash
# .env
SIM_VERSION=v1.2.3
```
One variable drives all three schema-coupled images, so they cannot drift apart. The `cron` service is deliberately excluded — it only makes HTTP calls and shares no schema, so it tracks `latest` unless you pin `SIM_CRON_VERSION`.
</Tab>
</Tabs>
## How migrations run
Migrations are Drizzle SQL files applied by a dedicated image.
- **Kubernetes** — an **init container on the app Deployment**. Every app pod waits for migrations to complete before it starts, so a failed migration blocks the rollout instead of starting an app against a mismatched schema. The container is idempotent, so it is a no-op on pods that start after the first.
- **Docker Compose** — a one-shot `migrations` service with `restart: no` that runs before the app.
Migrations are **forward-only**. There are no down-migrations, which is why the pre-upgrade backup below is not optional.
## Upgrade procedure
<Steps>
<Step>
### Read the release notes
Check the [releases page](https://github.com/simstudioai/sim/releases) for new required environment variables and breaking changes. When the chart's minor version moves, also read its `README.md` upgrade notes — chart upgrades occasionally rename or remove values keys.
</Step>
<Step>
### Take a backup
Snapshot the database immediately before upgrading. Because migrations are forward-only, this snapshot is your only rollback path for schema changes.
```bash
# Managed Postgres — take a manual snapshot
aws rds create-db-snapshot --db-instance-identifier sim-db \
--db-snapshot-identifier "sim-pre-upgrade-$(date +%Y%m%d)"
# Bundled Postgres — the Helm chart's database is named `sim` by default
# (Docker Compose uses `simstudio`; the cloud example values files override to `simstudio`)
kubectl exec -n simstudio statefulset/sim-postgresql -- \
pg_dump -U postgres -Fc sim > "pre-upgrade-$(date +%F).dump"
```
</Step>
<Step>
### Rehearse against real data
Migration surprises are usually data-shaped rather than schema-shaped, so a staging run against a copy of production data catches far more than a run against an empty database.
</Step>
<Step>
### Apply
<Tabs items={['Kubernetes (Helm)', 'Docker Compose']}>
<Tab value="Kubernetes (Helm)">
```bash
helm upgrade sim ./helm/sim \
--namespace simstudio \
--values my-values.yaml
```
Preview first if the chart version changed:
```bash
helm diff upgrade sim ./helm/sim -n simstudio --values my-values.yaml
```
Then watch the rollout:
```bash
kubectl rollout status -n simstudio deploy/sim-app --timeout=10m
kubectl logs -n simstudio deploy/sim-app -c migrations --tail=100
```
</Tab>
<Tab value="Docker Compose">
```bash
docker compose -f docker-compose.prod.yml pull
docker compose -f docker-compose.prod.yml up -d
docker compose -f docker-compose.prod.yml logs migrations
```
There is a short window where the app is unavailable while containers restart. Compose has no rolling-update mechanism — plan a maintenance window, or run Kubernetes if you need zero-downtime upgrades.
</Tab>
</Tabs>
</Step>
<Step>
### Verify
Run the [verification checklist](/platform/self-hosting/verify). At minimum: sign in, open a workflow, execute it, upload a file, and confirm the [background jobs](/platform/self-hosting/background-jobs) are still firing.
</Step>
</Steps>
## When a migration fails
The app pods will not become ready — this is by design.
```bash
kubectl logs -n simstudio deploy/sim-app -c migrations --tail=200
```
```bash
docker compose -f docker-compose.prod.yml logs migrations
```
Common causes:
| Symptom | Cause |
|---|---|
| `permission denied to create extension "vector"` | The database user lacks superuser rights. Create the pgvector extension manually as an admin, then re-run. |
| Connection refused / timeout | `DATABASE_URL` wrong, or the database is not reachable from the pod. Check network policy and credentials. |
| Lock timeout on a large table | A long-running query is blocking DDL. Drain traffic and retry during a quiet window. |
| Constraint violation | Pre-existing data conflicts with a new constraint. Capture the error, restore the pre-upgrade backup, and open an issue with the exact message. |
Do not manually edit the migrations table to skip a failed migration — the schema and Sim's expectations will diverge in ways that surface much later.
## Rolling back
**Application-only rollback** (no migrations ran, or the new migrations are additive):
```bash
helm rollback sim -n simstudio
```
```bash
# Docker Compose — set the previous tag and restart
docker compose -f docker-compose.prod.yml up -d
```
**Rollback after a schema change** requires restoring the database to the pre-upgrade backup, because migrations are forward-only:
1. Scale the app to zero.
2. Restore the pre-upgrade database snapshot.
3. Redeploy the previous image tag on all three images.
4. Verify.
This loses everything written since the snapshot. It is why the pre-upgrade backup and a staging rehearsal matter more here than in most systems.
<FAQ items={[
{ question: "Can I roll back an upgrade?", answer: "You can roll back the application with helm rollback. Schema changes cannot be rolled back — migrations are forward-only — so reverting a release that added migrations means restoring the pre-upgrade database backup and losing anything written since."},
{ question: "Can I skip versions?", answer: "Migrations apply in sequence, so jumping several releases generally works mechanically. It is riskier — those combinations get less testing, and you take several releases' worth of behavior changes at once. Upgrade in smaller steps where you can."},
{ question: "Is there downtime during an upgrade?", answer: "On Kubernetes with replicaCount > 1, the rolling update keeps the app available, though migrations run before the new pods start. On Docker Compose there is a restart window — Compose has no rolling-update mechanism." },
]} />
apps/docs/content/docs/en/platform/self-hosting/verify.mdx
+94
View File
@@ -0,0 +1,94 @@
---
title: Verify Your Install
description: A smoke test that exercises every subsystem in about ten minutes
---
import { Callout } from 'fumadocs-ui/components/callout'
import { FAQ } from '@/components/ui/faq'
Run this after a first install, after an upgrade, and after a restore. Each step exercises a different subsystem, so a failure tells you exactly where to look.
## Checklist
| # | Do this | Proves | If it fails |
|---|---|---|---|
| 1 | Open your Sim URL and create an account | App, database, TLS, migrations | `kubectl logs deploy/sim-app` — and check the `migrations` init container |
| 2 | Sign out and sign back in | Session handling, `BETTER_AUTH_SECRET`, `BETTER_AUTH_URL` | URLs must match your real origin exactly |
| 3 | Open a workflow and drag two blocks onto the canvas | Realtime websocket connection | Browser console for socket errors; see [Networking](/platform/self-hosting/networking) |
| 4 | Open the same workflow in a second browser window and edit | Cross-replica collaboration | With >1 replica this needs [Redis](/platform/self-hosting/redis) |
| 5 | Paste a model API key in settings and run a two-block workflow | Execution engine, credential encryption, outbound network | App logs; check `ENCRYPTION_KEY` is set and outbound egress is allowed |
| 6 | Upload a small file in Files | File storage end to end | With object storage configured: presigned URL + bucket CORS. On local disk: the upload proxies through the app |
| 7 | Upload a file larger than 50 MB | Multipart upload path (object storage only) | Confirm `ETag` is in the bucket's CORS exposed headers |
| 8 | Create a knowledge base and upload a PDF | Document parsing, embeddings, pgvector | Needs a hosted embedding provider — see below |
| 9 | Invite a teammate from workspace settings | Email delivery | App logs for the mailer; see [Email](/platform/self-hosting/email) |
| 10 | Connect an integration account | OAuth configuration | Redirect URI mismatch → see [Integrations & OAuth](/platform/self-hosting/integrations-oauth) |
| 11 | Create a workflow with a Schedule trigger set to every minute, deploy it, wait 2 minutes | **Background jobs** | Check the scheduler's logs — see [Background Jobs](/platform/self-hosting/background-jobs) |
| 12 | Trigger a workflow via the API with an API key | Public API and API-key auth | Check the key was created successfully in settings |
<Callout type="warn">
Step 11 is the one most people skip and most often discover broken weeks later. Scheduled workflows and every polling trigger depend on the scheduler, and a wrong or missing `CRON_SECRET` makes it fail silently from the app's side.
</Callout>
## Infrastructure checks
Before the UI walkthrough, confirm the deployment itself is healthy.
```bash
# Everything running?
kubectl get pods -n simstudio
# Migrations completed
kubectl logs -n simstudio deploy/sim-app -c migrations --tail=50
# Health endpoints
curl -fsS https://sim.yourdomain.com/api/health
# Background jobs scheduled
kubectl get cronjobs -n simstudio
# Redis configured (multi-replica deployments) — confirms the variable is set,
# not that Redis answers. Steps 3 and 4 below are the real reachability test.
kubectl exec -n simstudio deploy/sim-app -- printenv REDIS_URL
```
```bash
# Docker Compose
docker compose -f docker-compose.prod.yml ps
docker compose -f docker-compose.prod.yml logs migrations
curl -fsS http://localhost:3000/api/health
```
All six should be present on Compose: `simstudio`, `realtime`, `db`, `redis`, `cron`, and a completed `migrations`.
## Reading the failures
**Step 1 fails — app will not load.** Almost always migrations or database connectivity. Check the migrations init container first; a failed migration deliberately blocks the rollout.
**Step 2 fails — login loops or rejects.** `NEXT_PUBLIC_APP_URL` or `BETTER_AUTH_URL` does not match the origin you are browsing. Both must be the exact public URL, with scheme and no trailing slash.
**Step 3 fails — no live updates.** The reverse proxy is not passing websocket upgrades, or `/socket.io` is not routed to the realtime service. If realtime is on a separate hostname, `NEXT_PUBLIC_SOCKET_URL` must point at it and realtime's `ALLOWED_ORIGINS` must include the app origin.
**Step 4 fails — edits do not sync between windows.** With more than one replica, this is Redis. Confirm `REDIS_URL` is present on both pods.
**Step 5 fails — execution errors.** Check outbound connectivity to the model provider, then the app logs. If the error is about decrypting a credential, `ENCRYPTION_KEY` differs from the one that encrypted it.
**Step 6 or 7 fails.** With object storage configured, a CORS error in the browser console means the bucket policy does not allow your Sim origin; step 7 failing while step 6 passes specifically means `ETag` is missing from the exposed headers. On local-disk storage there is no CORS involved — uploads proxy through the app, so look at the app logs and the proxy body-size limit instead.
**Step 8 fails — knowledge base upload errors.** Knowledge bases need a hosted embedding provider — OpenAI, Azure OpenAI, or Gemini. There is no local embedding backend. If a key is set, check pgvector is installed on the database.
**Step 9 fails — no email arrives.** With no provider configured, emails are written to the app logs instead of sent — check there first to confirm the message was generated, then debug the provider.
**Step 11 fails — schedule never fires.** Read the scheduler's logs (`docker compose logs cron`, or `kubectl get cronjobs -n simstudio`). A `401` there means the app and the scheduler disagree on `CRON_SECRET`.
## After an upgrade
Re-run steps 1, 3, 5, 6, and 11 at minimum. Those cover the app, realtime, execution, storage, and background jobs — the five things a bad upgrade breaks.
## After a restore
Run the whole list, and pay special attention to **step 5 with an OAuth-backed integration**. That is what proves `ENCRYPTION_KEY` matches the backup. An app that loads and logs in but cannot decrypt credentials looks healthy right up until someone runs a real workflow.
<FAQ items={[
{ question: "How long does this take?", answer: "About ten minutes, most of it waiting for step 11's scheduled run. Run the infrastructure checks first — they take seconds and catch the majority of failed installs before you open a browser."},
{ question: "Do I need to run all of this after every upgrade?", answer: "No. Steps 1, 3, 5, 6, and 11 cover the app, realtime, execution, storage, and background jobs. Run the full list after a restore or a significant infrastructure change."},
]} />