Waleed f0fd48c2a4 fix(knowledge): bound chunking separators so one config can't stall processing (#6735)
* fix(knowledge): bound chunking separators so one config can't stall processing

`chunkingStrategyOptionsSchema.separators` accepted an arbitrary-length array
of arbitrary-length strings, next to a `pattern` field already capped at 500
chars. `RecursiveChunker` splits the whole document once per separator and
walks the list from the top for every oversized fragment, so a persisted
config with thousands of non-matching separators cost seconds of synchronous
CPU on every later document upload — work neither the processing `Promise.race`
timeout nor the after-the-fact chunk-count cap can interrupt. Measured on a
21.3 MB document: 632 ms at 100 separators, 6.1 s at 1000, 36.8 s at 5000.

- Bound `separators` to 32 entries of at most 100 characters on the write path.
  The largest built-in recipe (markdown) uses 16, so hand-tuned lists still fit.
- Keep the stored/read shape tolerant, so a config written before the bound
  still lists instead of failing response validation.
- Clamp in `RecursiveChunker` too, with a warning, so an already-persisted
  oversized list cannot reach the split loop. An over-long separator is dropped
  rather than truncated: a truncated separator matches where the configured one
  never did, silently re-cutting the document, while dropping it behaves like a
  separator that finds no match. A list left empty falls back to the recipe.
- Walk non-matching separators iteratively instead of recursing, so stack depth
  no longer tracks the separator count. Verified behavior-preserving against the
  previous implementation over 4000 randomized configs — byte-identical output.
- Validate in the create-base modal so the limit surfaces inline.

After the fix the same 21.3 MB document costs ~300 ms at every separator count.

* fix(knowledge): gate separator validation on the recursive strategy

- The separator refines ran for every strategy, but the field only renders for
  `recursive` and only that strategy submits it, so a value left behind by a
  strategy switch could block submit with no visible field to clear. Gated the
  same way the regex-pattern refine already is.
- Use absolute imports in the chunker test, per the repo convention.
2026-08-15 12:06:04 -07:00

Sim.ai Documentation Slack X

Ask DeepWiki Set Up with Cursor

Sim — Integrate, Context, Build, and Monitor AI agents

A workspace to build, deploy and manage AI agents and workflows.

Quickstart

Cloud-hosted: sim.ai

Open sim.ai

Self-hosted

git clone https://github.com/simstudioai/sim.git && cd sim
bun install
bun run setup

Open http://localhost:3000

The Sim platform — chat on the left, the visual workflow builder on the right

Capabilities

  • Connect 1,000+ integrations and every major LLM
  • Add Slack, Notion, HubSpot, Salesforce, databases, and more
  • Build agents visually, conversationally, or with code
  • Ingest files, knowledge bases, and structured table data
  • Monitor runs, logs, schedules, and workflow activity

One workspace, every surface

Chat and workflows are just the start — tables, files, and knowledge all live in the same workspace.

Tables in Sim — structured data your agents can query

Tables — a database, built in

Files in Sim — documents for your team and every agent

Files — one store for your team and every agent

Knowledge bases in Sim — synced docs your agents can search

Knowledge — your agents' memory

Self-hosting

Requirements: Bun and Docker.

bun run setup is an interactive wizard: it provisions the database, generates secrets, writes your .env files, connects a Chat API key, and starts Sim the way you choose:

  • Local dev — run from source to contribute or hack on Sim
  • Docker Compose — a self-contained instance for testing self-hosting
  • Kubernetes (Helm) — deploy to a local cluster

When it finishes, open http://localhost:3000.

Reconfigure an optional capability without rerunning the full wizard:

bun run setup status
bun run setup email
bun run setup storage
bun run setup sandbox
bun run setup jobs
bun run setup cache
bun run setup knowledge
bun run setup llm
bun run setup integration slack

bun run setup status detects the effective local-dev, Docker Compose, or current-context Helm configuration and reports configured, missing, or invalid capabilities and OAuth integrations without printing credential values. This is separate from bun run sim status, which reports whether installed services are running and healthy.

Manage your install with bun run sim:

bun run sim start | stop | restart   # bring your install up / down / cycle
bun run sim update                   # pull/rebuild and apply Compose images
bun run sim status                    # what's installed and healthy
bun run sim logs                      # follow logs
bun run sim doctor                    # diagnose configuration problems
bun run sim down                      # remove containers (data kept)
bun run sim reset                     # archive .env and wipe managed data

sim detects how you're running (Docker Compose, local dev, or Kubernetes) and acts accordingly.

Prefer a bare sim? Run bun link once — but note sim lands in ~/.bun/bin, which Homebrew's bun doesn't add to your PATH, so you may need export PATH="$HOME/.bun/bin:$PATH" in your shell profile.

Sim also supports local models via Ollama and vLLM. See the self-hosting docs for details.

Chat API Keys

Chat is a Sim-managed service. bun run setup connects a Chat API key for you — sign in when it opens your browser and the key is stored automatically. To view, create, or revoke keys later, go to sim.ai/selfhost/settings/chat-keys.

Environment Variables

See the environment variables reference for the full list, or apps/sim/.env.example for defaults.

Tech Stack

Next.js · Bun · PostgreSQL · Drizzle · Better Auth · Tailwind — and the rest of the stack

Contributing

We welcome contributions! Please see our Contributing Guide for details.

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Built by the Sim team in San Francisco

Languages
TypeScript 77%
MDX 20.8%
JavaScript 1.9%
CSS 0.1%