mirror of
https://github.com/simstudioai/sim.git
synced 2026-10-24 11:36:37 +08:00
Waleed
ef225f99ef
* fix(connectors): index Office documents and PDFs from SharePoint and OneDrive
The SharePoint and OneDrive connectors filtered their listings against a
12-item plain-text extension whitelist, so a document library of .docx, .pdf
or .xlsx files synced as "success, 0 documents" — no document, no failed row,
and no log line, which is indistinguishable from a wrong folder path. Both
whitelists had been unchanged since the connectors shipped, and Sim already
parses all of these formats for a manually uploaded knowledge base document.
Adds a shared `extractConnectorText` in connectors/utils that routes binary
document formats through the same `parseBuffer` the upload path uses, so the
OOXML zip-bomb guard and each parser's extraction limits apply. The
previously-accepted text formats stay on their exact existing path: sending
.csv through CsvParser would silently reformat every already-indexed connector
document on its next re-index.
Also logs a per-page count of files skipped for an unsupported extension.
Unsupported files are counted rather than turned into failed document rows, so
a library full of images does not fill the knowledge base with noise.
* fix(connectors): never index a degraded document extraction
`DocParser` and `PptxParser` never throw by design — on a legacy OLE `.doc`/`.ppt`
or a deck with no extractable text they return a placeholder sentence or scraped
ZIP internals so an interactive upload still shows the user something. Verified
against real OOXML fixtures: an image-only `.pptx` yields 1.9KB of
`[Content_Types].xml…` as "content", and a legacy `.ppt` yields "Unable to
extract text from PowerPoint file."
A connector sync would embed that into the vector index at scale, so it needs to
tell a real extraction from a fabricated one. Adds a declared `degraded` flag to
`FileParseMetadata`, set by exactly those two fallback paths, rather than having
callers sniff `extractionMethod`. `DocParser`'s plaintext branch stays unflagged:
a text file misnamed `.doc` is a genuine extraction.
`extractConnectorText` now raises `ConnectorTextExtractionError` when a parsed
format comes back degraded or blank, and SharePoint/OneDrive surface it as a
skipped document via the existing `markSkipped` path — so the file appears in the
knowledge base as a failed row telling the user to re-save it as DOCX/PPTX/XLSX,
instead of being silently dropped or indexed as junk.
The upload path is unaffected; it ignores the new flag.
* fix(file-parsers): register the document variants the parsers already handle
A document library holds whole format families, not just the headline extension
of each. These all extract correctly with the libraries already installed — they
were simply never registered, so every one of them was reported as an
unsupported file type:
docm dotx (WordprocessingML — mammoth reads word/document.xml regardless of
the package content type)
xlsm xlsb xltx ods (SheetJS reads every workbook container natively)
pptm potx (PresentationML)
odt odp (OpenDocument, via a new OpenDocumentParser)
Verified against real fixtures built with jszip and SheetJS rather than assumed:
officeparser identifies a Buffer by sniffing content with `file-type`, not by the
name we pass, so the routing had to be measured. `ods` goes to the spreadsheet
parser rather than OpenDocumentParser so its output keeps per-sheet structure.
`rtf` is deliberately excluded: nothing bundled extracts it, and DocParser's
plaintext branch would pass its control words through as if they were prose.
Converts the registry from `require()` inside per-parser `try/catch` blocks that
only logged to static imports. Every parser dependency is a regular, non-optional
one, so a resolution failure should fail loudly — the old form produced a silently
**empty** registry in which every format became `Unsupported file type`, with an
empty "Supported types are:" list as the only clue. The heavy extraction libraries
are still deferred inside the individual parsers, and connectors now import the
registry lazily so the ~60 connectors that never touch a file do not pull SheetJS.
Adds registry.test.ts, which exercises the real module: index.test.ts mocks
`@/lib/file-parsers` itself, so it validated its own fake routing table and the
real registry had no coverage at all. The new test gates every member of
SupportedFileType on having a registered parser that supports buffer parsing.
* fix(file-parsers): resolve the parser registry through a Map, not object keys
The registry rewrite switched extension lookup from
`Object.keys(parsers).includes(ext)` to a bracket read on an object literal,
which also resolves inherited keys. `PARSERS['constructor']` therefore returned
`Object` — truthy, with no parse methods — so a caller-supplied extension of
`constructor` fell through to "does not support buffer parsing" instead of being
rejected as an unsupported type, and `parseFile` would have raised a TypeError.
It also disagreed with `isSupportedFileType`, which used `Object.hasOwn` and
correctly returned false for the same input.
A Map has no prototype chain to walk, so lookup and support check now agree by
construction. `isSupportedFileType` also guards a non-string argument, which the
try/catch it replaced used to absorb.
A workspace to build, deploy and manage AI agents and workflows.
Quickstart
Cloud-hosted: sim.ai
Self-hosted
git clone https://github.com/simstudioai/sim.git && cd sim
bun install
bun run setup
Capabilities
- Connect 1,000+ integrations and every major LLM
- Add Slack, Notion, HubSpot, Salesforce, databases, and more
- Build agents visually, conversationally, or with code
- Ingest files, knowledge bases, and structured table data
- Monitor runs, logs, schedules, and workflow activity
One workspace, every surface
Chat and workflows are just the start — tables, files, and knowledge all live in the same workspace.
Tables — a database, built in |
Files — one store for your team and every agent |
Knowledge — your agents' memory |
Self-hosting
bun run setup is an interactive wizard: it provisions the database, generates secrets, writes your .env files, connects a Chat API key, and starts Sim the way you choose:
- Local dev — run from source to contribute or hack on Sim
- Docker Compose — a self-contained instance for testing self-hosting
- Kubernetes (Helm) — deploy to a local cluster
When it finishes, open http://localhost:3000.
Reconfigure an optional capability without rerunning the full wizard:
bun run setup status
bun run setup email
bun run setup storage
bun run setup sandbox
bun run setup jobs
bun run setup cache
bun run setup knowledge
bun run setup llm
bun run setup integration slack
bun run setup status detects the effective local-dev, Docker Compose, or current-context
Helm configuration and reports configured, missing, or invalid capabilities and OAuth
integrations without printing credential values. This is separate from bun run sim status,
which reports whether installed services are running and healthy.
Manage your install with bun run sim:
bun run sim start | stop | restart # bring your install up / down / cycle
bun run sim update # pull/rebuild and apply Compose images
bun run sim status # what's installed and healthy
bun run sim logs # follow logs
bun run sim doctor # diagnose configuration problems
bun run sim down # remove containers (data kept)
bun run sim reset # archive .env and wipe managed data
sim detects how you're running (Docker Compose, local dev, or Kubernetes) and acts accordingly.
Prefer a bare sim? Run bun link once — but note sim lands in ~/.bun/bin, which Homebrew's bun doesn't add to your PATH, so you may need export PATH="$HOME/.bun/bin:$PATH" in your shell profile.
Sim also supports local models via Ollama and vLLM. See the self-hosting docs for details.
Chat API Keys
Chat is a Sim-managed service. bun run setup connects a Chat API key for you — sign in when it opens your browser and the key is stored automatically. To view, create, or revoke keys later, go to sim.ai/selfhost/settings/chat-keys.
Environment Variables
See the environment variables reference for the full list, or apps/sim/.env.example for defaults.
Tech Stack
Next.js · Bun · PostgreSQL · Drizzle · Better Auth · Tailwind — and the rest of the stack
- Framework: Next.js (App Router)
- Runtime: Bun
- Database: PostgreSQL with Drizzle ORM
- Authentication: Better Auth
- Schema Validation: Zod
- UI: Shadcn, Tailwind CSS
- Streaming Markdown: Streamdown
- State Management: Zustand, TanStack Query
- Flow Editor: ReactFlow
- Docs: Fumadocs
- Monorepo: Turborepo
- Realtime: Socket.io
- Background Jobs: Trigger.dev
- Remote Code Execution: E2B
- Isolated Code Execution: isolated-vm
Contributing
We welcome contributions! Please see our Contributing Guide for details.
License
This project is licensed under the Apache License 2.0 - see the LICENSE file for details.
