23 Commits

Include renames

Author	SHA1	Message	Date
Waleed	34d65df7d6	fix(sdk): make the 0.2.0 SDK release safe to publish (#6616) ... The v2 SDK migration (#5273, #6564) shipped five breaking changes in both SDKs but got the release mechanics wrong in three separate ways, and left one of the two rewrites unable to complete a single successful call. Versions. packages/ts-sdk/package.json read 0.1.3 -- a patch digit added inside an unrelated compatibility commit, never deliberated. npm expands ^0.1.2 to >=0.1.2 <0.2.0, so every existing consumer would have picked the break up on a lockfile refresh: AsyncExecutionResult.jobId renamed to runId, executionId dropped from that interface, a failed sync run now throwing instead of resolving {success:false}, the request body reshaped, and the endpoint moved to /api/v2 with no fallback. 0.2.0 excludes every existing range, so the upgrade becomes opt-in. packages/python-sdk carries the identical break and was never bumped at all, so its publish job would have skipped green at the "version already exists" gate and left the repo and PyPI silently divergent; it moves 0.1.2 -> 0.2.0 in lockstep, along with the __version__ string in simstudio/__init__.py, which tracks pyproject and would otherwise have started lying. setup.py is left at 0.1.1: it is unchanged from main and demonstrably unread (0.1.2 published from pyproject while setup.py already said 0.1.1). It wants deleting, in its own commit. A 404 fallback was considered and rejected. The legacy 202 body's statusUrl points at /api/jobs/{jobId}, so mapping jobId onto runId would hand the caller an id that getWorkflowRun cannot resolve against that same old server -- a successful execute followed by an inexplicable failure on the next call is a worse contract than a clean 404. Both READMEs instead state the minimum server version and name the endpoint to check for. Cancelled runs. packages/python-sdk computed success as status != 'failed', so a run cancelled out of band reported success=True. The TypeScript SDK uses a closed whitelist and reports False, and before the migration both SDKs read the server's own value, which was False -- so this was a Python regression, not merely an inconsistency. Fixed by mirroring the whitelist. The v2 contract enumerates exactly completed|failed|paused|cancelled, so narrowing the blacklist to a whitelist cannot drop a live value, and a status added later now defaults to "not successful" rather than silently reporting True. WorkflowExecutionResult gains a status field because Python, unlike TypeScript, does not throw on 'failed' -- so success=False alone is ambiguous there in a way it is not in the TypeScript SDK, which is why status is not added to both. Rate-limit header. Found while auditing the two SDKs for further divergence, and the reason the Python bump could not have shipped as it stood: every authenticated v2 response now carries X-RateLimit-Reset as an ISO 8601 timestamp (recorded by v2RateLimits.publicApi, stamped by withRouteHandler). The Python SDK parsed it with int(), raising a bare ValueError that no handler in execute_workflow catches -- so every successful v2 execution raised instead of returning. None of the legacy endpoints the SDK previously called record a rate-limit snapshot, which is why the latent int() survived until the v2 move. The TypeScript SDK already branches on the format; _parse_reset_header mirrors it, including degrading an unrecognised value to 0, because a quota hint must not take down the call it rode in on. Timing metadata. The v2 rewrite stopped forwarding startedAt/endedAt, which main passed through and the TypeScript SDK still reports; restored under the same startTime/endTime keys the TypeScript SDK uses. Tests: cancelled/failed/paused status coverage, the ISO reset header, and the restored metadata keys, each verified red against the unfixed line first. The TypeScript suite gains matching cancelled/paused and ISO-reset pins -- they pass against today's source by design, and were confirmed to fail against a deliberately degraded copy so they are not toothless. Deliberately not included: a CI guard failing a PR that changes SDK source without a version bump. It would have caught this twice over, but it is a new script and workflow rather than a fix to the defect at hand. Review revision. bun.lock recorded packages/ts-sdk at 0.1.3 and was left stale by the first pass, so the repo asserted two versions for the same workspace package -- in a change whose whole thesis is that the version strings had diverged. It does not break CI (bun 1.3.14 accepts the mismatch under --frozen-lockfile, confirmed here), but 092311ea68 bumped the lock in lockstep with package.json, and the next unfrozen install would otherwise drop the line into an unrelated PR. _parse_reset_header gated the numeric branch on str.isdigit(), which accepts characters int() rejects ('²'.isdigit() is True, int('²') raises) -- and that int() sits outside the try, so the one function added to stop a quota hint raising could still raise, contradicting its own docstring. str.isdecimal() is exactly the set int() accepts. The tolerates-unparseable test is parametrized over both forms and was confirmed red on '²' against isdigit. Docs and docstrings: apps/docs api-reference/python.mdx mirrors the README's dataclass block and was the only copy left without the new status field. RateLimitInfo now names its units, because reset is epoch seconds for the legacy integer and milliseconds for the ISO form that v2 sends. execute_workflow's Args entry still described the pre-v2 body shape ("spread at root level"); every input is nested under input now, and this is the commit that ships that help() text to PyPI. The "declared last so positional construction keeps working" sentence was a maintainer's note that belongs in this message, not in every user's help(WorkflowExecutionResult).	2026-08-12 01:48:37 -07:00
Theodore Li	092311ea68	fix(api): restore migrated endpoint and SDK compatibility (#6564) ... * fix(api): restore migrated endpoint compatibility * fix(api): close remaining migration regressions * fix(files): validate ensured folder paths * fix(files): project folder path validation * fix(files): align archive regression fixture * fix(tables): avoid partial bulk update failures * fix(auth): project legacy knowledge audits	2026-08-11 20:36:37 -04:00
6 people	263e3ca67e	improvement(external-endpoints): v2 versions with clean signatures + updated docs based on openapi spec (#5273) ... * v0.6.29: login improvements, posthog telemetry (#4026) * feat(posthog): Add tracking on mothership abort (#4023) Co-authored-by: Theodore Li <theo@sim.ai> * fix(login): fix captcha headers for manual login (#4025) * fix(signup): fix turnstile key loading * fix(login): fix captcha header passing * Catch user already exists, remove login form captcha * improvement(external-endpoints): v2 versions with clean signatures + updated docs * feat(usage): accept X-API-Key on usage-logs list + export /api/users/me/usage-logs and /export now use checkHybridAuth — the same auth /api/users/me/usage-limits already accepts — so external monitors can read summary.bySourceCredits (the source breakdown of usage-limits' aggregate currentPeriodCost) instead of estimating Copilot spend by subtraction. Workspace-scoped keys are pinned to their own workspace's slice of the ledger: the filter defaults to the key's workspace and an explicit mismatch 403s. Both endpoints documented in openapi-core.json. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(billing): dedicated v2 usage endpoints; keep internal usage routes session-only Replaces the earlier X-API-Key enablement on /api/users/me/usage-logs with a dedicated public surface, so the internal Billing-settings endpoints can evolve with the UI while external monitors get a stable versioned contract: - GET /api/v2/billing/usage — current-billing-period summary with bySourceCredits (the source breakdown external monitors need to watch e.g. Copilot consumption without estimating by subtraction), plus limitCredits and plan - GET /api/v2/billing/usage/logs — cursor-paged credit ledger in the v2 envelope - workspace-scoped keys are pinned to their own workspace's slice; personal keys read the account ledger The public wire is credits-only: usage-logs rows now carry a hasCost boolean instead of dollarCost (the Billing UI only needed the >0 signal), and the rateLimit block is removed from the usage-limits response and docs (deploy-modal tab relabeled accordingly). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(docs): validate OpenAPI specs against the Zod contracts in CI The specs in apps/docs are hand-authored because they carry what Zod never defines — error envelopes, status codes, prose, examples — so they can't be generated; check:openapi validates them instead: - spec integrity: $refs resolve, operationIds unique, 2xx documented, no orphaned component schemas - v2 conventions: every /api/v2 operation documents 401 + 429 and every 4xx/5xx resolves to the canonical { error: { code, message } } envelope - contract cross-check: contracts are auto-discovered from lib/api/contracts/v2 (each carries its method + path); doc<->contract coverage both ways, query/body/response field diffs via z.toJSONSchema - examples: documented request/response examples must parse with the matching contract's actual Zod schemas First run caught real drift, fixed here: 16 stale orphaned schemas in the core spec, the v2 billing ops referencing v1-shaped error components, deploy/rollback examples missing the required nullable lifecycle keys, CreateTableBody missing folderId, a legacy-grammar delete-rows example, and four knowledge document ops missing their required workspaceId query param. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * fix(docs): recursive field diff in check:openapi + the deep drift it found A mutation test showed the doc<->contract field diff only compared top-level properties, so a typo inside the { data } envelope passed. The diff now descends through matching object properties and array items (both sides must expose a property set — passthrough contracts and prose-only docs end the descent instead of false-positive), with the Zod JSON-schema root doubling as the $defs context. Deep drift it immediately caught, fixed here: select-column config (options/multiple) missing from every tables column schema, AddColumnBody hand-rolling a third column shape (now composed from ColumnInput, with position/workflowGroupId as the per-op extensions the contracts actually admit), chunking strategyOptions undocumented, and the deployment lifecycle fields (activeDeployment/latestDeploymentAttempt) missing from DeploymentState. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * fix(security): close the triggerType rate-limit bypass on workflow execute Caller-supplied triggerType flowed unchecked into preprocessExecution, whose checkRateLimit default turns OFF for 'manual'/'chat' — so any API-key caller, and any anonymous public-API caller billed to the workspace owner, could execute unthrottled by sending {"triggerType":"manual"} (async runs also skipped the worker-side check via admissionCompleted). External callers may now only send the redundant 'api' value; internal JWT callers ('workflow'/'mcp') are unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * refactor(execution): extract enqueue/status/cancel into shared libs Prepares the v2 execution surface: handleAsyncExecution's queue logic moves to lib/workflows/executor/enqueue-execution.ts (slot/claim semantics encoded in a discriminated outcome, not HTTP statuses), the execution-status read to execution-status.ts, and the order-sensitive cancel machinery to lib/execution/cancel-workflow-execution.ts. The v1 routes re-render identically — their suites pass unmodified. Also: preprocessExecution gains rateLimitCounter ('sync'|'async') and its 429 now carries code RATE_LIMIT_EXCEEDED + retryAfterMs (previously indistinguishable from the concurrency 429 and Retry-After was discarded); and the duplicate cancel contract in contracts/logs.ts is unified on the full 5-value reason enum — its narrower copy made requestJson throw a client ZodError when cancelling a paused HITL run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(execution): callable execution service + structured error classifier executeWorkflowService composes the same libs the v1 route holds inline (call-chain guard, execution-id claim, LoggingSession, preprocessing, deployed-state load + file-field processing, timeout-bound executeWorkflowCore, output hydration/compaction) for the deployed-state caller class — the seam the v2 execute route and in-process internal callers share, making the HTTP endpoint syntactic sugar. classifyExecutionError stops discarding the block context that buildBlockExecutionError already attaches at throw sites: failed runs now yield {message, code, blockId, blockName, blockType} with a stable append-only code enum (TIMEOUT/CANCELLED/USAGE_LIMIT_EXCEEDED/ INVALID_INPUT/BLOCK_EXECUTION_FAILED/CHILD_WORKFLOW_FAILED/ OUTPUT_TOO_LARGE/EXECUTION_FAILED), so callers route on error class instead of substring-matching messages — the single place raw errors are interpreted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(api): POST /api/v2/workflows/[id]/execute Thin route over executeWorkflowService: X-API-Key or anonymous public-API auth (sync/stream only for anonymous), strict body with body-flag async (no mode headers on v2), SSE passthrough for stream, and the execution resource response — executionId always present, in-band run failures are status:'failed' with the structured {message, code, blockId, blockName, blockType} error, sync timeout is status:'failed' + TIMEOUT instead of v1's 408, and a Response block's payload stays inside output (authors never control response status/headers on this origin). Async debits the async bucket and the 202 statusUrl points at the v2 executions resource. Adds CLIENT_CLOSED_REQUEST/SERVICE_UNAVAILABLE to the v2 error codes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(api): v2 executions status + cancel with queued backfill GET /api/v2/workflows/[id]/executions/[executionId] is the single status URL for sync and async runs: before the async worker writes the durable log row, status is backfilled from the job queue (deterministic job id) as 'queued'/'running' — closing v1's 202-to-pickup 404 window — and failed runs carry the structured error object. POST .../cancel renders the shared cancellation lib in the v2 envelope with the tightened 5-value reason enum. Both authenticate via the shared resolveV2WorkflowAccess (X-API-Key, authz masked as 404, allowPersonalApiKeys honored). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(execution): workflow tool + MCP bridge run in-process workflow_executor (workflow-as-agent-tool) short-circuits in executeTool through WorkflowBlockHandler — the same invocation boundary canvas child workflows use — mirroring the deployed_block_executor precedent. The MCP serve bridge calls executeWorkflowService directly instead of fetching its own execute endpoint; deployment-version pinning, MCP response-size rejection, and the actor override become typed options instead of header sniffing. Both callers drop the double admission slot and duplicate top-level log row the HTTP hop cost, and failed child runs now surface the structured error + child executionId so parents and MCP clients can route on error class and hand providers a reproducible handle. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(infra): CORS + CSP coverage for the v2 execute path /api/v2/workflows/:id/execute gets the same wildcard-origin, credential-free CORS policy as v1 (the default credentialed policy would block browser API-key calls and open a cookie CSRF surface) with X-Sim-Stream-Protocol allowed and no X-Execution-Mode (async is body-selected on v2), plus the COEP/COOP/CSP header block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(ui): deploy modal + copilot advertise the v2 execute surface All 20 API-tab snippets move to POST /api/v2/workflows/{id}/execute with the nested {"input": ...} body, async as the "async": true body flag (X-Execution-Mode gone), status polling against the v2 executions resource, the third tab renamed Usage and pointed at /api/v2/billing/usage, and {data} envelope unwraps in the printed responses. Fixes the latent baseUrl derivation (endpoint.split('/api/workflows/')) that would have silently built garbage URLs under a v2 endpoint, and deletes dead code (exampleCommand across 3 sites, getAsyncExampleTitle). Copilot deploy/manage/serializer endpoint builders and the api_trigger bestPractices example follow (the latter also drops its hardcoded staging host). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * docs(api): document the v2 execution surface Adds execute, execution status, and cancel to openapi-v2-workflows.json with the structured ExecutionError schema (append-only code enum + block attribution) and the ExecutionResource contract, documenting the rules that differ from v1: modes are body-selected, a failed run is HTTP 200 with status 'failed', an executionId always means data (never the error envelope), queued status is visible immediately, and Response-block payloads stay inside output. Registers the three pages in the generated workflows meta.json and bumps the route-count baseline. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(api): gate the whole /api/v2 surface behind one flag; UI stays on v1 Every v2 route now runs exactly one check immediately after auth — v2ApiGateError — and answers 404 when the `v2-api` flag is off, so the surface is invisible until it is deliberately rolled out. The gate is keyed on userId only: a workspace/org-keyed check would have to read membership for a caller-supplied id before authorization runs, and its 404-vs-403 split would leak cohort membership (the trap the per-domain table gate worked around by running late). The two executions routes inherit it from the shared access resolver; the tables-specific gate is removed so no route checks twice. `tables-v2-api` stays, now gating only the internal predicate-grammar route /api/table/[tableId]/query — note v2 tables routes move to the unified flag, so enabling them is a `v2-api` decision now. Reverts the deploy modal, copilot handlers, and api_trigger example to the v1 execute endpoint: v1 works unchanged, and the UI must not advertise a surface most users would get a 404 from. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * fix(executor): restore child-cost aggregation dropped by the staging merge Staging's custom-block rewrite deleted `aggregateChildCost` from workflow-handler.ts, and git merged that file cleanly — but this branch's workflow-tool-runner.ts, added for the v2 execute migration, still imports it. A silent semantic conflict: no marker, broken build. Taking staging's rewrite is correct, so the helper is defined locally in its one remaining consumer rather than resurrected in the file staging just rewrote. Same four lines over the still-exported `calculateCostSummary`, so a failed child workflow keeps billing the hosted-key spend it consumed instead of reporting $0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(tables): make lib/table/orchestration the single implementation (#6134) * refactor(orchestration): move the shared error contract out of lib/workflows OrchestrationErrorCode and statusForOrchestrationError are the contract every lib/[resource]/orchestration module returns against, but they lived inside the workflows module, so resource-neutral code (lib/folders) already had to import from a workflow path. Moved to lib/core/orchestration/types. Adds a 'locked' class mapping to 423. Both tables and workflows have a lock that forbids a mutation, and each caller was translating that to a status itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(tables): make lib/table/orchestration the single implementation Column update was implemented four times — the UI route, v1, v2, and the copilot table tool — each calling the same column services but owning its own guards, error mapping, and audit. The copies had drifted, and the drift was the bug: v2 was missing both guards, only the copilot copy minted stable option ids, and only v1/v2 audited. performUpdateTableColumn, performDeleteTable, and performDeleteTableRow now own that logic; all ten call sites reduce to auth, parse, call, render. The guards are asserted once in lib/table/orchestration rather than four times against four routes. Behavior this consolidates, previously true on only some paths: - The typeChanging guard. updateColumnType early-returns on an unchanged type and drops any options sent with it, so restating the current type alongside new options silently discarded them. v2 had no guard at all and, since its contract shares v1's body schema, accepted options and ignored them. - The select-unique guard. Each write is its own locked transaction, so a rename or type change paired with a constraint write that is going to fail commits first and then throws, half-applying the schema change. - Stable select-option ids. Cells reference the option id, so an edit that re-sends an option by name has to reuse it or every cell holding it is orphaned. Only the copilot path did this; normalizeSelectOptionsInput moves to lib/table/select-options and now covers every caller. It preserves a supplied id, so it is a no-op for the fully-formed options the HTTP contracts accept. - required forwarded into the type and options writes, so a conversion validates against the constraint the same request is setting. - An audit on every successful update. The UI route and the copilot tool emitted none. - Single-row delete through the row service. v2 did a raw db.delete, skipping assertRowDelete and deleteOrderedRow, so a delete-locked table returned 200 and the row-count bookkeeping never ran. - The delete actor handed to deleteTable, which audits only when a row was actually archived. v1 and v2 omitted it and audited themselves outside that check, emitting TABLE_DELETED for a no-op delete of an archived table. Failure classes come back as OrchestrationErrorCode; v2 renders them through a new v2ErrorForOrchestration, mirroring statusForOrchestrationError on the v1 and UI surfaces, so a given failure maps to the same status everywhere. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(tables): bind the column-update tests to the orchestration function The base's route tests assert which column service each payload reaches — the behavior that now lives in performUpdateTableColumn. They mocked the `@/lib/table` barrel; the orchestration module imports the service directly, so they mock that too and keep asserting the same thing through the extracted implementation. The orchestration tests move onto the base's semantics: writes address the stable column id, a rename rides inside the write it accompanies rather than running first, and the currency guards replace the non-select options guard the service now owns. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(copilot): drop the column-type import the delegation made dead Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(tables): move the audit log out of the table service `lib/table/service.ts` wrote its own audit rows, so whether an operation was audited depended on which function a caller reached for rather than on a user having performed it. That is what let v1 and v2 audit a no-op delete, and what made `deleteTable`'s optional `actingUserId` double as an audit opt-out flag. Worse, most sites fell back to `actingUserId ?? createdBy`, so an unattributed call was logged against the table's *creator*. The copilot `mv` path passed no actor at all: renaming someone else's table recorded them as the renamer. Audit now lives in the orchestration functions — performDeleteTable, performRenameTable, performMoveTableToFolder, performUpdateTableLocks — and the services just write. Internal callers (folder cascade, import rollback) keep calling the service and are silent by construction rather than by remembering to omit an argument. Two services now return what the audit needs: `deleteTable` reports whether it actually archived a row, so a repeat delete logs nothing; `updateTableLocks` returns the before/after locks, since only the locked write can observe the transition its description names. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(tables): restore audit provenance and conflict status in orchestration Moving the audits into the orchestration functions dropped three things the routes had been carrying, and added one the orchestration now owns twice. - The v1 and v2 column-update routes passed `request` to `recordAudit`, so their audit rows recorded the caller's IP and user-agent. The orchestration function had no way to receive it. Every table orchestration function now takes an optional `OrchestrationRequestContext` and every HTTP route forwards it; the copilot and VFS callers, which have no request, omit it. - `classifyTableMutation` matched `TableConflictError` on "already exists" appearing in the message and reported it as `validation`, turning the UI route's 409 on a duplicate table rename into a 400. It now matches the type, the way `performRestoreTable` already did. - `captureServerEvent` ran on every delete while the audit was gated on a row actually being archived, so a repeat delete of an archived table still reported `table_deleted`. Both now hang off the same evidence. - The copilot delete path kept its own `captureServerEvent` from when the service did not emit one, double-counting every copilot table delete. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a * fix(tables): say which type a no-op column update restated A copilot `update_column` payload whose only content was the column's current type used to return success with the live schema, while the v1, v2, and UI routes rejected the same payload with "No updates specified". Delegating to `performUpdateTableColumn` unified them onto the routes' rejection — correct, but the message tells the caller its request was empty when it named a type. The orchestration function now reports the same thing `updateColumnType` reports when it loses this race concurrently: the column is already that type, re-issue without the type change. An empty payload still reads "No updates specified". Drops the copilot's `outcome.table ?? tableForUpdate` fallback with it — the comment described the no-op that can no longer reach that line, and a success always carries a table. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a * refactor(tables): classify failures by type instead of by message text The table module decided HTTP statuses by searching error messages for phrases. `VALIDATION_MESSAGE_FRAGMENTS` and `ROW_WRITE_ERROR_PATTERNS` held 32 substrings between them, and fifteen more lists were inlined in routes — 83 matchers over 17 files, each its own copy of the guesswork and already drifted apart. It made message wording load-bearing: `TableRowLimitError`'s own doc comment noted that its text had to contain "row limit" for a route to answer 400, and adding "already exists" to a rename message silently demoted a 409 to a 400 (the bug fixed one commit ago, by adding another special case). Services now throw `OrchestrationError`, which carries the transport-neutral `OrchestrationErrorCode` the layers above already speak. Classification is one `instanceof` in `orchestrationErrorResponse` (UI + v1) and `v2CaughtOrchestrationError` (v2). Every pattern list is gone. Wording is free to change; an unclassified error still becomes a generic 500, which is what an unexpected fault should be. `asOrchestrationError` walks the `cause` chain rather than testing the caught value directly: drizzle wraps a throw raised inside a transaction callback in a `DrizzleQueryError` whose own message is the failed SQL, so a bare `instanceof` would drop every failure raised inside `withLockedTable`. That is the same reason `rootErrorMessage` had to dig for a root cause before. Three throws stay bare `Error` deliberately — `Table ID mismatch`, `Workspace ID mismatch`, and `Failed to build upsert conflict predicate` are internal invariants no consumer classified, and they keep falling through to a 500. `Insufficient capacity` was in the pattern list with no producer anywhere in the codebase. Status changes, all deliberate: - `'forbidden'` joins the code union so the table-row-limit ceiling keeps its 403; without it this refactor would have flattened it to 400. - import-async's table-limit rejection: 400 -> 403, matching the two other create routes it had drifted from. - Renaming a table to an invalid name: 500 -> 400. `validateTableName` messages don't contain "Invalid", so no matcher ever caught them. - Restoring a table that isn't archived, or into an archived workspace: 500 -> 400. - A duplicate *column* name stays `validation`/400 rather than becoming a 409 like a duplicate table name. Both v1 and the orchestration have always answered 400 for it; changing a published status is not this refactor's job. The twelve tests that changed were asserting the substring mechanism itself, constructing plain `Error`s with magic strings. They now assert the real contract, plus new cases pinning that identical wording carrying no classification stays internal and keeps its message off the wire. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * feat(api): add v2 endpoints for MCP servers, skills, custom tools, folders, and credentials (#6150) * feat(api): add v2 endpoints for MCP servers, skills, custom tools, folders, and credentials * fix(api): correct credential role, skill permission bar, MCP url identity, and custom-tool conflict mapping * fix(api): align credential mutation gating, provider-outage status, and unique-violation conflicts * fix(api): close unique-violation, revival, orphan-write, and env-rename gaps * fix(api): treat every provider-outage code as unavailable on create and update * fix(credentials): use the shared outage predicate on the session update path * fix(contracts): anchor the predicate double-cast annotation to the cast `check:api-validation:strict` counted 9 unannotated double-casts against a baseline of 8, failing CI. The predicate leaf schema was annotated, but the annotation sat above the declaration while the checker anchors on the line carrying the cast — five lines below, at the close of the object literal. The scanner walks back at most three lines and stops at the first non-comment one, so it hit `value: z.unknown().optional(),` and never saw the reason. Splitting the object schema from the cast puts them adjacent, so the existing reason binds. No behavior change — the cast, the schema, and the reasoning are unchanged. Also lowers the rawJsonReads ratchet 6 -> 5 to match the current count, which had drifted down; leaving it high lets a removed raw read silently come back. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(skills): point the orchestration error contract at its moved module #6150 branched before #6134, so skill-lifecycle.ts imports @/lib/workflows/orchestration/types — the module #6134 moved to @/lib/core/orchestration/types. Git merged a file deletion on one side with a new file referencing it on the other: no textual conflict, broken build. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(knowledge): make lib/knowledge/orchestration the single implementation (#6154) * refactor(knowledge): make lib/knowledge/orchestration the single implementation Knowledge base create was implemented four times — the internal route, v1, v2, and the copilot tool — and the orchestration around the shared write had drifted. Extract it the same way lib/table/orchestration was: services write, orchestration decides which writes run, guards them, audits them, and returns a transport-neutral failure. Behavior converged, not preserved: - One chunking default (DEFAULT_CHUNKING_CONFIG). The agent defaulted minSize to 1 against the API's 100, so identical input produced differently-chunked knowledge bases depending on who created it. The agent path now chunks at 100. - Every successful mutation is audited inside the orchestration function. The copilot tool called recordAudit zero times, so agent-created knowledge bases, document uploads, updates and deletes left no audit trail at all. - Failures classify by class, not by message text. The knowledge service errors are OrchestrationError subclasses and storage-quota rejections throw a shared StorageLimitExceededError, replacing four separate message greps for "already exists" / "does not have permission" / "storage limit". delete_connector reported the opposite of what happened. It reached the route through an internal HTTP self-call that sent no query string, so the route's keep-documents default always applied while the agent told the user the documents had been removed. The self-call is gone — all four connector operations run in-process — and the orchestration returns the real counts. Also: - OrchestrationErrorCode gains 'payload_too_large' (413 / PAYLOAD_TOO_LARGE). Without it, dropping the storage-limit message match would have regressed the documented 413 on knowledge base create and document upload to a 500. - messageForOrchestrationError renders a route's own wording for an unclassified fault, so a driver's message no longer reaches the client on a 500. - v1 and v2 knowledge base update now forward actorUserId, which the service requires for a workspace move; both omitted it. - The connector DELETE route reads deleteDocuments through parseRequest. Its contract declared z.boolean(), which would have rejected the string a query param actually is. - Drop the 409 from POST /api/v2/knowledge/{id}/documents in the OpenAPI spec. Nothing on the upload path throws a conflict; it was only ever reachable by the message match this change removes. Behavior change worth noting: a v1/v2 PUT carrying only the workspaceId scope field and no actual updates now returns 400 rather than 200 with the unchanged knowledge base. Deliberately deferred: document update remains internal-only. Extracting performUpdateKnowledgeDocument makes exposing it on v1/v2 a contract and a route away, but that is a new public surface rather than part of this consolidation. * fix(knowledge): make connector create atomic and stop flattening failures Review round 1 on #6154. - Resolve the billing payer before the connector is committed, not after. A malformed attribution header rejected post-commit left a live connector behind a 500, and a retry created a duplicate plus duplicate sync work. Manual sync resolves before writing its audit for the same reason. - Let the source-config validator carry its own failure class. Collapsing every rejection to `validation` flattened the connector PATCH route's 401 (stale stored credential) and 409 (missing workspace context) into a 400. - Add `unauthorized` to OrchestrationErrorCode. It is the class that 401 was already expressing on this route, and the v2 vocabulary already had UNAUTHORIZED; only the shared union was missing it. - Report a knowledge base that exists but failed to archive as failed, with the reason, rather than as not found. The copilot delete loop folded every non-not-found failure into `notFound`, telling the user it was never there. - Route copilot failures through the same message helper the HTTP surfaces use, so an unclassified fault's raw text (a driver's failed SQL) no longer reaches the agent verbatim while the UI and public APIs get the generic wording. * feat(api): expand the public v2 files surface (#6160) * feat(api): expand the public v2 files surface Adds folder support, rename/restore, move, bulk archive, share, and content replace to /api/v2/files, so managing files by API no longer stops at upload + download + archive-one. Routes are thin: auth -> parse -> perform* -> serialize. Share and content replace get their orchestration extracted first so the session routes and the public ones cannot diverge on the effective-authType resolution, the EE public-sharing gate, or the storage-quota classification. Presigned upload stays session-only: presign does an advisory quota check and the real debit happens in the separate register step, so a caller that never registers leaves unaccounted bytes with no reaper. The buffered multipart path debits inside uploadWorkspaceFile's own transaction. * fix(files): classify folder and content failures instead of 500ing them Bugbot round 1. The v2 routes map errorCode straight to a status, so every manager failure that arrived unclassified became a 500 for what is really a caller-fixable 400 or 404. - Folder manager throws OrchestrationError: missing target/folder -> not_found, reparent cycle / self-parent / restore-into-archived-workspace -> validation. - File manager does the same for the in-transaction 'File not found' paths that the earlier pass missed. - updateWorkspaceFileContent's outer catch re-wrapped everything in a bare Error, which stripped the class off StorageLimitExceededError and the new not_found alike. It now rethrows a classified failure untouched and attaches cause to the generic wrap, so asOrchestrationError can still walk the chain. - Every remaining perform* gained the asOrchestrationError branch. - renameWorkspaceFile returned the pre-update read, so the v2 PATCH reported a stale updatedAt; it now returns the timestamp it actually wrote. Docs: upload auto-suffixes a duplicate name rather than rejecting it, matching the in-app uploader. The description claimed 409 and was simply wrong. * fix(files): surface a failed upload read-back as the real error getWorkspaceFile swallows a query failure and returns null unless throwOnError is set, so a transient blip on the post-upload read reported as 'file could not be read back'. Distinguish the two: a real null after a just-committed write is an invariant break, a query failure is itself. * revert(api): drop the dedicated v2 file-folder routes File folders already live in the shared folder table as resourceType 'file' (#6045 cut them over, #6051 dropped workspace_file_folders), and the remaining file-specific folder machinery is being folded into the generic folder engine. Publishing /api/v2/files/folders/** would pin that transitional split into a public contract we'd then have to keep or break. Files stay folder-aware — folderId/folderPath on the projection, folderId on upload, and the move route — because a folder id is a folder.id and survives the unification untouched. Folder management belongs on /api/v2/folders once that surface serves resourceType 'file'; until then there is no v2 way to enumerate file folders, which is the deliberate gap. The orchestration classification fixes stay: the internal routes and the copilot file-folder tools still call those perform* functions. * fix(files): classify upload failures instead of matching their wording Bugbot round 2. uploadWorkspaceFile had the same outer-catch rewrap that updateWorkspaceFileContent did, so a blown storage quota reached the route as a bare Error and the v2 handler recovered the status by substring-matching the message. Any rewording silently demoted a 413 to a 500. - uploadWorkspaceFile rethrows a classified failure untouched and attaches cause to the generic wrap. - FileConflictError is now an OrchestrationError('conflict'), so a duplicate name classifies like every other conflict. Its 'FILE_EXISTS' discriminator had no readers and is gone; the instanceof checks elsewhere still hold. - The v2 upload handler uses v2CaughtOrchestrationError, dropping all three string matches. Also documents that bulk-archive is best-effort: unknown or already-archived ids are skipped rather than failing the call, and deletedItems is what actually happened. That asymmetry with the single-id DELETE was undocumented. * feat(api): add search, filtering, and sorting to the v2 list endpoints (#6189) * feat(api): add search, filtering, and sorting to the v2 list endpoints One convention across every v2 list, documented on lib/api/contracts/v2/shared.ts: `search` (case-insensitive substring on the resource's natural name field), `sortBy` + `sortOrder` (per-resource enum, never a free string), and enumerated resource-specific filters. Reuses the sortBy/sortOrder pair v2 logs and v2 knowledge-documents already ship rather than inventing a third dialect alongside the Logs filters and the Tables predicate grammar. Every filter and sort is pushed into SQL. GET /api/v2/files previously read the whole scope and sorted/sliced it in JS; it now goes through a new queryWorkspaceFiles that filters, orders, and bounds the page in one query. Cursors are stamped with the sort they were minted under, so replaying one under a different sort is a 400 instead of silently duplicated or skipped rows. * fix(api): validate v2 cursor key values and compare timestamps at ms precision Two review findings, fixed at the root by making a keyset key own its cursor codec instead of hand-writing a decoder per sort. Cursor key values are caller-controlled, and matching the sort stamp and key count was not enough: an unparseable timestamp or a non-numeric size reached the query as an Invalid Date or NaN and surfaced as a 500. Each key now type- checks its own value and rejects a cursor it cannot hold, which both routes render as the documented 400. Timestamp keys now order and compare on date_trunc('milliseconds', col). Postgres keeps microseconds and defaultNow() populates them, but a cursor value round-trips through a millisecond-only JS Date — comparing the raw column against the truncated value re-admitted the page's own last row, duplicating it and stalling pagination outright at a page size of one. Reachable today via workspace_files.updated_at, which insertFileMetadata leaves to defaultNow(). * feat(api): complete the v2 workflows resource with versions and CRUD (#6184) * feat(api): complete the v2 workflows resource with versions and CRUD Adds version listing/detail plus create, update, and delete to the v2 workflows surface, which previously covered only execution and deployment. - GET /api/v2/workflows/[id]/versions — cursor-paginated, newest first - GET /api/v2/workflows/[id]/versions/[version] — version + pinned state - POST /api/v2/workflows, PATCH and DELETE /api/v2/workflows/[id] All six delegate to the existing orchestration and persistence helpers; no new domain logic. * fix(api): check folder containment before lock state; reject malformed version cursors assertFolderMutable walks a folder's ancestor chain without filtering on workspace, so inspecting it before containment let a caller tell a locked folder in someone else's workspace (423) from a nonexistent one (400). Create and update now assert containment first, matching the ordering import-workflow.ts already uses. A version cursor that decodes to JSON without a numeric version filtered every row out and returned an empty page with nextCursor null, which reads as a clean end-of-list. Malformed cursors are now a 400. * refactor(api): page workflow versions in the persistence helper listWorkflowVersions read every version row and the route filtered and sliced the result in memory, so the response was bounded but the query was not. It now takes optional limit/afterVersion, turning the cursor into a real keyset query; the route asks for limit + 1 and only trims the has-more probe. Both params are optional, so the internal, v1 admin, and copilot callers are unchanged. Also restores the untouched GET handler in [id]/route.ts to its original formatting — collapsing its signature had re-indented the whole body and buried the actual additions in whitespace churn. * feat(api): expand v2 tables with stateless multipart transfers (#6188) * feat(api): expand the public v2 tables surface Adds 16 operations so a v2 caller can do what the internal surface can: rename/move/lock a table, restore it, manage saved views, run enrichment columns, look up rows, and import/export with observable job control. Extracts lib/table/orchestration/import.ts (performTableCsvImport, performCreateTableFromCsv) and lib/table/export-stream.ts from the first-party routes, then repoints those routes at them, so v1 and v2 cannot drift on what an import or export actually does. events/stream, metadata and dispatches stay internal — they are editor state, not public API. * fix(api): make v2 table PATCH all-or-nothing and name the lock in every 423 Greptile P1: PATCH applied locks, rename and move as three sequential transactions, so a folder rejected mid-request left the earlier writes persisted while the response reported failure — and the schema-changed signal was skipped, leaving open clients on stale state. Every rejectable condition now runs before the first write, and the signal fires whenever anything did land. Cursor: v2TableLockError dropped the lock kind, so async import, column run, enrichment and table mutations returned a bare LOCKED. A table has four independent locks, so the caller could not tell which to clear. * fix(api): report the lock kind on classified 423s too, not just thrown ones The previous commit named the lock only where the rejection was thrown and caught at the route boundary. Where it instead arrives as a classified `errorCode: 'locked'` outcome — delete table, delete row, update column, and the table mutations — the kind was dropped, so those 423s stayed unactionable while their neighbours improved. The orchestration results now carry `lock`, and a shared `v2TableOrchestrationError` renders both arrival paths into the same `{ code, message, details: { lock } }` body. `details` is omitted rather than sent null when the kind is unknown, so a caller branching on it sees absence instead of a phantom value. * fix(api): make async table imports observable, not just startable `POST /import-async` pointed callers at `GET /api/v2/tables/jobs` to track progress, but that endpoint filters to `type = 'export'` — imports are derived onto the table itself, one write job at a time, and exports get a separate list precisely because they are excluded from that derivation. The public Table shape omitted those derived fields, so an async import could be started and cancelled but never observed to completion, failure, or progress. That is the gap the import/export/job-control set was meant to close. Table now carries `job` — id, type, status, rowsProcessed, error, or null when idle — and the import-async docs point at the table rather than the export list. * feat(api): make v2 table PATCH state which operations landed on failure Greptile held the PR at 4/5 on the residual non-atomicity and named two acceptable resolutions: make PATCH atomic, or have the contract adopt and expose partial-success explicitly. Atomicity would mean threading one transaction through renameTable, moveTableToFolder and updateTableLocks — three shared service functions with four non-test callers including the first-party route and two copilot tools — and deferring their per-operation audits to commit time. That is a refactor of shared write paths well outside this PR. So the contract states it instead. Every rejectable condition is already pre-validated, so a failure here is a genuine fault; when one follows a successful operation the error now carries `details.applied` listing what is live. Absent when nothing applied, so its presence always means "these changes took effect despite the error". Documented on the operation. `v2ErrorForOrchestration` gained the optional `details` this needs. * fix(api): make table lock flags read-only on the public v2 surface The new PATCH /api/v2/tables/[tableId] accepted a `locks` object, gated on workspace admin plus the table-locks feature. That still lets an API key clear the guard placed there to stop it: `write` is the floor for the endpoint, and admin keys are ordinary API keys, so a lock is no longer a boundary the key cannot cross. Locks stay readable on the table resource and enforcement is unchanged (a locked verb still returns 423). Changing one is now a first-party admin action only. The v2 body is declared here rather than reusing the first-party updateTableBodySchema, which keeps its `locks` field so the UI can still toggle them. It is .strict(), so a request carrying `locks` is rejected with a 400 naming the field instead of silently succeeding without applying it. * fix(api): keep reporting applied operations when the PATCH re-read fails The composite table PATCH promises that `error.details.applied` names the operations that are live despite an error, but `applied` was scoped inside the try. A rename or move that committed and was then followed by a throw in the final re-read — or a re-read finding the table archived — returned a bare 500/404 with no details, telling the caller nothing had landed. It would then retry into a duplicate-name conflict or repeat the move. `applied` is now function-scoped so every post-write exit carries it: the 404 on a missing re-read, a thrown lock error, a classified orchestration error, and the generic 500. `v2TableLockError` gains the same `extraDetails` parameter `v2TableOrchestrationError` already had. * feat(api): add workflow group writes to the v2 tables surface v2 exposed GET /groups but none of the writes, so the public API could run an enrichment or workflow column and read its binding, but never create one. A caller could add a plain data column and trigger the machine; wiring the two together still required the UI. Adds POST/PATCH/DELETE on /api/v2/tables/[tableId]/groups. The group is the unit that fills columns — one group feeds several — so creating one creates its output columns in the same call, matching the first-party shape rather than inverting it onto the column endpoint. Four departures from the first-party body, all public-surface concerns: - group.id is optional and server-generated. The UI mints an id to render optimistically; a public caller has no such need and a client-chosen id is a collision waiting to happen. - outputColumns[].workflowGroupId is dropped from the body and stamped from the resolved group, so it cannot disagree with it. - autoRun defaults to false. First-party defaults true so a UI add fills cells immediately; here it would make one POST fan out a metered run across every existing row. - A group naming neither a workflowId (type manual) nor an enrichmentId (type enrichment) is a 400 rather than a half-specified group the route has to guess about. Also rejects an outputColumns entry no group output feeds — the two arrays are joined by column name, and the first-party client builds both from one picker so it cannot desync, but a public caller can. Workspace containment on workflowId is asserted before it is persisted, on create and on any update that re-points the group; without it a table becomes a way to invoke workflows the key cannot otherwise reach. * improvement(api): make v2 table import and export async-only Drops the three synchronous entry points: POST /tables/[tableId]/import, POST /tables/import-csv, and GET /tables/[tableId]/export. Sync import tied a write to the lifetime of an HTTP request. The body *was* the data, so it carried a 10 MB cap that Next silently truncates past — a partial import reporting success. It also had no job, so a timeout mid-write left rows in place with nothing to poll and nothing to cancel. The async path reads the file from storage instead: upload via POST /api/v2/files for a key, start with POST /import-async, watch GET /tables/[tableId] -> job, stop with POST /job/cancel. Sync export carried no such hazard, but one shape per operation beats two: with both removed the surface has exactly one way to move a table in or out, and the CLI wraps the extra calls. This also removes the last multipart handling in v2 tables. Those were the only routes bypassing parseRequest — form fields were parsed by hand against separate form schemas, outside the contract system every other v2 write goes through. Create-a-table-from-CSV is now two calls: POST /tables, then /import-async with createColumns. csvImportModeSchema is append|replace, so there is no single-call create. Route baseline 1064 -> 1061. * docs(api): correct the import-async note about upload size limits The docstring claimed there is no synchronous upload endpoint and so no request-body size cliff. Both are wrong: POST /api/v2/files is a synchronous multipart upload with a 100 MB cap, and it is the only v2 upload path (presigned is deliberately absent). What async-only actually bought: the cap went 10 MB -> 100 MB, it fails on an explicit size check and a bounded body read rather than a proxy cap that silently truncates, authorization completes before any body is buffered, and the table write is a job that can be watched and cancelled. * feat(api): unify file and table transfers * improvement(api): make multipart transfers stateless * fix(api): make table import completion retries idempotent * feat(v2-tables): paginate the table list `GET /api/v2/tables` returned every table in the workspace in one response — it used the cursor envelope but hardcoded `nextCursor: null`, and had no `limit`. That was defensible when tables were only created through the UI; `POST /api/v2/tables` is public now, so a script can create them in bulk and the list has no way to ask for less. Adds `queryTables` alongside `listTables` rather than changing it, so the internal callers that genuinely want the whole scope are untouched — the same split `queryWorkspaceFiles` / `listWorkspaceFiles` already uses. Filter, order and slice all run in the query, so a `search` never costs a full-workspace read. A cursor whose values don't bind raises a validation error instead of being coerced to "no filter", which would have silently served page 1 under a resumed cursor. The keyset closes on `id` so a page boundary inside a run of equal names or timestamps stays stable. The shared `LimitQuery` doc component said "Maximum rows to return"; it now serves the table list too, so the wording is resource-neutral. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(api): add multipart knowledge document uploads * fix(api): keep usage admission at knowledge upload session creation * feat(knowledge): wire knowledge base uploads to multipart sessions * fix(knowledge): refuse to abort an upload once a document is bound * fix(uploads): prevent multipart cleanup races * Unify file creation and signed upload sessions (#6264) * feat(uploads): unify signed upload sessions * fix(uploads): preserve attachment storage semantics * feat(files): add authored file creation * fix(uploads): omit hoisted S3 metadata headers * feat(api): add file metadata endpoint * improvement(api): scope folders to resource paths (#6284) * improvement(api): scope folders to resource paths * fix(files): serialize folder resolution with uploads * fix(files): release folder lock before upload setup * fix(api): normalize folder paths and unblock resource mutations * fix(api): make resource cleanup and metadata consistent * improvement(uploads): persist multipart sessions in postgres * fix(db): store table row trigger timestamps in UTC * improvement(api): default folder deletion to non-recursive * fix(billing): unify chat usage source * improvement(logs): expose trace spans on log detail * fix(logs): parse list trace spans * improvement(api): replace workflow jobs with execution resources (#6294) * improvement(api): replace workflow jobs with execution resources * fix(api): preserve legacy jobs while preferring v2 executions * fix(api): make execution polling resume-aware * fix(ui): hide async examples for public workflows * fix(api): bridge resume queue visibility lag * feat(api): add v2 workflow resume endpoint * fix(api): project pending resume attempts * fix(api): prefer terminal logs over stale resumes * improvement(api): unify v2 resource query layers (#6319) * improvement(api): unify v2 resource query layers * fix(api): address v2 review findings * fix(api): preserve cancelled queue status * fix(api): guard cancelled job transitions * fix(api): close v2 resume and log gaps * feat(api): rename v2 executions to runs * feat(api): split credentials and secrets * feat(api): add workspace metadata and email attribution * improvement(api): consolidate public v2 route handling * improvement(files): centralize operations across APIs and Copilot (#6392) * improvement(files): unify rename authorization * chore(skills): add file operation migration guide * improvement(files): consolidate file operation authorization * improvement(files): extract shared operation foundation * improvement(api): simplify internal route declarations * improvement(files): centralize application authorization * refactor(api): share workspace file name validation * refactor(files): centralize copilot application calls * docs(skills): generalize application operation migration * improvement(api): centralize remaining v2 resource operations (#6412) * improvement(api): centralize v2 resource operations * fix(api): preserve custom tool conflict errors * improvement(api): migrate policy-sensitive v2 reads (#6410) * improvement(workflows): centralize v2 application operations (#6411) * refactor(api): migrate v2 knowledge operations (#6413) * refactor(api): migrate v2 knowledge operations * fix(knowledge): fail upload completion on dispatch errors * fix(knowledge): preserve upload retry and VFS errors * improvement(tables): centralize v2 application operations (#6414) * improvement(tables): centralize v2 application operations * fix(tables): preserve run validation and signals * feat(auth): add scoped internal executor delegation (#6459) * feat(auth): add scoped internal executor delegation * fix(auth): derive delegation lifetime from one timestamp * Include share status in file metadata * feat(auth): centralize delegated identity policy (#6462) * improvement(copilot): consolidate application adapters (#6450) * improvement(api): harden application route boundaries (#6451) * improvement(api): harden application route boundaries * fix(folders): reject creates at workspace cap * fix(knowledge): enforce trusted workspace scope (#6452) * fix(knowledge): enforce trusted workspace scope * refactor(knowledge): declare v2 body lifecycle * finish knowledge application migration * refactor(knowledge): compose copilot batch commands * fix(knowledge): parse connector query flags * fix(knowledge): finalize partial batch effects * fix(knowledge): align merged application boundaries * fix(knowledge): close application boundary review gaps * style(knowledge): satisfy branch biome checks * fix(knowledge): page connector documents in editor * refactor: enforce Copilot table application boundary (#6453) * refactor: enforce copilot table application boundary * fix(tables): finish application boundary migration * fix(tables): restore scoped copilot imports * fix(tables): compose copilot commands atomically * fix(tables): preserve workflow group scheduling * fix(tables): complete fixed copilot composition * fix(tables): reject enrichment output mutation * fix(tables): complete authorized application boundary * fix(workflows): migrate Copilot application boundary (#6455) * fix(workflows): migrate Copilot application boundary * fix(workflows): finish delegated application migration * fix(workflows): encode VFS folder aliases * fix(workflows): close application composition gaps * fix(workflows): preserve VFS validation errors * fix(workflows): complete application boundary migration * test(workflows): format canonical binding coverage * fix(workflows): scope executor metadata reads * fix(workflows): bind executor metadata targets * improvement(skills): align application operation guidance (#6532) * feat(api): expose v2 resource owners * fix(api): distinguish visible resource authorization failures (#6537) * feat(api): generate v2 OpenAPI from contracts (#6509) * feat(api): generate v2 OpenAPI from contracts * fix(api): preserve string boolean wire defaults * fix(api): document file download headers * fix(docs): use TypeScript CLI with Next.js * fix(docs): avoid client-rendered theme script * fix(api): document departed audit default * feat(api): replace legacy core docs with v2 * feat(api): generate v2 OpenAPI from contracts * feat(api): refine generated v2 OpenAPI docs * fix(docs): align localized v2 execution examples * fix(ci): restore Helm diff and sync audit mock * fix CI regressions after staging merge --------- Co-authored-by: Waleed <walif6@gmail.com> Co-authored-by: Theodore Li <theodoreqili@gmail.com> Co-authored-by: Siddharth Ganesan <33737564+Sg312@users.noreply.github.com> Co-authored-by: Theodore Li <theo@sim.ai> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-08-11 05:45:25 -04:00
Vikhyath MondretiandSiddharth Ganesan	117fe3137b	feat(code): cli sandboxes, enterprise timeouts, secrets projections, resolver lift, workflow exec cancellations (#6247) ... * feat(code): cli sandboxes, enterprise timeouts, secrets projections, resolver lift * fix(execution): harden compatibility and secret diagnostics * fix(execution): harden generated JavaScript literals * fix(execution): align timeout cleanup semantics * fix(tables): decouple stale job cleanup * fix(execution): drain stale workflow backlog * test(sandbox): make deadline assertions timing-safe * fix(execution): lock cleanup candidate batches * fix(execution): preserve cleanup failure metrics * cancel route fixes * separate out mship template and func template * fix * fix(execution): harden secret projection and block runs * fix(workflow): validate draft execution state * run from block ui disabling * feat(copilot): expose Sim sandboxes to mothership * feat(copilot): expose sandbox capability catalog in VFS * Updates * fix legacy logs showing up * fix(copilot): keep sandbox config visible * fix model provenance issues * fix lint' * more lint * more * test(files): align provenance copy query order * consolidate migrations, rollout compat * integration projections * update skills * fix * add provenance linters * fix: address review and compatibility regressions * fix: make tool boundary audit Bun 1.3 compatible --------- Co-authored-by: Siddharth Ganesan <siddharthganesan@gmail.com>	2026-08-05 19:22:04 -07:00
WaleedandWaleed Latif	a042f0bd0f	chore(deps): fix OTel version split, drop dead deps, declare emcn peers (#5994) ... * chore(deps): fix OTel version split, drop dead deps, declare emcn peers - Pin @opentelemetry/{resources,sdk-metrics,sdk-trace-base,sdk-trace-node} to exact 2.7.1 so they match sdk-node's pins instead of floating to 2.8.0. The carets meant app code built spans with 2.8.0 and passed them into NodeSDK from 2.7.1, which only worked by duck-typing. - Declare the 13 packages @sim/emcn imports but never declared, as peers mirrored into devDeps. @radix-ui/react-dismissable-layer had no declaration anywhere in the repo and resolved only transitively. - Remove ffmpeg-static: its binary downloads via postinstall, but it is not in trustedDependencies and Docker installs with --ignore-scripts, so the accessSync branch never succeeded and both call sites always fell through to system ffmpeg. - Remove critters + experimental.optimizeCss: Next only loads critters from the Pages Router renderer, and apps/sim is App Router only. - Make simstudio-ts-sdk zero-dependency by dropping node-fetch for native fetch; engines >=18. - Remove unused @vercel/og and postgres from docs, dotenv/inquirer/listr2 from the CLI, and yaml from the root. - Move @aws-sdk/client-appconfig from the root to apps/sim, its only consumer. - Delete the apps/sim overrides block; Bun only honors top-level overrides. - Bump free-email-domains 1.2.25 -> 1.9.70 (4,779 -> 13,059 domains). - Validate object/array variables with JSON.parse instead of JSON5, matching what the executor actually parses. - Swap the changelog GitHub icon off lucide to GithubOutlineIcon, matching the navbar chip on the same page. - Unify @types/node on 24.2.1 and lucide-react on ^0.511.0; bump chalk to 5 and image-size to 2. * fix(deps): complete the OTel pin, restore SDK error detail, revert email list Follow-ups from an independent audit of the previous commit. - Pin @opentelemetry/sdk-node and the three otlp-http exporters to exact 0.217.0. Pinning only their four dependents was self-reversing: sdk-node 0.219.0 requires core 2.8.0 exactly, so the next update would have silently rebuilt the split this PR removes. - Declare @opentelemetry/core (2.7.1). It is imported by lib/copilot/request/go/propagation.ts but resolved only by hoisting, and it is the OTel package with the most version churn in the tree. - Pin @radix-ui/react-dismissable-layer to exact 1.1.13 in @sim/emcn. All five transitive parents pin it exactly; a caret would fork a second copy on 1.1.14, which is the duplicate-context bug the declaration prevents. - Surface error.cause in simstudio-ts-sdk. Native fetch reports network failures as a bare "fetch failed" and puts the reason on cause, so every DNS/TLS/refused error was reaching callers with no diagnostic content. - Revert free-email-domains to 1.2.25. Upstream now merges the free-domain list with two disposable-email blocklists, so 1.9.70 classifies real organization domains as free — UK charities, some companies and universities, and the JP/KR ISP domains APAC SMBs use for business mail. The demo form blocks submission on that check, so a false positive costs the booking entirely. Worth doing deliberately, not inside a deps change. - Lower packages/cli engines to >=18. chalk 5 and commander 11 both accept >=16 and the source uses no Node 20 API, so >=20 only produced EBADENGINE for Node 18 users. --------- Co-authored-by: Waleed Latif <waleed@simstudio.ai>	2026-07-27 19:16:32 -07:00
Waleed	d078c84ee6	chore(typescript): upgrade to TypeScript 7 (native Go compiler) (#5521) ... * chore(typescript): upgrade to TypeScript 7 (native Go compiler) Bumps typescript to ^7.0.2 across every workspace package. Full bun run type-check/lint/build/test all pass; apps/sim's type-check (the one needing an 8GB heap bump) drops from ~55s to ~7s wall time. Migration fixes required by TS7's stricter defaults: - baseUrl removed: drop it from 5 tsconfigs (paths already resolved relative to tsconfig dir, so behavior is unchanged) and prefix the one bare (non-relative) paths entry each in apps/sim and apps/realtime with './' - moduleResolution=node10 removed: switch packages/cli and packages/ts-sdk to "bundler", matching the rest of the monorepo - types now defaults to [] instead of auto-including every @types/* package: add "types": ["node"] to the shared base tsconfig (this is fundamentally a Node monorepo, so this restores prior behavior in one place instead of duplicating it per-package), add explicit @types/node deps to packages that now rely on it transitively via @sim/db/@sim/logger, and add "declare module '*.css'" to the two packages with plain (non-module) CSS side-effect imports that TS7's stricter checker now flags - packages/logger's isomorphic `typeof window` check no longer needs DOM lib in every consumer: replaced with `'window' in globalThis` - packages/testing and apps/realtime's fetch/DOM mocks need DOM lib where they're compiled, since they model the browser Fetch API - the `typescript` npm package no longer exports the classic Compiler API from its main entry (moved to unstable/ast subpaths); apps/sim's Function-block route used it at runtime to strip import statements from user code, so that one call site now uses Microsoft's official transition package, @typescript/typescript6 - Next.js 16.2.6's own TypeScript-detection heuristic hardcodes a path TS7 no longer ships, and its auto-install fallback assumes npm/pnpm; added @typescript/native-preview as a devDependency to apps/sim and apps/docs so Next detects a valid native compiler instead of trying (and failing) to auto-install one Not merging yet: TS 7.0.2 published today and is still inside this repo's bunfig.toml minimumReleaseAge (7-day) supply-chain gate, so `bun install` will fail for everyone until 2026-07-15. Opening this now to get it through review; hold the actual merge until then. * fix(typescript): address Greptile review findings on TS7 upgrade - packages/logger: 'window' in globalThis treats a shim that leaves globalThis.window explicitly undefined as browser-only, silently dropping production server logs. Restore the original typeof !== 'undefined' semantics via an inline cast instead, so it stays correct without requiring DOM lib in every consumer. - packages/ts-sdk, packages/cli: both are tsc-built, published as Node ESM (package.json "type": "module" with an "exports" map). "moduleResolution": "bundler" is too permissive for that target - it accepts import patterns (e.g. extensionless relative imports) that Node's actual ESM resolver rejects at runtime. Switch both to "module"/"moduleResolution": "nodenext", the correct pairing for a published Node ESM package. Verified real tsc builds (not just --noEmit) still succeed for both. * chore(bunfig): temporarily disable minimumReleaseAge gate for TS7 install TS 7.0.2 published today, still inside the 7-day gate. Lowering to 0 to unblock this merge; will restore to 604800 in an immediate follow-up commit right after merging.	2026-07-08 16:41:23 -07:00
Waleed	a7b0bd311d	fix(deps): upgrade vitest to ^4.1.0 to patch critical Vitest UI advisory (GHSA-5xrq-8626-4rwp) (#4837) ... * fix(deps): upgrade vitest to ^4.1.0 to patch critical Vitest UI advisory (GHSA-5xrq-8626-4rwp) - Bump vitest and @vitest/coverage-v8 to ^4.1.0 across all workspaces (only patched release for the critical 'Vitest UI server arbitrary file read/execute' advisory; no 3.x backport exists) - Widen @sim/testing peer range to ^3.0.0 || ^4.0.0 - Migrate constructor mocks to class expressions: vitest 4 uses Reflect.construct for mocks invoked with new, and arrow/function implementations are not constructable (function expressions also get reverted to arrows by biome's useArrowFunction) - Remove deprecated test.poolOptions from apps/sim/vitest.config.ts (options are now top-level in vitest 4) * fix(deps): exclude vulnerable vitest 4.0.x from @sim/testing peer range Tighten the v4 arm of the peer range to >=4.1.0 <5.0.0 so the peer requirement cannot be satisfied by the unpatched 4.0.x builds that GHSA-5xrq-8626-4rwp affects. * fix(testing): make vitest 4 constructor mocks type-check cleanly - logging-session & mcp-oauth mocks: a class passed to mockImplementation has a construct signature that isn't assignable to its (...args) => any parameter, failing tsc. Use named function declarations instead (constructable via Reflect.construct, assignable to mockImplementation, and not rewritten to arrows by biome's useArrowFunction). - database.mock.ts: vitest 4's generic vi.fn typings no longer break the self-referential cycle on the transaction callback's tx param; loosen tx and annotate the callback's return type to resolve the implicit-any errors. * test(isolated-vm): de-flake queue-capacity scheduler tests The 'queue is full' and 'per-owner queued limit' tests relied on 'await sleep(1)' to assume the first request had reached the queue before submitting the overflow request. The first request only enqueues after an async spawn-failure chain (acquireWorker -> spawn exit -> resolve null -> enqueue), which isn't guaranteed within 1ms under CI load — the overflow request then found an empty queue and hit the 200ms queue-wait timeout instead of the capacity rejection. Replace the wall-clock barrier with a deterministic, event-driven one: hold the single global concurrency slot (IVM_MAX_CONCURRENT=1) with an active worker and await an explicit 'dispatched' signal (fired when the worker receives its execute message, after the scheduler counts it active). The follow-up requests then deterministically hit the synchronous enqueue path. Also drops the queue-wait timeout from 200ms to 50ms, so the tests run faster.	2026-06-01 16:11:35 -07:00
Waleed	8d7bbbc670	chore(utils): migrate to shared random/ID utilities and add enforcement linting (#4623) ... * chore(utils): migrate to shared random/ID utilities and add enforcement linting - Replace all Math.random(), crypto.randomUUID(), crypto.randomBytes(), nanoid, and uuid usages with shared @sim/utils/random and @sim/utils/id helpers across 72 files - Add new @sim/utils exports: deepClone, omit, filterUndefined (object), truncate (string), backoffWithJitter, parseRetryAfter (retry), getErrorMessage (errors) - Sweep all getErrorMessage, sleep, deepClone callsites across 500+ files to use shared utilities - Add Biome noRestrictedImports rule to catch nanoid, uuid, and crypto named imports at lint time - Add scripts/check-utils-enforcement.ts to catch Math.random and crypto.* global property access - Add check:utils script to package.json * chore(utils): replace deepClone wrapper with structuredClone built-in deepClone() was a one-line wrapper around structuredClone(), which is universally available in Node 17+ and all modern browsers. Removing the abstraction reduces indirection and means contributors don't need to learn a project-specific name for a well-known built-in. - Remove deepClone from packages/utils/src/object.ts and index.ts - Replace all 17 call sites with structuredClone() directly - Update check:utils script suggestion text - Update CLAUDE.md and global.md docs * fix(utils): add missing biome noRestrictedImports rule and correct truncate docs - Add noRestrictedImports to biome.json under style — bans nanoid and uuid package imports at lint time (crypto.randomUUID/randomBytes are caught by the check:utils grep script which handles global property access) - Correct truncate() TSDoc and parameter name: sliceLength makes it clear that total output length is sliceLength + suffix.length, matching the behavior all callers were already written to expect * fix(utils): add missing getErrorMessage imports at 4 call sites The sweep agents added getErrorMessage calls without the corresponding import in 4 files, causing test failures. Added the missing imports. * fix(utils): fix build errors from getErrorMessage sweep and retry.ts Turbopack issue - Fix retry.ts cross-file import: Turbopack cannot resolve './random.js' for internal package imports; inline the jitter crypto call directly - Add missing getErrorMessage imports to 32 files where the sweep added calls without the corresponding import (caught by type-check and test runs) - Remove accidental getErrorMessage import from crowdstrike/query/route.ts which has its own domain-specific getErrorMessage for parsing CrowdStrike's JSON error format - Fix use-sub-block-value.ts type error from structuredClone narrowing: add 'as T' cast at emitValue callsite (safe — valueCopy is always a structural copy of newValue) * fix(tools): use toError in crowdstrike catch block instead of local getErrorMessage The catch block was calling the local getErrorMessage function which parses CrowdStrike API JSON responses, not JavaScript Error objects. Use toError(error).message to correctly extract the message from a caught value in this context.	2026-05-15 17:31:27 -07:00
Vikhyath Mondreti	43f53bb7d3	feat(execution): payload size bottlenecks with lazy execution value hydration, safer materialization, and batched parallel execution (#4560) ... * improvement(resolver): lazy resolution for underlying fields greater than 10MB * progress * feat(parallel): batching * codegen to allow inline substitution * address comments * ui inconsistencies * cleanup redundant code * address more comments * address comments * replace helper * fix tests	2026-05-12 14:57:32 -07:00
Vikhyath Mondreti	b8959eb20d	improvement(repo): zod based client-server boundary (#4355) ... * improvement(repo): centralized zod contracts (#4336) * improvement(repo): zod schema contracts * type checks * fix(notion): correctly register tool (#4337) * fix func blokc * more improvements * fix tests * type check * remove v3 refs * minor type improvements * address comments * update jira contract * remove validateJsonBody * improvement(repo): consolidation of boundary helpers + better unknown usage (#4352) * improvement(repo): consolidation of boundary helpers + better unknown usage * address comments * improve file transfer error messaging * fix docs listing schema drift * fix inocrrect type casting * address council comments * remove prefix	2026-04-30 12:16:28 -07:00
9 people	5b9f0d73c2	feat(mothership): mothership (#3411) ... * Fix lint * improvement(sidebar): loading * fix(sidebar): use client-generated UUIDs for stable optimistic updates (#3439) * fix(sidebar): use client-generated UUIDs for stable optimistic updates * fix(folders): use zod schema validation for folder create API Replace inline UUID regex with zod schema validation for consistency with other API routes. Update test expectations accordingly. * fix(sidebar): add client UUID to single workflow duplicate hook The useDuplicateWorkflow hook was missing newId: crypto.randomUUID(), causing the same temp-ID-swap issue for single workflow duplication from the context menu. * fix(folders): avoid unnecessary Set re-creation in replaceOptimisticEntry Only create new expandedFolders/selectedFolders Sets when tempId differs from data.id. In the common happy path (client-generated UUIDs), this avoids unnecessary Zustand state reference changes and re-renders. * Mothership block logs * Fix mothership block logs * improvement(knowledge): make connector-synced document chunks readonly (#3440) * improvement(knowledge): make connector-synced document chunks readonly * fix(knowledge): enforce connector chunk readonly on server side * fix(knowledge): disable toggle and delete actions for connector-synced chunks * Job exeuction logs * Job logs * fix(connectors): remove unverifiable requiredScopes for Linear connector * fix(connectors): remove legacy requiredScopes from Jira and Confluence connectors Jira and Confluence OAuth tokens don't return legacy scope names like read:jira-work or read:confluence-content.all, causing the 'Update access' banner to always appear. Set requiredScopes to empty array like Linear. * feat(tasks): add rename to task context menu (#3442) * Revert "fix(connectors): remove legacy requiredScopes from Jira and Confluence connectors" This reverts commit a0be3ff414. * fix(connectors): restore Linear connector requiredScopes Linear OAuth does return scopes in the token response. The previous fix of emptying requiredScopes was based on an incorrect assumption. Restoring requiredScopes: ['read'] as it should work correctly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(knowledge): pass workspaceId to useOAuthCredentials in connector card The ConnectorCard was calling useOAuthCredentials(providerId) without a workspaceId, causing the credentials API to return an empty array. This meant the credential lookup always failed, getMissingRequiredScopes received undefined, and the "Update access" banner always appeared. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix oauth link callback from mothership task * feat(connectors): add Fireflies connector and API key auth support (#3448) * feat(connectors): add Fireflies connector and API key auth support Extend the connector system to support both OAuth and API key authentication via a discriminated union (`ConnectorAuthConfig`). Add Fireflies as the first API key connector, syncing meeting transcripts via the Fireflies GraphQL API. Schema changes: - Make `credentialId` nullable (null for API key connectors) - Add `encryptedApiKey` column (AES-256-GCM encrypted, null for OAuth) This eliminates the `'_apikey_'` sentinel and inline `sourceConfig._encryptedApiKey` patterns, giving each auth mode its own clean column. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(fireflies): allow 0 for maxTranscripts (means unlimited) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * Add context * fix(fireflies): correct types from live API validation (#3450) * fix(fireflies): correct types from live API validation - speakers.id is number, not string (API returns 0, 1, 2...) - summary.action_items is a single string, not string[] - Update formatTranscriptContent to handle action_items as string Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(fireflies): correct tool types from live API validation - FirefliesSpeaker.id: string -> number - FirefliesSentence.speaker_id: string -> number - FirefliesSpeakerAnalytics.speaker_id: string -> number - FirefliesSummary.action_items: string[] -> string - FirefliesSummary.outline: string[] -> string - FirefliesSummary.shorthand_bullet: string[] -> string - FirefliesSummary.bullet_gist: string[] -> string - FirefliesSummary.topics_discussed: string[] -> string Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * feat(knowledge): add connector tools and expand document metadata (#3452) * feat(knowledge): add connector tools and expand document metadata * fix(knowledge): address PR review feedback on new tools * fix(knowledge): remove unused params from get_document transform * refactor, improvement * fix: correct knowledge block canonical pair pattern and subblock migration - Rename manualDocumentId to documentId (advanced subblock ID should match canonicalParamId, consistent with airtable/gmail patterns) - Fix documentSelector.dependsOn to reference knowledgeBaseSelector (basic depends on basic, not advanced) - Remove unnecessary documentId migration (ID unchanged from main) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * lint * fix: resolve post-merge test and lint failures - airtable: sync tableSelector condition with tableId (add getSchema) - backfillCanonicalModes test: add documentId mode to prevent false backfill - schedule PUT test: use invalid action string now that disable is valid - schedule execute tests: add ne mock, sourceType field, use mockReturnValueOnce for two db.update calls - knowledge tools: fix biome formatting (single-line arrow functions) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fixes * Fixes * Clean vfs * Fix * Fix lint * fix(connectors): add rate limiting, concurrency controls, and bug fixes (#3457) * fix(connectors): add rate limiting, concurrency controls, and bug fixes across knowledge connectors - Add Retry-After header support to fetchWithRetry for all 18 connectors - Batch concurrent API calls (concurrency 5) in Dropbox, Google Docs, Google Drive, OneDrive, SharePoint - Batch concurrent API calls (concurrency 3) in Notion to match 3 req/s limit - Cache GitHub tree in syncContext to avoid re-fetching on every pagination page - Batch GitHub blob fetches with concurrency 5 - Fix GitHub base64 decoding: atob() → Buffer.from() for UTF-8 safety - Fix HubSpot OAuth scope: 'tickets' → 'crm.objects.tickets.read' (v3 API) - Fix HubSpot syncContext key: totalFetched → totalDocsFetched for consistency - Add jitter to nextSyncAt (10% of interval, capped at 5min) to prevent thundering herd - Fix Date consistency in connector DELETE route * fix(connectors): address PR review feedback on retry and SharePoint batching - Remove 120s cap on Retry-After — pass all values through to retry loop - Add maxDelayMs guard: if Retry-After exceeds maxDelayMs, throw immediately instead of hammering with shorter intervals (addresses validate timeout concern) - Add early exit in SharePoint batch loop when maxFiles limit is reached to avoid unnecessary API calls * fix(connectors): cap Retry-After at maxDelayMs instead of aborting Match Google Cloud SDK behavior: when Retry-After exceeds maxDelayMs, cap the wait to maxDelayMs and log a warning, rather than throwing immediately. This ensures retries are bounded in duration while still respecting server guidance within the configured limit. * fix(connectors): add early-exit guard to Dropbox, Google Docs, OneDrive batch loops Match the SharePoint fix — skip remaining batches once maxFiles limit is reached to avoid unnecessary API calls. * improvement(turbo): align turborepo config with best practices (#3458) * improvement(turbo): align turborepo config with best practices * fix(turbo): address PR review feedback * fix(turbo): add lint:check task for read-only lint+format CI checks lint:check previously delegated to format:check which only checked formatting. Now it runs biome check (no --write) which enforces both lint rules and formatting without mutating files. * upgrade turbo * improvement(perf): apply react and js performance optimizations across codebase (#3459) * improvement(perf): apply react and js performance optimizations across codebase - Parallelize independent DB queries with Promise.all in API routes - Defer PostHog and OneDollarStats via dynamic import() to reduce bundle size - Use functional setState in countdown timers to prevent stale closures - Replace O(n*m) .filter().find() with Set-based O(n) lookups in undo-redo - Use .toSorted() instead of .sort() for immutable state operations - Use lazy initializers for useState(new Set()) across 20 components - Remove useMemo wrapping trivially cheap expressions (typeof, ternary, template strings) - Add passive: true to scroll event listener * fix(perf): address PR review feedback - Extract IIFE Set patterns to named consts for readability in use-undo-redo - Hoist Set construction above loops in BATCH_UPDATE_PARENT cases - Add .catch() error handler to PostHog dynamic import - Convert session-provider posthog import to dynamic import() to complete bundle split * fix(analytics): add .catch() to onedollarstats dynamic import * improvement(resource): tables, files * improvement(resources): all outer page structure complete * refactor(queries): comprehensive TanStack Query best practices audit (#3460) * refactor: comprehensive TanStack Query best practices audit and migration - Add AbortSignal forwarding to all 41 queryFn implementations for proper request cancellation - Migrate manual fetch patterns to useMutation hooks (useResetPassword, useRedeemReferralCode, usePurchaseCredits, useImportWorkflow, useOpenBillingPortal, useAllowedMcpDomains) - Migrate standalone hooks to TanStack Query (use-next-available-slot, use-mcp-server-test, use-webhook-management, use-referral-attribution) - Fix query key factories: add missing `all` keys, replace inline keys with factory methods - Fix optimistic mutations: use onSettled instead of onSuccess for cache reconciliation - Replace overly broad cache invalidations with targeted key invalidation - Remove keepPreviousData from static-key queries where it provides no benefit - Add staleTime to queries missing explicit cache duration - Fix `any` type in UpdateSettingParams with proper GeneralSettings typing - Remove dead code: loadingWebhooks/checkedWebhooks from subblock store, unused helper functions - Update settings components (general, debug, referral-code, credit-balance, subscription, mcp) to use mutation state instead of manual useState for loading/error/success Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove unstable mutation object from useCallback deps openBillingPortal mutation object is not referentially stable, but .mutate() is stable in TanStack Query v5. Remove from deps to prevent unnecessary handleBadgeClick recreations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add missing byWorkflows invalidation to useUpdateTemplate The onSettled handler was missing the byWorkflows() invalidation that was dropped during the onSuccess→onSettled migration. Without this, the deploy modal (useTemplateByWorkflow) would show stale data after a template update. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: add TanStack Query best practices to CLAUDE.md and cursor rules Add comprehensive React Query best practices covering: - Hierarchical query key factories with intermediate plural keys - AbortSignal forwarding in all queryFn implementations - Targeted cache invalidation over broad .all invalidation - onSettled for optimistic mutation cache reconciliation - keepPreviousData only on variable-key queries - No manual fetch in components rule - Stable mutation references in useCallback deps Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review feedback - Fix syncedRef regression in use-webhook-management: only set syncedRef.current=true when webhook is found, so re-sync works after webhook creation (e.g., post-deploy) - Remove redundant detail(id) invalidation from useUpdateTemplate onSettled since onSuccess already populates cache via setQueryData Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address second round of PR review feedback - Reset syncedRef when blockId changes in use-webhook-management so component reuse with a different block syncs the new webhook - Add response.ok check in postAttribution so non-2xx responses throw and trigger TanStack Query retry logic Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: use lists() prefix invalidation in useCreateWorkspaceCredential Use workspaceCredentialKeys.lists() instead of .list(workspaceId) so filtered list queries are also invalidated on credential creation, matching the pattern used by update and delete mutations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address third round of PR review feedback - Add nullish coalescing fallback for bonusAmount in referral-code to prevent rendering "undefined" when server omits the field - Reset syncedRef when queryEnabled becomes false so webhook data re-syncs when the query is re-enabled without component remount Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address fourth round of PR review feedback - Add AbortSignal to testMcpServerConnection for consistency - Wrap handleTestConnection in try/catch for mutateAsync error handling - Replace broad subscriptionKeys.all with targeted users()/usage() invalidation - Add intermediate users() key to subscription key factory for prefix matching - Add comment documenting syncedRef null-webhook behavior - Fix api-keys.ts silent error swallowing on non-ok responses - Move deployments.ts cache invalidation from onSuccess to onSettled Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: achieve full TanStack Query best practices compliance - Add intermediate plural keys to api-keys, deployments, and schedules key factories for prefix-based invalidation support - Change copilot-keys from refetchQueries to invalidateQueries - Add signal parameter to organization.ts fetch functions (better-auth client does not support AbortSignal, documented accordingly) - Move useCreateMcpServer invalidation from onSuccess to onSettled Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * ran lint * Fix tables row count * Update mothership to match copilot in logs * improvement(resource): layout * fix(knowledge): compute KB tokenCount from documents instead of stale column (#3463) The knowledge_base.token_count column was initialized to 0 and never updated. Replace with COALESCE(SUM(document.token_count), 0) in all read queries, which already JOIN on documents with GROUP BY. * improvement(resources): layout and items * feat(knowledge): add v1 knowledge base API, Obsidian/Evernote connectors, and docs (#3465) * feat(knowledge): add v1 knowledge base API, Obsidian/Evernote connectors, and docs - Add v1 REST API for knowledge bases (CRUD, document management, vector search) - Add Obsidian and Evernote knowledge base connectors - Add file type validation to v1 file and document upload endpoints - Update OpenAPI spec with knowledge base endpoints and schemas - Add connectors documentation page - Apply query hook formatting improvements * fix(knowledge): address PR review feedback - Remove validateFileType from v1/files route (general file upload, not document-only) - Reject tag filters when searching multiple KBs (tag defs are KB-specific) - Cache tag definitions to avoid duplicate getDocumentTagDefinitions call - Fix Obsidian connector silent empty results when syncContext is undefined * improvement(connectors): add syncContext to getDocument, clean up caching - Update docs to say 20+ connectors - Add syncContext param to ConnectorConfig.getDocument interface - Use syncContext in Evernote getDocument to cache tag/notebook maps - Replace index-based cache check with Map keyed by KB ID in search route * fix(knowledge): address second round of PR review feedback - Fix Zod .default('text') overriding tag definition's actual fieldType - Fix encodeURIComponent breaking multi-level folder paths in Obsidian - Use 413 instead of 400 for file-too-large in document upload - Add knowledge-bases to API reference docs navigation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(knowledge): prevent cross-workspace KB access in search Filter accessible KBs by matching workspaceId from the request, preventing users from querying KBs in other workspaces they have access to but didn't specify. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(knowledge): audit resourceId, SSRF protection, recursion depth limit - Fix recordAudit using knowledgeBaseId instead of newDocument.id - Add SSRF validation to Obsidian connector (reject private/loopback URLs) - Add max recursion depth (20) to listVaultFiles Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(obsidian): remove SSRF check that blocks localhost usage The Obsidian connector is designed to connect to the Local REST API plugin running on localhost (127.0.0.1:27124). The SSRF check was incorrectly blocking this primary use case. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * improvement(resources): segmented API * fix(execution): ensure background tasks await post-execution DB status updates (#3466) The fire-and-forget IIFE in execution-core.ts for post-execution logging could be abandoned when trigger.dev tasks exit, leaving executions permanently stuck in "running" status. Store the promise on LoggingSession so background tasks can optionally await it before returning. * improvement(resource): sorting and icons * fix(resource): sorting * improvement(settings): fix mcp modal, add option to edit JSON and add Sim as an MCP client (#3467) * improvement(settings): fix mcp modal, add option to edit JSON and add Sim as an MCP client * added docs link in sidebar * ack comments * ack comments * fixed error msg * feat(mothership): billing (#3464) * Billing update * more billing improvements * credits UI * credit purchase safety * progress * ui improvements * fix cancel sub * fix types * fix daily refresh for teams * make max features differentiated * address bugbot comments * address greptile comments * revert isHosted * address more comments * fix org refresh bar * fix ui rounding * fix minor rounding * fix upgrade issue for legacy plans * fix formatPlanName * fix email dispay names * fix legacy team reference bugs * referral bonus in credits * fix org upgrade bug * improve logs * respect toggle for paid users * fix landing page pro features and usage limit checks * fixed query and usage * add unit test * address more comments * enterprise guard * fix limits bug * pass period start/end for overage * fix(sidebar): restore drag-and-drop for workflows and folders (#3470) * fix(sidebar): restore drag-and-drop for workflows and folders Made-with: Cursor * update docs, unrelated * improvement(tables): consolidation * feat(schedules): add schedule creator modal for standalone jobs Add modal to create standalone scheduled jobs from the Schedules page. Includes POST API endpoint, useCreateSchedule mutation hook, and full modal with schedule type selection, timezone, lifecycle, and live preview. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(schedules): add edit support with context menu for standalone jobs * style(schedules): apply linter formatting * improvement: tables, favicon * feat(files): inline file viewer with text editing (#3475) * feat(files): add inline file viewer with text editing and create file modal Add file preview/edit functionality to the workspace files page. Text files (md, json, txt, yaml, etc.) open in an editable textarea with Cmd/Ctrl+S save. PDFs render in an iframe. New file button creates empty .md files via a modal. Uses ResourceHeader breadcrumbs and ResourceOptionsBar for save/download/delete. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * improvement(files): add UX polish, PR review fixes, and context menu - Add unsaved changes guard modal (matching credentials manager pattern) - Add delete confirmation modal for both viewer and context menu - Add save status feedback (Save → Saving... → Saved) - Add right-click context menu with Open, Download, Delete actions - Add 50MB file size limit on content update API - Add storage quota check before content updates - Add response.ok guard on download to prevent corrupt files - Add skeleton loading for pending file selection (prevents flicker) - Fix updateContent in handleSave dependency array Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): propagate save errors and remove redundant sizeDiff - Remove try/catch in TextEditor.handleSave so errors propagate to parent, which correctly shows save failure status - Remove redundant inner sizeDiff declaration that shadowed outer scope Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): remove unused textareaRef Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): move Cmd+S to parent, add save error feedback, hide save for non-text files - Move Cmd+S keyboard handler from TextEditor to Files so it goes through the parent handleSave with proper status management - Add 'error' save status with red "Save failed" label that auto-resets - Only show Save button for text-editable file types (md, txt, json, etc.) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * improvement(files): add save tooltip, deduplicate text-editable extensions - Add Tooltip on Save button showing Cmd+S / Ctrl+S shortcut - Export TEXT_EDITABLE_EXTENSIONS from file-viewer and reuse in files.tsx instead of duplicating the list inline Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: extract isMacPlatform to shared utility Move isMacPlatform() from global-commands-provider.tsx to lib/core/utils/platform.ts so it can be reused by files.tsx tooltip without duplication. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(files): deduplicate delete modal, use shared formatFileSize - Extract DeleteConfirmModal component to eliminate duplicate modal markup between viewer and list modes - Replace local formatFileSize with shared utility from file-utils.ts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): fix a11y label lint error and remove mutation object from useCallback deps Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): add isDirty guard on handleSave, return proper HTTP status codes Prevents "Saving → Saved" flash when pressing Cmd+S with no changes. Returns 404 for file-not-found and 402 for quota-exceeded instead of 500. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): reset isDirty/saveStatus on delete and discard, remove deprecated navigator.platform - Clear isDirty and saveStatus when deleting the currently-viewed file to prevent spurious beforeunload prompts - Reset saveStatus on discard to prevent stale "Save failed" when opening another file - Remove deprecated navigator.platform, userAgent fallback covers all cases Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): prevent concurrent saves on rapid Cmd+S, add YAML MIME types - Add saveStatus === 'saving' guard to handleSave to prevent duplicate concurrent PUT requests from rapid keyboard shortcuts - Add yaml/yml MIME type mappings to getMimeTypeFromExtension Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(files): reuse shared extension constants, parallelize cancelQueries - Replace hand-rolled SUPPORTED_EXTENSIONS with composition from existing SUPPORTED_DOCUMENT/AUDIO/VIDEO_EXTENSIONS in validation.ts - Parallelize sequential cancelQueries calls in delete mutation onMutate Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): guard handleCreate against duplicate calls while pending Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): show upload progress on the Upload button, not New file Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(files): use ref-based guard for create pending state to avoid stale closure The uploadFile.isPending check was stale because the mutation object is excluded from useCallback deps (per codebase convention). Using a ref ensures the guard works correctly across rapid Enter key presses. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * cleanup(files): use shared icon import, remove no-op props, wrap handler in useCallback Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * improvement: tables, dropdown * improvement(docs): align sidebar method badges and polish API reference styling (#3484) * improvement(docs): align sidebar method badges and polish API reference styling * fix(docs): revert className prop on DocsPage for CI compatibility * fix(docs): restore oneOf schema for delete rows and use rem units in CSS * fix(docs): replace :has() selectors with direct className for reliable prod layout The API docs layout was intermittently narrow in production because CSS :has(.api-page-header) selectors are unreliable in Tailwind v4 production builds. Apply className="openapi-page" directly to DocsPage and replace all 64 :has() selectors with .openapi-page class targeting. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(docs): bypass TypeScript check for className prop on DocsPage Use spread with type assertion to pass className to DocsPage, working around a CI type resolution issue where the prop exists at runtime but is not recognized by TypeScript in the Vercel build environment. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(docs): use inline style tag for grid layout, revert CSS to :has() selectors The className prop on DocsPage doesn't exist in the fumadocs-ui version resolved on Vercel, so .openapi-page was never applied and all 64 CSS rules broke. Revert to :has(.api-page-header) selectors for styling and use an inline <style> tag for the critical grid-column layout override, which is SSR'd and doesn't depend on any CSS selector matching. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(docs): add pill styling to footer navigation method badges The footer nav badges (POST, GET, etc.) had color from data-method rules but lacked the structural pill styling (padding, border-radius, font-size). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * fix(docs): use named grid lines instead of numeric column indices (#3487) Root cause: the fumadocs grid template has 3 columns in production but 5 columns in local dev. Our CSS used `grid-column: 3 / span 2` which targeted the wrong column in the 3-column grid, placing content in the near-zero-width TOC column instead of the main content column. Fix: use `grid-column: main-start / toc-end` which uses CSS named grid lines from grid-template-areas, working regardless of column count. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * improvement(resource): layout * improvement: icon, resource header options * improvement: icons * fix(files): icon * feat(tables): column operations, row ordering, V1 API (#3488) * feat(tables): add column operations, row ordering, V1 columns API, and OpenAPI spec Adds column rename/delete/type change/constraint updates to the tables module, row ordering via position column, UI metadata schema, V1 public API for column operations with rate limiting and audit logging, and OpenAPI documentation. Key changes: - Service-layer column operations with validation (name pattern, type compatibility, unique/required constraints) - Position column on user_table_rows with composite index for efficient ordering - V1 /api/v1/tables/{tableId}/columns endpoint (POST/PATCH/DELETE) with rate limiting and audit - Shared Zod schemas extracted to table/utils.ts using COLUMN_TYPES constant - Targeted React Query invalidation (row vs schema mutations) with consistent onSettled usage - OpenAPI 3.1.0 spec for columns endpoint with code samples - Position field added to all row response mappings for consistency - Sort fallback to position ordering when buildSortClause returns null Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(tables): use specific error prefixes instead of broad "Cannot" match Prevents internal TypeErrors (e.g. "Cannot read properties of undefined") from leaking as 400 responses. Now matches only domain-specific errors: "Cannot delete the last column" and "Cannot set column". Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(tables): reject Infinity and NaN in number type compatibility check Number.isFinite rejects Infinity, -Infinity, and NaN, preventing non-finite values from passing column type validation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(tables): invalidate table list on row create/delete for stale rowCount Row create and delete mutations now invalidate the table list cache since it includes a computed rowCount. Row updates (which don't change count) continue to only invalidate row queries. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(tables): add column name length check, deduplicate name gen, reset pagination on clear - Add MAX_COLUMN_NAME_LENGTH validation to addTableColumn (was missing, renameColumn already had it) - Extract generateColumnName helper to eliminate triplicated logic across handleAddColumn, handleInsertColumnLeft, handleInsertColumnRight - Reset pagination to page 0 when clearing sort/filter to prevent showing empty pages after narrowing filters are removed Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: hoist tableId above try block in V1 columns route, add detail invalidation to invalidateRowCount - V1 columns route: `tableId` was declared inside `try` but referenced in `catch` logger.error, causing undefined in error logs. Hoisted `await params` above try in all three handlers (POST, PATCH, DELETE). - invalidateRowCount: added `tableKeys.detail(tableId)` invalidation since the single-table GET response includes `rowCount`, which becomes stale after row create/delete without this. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add position to all row mutation responses, remove dead filter code - Add `position` field to POST (single + batch) and PATCH row responses across both internal and V1 routes, matching GET responses and OpenAPI spec. - Remove unused `filterConfig`, `handleFilterToggle`, `handleFilterClear`, and `activeFilters` — dead code left over from merge conflict resolution. `handleFilterApply` (the one actually wired to JSX) is preserved. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: invalidateTableSchema now also invalidates table list cache Column add/rename/delete/update mutations now invalidate tableKeys.list() since the list endpoint returns schema.columns for each table. Without this, the sidebar table list would show stale column schemas until staleTime expires. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: replace window.prompt/confirm with emcn Modal dialogs Replace non-standard browser dialogs with proper emcn Modal components to match the existing codebase pattern (e.g. delete table confirmation). - Column rename: Modal with Input field + Enter key support - Column delete: Modal with destructive confirmation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * update schedule creation ui and run lint * improvement: logs * improvement(tables): multi-select and efficiencies * Table tools * improvement(folder-selection): folder deselection + selection order should match visual * fix(selections): more nested folder inaccuracies * Tool updates * Store tool call results * fix(landing): wire agent input to mothership * feat(mothership): resource viewer * fix tests * fix(streaming): smoother streaming with throttled rendering, ResizeObserver scroll, and batched updates (#3471) * fix(streaming): smoother streaming with throttled rendering, ResizeObserver scroll, and batched updates - Add useThrottledValue hook (100ms trailing-edge throttle) to gate DOM re-renders during streaming across all chat surfaces - Replace 100ms setInterval scroll polling with ResizeObserver-based auto-scroll, programmatic scroll timestamp tracking, and nested [data-scrollable] region handling - Extract processContentBuffer from inline content handler for cleaner code organization in copilot SSE handlers - Add RAF-based update batching (50ms max interval) to floating chat and home chat streaming paths - Add useProgressiveList hook for progressive rendering of long conversation histories via requestAnimationFrame Made-with: Cursor * ack PR comments * fix search modal * more comments * ack comments * count * ack comments * ack comment * improvement(mothership): worklfow resource * Fix tool call persistence in chat * Tool results * Fix error status * File uploads to mothership * feat(templates): landing page templates workflow states * improvement(mothership): chat stability * improvement(mothership): chat history and stability * improvement(tables): click-to-select navigation, inline rename, column resize (#3496) * improvement(tables): click-to-select navigation, inline rename, column resize * fix(tables): address PR review comments - Add doneRef guard to useInlineRename preventing Enter+blur double-fire - Fix PATCH error handler: return 500 for non-validation errors, fix unreachable logger.error - Stop click propagation on breadcrumb rename input * fix(tables): add rows-affected check in renameTable service Prevents silent no-op when tableId doesn't match any record. * fix(tables): useMemo deps + placeholder memo initialCharacter check - Use primitive editingId/editValue in useMemo deps instead of whole useInlineRename object (which creates a new ref every render) - Add initialCharacter comparison to placeholderPropsAreEqual, matching the existing pattern in dataRowPropsAreEqual * fix(tables): address round 2 review comments - Mirror name validation (regex + max length) in PatchTableSchema so validateTableName failures return 400 instead of 500 - Add .returning() + rows-affected check to renameWorkspaceFile, matching the renameTable pattern - Check response.ok before parsing JSON in useRenameWorkspaceFile, matching the useRenameTable pattern * refactor(tables): reuse InlineRenameInput in BreadcrumbSegment Replace duplicated inline input markup with the shared component. Eliminates redundant useRef, useEffect, and input boilerplate. * fix(tables): set doneRef in cancelRename to prevent blur-triggered save Escape → cancelRename → input unmounts → blur → submitRename would save instead of canceling. Now cancelRename sets doneRef like submitRename does, blocking the subsequent blur handler. * fix(tables): pointercancel cleanup + typed FileConflictError - Add pointercancel handler to column resize to prevent listener leaks when system interrupts the pointer (touch-action override, etc.) - Replace stringly-typed error.message.includes('already exists') with FileConflictError class for refactor-safe 409 status detection * fix(tables): stable useCallback dep + rename shadowed variable - Use listRename.startRename (stable ref) instead of whole listRename object in handleContextMenuRename deps - Rename inner 'target' to 'origin' in arrow-key handler to avoid shadowing the outer HTMLElement 'target' * fix(tables): move class below imports, stable submitRename, clear editingCell - Move FileConflictError below import statements (import-first convention) - Make submitRename a stable useCallback([]) by reading editingId and editValue through refs (matches existing onSaveRef pattern) - Add setEditingCell(null) to handleEmptyRowClick for symmetry with handleCellClick * feat(tables): persist column widths in table metadata Column widths now survive navigation and page reloads. On resize-end, widths are debounced (500ms) and saved to the table's metadata field via a new PUT /api/table/[tableId]/metadata endpoint. On load, widths are seeded from the server once via React Query. * fix type checking for file viewer * fix(tables): address review feedback — 4 fixes 1. headerRename.onSave now uses the fileId parameter directly instead of the selectedFile closure, preventing rename-wrong-file race 2. updateMetadataMutation uses ref pattern matching mutateRef/createRef 3. Type-to-enter filters non-numeric chars for number columns, non-date chars for date columns 4. renameValue only passed to actively-renaming ColumnHeaderMenu, preserving React.memo for other columns * fix(tables): position-based gap rows, insert above/below, consistency fixes - Fix gap row insert shifting: only shift rows when target position is occupied, preventing unnecessary displacement of rows below - Switch to position-based indexing throughout (positionMap, maxPosition) instead of array-index for correct sparse position handling - Add insert row above/below to context menu - Use CellContent for pending values in PositionGapRows (matching PlaceholderRows) - Add belowHeader selection overlay logic to PositionGapRows - Remove unnecessary 500ms debounce on column width persistence Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix cells nav w keyboard * added preview panel for html, markdown rendering, completed table --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * fix(tables): one small tables ting (#3497) * feat(exa-hosted-key): Restore exa hosted key (#3499) Co-authored-by: Theodore Li <theo@sim.ai> * improvement(ui): consistent styling * styling alignment * improvements(tables): styling improvements * improve resizer for file preview for html files * updated document icon * fix(credentials): exclude regular login methods from credential sync * update docs * upgrade turbo * improvement: tables, chat * Fix table column delete * small table rename bug, files updates not persisting * Table batch ops * fix(credentials): block usage at execution layer without perms + fix invites * feat(hosted-key-services) Add hosted key for multiple services (#3461) * feat(hosted keys): Implement serper hosted key * Handle required fields correctly for hosted keys * Add rate limiting (3 tries, exponential backoff) * Add custom pricing, switch to exa as first hosted key * Add telemetry * Consolidate byok type definitions * Add warning comment if default calculation is used * Record usage to user stats table * Fix unit tests, use cost property * Include more metadata in cost output * Fix disabled tests * Fix spacing * Fix lint * Move knowledge cost restructuring away from generic block handler * Migrate knowledge unit tests * Lint * Fix broken tests * Add user based hosted key throttling * Refactor hosted key handling. Add optimistic handling of throttling for custom throttle rules. * Remove research as hosted key. Recommend BYOK if throtttling occurs * Make adding api keys adjustable via env vars * Remove vestigial fields from research * Make billing actor id required for throttling * Switch to round robin for api key distribution * Add helper method for adding hosted key cost * Strip leading double underscores to avoid breaking change * Lint fix * Remove falsy check in favor for explicit null check * Add more detailed metrics for different throttling types * Fix _costDollars field * Handle hosted agent tool calls * Fail loudly if cost field isn't found * Remove any type * Fix type error * Fix lint * Fix usage log double logging data * Fix test * Add browseruse hosted key * Add firecrawl and serper hosted keys * feat(hosted key): Add exa hosted key (#3221) * feat(hosted keys): Implement serper hosted key * Handle required fields correctly for hosted keys * Add rate limiting (3 tries, exponential backoff) * Add custom pricing, switch to exa as first hosted key * Add telemetry * Consolidate byok type definitions * Add warning comment if default calculation is used * Record usage to user stats table * Fix unit tests, use cost property * Include more metadata in cost output * Fix disabled tests * Fix spacing * Fix lint * Move knowledge cost restructuring away from generic block handler * Migrate knowledge unit tests * Lint * Fix broken tests * Add user based hosted key throttling * Refactor hosted key handling. Add optimistic handling of throttling for custom throttle rules. * Remove research as hosted key. Recommend BYOK if throtttling occurs * Make adding api keys adjustable via env vars * Remove vestigial fields from research * Make billing actor id required for throttling * Switch to round robin for api key distribution * Add helper method for adding hosted key cost * Strip leading double underscores to avoid breaking change * Lint fix * Remove falsy check in favor for explicit null check * Add more detailed metrics for different throttling types * Fix _costDollars field * Handle hosted agent tool calls * Fail loudly if cost field isn't found * Remove any type * Fix type error * Fix lint * Fix usage log double logging data * Fix test --------- Co-authored-by: Theodore Li <teddy@zenobiapay.com> * Fail fast on cost data not being found * Add hosted key for google services * Add hosting configuration and pricing logic for ElevenLabs TTS tools * Add linkup hosted key * Add jina hosted key * Add hugging face hosted key * Add perplexity hosting * Add broader metrics for throttling * Add skill for adding hosted key * Lint, remove vestigial hosted keys not implemented * Revert agent changes * fail fast * Fix build issue * Fix build issues * Fix type error * Remove byok types that aren't implemented * Address feedback * Use default model when model id isn't provided * Fix cost default issues * Remove firecrawl error suppression * Restore original behavior for hugging face * Add mistral hosted key * Remove hugging face hosted key * Fix pricing mismatch is mistral and perplexity * Add hosted keys for parallel and brand fetch * Add brandfetch hosted key * Update types * Change byok name to parallel_ai * Add telemetry on unknown models --------- Co-authored-by: Theodore Li <theo@sim.ai> * improvement(settings): SSR prefetch, code splitting, dedicated skeletons * fix: bust browser cache for workspace file downloads The downloadFile function was using a plain fetch() that honored the aggressive cache headers, causing newly created files to download empty. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(settings): use emcn Skeleton in extracted skeleton files * fix(settings): extract shared response mappers to prevent server/client shape drift Addresses PR review feedback — prefetch.ts duplicated response mapping logic from client hooks. Extracted mapGeneralSettingsResponse and mapUserProfileResponse as shared functions used by both client fetch and server prefetch. * update byok page * fix(settings): include theme sync in client-side prefetch queryFn Hover-based prefetchGeneralSettings now calls syncThemeToNextThemes, matching the useGeneralSettings hook behavior so theme updates aren't missed when prefetch refreshes stale cache. * fix(byok): use EMCN Input for search field instead of ui Input Replace @/components/ui Input with the already-imported EmcnInput for design-system consistency. * fix(byok): use ui Input for search bar to match other settings pages * fix(settings): use emcn Input for file input in general settings * improvement(settings): add search bar to skeleton loading states Skeletons now include the search bar (and action button where applicable) so the layout matches the final component 1:1. Eliminates layout shift when the dynamic chunk loads — search bar area is already reserved by the skeleton. * fix(settings): align skeleton layouts with actual component structures - Fix list item gap from 12px to 8px across all skeletons (API keys, custom tools, credentials, MCP) - Add OAuth icon placeholder to credential skeleton - Fix credential button group gap from 8px to 4px - Remove incorrect gap-[4px] from credential-sets text column - Rebuild debug skeleton to match real layout (description + input/button row) - Add scrollable wrapper to BYOK skeleton with more representative item count * chore: lint fixes * improvement(sidebar): match workspace switcher popover width to sidebar Use Radix UI's built-in --radix-popover-trigger-width CSS variable instead of hardcoded 160px so the popover matches the trigger width and responds to sidebar resizing. * revert hardcoded ff * fix: copilot, improvement: tables, mothership * feat: inline chunk editor and table batch ops with undo/redo (#3504) * feat: inline chunk editor and table batch operations with undo/redo Replace modal-based chunk editing/creation with inline editor following the files tab pattern (state-based view toggle with ResourceHeader). Add batch update API endpoint, undo/redo support, and Popover-based context menus for tables. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove icons from table context menu PopoverItems Icons were incorrectly carried over from the DropdownMenu migration. PopoverItems in this codebase use text-only labels. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: restore DropdownMenu for table context menu The table-level context menu was incorrectly migrated to Popover during conflict resolution. Only the row-level context menu uses Popover; the table context menu should remain DropdownMenu with icons, matching the base branch. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: bound cross-page chunk navigation polling to max 50 retries Prevent indefinite polling if page data never loads during chunk navigation across page boundaries. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: navigate to last page after chunk creation for multi-page documents After creating a chunk, navigate to the last page (where new chunks append) before selecting it. This prevents the editor from showing "Loading chunk..." when the new chunk is not on the current page. The loading state breadcrumb remains as an escape hatch for edge cases. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add duplicate rowId validation to BatchUpdateByIdsSchema Adds a .refine() check to reject duplicate rowIds in batch update requests, consistent with the positions uniqueness check on batch insert. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review comments - Fix disableEdit logic: use || instead of && so connector doc chunks cannot be edited from context menu (row click still opens viewer) - Add uniqueness validation for rowIds in BatchUpdateByIdsSchema - Fix inconsistent bg token: bg-background → bg-[var(--bg)] in Pagination Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove duplicate rowId uniqueness refine on BatchUpdateByIdsSchema The refine was applied both on the inner updates array and the outer object. Keep only the inner array refine which is cleaner. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address additional PR review comments - Fix stale rowId after create-row redo: patch undo stack with new row ID using patchUndoRowId so subsequent undo targets the correct row - Fix text color tokens in Pagination: use CSS variable references (text-[var(--text-body)], text-[var(--text-secondary)]) instead of Tailwind semantic tokens for consistency with the rest of the file Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove dead code and fix type errors in table context menu Remove unused `onAddData` prop and `isEmptyCell` variable from row context menu (introduced in PR but never wired to JSX). Fix type errors in optimistic update spreads by removing unnecessary `as Record<string, unknown>` casts that lost the RowData type. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: prevent false "Saved" status on invalid content and mark fire-and-forget goToPage calls ChunkEditor.handleSave now throws on empty/oversized content instead of silently returning, so the parent's catch block correctly sets saveStatus to 'error'. Also added explicit `void` to unawaited goToPage(1) calls in filter handlers to signal intentional fire-and-forget. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: handle stale totalPages in handleChunkCreated for new-page edge case When creating a chunk that spills onto a new page, totalPages in the closure is stale. Now polls displayChunksRef for the new chunk, and if not found, checks totalPagesRef for an updated page count and navigates to the new last page before continuing to poll. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * Streaming fix -- need to test more * Make mothership block use long input instead of prompt input * improvement(billing): isAnnual metadata + docs updates (#3506) * improvement(billing): on demand toggling and infinite limits * store stripe metadata to distinguish annual vs monthly * udpate docs * address bugbot * Add piping * feat(clean-hosted-keys) Remove eleven labs, browseruse. Tweak firecrawl and mistral key impl (#3503) * Remove eleven labs, browseruse, and firecrawl * Remove creditsUsed output * Add back mistral hosting for mistral blocks * Add back firecrawl since they queue up concurrent requests * Fix price calculation, remove agent since its super long running and will clog up queue * Define hosting per tool * Remove redundant token finding --------- Co-authored-by: Theodore Li <theo@sim.ai> * Update vfs to handle hosted keys * improvement(tables): fix cell editing flash, batch API docs, and UI polish (#3507) * fix: show text cursor in chunk editor and ensure textarea fills container Add cursor-text to the editor wrapper so the whole area shows a text cursor. Click on empty space focuses the textarea. Changed textarea from h-full/w-full to flex-1/min-h-0 so it properly fills the flex container. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * improvement(tables): fix cell editing flash, add batch API docs, and UI polish Fix stale-data flash when saving inline cell edits by using TanStack Query's isPending+variables pattern instead of manual cache writes. Also adds OpenAPI docs for batch table endpoints, DatePicker support in row modal, duplicate row in context menu, and styling improvements. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove dead resolveColumnFromEvent callback Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: unify paste undo into single create-rows action Batch-created rows from paste now push one `create-rows` undo entry instead of N individual `create-row` entries, so a single Ctrl+Z reverses the entire paste operation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: validate dates in inline editor and displayToStorage InlineDateEditor now validates computed values via Date.parse before saving, preventing invalid strings like "hello" from being sent to the server. displayToStorage now rejects out-of-range month/day values (e.g. 13/32) instead of producing invalid YYYY-MM-DD strings. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: accept ISO date format in inline date editor Fall back to raw draft input when displayToStorage returns null, so valid ISO dates like "2024-03-15" pasted or typed directly are accepted instead of silently discarded. Date.parse still validates the final value. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add ISO date support to displayToStorage and fix picker Escape displayToStorage now recognizes YYYY-MM-DD input directly, so ISO dates typed or pasted work correctly for both saving and picker sync. DatePicker Escape now refocuses the input instead of saving, so the user can press Escape again to cancel or Enter to confirm — matching the expected cancel behavior. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove dead paste boundary check The totalR guard in handlePaste could never trigger since totalR included pasteRows.length, making targetRow always < totalR. Remove the unused variable and simplify the selection focus calc. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * update openapi --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * fix dysfunctional unique operation in tables * feat(autosave): files and chunk editor autosave with debounce + refetch (#3508) * feat(files): debounced autosave while editing * address review comments * more comments * fix: unique constraint check crash and copilot table initial rows - Fix TypeError in updateColumnConstraints: db.execute() returns a plain array with postgres-js, not { rows: [...] }. The .rows.length access always crashed, making "Set unique" completely broken. - Add initialRowCount: 20 to copilot table creation so tables created via chat have the same empty rows as tables created from the UI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix signaling * revert: remove initialRowCount from copilot table creation Copilot populates its own data after creating a table, so pre-creating 20 empty rows causes data to start at position 21 with empty rows above. initialRowCount only makes sense for the manual UI creation flow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * improvement: chat, workspace header * chat metadata * Fix schema mismatch (#3510) Co-authored-by: Theodore Li <theo@sim.ai> * Fixes * fix: manual table creation starts with 1 row, 1 column Manual tables now create with a single 'name' column and 1 row instead of 2 columns and 20 rows. Copilot tables remain at 0 rows, 0 columns. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: horizontal scroll in embedded table by replacing overflow-hidden with overflow-clip Cell content spans used Tailwind's `truncate` (overflow: hidden), creating scroll containers that consumed trackpad wheel events on macOS without propagating to the actual scroll ancestor. Replaced with overflow-clip which clips identically but doesn't create a scroll container. Also moved focus target from outer container to the scroll div for correctness. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix tool call ordering * Fix tests * feat: add task multi-select, context menu, and subscription UI updates Add shift-click range selection, cmd/ctrl-click toggle, and right-click context menu for tasks in sidebar matching workflow/folder patterns. Update subscription settings tab UI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(credentials): autosync behaviour cross workspace (#3511) * fix(credentials): autosync behaviour cross workspace * address comments * fix(api-key-reminder) Add reminder on hosted keys that api key isnt needed (#3512) * Add reminder on hosted keys that api key isnt needed * Fix test case --------- Co-authored-by: Theodore Li <theo@sim.ai> * improvement: sidebar, chat * Usage limit * Plan prompt * fix(sidebar): workspace header collapse * fix(sidebar): task navigation * Subagent tool call persistence * Don't drop suabgent text * improvement(ux): streaming * improvement: thinking * fix(random): optimized kb connector sync engine, rerenders in tables, files, editors, chat (#3513) * optimized kb connector sync engine, rerenders in tables, files, editors, chat * refactor(sidebar): rename onTaskClick to onMultiSelectClick for clarity Made-with: Cursor * ack comments, add docsFailed * feat(email-footer) Add "sent with sim ai" for free users (#3515) * Add "sent with sim ai" for free users * Only add prompt injection on free tier * Add try catch around billing info fetch --------- Co-authored-by: Theodore Li <theo@sim.ai> * improvement: modals * ran migrations * fix(mothership): fix hardcoded workflow color, tables drag line overflowing * feat(mothership): file attachment indicators, persistence, and chat input improvements - Show image thumbnails and file-icon cards above user messages in mothership chat - Persist file attachment metadata (key, filename, media_type, size) in DB with user messages - Restore attachments from history via /api/files/serve/ URLs so they survive refresh/navigation - Unify all chat file inputs to use shared CHAT_ACCEPT_ATTRIBUTE constant - Fix file thumbnail overflow: use flex-wrap instead of hidden horizontal scroll - Compact attachment cards in floating workflow chat messages Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * improvement: search modal * improvement(usage): free plan to 1000 credits (#3516) * improvement(billing): free plan to five dollars * fix comment * remove per month terminology from marketing * generate migration * remove migration * add migration back * feat(workspace): add workspace color changing, consolidate update hooks, fix popover dismiss - Add workspace color change via context menu, reusing workflow ColorGrid UI - Consolidate useUpdateWorkspaceName + useUpdateWorkspaceColor into useUpdateWorkspace - Fix popover hover submenu dismiss by using DismissableLayerBranch with pointerEvents - Remove passthrough wrapper for export, reuse Workspace type for capturedWorkspaceRef - Reorder log columns: workflow first, merge date+time into single column Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update oauth cred tool * fix(diff-controls): fixed positioning for copilot diff controls * fix(font): added back old font for emcn code editor * improvement: panel, special tags * improvement: chat * improvement: loading and file dropping * feat(templates): create home templates * fix(uploads): resolve .md file upload rejection and deduplicate file type utilities Browsers report empty or application/octet-stream MIME types for .md files, causing copilot uploads to be rejected. Added resolveFileType() utility that falls back to extension-based MIME resolution at both client and server boundaries. Consolidated duplicate MIME mappings into module-level constants, removed duplicate isImageFileType from copilot module, and replaced hardcoded ALLOWED_EXTENSIONS with composition from shared validation constants. Also switched file attachment previews to use shared getDocumentIcon utility. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(home): prevent initial view from being scrollable Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * autofill fixes * added back integrations page, reverted secrets page back to old UI * Fix workspace dropdown getting cut off when sidebar is collapsed * fix(mothership): lint (#3517) * fix(mothership): lint * fix typing * fix tests * fix stale query * fix plan display name * Feat/add mothership manual workflow runs (#3520) * Add run and open workflow buttons in workflow preview * Send log request message after manual workflow run * Make edges in embedded workflow non-editable * Change chat to pass in log as additional context * Revert "Change chat to pass in log as additional context" This reverts commit e957dffb2f. * Revert "Send log request message after manual workflow run" This reverts commit 0fb92751f0. * Move run and workflow icons to tab bar * Simplify boolean condition --------- Co-authored-by: Theodore Li <theo@sim.ai> * feat(resource-tab-scroll): Allow vertical scrolling to scroll resource tab * fix(remove-speed-hosted-key) Remove maps speed limit hosted key, it's deprecated (#3521) Co-authored-by: Theodore Li <theo@sim.ai> * improvement: home, sidebar * fix(download-file): render correct file download link for mothership (#3522) * fix(download-file): render correct file download link for mothership * Fix uunecessary call * Use simple strip instead of db lookup and moving behavior * Make regex strip more strict --------- Co-authored-by: Theodore Li <theo@sim.ai> * improvement: schedules, auto-scroll * fix(settings): navigate back to origin page instead of always going home Use sessionStorage to store the return URL when entering settings, and use router.replace for tab switches so history doesn't accumulate. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(schedules): release lastQueuedAt lock on all exit paths to prevent stuck schedules Multiple error/early-return paths in executeScheduleJob and executeJobInline were exiting without clearing lastQueuedAt, causing the dueFilter to permanently skip those schedules — resulting in stale "X hours ago" display for nextRunAt. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(mothership): inline rename for resource tabs + workspace_file rename tool - Add double-click inline rename on file and table resource tabs - Wire useInlineRename + useRenameWorkspaceFile/useRenameTable mutations - Add rename operation to workspace_file copilot tool (schema, server, router) - Add knowledge base resource support (type, extraction, rendering, actions) - Accept optional className on InlineRenameInput for context-specific sizing Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * revert: remove inline rename UI from resource tabs Keep the workspace_file rename tool for the mothership agent. Only the UI-side inline rename (double-click tabs) is removed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(mothership): knowledge base resource extraction + Resource/ResourceTable refactor - Extract KB resources from knowledge subagent respond format (knowledge_bases array) - Add knowledge_base tool to RESOURCE_TOOL_NAMES and TOOL_UI_METADATA - Extract ResourceTable as independently composable memoized component - Move contentOverride/overlay to Resource shell level (not table primitive) - Remove redundant disableHeaderSort and loadingRows props - Rename internal sort state for clarity (sort → internalSort, sortOverride → externalSort) - Export ResourceTable and ResourceTableProps from barrel Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(logs) Run workflows client side in mothership to transmit logs (#3529) * Run workflows client side in mothership to transmit logs * Initialize set as constant, prevent duplicate execution * Fix lint --------- Co-authored-by: Theodore Li <theo@sim.ai> * fix(import) fix missing file * fix(resource): Hide resources that have been deleted (#3528) * Hide resources that have been deleted * Handle table, workflow not found * Add animation to prevent flash when previous resource was deleted * Fix animation playing on every switch * Run workflows client side in mothership to transmit logs * Fix race condition for animation * Use shared workflow tool util file --------- Co-authored-by: Theodore Li <theo@sim.ai> * fix: chat scrollbar on sidebar collapse/open * edit existing workflow should bring up artifact * fix(agent) subagent and main agent text being merged without spacing * feat(mothership): remove resource-level delete tools from copilot Remove delete operations for workflows, folders, tables, and files from the mothership copilot to prevent destructive actions via AI. Row-level and column-level deletes are preserved. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: stop sidebar from auto-collapsing when resource panel appears (#3540) The sidebar was forcibly collapsed whenever a resource (e.g. workflow) first appeared in the resource panel during a task. This was disruptive on larger screens where users want to keep both the sidebar and resource panel visible simultaneously. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * fix(mothership): insert copilot-created workflows at top of list (#3537) * feat(mothership): remove resource-level delete tools from copilot Remove delete operations for workflows, folders, tables, and files from the mothership copilot to prevent destructive actions via AI. Row-level and column-level deletes are preserved. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(mothership): insert copilot-created workflows at top of list * fix(mothership): server-side top-insertion sort order and deduplicate registry logic * fix(mothership): include folder sort orders when computing top-insertion position * fix(mothership): use getNextWorkflowColor instead of hardcoded color --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * fix(stop) Add stop of motehership ran workflows, persist stop messages (#3538) * Connect play stop workflow in embedded view to workflow * Fix stop not actually stoping workflow * Fix ui not showing stopped by user * Lint fix * Plumb cancellation through system * Stopping mothership chat stops workflow * Remove extra fluff * Persist blocks on cancellation * Add root level stopped by user --------- Co-authored-by: Theodore Li <theo@sim.ai> * fix(autolayout): targetted autolayout heuristic restored (#3536) * fix(autolayout): targetted autolayout heuristic restored * fix autolayout boundary cases * more fixes * address comments * on conflict updates * address more comments * fix relative position scope * fix tye omission * address bugbot comment * Credential tags * Credential id field * feat(mothership): server-persisted unread task indicators via SSE (#3549) * feat(mothership): server-persisted unread task indicators via SSE Replace fragile client-side polling + timer-based green flash with server-persisted lastSeenAt semantics, real-time SSE push via Redis pub/sub, and dot overlay UI on the Blimp icon. - Add lastSeenAt column to copilotChats for server-persisted read state - Add Redis/local pub/sub singleton for task status events (started, completed, created, deleted, renamed) - Add SSE endpoint (GET /api/mothership/events) with heartbeat and workspace-scoped filtering - Add mark-read endpoint (POST /api/mothership/chats/read) - Publish SSE events from chat, rename, delete, and auto-title handlers - Add useTaskEvents hook for client-side SSE subscription - Add useMarkTaskRead mutation with optimistic update - Replace timer logic in sidebar with TaskStatus state machine (running/unread/idle) and dot overlay using brand color variables - Mark tasks read on mount and stream completion in home page - Fix security: add userId check to delete WHERE clause - Fix: bump updatedAt on stream completion - Fix: set lastSeenAt on rename to prevent false-positive unread Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review feedback - Return 404 when delete finds no matching chat (was silent no-op) - Move log after ownership check so it only fires on actual deletion - Publish completed SSE event from stop route so sidebar dot clears on abort Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: backfill last_seen_at in migration to prevent false unread dots Existing rows would have last_seen_at = NULL after migration, causing all past completed tasks to show as unread. Backfill sets last_seen_at to updated_at for all existing rows. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: timestamp mismatch on task creation + wasSendingRef leak across navigation - Pass updatedAt explicitly alongside lastSeenAt on chat creation so both use the same JS timestamp (DB defaultNow() ran later, causing updatedAt > lastSeenAt → false unread) - Reset wasSendingRef when chatId changes to prevent a stale true from task A triggering a redundant markRead on task B Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: mark-read fires for inline-created chats + encode workspaceId in SSE URL Expose resolvedChatId from useChat so home.tsx can mark-read even when chatId prop stays undefined after replaceState URL update. Also URL-encode workspaceId in EventSource URL as a defensive measure. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: auto-focus home input on initial view + fix sidebar task click handling Auto-focus the textarea when the initial home view renders. Also fix sidebar task click to always call onMultiSelectClick so selection state stays consistent. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: auto-title sets lastSeenAt + move started event inside DB guard Auto-title now sets both updatedAt and lastSeenAt (matching the rename route pattern) to prevent false-positive unread dots. Also move the 'started' SSE event inside the if(updated) guard so it only fires when the DB update actually matched a row. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * modified tasks multi select to be just like workflows * fix * refactor: extract generic pub/sub and SSE factories + fixes - Extract createPubSubChannel factory (lib/events/pubsub.ts) to eliminate duplicated Redis/EventEmitter boilerplate between task and MCP pub/sub - Extract createWorkspaceSSE factory (lib/events/sse-endpoint.ts) to share auth, heartbeat, and cleanup logic across SSE endpoints - Fix auto-title race suppressing unread status by removing updatedAt/lastSeenAt from title-only DB update - Fix wheel event listener leak in ResourceTabs (RefCallback cleanup was silently discarded) - Fix getFullSelection() missing taskIds (inconsistent with hasAnySelection) - Deduplicate SSE_RESPONSE_HEADERS to spread from shared SSE_HEADERS - Hoist isSttAvailable to module-level constant to avoid per-render IIFE Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * feat(logs): add workflow trigger type for sub-workflow executions (#3554) * feat(logs): add workflow trigger type for sub-workflow executions * fix(logs): align workflow filter color with blue-secondary badge variant * feat(tab) allow user to control resource tabs * Make resources persist to backend * Use colored squares for workflows * Add click and drag functionality to resource * Fix expanding panel logic * Reduce duplication, reading resource also opens up resource panel * Move resource dropdown to own file * Handle renamed resources * Clicking already open tab should just switch to tab --------- Co-authored-by: Theodore Li <theo@sim.ai> * Fix new resource tab button not appearing on tasks * improvement(ui): dropdown menus, icons, globals * improvement: notifications, terminal, globals * reverted task logic * feat(context) pass resource tab as context (#3555) * feat(context) add currenttly open resource file to context for agent * Simplify resource resolution * Skip initialize vfs * Restore ff * Add back try catch * Remove redundant code * Remove json serialization/deserialization loop --------- Co-authored-by: Theodore Li <theo@sim.ai> * Feat(references) add at to reference sim resources(#3560) * feat(chat) add at sign * Address bugbot issues * Remove extra chatcontext defs * Add table and file to schema * Add icon to chip for files --------- Co-authored-by: Theodore Li <theo@sim.ai> * improvement(refactor): move to soft deletion of resources + reliability improvements (#3561) * improvement(deletion): migrate to soft deletion of resources * progress * scoping fixes * round of fixes * deduplicated name on workflow import * fix tests * add migration * cleanup dead code * address bugbot comments * optimize query * feat(sim-mailer): email inbox for mothership with chat history and plan gating (#3558) * feat(sim-mailer): email inbox for mothership with chat history and plan gating * revert hardcoded ff * fix(inbox): address PR review comments - plan enforcement, idempotency, webhook auth - Enforce Max plan at API layer: hasInboxAccess() now checks subscription tier (>= 25k credits or enterprise) - Add idempotency guard to executeInboxTask() to prevent duplicate emails on Trigger.dev retries - Add AGENTMAIL_WEBHOOK_SECRET env var for webhook signature verification (Bearer token) * improvement(inbox): harden security and efficiency from code audit - Use crypto.timingSafeEqual for webhook secret comparison (prevents timing attacks) - Atomic claim in executor: WHERE status='received' prevents duplicate processing on retries - Parallelize hasInboxAccess + getUserEntityPermissions in all API routes (reduces latency) - Truncate email body at webhook insertion (50k char limit, prevents unbounded DB storage) - Harden escapeAttr with angle bracket and single quote escaping - Rename use-inbox.ts to inbox.ts (matches hooks/queries/ naming convention) * fix(inbox): replace Bearer token auth with proper Svix HMAC-SHA256 webhook verification - Use per-workspace webhook secret from DB instead of global env var - Verify AgentMail/Svix signatures: HMAC-SHA256 over svix-id.timestamp.body - Timing-safe comparison via crypto.timingSafeEqual - Replay protection via timestamp tolerance (5 min window) - Join mothershipInboxWebhook in workspace lookup (zero additional DB calls) - Remove dead AGENTMAIL_WEBHOOK_SECRET env var - Select only needed workspace columns in webhook handler * fix(inbox): require webhook secret — reject requests when secret is missing Previously, if the webhook secret was missing from the DB (corrupted state), the handler would skip verification entirely and process the request unauthenticated. Now all three conditions are hard requirements: secret must exist in DB, Svix headers must be present, and signature must verify. * fix(inbox): address second round of PR review comments - Exclude rejected tasks from rate limit count to prevent DoS via spam - Strip raw HTML from LLM output before marked.parse to prevent XSS in emails - Track responseSent flag to prevent duplicate emails when DB update fails after send * fix(inbox): address third round of PR review comments - Use dynamic isHosted from feature-flags instead of hardcoded true - Atomic JSON append for chat message persistence (eliminates read-modify-write race) - Handle cutIndex === 0 in stripQuotedReply (body starts with quote) - Clean up orphan mothershipInboxWebhook row on enableInbox rollback - Validate status query parameter against enum in tasks API * fix(inbox): validate cursor param, preserve code blocks in HTML stripping - Validate cursor date before using in query (return 400 for invalid) - Split on fenced code blocks before stripping HTML tags to preserve code examples in email responses * fix(inbox): return 500 on webhook server errors to enable Svix retries * fix(inbox): remove isHosted guard from hasInboxAccess — feature flag is sufficient * fix(inbox): prevent double-enable from deleting webhook secret row * fix(inbox): null-safe stripThinkingTags, encode URL params, surface remove-sender errors - Guard against null result.content in stripThinkingTags - Use encodeURIComponent on all AgentMail API path parameters - Surface handleRemoveSender errors to the user instead of swallowing * improvement(inbox): remove unused types, narrow SELECT queries, fix optimistic ID collision Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(inbox): add keyboard accessibility to clickable task rows * fix(inbox): use Svix library for webhook verification, fix responseSent flag, prevent inbox enumeration - Replace manual HMAC-SHA256 verification with official Svix library per AgentMail docs - Fix responseSent flag: only set true when email delivery actually succeeds - Return consistent 401 for unknown inbox and bad signature to prevent enumeration - Make AgentMailInbox.organization_id optional to match API docs * chore(db): rebase inbox migration onto feat/mothership-copilot (0172 → 0173) Sync schema with target branch and regenerate migration as 0173 to avoid conflicts with 0172_silky_magma on feat/mothership-copilot. * fix(db): rebase inbox migration to 0173 after feat/mothership-copilot divergence Target branch added 0172_silky_magma, so our inbox migration is now 0173_youthful_stryfe. * fix(db): regenerate inbox migration after rebase on feat/mothership-copilot * fix(inbox): case-insensitive email match and sanitize javascript: URIs in email HTML - Use lower() in isSenderAllowed SQL to match workspace members regardless of email case stored by auth provider - Strip javascript:, vbscript:, and data: URIs from marked HTML output to prevent XSS in outbound email responses * fix(inbox): case-insensitive email match in resolveUserId Consistent with the isSenderAllowed fix — uses lower() so mixed-case stored emails match correctly, preventing silent fallback to workspace owner. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * Kb args * refactor(resource): remove logs-specific escape hatches from Resource abstraction Logs now composes ResourceHeader + ResourceOptionsBar + ResourceTable directly instead of using Resource with contentOverride/overlay escape hatches. Removes contentOverride, onLoadMore, hasMore, isLoadingMore from ResourceProps. Adds ColumnOption to barrel export and fixes table.tsx internal import. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(sim-mailer): download email attachments and pass to LLM as multimodal content Attachments were only passed as metadata text in the email body. Now downloads actual file bytes from AgentMail, converts via createFileContent (same path as interactive chat), and sends as fileAttachments to the orchestrator. Also parallelizes attachment fetching with workspace context loading, and downloads multiple attachments concurrently via Promise.allSettled. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(connector): add Gmail knowledge base connector with thread-based sync and filtering Syncs email threads from Gmail into knowledge bases with configurable filters: label scoping, date range presets, promotions/social exclusion, Gmail search syntax support, and max thread caps to keep KB size manageable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(connector): add Outlook knowledge base connector with conversation grouping and filtering Syncs email conversations from Outlook/Office 365 via Microsoft Graph API. Groups messages by conversationId into single documents. Configurable filters: folder selection, date range presets, Focused Inbox, KQL search syntax, and max conversation caps. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * cleanup resource definition * feat(connectors): add 8 knowledge base connectors — Zendesk, Intercom, ServiceNow, Google Sheets, Microsoft Teams, Discord, Google Calendar, Reddit Each connector syncs documents into knowledge bases with configurable filtering: - Zendesk: Help Center articles + support tickets with status/locale filters - Intercom: Articles + conversations with state filtering - ServiceNow: KB articles + incidents with state/priority/category filters - Google Sheets: Spreadsheet tabs as LLM-friendly row-by-row documents - Microsoft Teams: Channel messages (Slack-like pattern) via Graph API - Discord: Channel messages with bot token auth - Google Calendar: Events with date range presets and attendee metadata - Reddit: Subreddit posts with top comments, sort/time filters All connectors validated against official API docs with bug fixes applied. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(inbox): fetch real attachment binary from presigned URL and persist for chat display The AgentMail attachment endpoint returns JSON metadata with a download_url, not raw binary. We were base64-encoding the JSON text and sending it to the LLM, causing provider rejection. Now we parse the metadata, fetch the actual file from the presigned URL, upload it to copilot storage, and persist it on the chat message so images render inline with previews. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * added agentmail domain for mailer * added docs for sim mailer * fix(resource) handle resource deletion deletion (#3568) * Add handle dragging tab to input chat * Add back delete tools * Handle deletions properly with resources view * Fix lint * Add permisssions checking * Skip resource_added event when resource is deleted * Pass workflow id as context --------- Co-authored-by: Theodore Li <theo@sim.ai> * update docs styling, add delete confirmation on inbox * Fix fast edit route * updated docs styling, added FAQs, updated content * upgrade turbo * fix(knowledge) use consistent empty state for documents page Replace the centered "No documents yet" text with the standard Resource table empty state (column headers + create row), matching all other resource pages. Move "Upload documents" from header action to table create row as "New documents". Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(notifications): polish modal styling, credential display, and trigger filters (#3571) * fix(notifications): polish modal styling, credential display, and trigger filters - Show credential display name instead of raw account ID in Slack account selector - Fix label styling to use default Label component (text-primary) for consistency - Fix modal body spacing with proper top padding after tab bar - Replace list-card skeleton with form-field skeleton matching actual layout - Replace custom "Select a Slack account first" box with disabled Combobox (dependsOn pattern) - Use proper Label component in WorkflowSelector with consistent gap spacing - Add overflow badge pattern (slice + +N) to level and trigger filter badges - Use dynamic trigger options from getTriggerOptions() instead of hardcoded CORE_TRIGGER_TYPES - Relax API validation to accept integration trigger types (z.string instead of z.enum) - Deduplicate account rows from credential leftJoin in accounts API - Extract getTriggerOptions() to module-level constants to avoid per-render calls Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(notifications): address PR review feedback - Restore accountId in displayName fallback chain (credentialDisplayName || accountId || providerId) - Add .default([]) to triggerFilter in create schema to preserve backward compatibility - Treat empty triggerFilter as "match all" in notification matching logic - Remove unreachable overflow badge for levelFilter (only 2 possible values) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(settings): add spacing to Sim Keys toggle and replace Sim Mailer icon with Send Add 24px top margin to the "Allow personal Sim keys" toggle so it doesn't sit right below the empty state. Replace the Mail envelope icon for Sim Mailer with a new Send (paper plane) icon matching the emcn icon style. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * standardize back buttons in settings * feat(restore) Add restore endpoints and ui (#3570) * Add restore endpoints and ui * Derive toast from notification * Auth user if workspaceid not found * Fix recently deleted ui * Add restore error toast * Fix deleted at timestamp mismatch --------- Co-authored-by: Theodore Li <theo@sim.ai> * fix type errors * Lint * improvements: ui/ux around mothership * reactquery best practices, UI alignment in restore * clamp logs panel * subagent thinking text * fix build, speedup tests by up to 40% * Fix fast edit * Add download file shortcut on mothership file view * fix: SVG file support in mothership chat and file serving - Send SVGs as document/text-xml to Claude instead of unsupported image/svg+xml, so the mothership can actually read SVG content - Serve SVGs inline with proper content type and CSP sandbox so chat previews render correctly - Add SVG preview support in file viewer (sandboxed iframe) - Derive IMAGE_MIME_TYPES from MIME_TYPE_MAPPING to reduce duplication - Add missing webp to contentTypeMap, SAFE_INLINE_TYPES, binaryExtensions - Consolidate PREVIEWABLE_EXTENSIONS into preview-panel exports Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: replace image/* wildcard with explicit supported types in file picker The image/* accept attribute allowed users to select BMP, TIFF, HEIC, and other image types that are rejected server-side. Replace with the exact set of supported image MIME types and extensions to match the copilot upload validation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Context tags * Fix lint * improvement: chat and terminal --------- Co-authored-by: Emir Karabeg <emirkarabeg@berkeley.edu> Co-authored-by: Waleed <walif6@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Theodore Li <teddy@zenobiapay.com> Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai> Co-authored-by: Theodore Li <theodoreqili@gmail.com> Co-authored-by: Theodore Li <theo@sim.ai>	2026-03-13 21:02:08 -07:00
Waleed	1bf5ed4586	improvement(docs): add quick reference page and update SDK documentation (#2994) ... * docs(sdk): update README to reflect new interface * improvement(docs): add quick reference page and update SDK documentation * docs(copilot): update copilot documentation with all features	2026-01-25 02:21:02 -08:00
Waleed	dc0ed842c4	fix(sdk): improve input handling and separate input from options (#2993) ... * fix(sdk): improve input handling and separate input from options * fix(sdk): treat null as no input for consistency with Python SDK	2026-01-25 00:50:09 -08:00
Waleed	71130c8b0a	improvement(monorepo): added tsconfig package, resolved type errors in testing package (#2613)	2025-12-28 00:36:48 -08:00
Waleed	7b2792122d	fix(sdks): bump sdk versions (#2142)	2025-11-30 19:47:58 -08:00
3 people	fe9ebbf81b	improvement(api-keys): move to workspace level (#1765) ... * fix(billing): should allow restoring subscription (#1728) * fix(already-cancelled-sub): UI should allow restoring subscription * restore functionality fixed * fix * improvement(api-keys): move to workspace level * remove migration to prep merge * remove two more unused cols * prep staging merge * add migration back --------- Co-authored-by: Waleed <walif6@gmail.com> Co-authored-by: Siddharth Ganesan <33737564+Sg312@users.noreply.github.com>	2025-10-30 11:42:58 -07:00
Waleed	8f06aec68b	fix(vulns): fix various vulnerabilities and enhanced code security (#1611) ... * fix(vulns): fix SSRF vulnerabilities * cleanup * cleanup * regen docs * remove unused deps * fix failing tests * cleanup * update deps * regen bun lock	2025-10-11 22:14:31 -07:00
Waleedandwaleed	2d49892aaa	feat(deployed-chat): added file upload to workflow execute API, added to deployed chat, updated chat panel (#1588) ... * feat(deployed-chat): updated chat panel UI, deployed chat and API can now accept files * added nested tag dropdown for files * added duplicate file validation to chat panel * update docs & SDKs * fixed build * rm extraneous comments * ack PR comments, cut multiple DB roundtrips for permissions & api key checks in api/workflows * allow read-only users to access deployment info, but not take actions * add downloadable file to logs for files passed in via API * protect files/serve route that is only used client-side --------- Co-authored-by: waleed <waleed>	2025-10-10 17:19:51 -07:00
Waleed	1a0fdb32fe	chore(docs): update docs (#1578)	2025-10-08 00:55:18 -07:00
Waleed	9d45b8df1e	fix(ts-sdk): fix job to publish ts sdk (#1576)	2025-10-08 00:46:30 -07:00
Waleed	872e034312	feat(chat-streaming): added a stream option to workflow execute route, updated SDKs, updated docs (#1565) ... * feat(chat-stream): updated workflow id execute route to support streaming via API * enable streaming via api * added only text stream option * cleanup deployed preview componnet * updated selectedOutputIds to selectedOutput * updated TS and Python SDKs with async, rate limits, usage, and streaming API routes * stream non-streaming blocks when streaming is specified * fix(chat-panel): add onBlockComplete handler to chat panel to stream back blocks as they complete * update docs * cleanup * ack PR comments * updated next config * removed getAssetUrl in favor of local assets * resolve merge conflicts * remove extra logic to create sensitive result * simplify internal auth * remove vercel blob from CSP + next config	2025-10-07 15:10:37 -07:00
Waleed Latifandwaleedlatif	ae43381d84	feat(domain): drop the 'studio' (#818) ... * feat(domain): drop the * change all references for Sim Studio to Sim * change back license and notice * lint --------- Co-authored-by: waleedlatif <waleedlatif@waleedlatifs-MacBook-Pro.local>	2025-07-29 12:51:43 -07:00
+111 people	76df2b9cd9	fix(sockets): added throttling, refactor entire socket server, added tests (#534) ... * refactor(kb): use chonkie locally (#475) * feat(parsers): text and markdown parsers (#473) * feat: text and markdown parsers * fix: don't readfile on buffer, convert buffer to string instead * fix(knowledge-wh): fixed authentication error on webhook trigger fix(knowledge-wh): fixed authentication error on webhook trigger * feat(tools): add huggingface tools/blcok (#472) * add hugging face tool * docs: add Hugging Face tool documentation * fix: format and lint Hugging Face integration files * docs: add manual intro section to Hugging Face documentation * feat: replace Record<string, any> with proper HuggingFaceRequestBody interface * accidental local files added * restore some docs * make layout full for model field * change huggingface logo * add manual content * fix lint --------- Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local> * fix(knowledge-ux): fixed ux for knowledge base (#478) fix(knowledge-ux): fixed ux for knowledge base (#478) * fix(billing): bump better-auth version & fix existing subscription issue when adding seats (#484) * bump better-auth version & fix existing subscription issue Bwhen adding seats * ack PR comments * fix(env): added NEXT_PUBLIC_APP_URL to .env.example (#485) * feat(subworkflows): workflows as a block within workflows (#480) * feat(subworkflows) workflows in workflows * revert sync changes * working output vars * fix greptile comments * add cycle detection * add tests * working tests * works * fix formatting * fix input var handling * add images --------- Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local> Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net> * fix(kb): fixed kb race condition resulting in no chunks found (#487) * fix: added all blocks activeExecutionPath (#486) * refactor(chunker): replace chonkie with custom TextChunker (#479) * refactor(chunker): replace chonkie with custom TextChunker implementation and update document processing logic * chore: cleanup unimplemented types * fix: KB tests updated * fix(tab-sync): sync between tabs on change (#489) * fix(tab-sync): sync between tabs on change * refactor: optimize JSON.stringify operations that are redundant * fix(file-upload): upload presigned url to kb for file upload instead of the whole file, circumvents 4.5MB serverless func limit (#491) * feat(folders): folders to manage workflows (#490) * feat(subworkflows) workflows in workflows * revert sync changes * working output vars * fix greptile comments * add cycle detection * add tests * working tests * works * fix formatting * fix input var handling * fix(tab-sync): sync between tabs on change * feat(folders): folders to organize workflows * address comments * change schema types * fix lint error * fix typing error * fix race cond * delete unused files * improved UI * updated naming conventions * revert unrelated changes to db schema * fixed collapsed sidebar subfolders * add logs filters for folders --------- Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local> Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net> Co-authored-by: Waleed Latif <walif6@gmail.com> * revert tab sync * improvement(folders): added multi-select for moving folders (#493) * added multi-select for folders * allow drag into root * remove extraneous comments * instantly create worfklow on plus * styling improvements, fixed flicker * small improvement to dragover container * ack PR comments * fix(deployed-chat): made the chat mobile friendly (#494) * improvement(ui/ux): chat deploy (#496) * improvement(ui/ux): chat deploy experience * improvement(ui/ux): chat fontweight * feat(gmail): added option to access raw gmail from gmail polling service (#495) * added option to grab raw gmail from gmail polling service * safe json parse for function block execution to prevent vars in raw email from being resolved as sim studio vars * added tests * remove extraneous comments * fix(ui): fix the UI for folder deletion, huggingface icon, workflow block icon, standardized alert dialog (#498) * fixed folder delete UI * fixed UI for workflow block, huggingface, & added alert dialog for deleting folders * consistently style all alert dialogs * fix(reset-data): remove reset all data button from settings modal along with logic (#499) * fix(airtable): fixed airtable oauth token refresh, added tests (#502) * fixed airtable token refresh, added tests * added helpers for refreshOAuthToken function * feat(registration): disable registration + handle env booleans (#501) * feat: disable registration + handle env booleans * chore: removing pre-process because we need to use util * chore: format * feat(providers): added azure openai (#503) * added azure openai * fix request params being passed through agent block for azure * remove o1 from azure-openai models list * fix: add vscode settings to gitignore * feat(file-upload): generalized storage to support azure blob, enhanced error logging in kb, added xlsx parser (#506) * added blob storage option for azure, refactored storage client to be provider agnostic, tested kb & file upload and s3 is undisrupted, still have to test blob * updated CORS policy for blob, added azure blob-specific headers * remove extraneous comments * add file size limit and timeout * added some extra error handling in kb add documents * grouped envvars * ack PR comments * added sheetjs and xlsx parser * fix(folders): modified folder deletion to delete subfolders & workflows in it instead of moving to root (#508) * modified folder deletion to delete subfolders & workflows in it instead of moving to root * added additional testing utils * ack PR comments * feat: api response block and implementation * improvement(local-storage): remove use of local storage except for oauth and last active workspace id (#497) * remove local storage usage * remove migration for last active workspace id * Update apps/sim/app/w/[id]/components/workflow-block/components/sub-block/components/file-selector/components/jira-issue-selector.tsx Add fallback for required scopes Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * add url builder util * fi * fix lint * lint * modify pre commit hook * fix oauth * get last active workspace working again * new workspace logic works * fetch locks * works now * remove empty useEffect * fix loading issue * skip empty workflow syncs * use isWorkspace in transition flag * add logging * add data initialized flag * fix lint * fix: build error by create a server-side utils * remove migration snapshots * reverse search for workspace based on workflow id * fix lint * improvement: loading check and animation * remove unused utils * remove console logs --------- Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Co-authored-by: Emir Karabeg <emirkarabeg@berkeley.edu> Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@vikhyaths-air.lan> * feat(multi-select): simplified chat to always return readable stream, can select multiple outputs and get response streamed back in chat panel & deployed chat (#507) * improvement: all workflow executions return ReadableStream & use sse to support multiple streamed outputs in chats * fixed build * remove extraneous comments * general improvemetns * ack PR comments * fixed built * improvement(workflow-state): split workflow state into separate tables (#511) * new tables to track workflow state * fix lint * refactor into separate tables * fix typing * fix lint * add tests * fix lint * add correct foreign key constraint * add self ref * remove unused checks * fix types * fix type --------- Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net> * feat(models): added new openai models, updated model pricing, added new groq model (#513) * fix(autocomplete): fixed extra closing tag on tag dropdown autocomplete (#514) * chore: enable input format again * fix: process the input made on api calls with proper extraction * feat: add json-object for ai generation for response block and others * chore: add documentation for response block * chore: rollback temp fix and uncomment original input handler * chore: add missing mock for response handler * chore: add missing mock * chore: greptile recommendations * added cost tracking for router & evaluator blocks, consolidated model information into a single file, hosted keys for evaluator & router, parallelized unit tests (#516) * fix(deployState): deploy not persisting bug (#518) * fix(undeploy-bug): fix deployment persistence failing bug * fix lint --------- Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local> * fix decimal entry issues * remove unused files * fix(db): decimal position entry issues (#520) * fix decimal entry issues * remove unused files --------- Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net> * fix lint * fix test * improvement(kb): added configurability for chunks, query across multiple knowledge bases (#512) * refactor: consolidate create modal file * fix: identify dead processes * fix: mark failed in DB after processing timeout * improvement: added overlap chunks and fixed modal UI * feat: multiselect logic * fix: biome changes for css ordering warn instead of error * improvement: create chunk ui * fix: removed unused schema columns * fix: removed references to deleted columns * improvement: sped up vector search time * feat: multi-kb search * add bulk endpoint to disable/delete multiple chunks * add bulk endpoint to disable/delete multiple chunks * fix: removed unused schema columns * fix: removed references to deleted columns * made endpoints for knowledge more RESTful, added tests * added batch operations for delete/enable/disable docs, alr have this for chunks * added migrations * added migrations --------- Co-authored-by: Waleed Latif <walif6@gmail.com> * fix(models): remove temp from models that don't support it * feat(sdk): added ts and python SDKs + docs (#524) * added ts & python sdk, renamed cli from simstudio to cli * added docs * ack PR comments * improvements * fixed issue where it goes to random workspace when you click reload fixed lint issue * feat: better response builder + doc update * fix(auth): added preview URLs to list of trusted origins (#525) * trusted origins * lint error * removed localhost * ran lint --------- Co-authored-by: Waleed Latif <walif6@gmail.com> * fix(sdk): remove dev script from SDK * PR: changes for migration * add changes on top of db migration changes * fix: allow removing single input field * improvement(permissions): workspace permissions improvements, added provider and reduced API calls by 85% (#530) * improved permissions UI & access patterns, show outstanding invites * added logger * added provider for workspace permissions, 85% reduction in API calls to get user permissions and improved performance for invitations * ack PR comments * cleanup * fix disabled tooltips * improvement(tests): parallelized tests and build fixes (#531) * added provider for workspace permissions, 85% reduction in API calls to get user permissions and improved performance for invitations * parallelized more tests, fixed test warnings * removed waitlist verification route, use more utils in tests * fixed build * ack PR comments * fix * fix(kb): reduced params in kb block, added advanced mode to starter block, updated docs * feat(realtime): sockets + normalized tables + deprecate sync (#523) * feat: implement real-time collaborative workflow editing with Socket.IO - Add Socket.IO server with room-based architecture for workflow collaboration - Implement socket context for client-side real-time communication - Add collaborative workflow hook for synchronized state management - Update CSP to allow socket connections to localhost:3002 - Add fallback authentication for testing collaborative features - Enable real-time broadcasting of workflow operations between tabs - Support multi-user editing of blocks, edges, and workflow state Key components: - socket-server/: Complete Socket.IO server with authentication and room management - contexts/socket-context.tsx: Client-side socket connection and state management - hooks/use-collaborative-workflow.ts: Hook for collaborative workflow operations - Workflow store integration for real-time state synchronization Status: Basic collaborative features working, authentication bypass enabled for testing * feat: complete collaborative subblock editing implementation ✅ All collaborative features now working perfectly: - Real-time block movement and positioning - Real-time subblock value editing (text fields, inputs) - Real-time edge operations and parent updates - Multi-user workflow rooms with proper broadcasting - Socket.IO server with room-based architecture - Permission bypass system for testing 🔧 Technical improvements: - Modified useSubBlockValue hook to use collaborative event system - All subblock setValue calls now dispatch 'update-subblock-value' events - Collaborative workflow hook handles all real-time operations - Socket server processes and persists all operations to database - Clean separation between local and collaborative state management 🧪 Tested and verified: - Multiple browser tabs with different fallback users - Block dragging and positioning updates in real-time - Subblock text editing reflects immediately across tabs - Workflow room management and user presence - Database persistence of all collaborative operations Status: Full collaborative workflow editing working with fallback authentication * feat: implement proper authentication for collaborative Socket.IO server ✅ **Authentication System Complete**: - Removed all fallback authentication code and bypasses - Socket server now requires valid Better Auth session cookies - Proper session validation using auth.api.getSession() - Authentication errors properly handled and logged - User info extracted from session: userId, userName, email, organizationId 🔧 **Technical Implementation**: - Updated CSP to allow WebSocket connections (ws://localhost:3002) - Socket authentication middleware validates session tokens - Proper error handling for missing/invalid sessions - Permission system enforces workflow access controls - Clean separation between authenticated and unauthenticated states 🧪 **Testing Status**: - Socket server properly rejects unauthenticated connections - Authentication errors logged with clear messages - CSP updated to allow both HTTP and WebSocket protocols - Ready for testing with authenticated users Status: Production-ready collaborative authentication system * feat: complete authentication integration for collaborative Socket.IO system 🎉 **PRODUCTION-READY COLLABORATIVE SYSTEM** ✅ **Authentication Integration Complete**: - Fixed Socket.IO client to send credentials (withCredentials: true) - Updated server CORS to accept credentials with specific origin - Removed all fallback authentication bypasses - Proper Better Auth session validation working 🔧 **Technical Fixes**: - Socket client: Enable withCredentials for cookie transmission - Socket server: Accept credentials with origin 'http://localhost:3000' - Better Auth cookie utility integration for session parsing - Comprehensive authentication middleware with proper error handling 🧪 **Verified Working Features**: - ✅ Real user authentication (Vikhyath Mondreti authenticated) - ✅ Multi-user workflow rooms (2+ users in same workflow) - ✅ Permission system enforcing workflow access controls - ✅ Real-time subblock editing across browser tabs - ✅ Block movement and positioning updates - ✅ Automatic room cleanup and management - ✅ Database persistence of all collaborative operations 🚀 **Status**: Complete enterprise-grade collaborative workflow editing system - No more fallback users - production authentication - Multi-tab collaboration working perfectly - Secure access control with Better Auth integration - Real-time updates for all workflow operations * remove sync system and move to server side * fix lint * delete unused file * added socketio dep * fix subblock persistence bug * working deletion of workflows * fix lint * added railway * add debug logging for railway deployment * improve typing * fix lint * working subflow persistence * fix lint * working cascade deletion * fix lint * working subflow inside subflow * works * fix lint * prevent subflow in subflow * fix lint * add additional logs, add localhost as allowedOrigin * add additional logs, add localhost as allowedOrigin * fix type error * remove unused code * fix lint * fix tests * fix lint * fix build error * workign folder updates * fix typing issue * fix lint * fix typing issues * lib/ * fix tests * added old presence component back, updated to use one-time-token better auth plugin for socket server auth, tested * fix errors * fix bugs * add migration scripts to run * fix lint * fix deploy tests * fix lint * fix minor issues * fix lint * fix migration script * allow comma separateds id file input to migration script * fix lint * fixed * fix lint * fix fallback case * fix type errors * address greptile comments * fix lint * fix script to generate new block ids * fix lint --------- Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net> Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@vikhyaths-air.lan> Co-authored-by: Waleed Latif <walif6@gmail.com> Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local> * fix(sockets): updated CSP * remove unecessary logs * fix lint * added throttling, refactor entire socket server, added tests * improvements * remove self monitoring func, add block name event * working isWide, isAdvanced toggles with sockets * fix lint * fix duplicate key issue for user avatar * fix lint * fix user presence * working parallel badges / loop badges updates * working connection output persistence * fix lint * fix build errors * fix lint * logs removed * fix cascade var name update bug * works * fix lint * fix parallel blocks * fix placeholder * fix test * fixed tests --------- Co-authored-by: Aditya Tripathi <aditya@climactic.co> Co-authored-by: Adam Gough <77861281+aadamgough@users.noreply.github.com> Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com> Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-MacBook-Air.local> Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@Vikhyaths-Air.attlocal.net> Co-authored-by: Emir Karabeg <emirkarabeg@berkeley.edu> Co-authored-by: Emir Karabeg <78010029+emir-karabeg@users.noreply.github.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Co-authored-by: Vikhyath Mondreti <vikhyathmondreti@vikhyaths-air.lan> Co-authored-by: Ajit Kadaveru <ajit.kadaveru@berkeley.edu>	2025-06-24 17:44:30 -07:00