12 Commits

Include renames

Author	SHA1	Message	Date
6 people	263e3ca67e	improvement(external-endpoints): v2 versions with clean signatures + updated docs based on openapi spec (#5273) ... * v0.6.29: login improvements, posthog telemetry (#4026) * feat(posthog): Add tracking on mothership abort (#4023) Co-authored-by: Theodore Li <theo@sim.ai> * fix(login): fix captcha headers for manual login (#4025) * fix(signup): fix turnstile key loading * fix(login): fix captcha header passing * Catch user already exists, remove login form captcha * improvement(external-endpoints): v2 versions with clean signatures + updated docs * feat(usage): accept X-API-Key on usage-logs list + export /api/users/me/usage-logs and /export now use checkHybridAuth — the same auth /api/users/me/usage-limits already accepts — so external monitors can read summary.bySourceCredits (the source breakdown of usage-limits' aggregate currentPeriodCost) instead of estimating Copilot spend by subtraction. Workspace-scoped keys are pinned to their own workspace's slice of the ledger: the filter defaults to the key's workspace and an explicit mismatch 403s. Both endpoints documented in openapi-core.json. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(billing): dedicated v2 usage endpoints; keep internal usage routes session-only Replaces the earlier X-API-Key enablement on /api/users/me/usage-logs with a dedicated public surface, so the internal Billing-settings endpoints can evolve with the UI while external monitors get a stable versioned contract: - GET /api/v2/billing/usage — current-billing-period summary with bySourceCredits (the source breakdown external monitors need to watch e.g. Copilot consumption without estimating by subtraction), plus limitCredits and plan - GET /api/v2/billing/usage/logs — cursor-paged credit ledger in the v2 envelope - workspace-scoped keys are pinned to their own workspace's slice; personal keys read the account ledger The public wire is credits-only: usage-logs rows now carry a hasCost boolean instead of dollarCost (the Billing UI only needed the >0 signal), and the rateLimit block is removed from the usage-limits response and docs (deploy-modal tab relabeled accordingly). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(docs): validate OpenAPI specs against the Zod contracts in CI The specs in apps/docs are hand-authored because they carry what Zod never defines — error envelopes, status codes, prose, examples — so they can't be generated; check:openapi validates them instead: - spec integrity: $refs resolve, operationIds unique, 2xx documented, no orphaned component schemas - v2 conventions: every /api/v2 operation documents 401 + 429 and every 4xx/5xx resolves to the canonical { error: { code, message } } envelope - contract cross-check: contracts are auto-discovered from lib/api/contracts/v2 (each carries its method + path); doc<->contract coverage both ways, query/body/response field diffs via z.toJSONSchema - examples: documented request/response examples must parse with the matching contract's actual Zod schemas First run caught real drift, fixed here: 16 stale orphaned schemas in the core spec, the v2 billing ops referencing v1-shaped error components, deploy/rollback examples missing the required nullable lifecycle keys, CreateTableBody missing folderId, a legacy-grammar delete-rows example, and four knowledge document ops missing their required workspaceId query param. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * fix(docs): recursive field diff in check:openapi + the deep drift it found A mutation test showed the doc<->contract field diff only compared top-level properties, so a typo inside the { data } envelope passed. The diff now descends through matching object properties and array items (both sides must expose a property set — passthrough contracts and prose-only docs end the descent instead of false-positive), with the Zod JSON-schema root doubling as the $defs context. Deep drift it immediately caught, fixed here: select-column config (options/multiple) missing from every tables column schema, AddColumnBody hand-rolling a third column shape (now composed from ColumnInput, with position/workflowGroupId as the per-op extensions the contracts actually admit), chunking strategyOptions undocumented, and the deployment lifecycle fields (activeDeployment/latestDeploymentAttempt) missing from DeploymentState. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * fix(security): close the triggerType rate-limit bypass on workflow execute Caller-supplied triggerType flowed unchecked into preprocessExecution, whose checkRateLimit default turns OFF for 'manual'/'chat' — so any API-key caller, and any anonymous public-API caller billed to the workspace owner, could execute unthrottled by sending {"triggerType":"manual"} (async runs also skipped the worker-side check via admissionCompleted). External callers may now only send the redundant 'api' value; internal JWT callers ('workflow'/'mcp') are unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * refactor(execution): extract enqueue/status/cancel into shared libs Prepares the v2 execution surface: handleAsyncExecution's queue logic moves to lib/workflows/executor/enqueue-execution.ts (slot/claim semantics encoded in a discriminated outcome, not HTTP statuses), the execution-status read to execution-status.ts, and the order-sensitive cancel machinery to lib/execution/cancel-workflow-execution.ts. The v1 routes re-render identically — their suites pass unmodified. Also: preprocessExecution gains rateLimitCounter ('sync'|'async') and its 429 now carries code RATE_LIMIT_EXCEEDED + retryAfterMs (previously indistinguishable from the concurrency 429 and Retry-After was discarded); and the duplicate cancel contract in contracts/logs.ts is unified on the full 5-value reason enum — its narrower copy made requestJson throw a client ZodError when cancelling a paused HITL run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(execution): callable execution service + structured error classifier executeWorkflowService composes the same libs the v1 route holds inline (call-chain guard, execution-id claim, LoggingSession, preprocessing, deployed-state load + file-field processing, timeout-bound executeWorkflowCore, output hydration/compaction) for the deployed-state caller class — the seam the v2 execute route and in-process internal callers share, making the HTTP endpoint syntactic sugar. classifyExecutionError stops discarding the block context that buildBlockExecutionError already attaches at throw sites: failed runs now yield {message, code, blockId, blockName, blockType} with a stable append-only code enum (TIMEOUT/CANCELLED/USAGE_LIMIT_EXCEEDED/ INVALID_INPUT/BLOCK_EXECUTION_FAILED/CHILD_WORKFLOW_FAILED/ OUTPUT_TOO_LARGE/EXECUTION_FAILED), so callers route on error class instead of substring-matching messages — the single place raw errors are interpreted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(api): POST /api/v2/workflows/[id]/execute Thin route over executeWorkflowService: X-API-Key or anonymous public-API auth (sync/stream only for anonymous), strict body with body-flag async (no mode headers on v2), SSE passthrough for stream, and the execution resource response — executionId always present, in-band run failures are status:'failed' with the structured {message, code, blockId, blockName, blockType} error, sync timeout is status:'failed' + TIMEOUT instead of v1's 408, and a Response block's payload stays inside output (authors never control response status/headers on this origin). Async debits the async bucket and the 202 statusUrl points at the v2 executions resource. Adds CLIENT_CLOSED_REQUEST/SERVICE_UNAVAILABLE to the v2 error codes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(api): v2 executions status + cancel with queued backfill GET /api/v2/workflows/[id]/executions/[executionId] is the single status URL for sync and async runs: before the async worker writes the durable log row, status is backfilled from the job queue (deterministic job id) as 'queued'/'running' — closing v1's 202-to-pickup 404 window — and failed runs carry the structured error object. POST .../cancel renders the shared cancellation lib in the v2 envelope with the tightened 5-value reason enum. Both authenticate via the shared resolveV2WorkflowAccess (X-API-Key, authz masked as 404, allowPersonalApiKeys honored). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(execution): workflow tool + MCP bridge run in-process workflow_executor (workflow-as-agent-tool) short-circuits in executeTool through WorkflowBlockHandler — the same invocation boundary canvas child workflows use — mirroring the deployed_block_executor precedent. The MCP serve bridge calls executeWorkflowService directly instead of fetching its own execute endpoint; deployment-version pinning, MCP response-size rejection, and the actor override become typed options instead of header sniffing. Both callers drop the double admission slot and duplicate top-level log row the HTTP hop cost, and failed child runs now surface the structured error + child executionId so parents and MCP clients can route on error class and hand providers a reproducible handle. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(infra): CORS + CSP coverage for the v2 execute path /api/v2/workflows/:id/execute gets the same wildcard-origin, credential-free CORS policy as v1 (the default credentialed policy would block browser API-key calls and open a cookie CSRF surface) with X-Sim-Stream-Protocol allowed and no X-Execution-Mode (async is body-selected on v2), plus the COEP/COOP/CSP header block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(ui): deploy modal + copilot advertise the v2 execute surface All 20 API-tab snippets move to POST /api/v2/workflows/{id}/execute with the nested {"input": ...} body, async as the "async": true body flag (X-Execution-Mode gone), status polling against the v2 executions resource, the third tab renamed Usage and pointed at /api/v2/billing/usage, and {data} envelope unwraps in the printed responses. Fixes the latent baseUrl derivation (endpoint.split('/api/workflows/')) that would have silently built garbage URLs under a v2 endpoint, and deletes dead code (exampleCommand across 3 sites, getAsyncExampleTitle). Copilot deploy/manage/serializer endpoint builders and the api_trigger bestPractices example follow (the latter also drops its hardcoded staging host). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * docs(api): document the v2 execution surface Adds execute, execution status, and cancel to openapi-v2-workflows.json with the structured ExecutionError schema (append-only code enum + block attribution) and the ExecutionResource contract, documenting the rules that differ from v1: modes are body-selected, a failed run is HTTP 200 with status 'failed', an executionId always means data (never the error envelope), queued status is visible immediately, and Response-block payloads stay inside output. Registers the three pages in the generated workflows meta.json and bumps the route-count baseline. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * feat(api): gate the whole /api/v2 surface behind one flag; UI stays on v1 Every v2 route now runs exactly one check immediately after auth — v2ApiGateError — and answers 404 when the `v2-api` flag is off, so the surface is invisible until it is deliberately rolled out. The gate is keyed on userId only: a workspace/org-keyed check would have to read membership for a caller-supplied id before authorization runs, and its 404-vs-403 split would leak cohort membership (the trap the per-domain table gate worked around by running late). The two executions routes inherit it from the shared access resolver; the tables-specific gate is removed so no route checks twice. `tables-v2-api` stays, now gating only the internal predicate-grammar route /api/table/[tableId]/query — note v2 tables routes move to the unified flag, so enabling them is a `v2-api` decision now. Reverts the deploy modal, copilot handlers, and api_trigger example to the v1 execute endpoint: v1 works unchanged, and the UI must not advertise a surface most users would get a 404 from. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiHhAk2R1NryaS3R8n2yFz * fix(executor): restore child-cost aggregation dropped by the staging merge Staging's custom-block rewrite deleted `aggregateChildCost` from workflow-handler.ts, and git merged that file cleanly — but this branch's workflow-tool-runner.ts, added for the v2 execute migration, still imports it. A silent semantic conflict: no marker, broken build. Taking staging's rewrite is correct, so the helper is defined locally in its one remaining consumer rather than resurrected in the file staging just rewrote. Same four lines over the still-exported `calculateCostSummary`, so a failed child workflow keeps billing the hosted-key spend it consumed instead of reporting $0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(tables): make lib/table/orchestration the single implementation (#6134) * refactor(orchestration): move the shared error contract out of lib/workflows OrchestrationErrorCode and statusForOrchestrationError are the contract every lib/[resource]/orchestration module returns against, but they lived inside the workflows module, so resource-neutral code (lib/folders) already had to import from a workflow path. Moved to lib/core/orchestration/types. Adds a 'locked' class mapping to 423. Both tables and workflows have a lock that forbids a mutation, and each caller was translating that to a status itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(tables): make lib/table/orchestration the single implementation Column update was implemented four times — the UI route, v1, v2, and the copilot table tool — each calling the same column services but owning its own guards, error mapping, and audit. The copies had drifted, and the drift was the bug: v2 was missing both guards, only the copilot copy minted stable option ids, and only v1/v2 audited. performUpdateTableColumn, performDeleteTable, and performDeleteTableRow now own that logic; all ten call sites reduce to auth, parse, call, render. The guards are asserted once in lib/table/orchestration rather than four times against four routes. Behavior this consolidates, previously true on only some paths: - The typeChanging guard. updateColumnType early-returns on an unchanged type and drops any options sent with it, so restating the current type alongside new options silently discarded them. v2 had no guard at all and, since its contract shares v1's body schema, accepted options and ignored them. - The select-unique guard. Each write is its own locked transaction, so a rename or type change paired with a constraint write that is going to fail commits first and then throws, half-applying the schema change. - Stable select-option ids. Cells reference the option id, so an edit that re-sends an option by name has to reuse it or every cell holding it is orphaned. Only the copilot path did this; normalizeSelectOptionsInput moves to lib/table/select-options and now covers every caller. It preserves a supplied id, so it is a no-op for the fully-formed options the HTTP contracts accept. - required forwarded into the type and options writes, so a conversion validates against the constraint the same request is setting. - An audit on every successful update. The UI route and the copilot tool emitted none. - Single-row delete through the row service. v2 did a raw db.delete, skipping assertRowDelete and deleteOrderedRow, so a delete-locked table returned 200 and the row-count bookkeeping never ran. - The delete actor handed to deleteTable, which audits only when a row was actually archived. v1 and v2 omitted it and audited themselves outside that check, emitting TABLE_DELETED for a no-op delete of an archived table. Failure classes come back as OrchestrationErrorCode; v2 renders them through a new v2ErrorForOrchestration, mirroring statusForOrchestrationError on the v1 and UI surfaces, so a given failure maps to the same status everywhere. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(tables): bind the column-update tests to the orchestration function The base's route tests assert which column service each payload reaches — the behavior that now lives in performUpdateTableColumn. They mocked the `@/lib/table` barrel; the orchestration module imports the service directly, so they mock that too and keep asserting the same thing through the extracted implementation. The orchestration tests move onto the base's semantics: writes address the stable column id, a rename rides inside the write it accompanies rather than running first, and the currency guards replace the non-select options guard the service now owns. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(copilot): drop the column-type import the delegation made dead Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(tables): move the audit log out of the table service `lib/table/service.ts` wrote its own audit rows, so whether an operation was audited depended on which function a caller reached for rather than on a user having performed it. That is what let v1 and v2 audit a no-op delete, and what made `deleteTable`'s optional `actingUserId` double as an audit opt-out flag. Worse, most sites fell back to `actingUserId ?? createdBy`, so an unattributed call was logged against the table's *creator*. The copilot `mv` path passed no actor at all: renaming someone else's table recorded them as the renamer. Audit now lives in the orchestration functions — performDeleteTable, performRenameTable, performMoveTableToFolder, performUpdateTableLocks — and the services just write. Internal callers (folder cascade, import rollback) keep calling the service and are silent by construction rather than by remembering to omit an argument. Two services now return what the audit needs: `deleteTable` reports whether it actually archived a row, so a repeat delete logs nothing; `updateTableLocks` returns the before/after locks, since only the locked write can observe the transition its description names. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(tables): restore audit provenance and conflict status in orchestration Moving the audits into the orchestration functions dropped three things the routes had been carrying, and added one the orchestration now owns twice. - The v1 and v2 column-update routes passed `request` to `recordAudit`, so their audit rows recorded the caller's IP and user-agent. The orchestration function had no way to receive it. Every table orchestration function now takes an optional `OrchestrationRequestContext` and every HTTP route forwards it; the copilot and VFS callers, which have no request, omit it. - `classifyTableMutation` matched `TableConflictError` on "already exists" appearing in the message and reported it as `validation`, turning the UI route's 409 on a duplicate table rename into a 400. It now matches the type, the way `performRestoreTable` already did. - `captureServerEvent` ran on every delete while the audit was gated on a row actually being archived, so a repeat delete of an archived table still reported `table_deleted`. Both now hang off the same evidence. - The copilot delete path kept its own `captureServerEvent` from when the service did not emit one, double-counting every copilot table delete. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a * fix(tables): say which type a no-op column update restated A copilot `update_column` payload whose only content was the column's current type used to return success with the live schema, while the v1, v2, and UI routes rejected the same payload with "No updates specified". Delegating to `performUpdateTableColumn` unified them onto the routes' rejection — correct, but the message tells the caller its request was empty when it named a type. The orchestration function now reports the same thing `updateColumnType` reports when it loses this race concurrently: the column is already that type, re-issue without the type change. An empty payload still reads "No updates specified". Drops the copilot's `outcome.table ?? tableForUpdate` fallback with it — the comment described the no-op that can no longer reach that line, and a success always carries a table. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a * refactor(tables): classify failures by type instead of by message text The table module decided HTTP statuses by searching error messages for phrases. `VALIDATION_MESSAGE_FRAGMENTS` and `ROW_WRITE_ERROR_PATTERNS` held 32 substrings between them, and fifteen more lists were inlined in routes — 83 matchers over 17 files, each its own copy of the guesswork and already drifted apart. It made message wording load-bearing: `TableRowLimitError`'s own doc comment noted that its text had to contain "row limit" for a route to answer 400, and adding "already exists" to a rename message silently demoted a 409 to a 400 (the bug fixed one commit ago, by adding another special case). Services now throw `OrchestrationError`, which carries the transport-neutral `OrchestrationErrorCode` the layers above already speak. Classification is one `instanceof` in `orchestrationErrorResponse` (UI + v1) and `v2CaughtOrchestrationError` (v2). Every pattern list is gone. Wording is free to change; an unclassified error still becomes a generic 500, which is what an unexpected fault should be. `asOrchestrationError` walks the `cause` chain rather than testing the caught value directly: drizzle wraps a throw raised inside a transaction callback in a `DrizzleQueryError` whose own message is the failed SQL, so a bare `instanceof` would drop every failure raised inside `withLockedTable`. That is the same reason `rootErrorMessage` had to dig for a root cause before. Three throws stay bare `Error` deliberately — `Table ID mismatch`, `Workspace ID mismatch`, and `Failed to build upsert conflict predicate` are internal invariants no consumer classified, and they keep falling through to a 500. `Insufficient capacity` was in the pattern list with no producer anywhere in the codebase. Status changes, all deliberate: - `'forbidden'` joins the code union so the table-row-limit ceiling keeps its 403; without it this refactor would have flattened it to 400. - import-async's table-limit rejection: 400 -> 403, matching the two other create routes it had drifted from. - Renaming a table to an invalid name: 500 -> 400. `validateTableName` messages don't contain "Invalid", so no matcher ever caught them. - Restoring a table that isn't archived, or into an archived workspace: 500 -> 400. - A duplicate *column* name stays `validation`/400 rather than becoming a 409 like a duplicate table name. Both v1 and the orchestration have always answered 400 for it; changing a published status is not this refactor's job. The twelve tests that changed were asserting the substring mechanism itself, constructing plain `Error`s with magic strings. They now assert the real contract, plus new cases pinning that identical wording carrying no classification stays internal and keeps its message off the wire. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * feat(api): add v2 endpoints for MCP servers, skills, custom tools, folders, and credentials (#6150) * feat(api): add v2 endpoints for MCP servers, skills, custom tools, folders, and credentials * fix(api): correct credential role, skill permission bar, MCP url identity, and custom-tool conflict mapping * fix(api): align credential mutation gating, provider-outage status, and unique-violation conflicts * fix(api): close unique-violation, revival, orphan-write, and env-rename gaps * fix(api): treat every provider-outage code as unavailable on create and update * fix(credentials): use the shared outage predicate on the session update path * fix(contracts): anchor the predicate double-cast annotation to the cast `check:api-validation:strict` counted 9 unannotated double-casts against a baseline of 8, failing CI. The predicate leaf schema was annotated, but the annotation sat above the declaration while the checker anchors on the line carrying the cast — five lines below, at the close of the object literal. The scanner walks back at most three lines and stops at the first non-comment one, so it hit `value: z.unknown().optional(),` and never saw the reason. Splitting the object schema from the cast puts them adjacent, so the existing reason binds. No behavior change — the cast, the schema, and the reasoning are unchanged. Also lowers the rawJsonReads ratchet 6 -> 5 to match the current count, which had drifted down; leaving it high lets a removed raw read silently come back. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(skills): point the orchestration error contract at its moved module #6150 branched before #6134, so skill-lifecycle.ts imports @/lib/workflows/orchestration/types — the module #6134 moved to @/lib/core/orchestration/types. Git merged a file deletion on one side with a new file referencing it on the other: no textual conflict, broken build. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(knowledge): make lib/knowledge/orchestration the single implementation (#6154) * refactor(knowledge): make lib/knowledge/orchestration the single implementation Knowledge base create was implemented four times — the internal route, v1, v2, and the copilot tool — and the orchestration around the shared write had drifted. Extract it the same way lib/table/orchestration was: services write, orchestration decides which writes run, guards them, audits them, and returns a transport-neutral failure. Behavior converged, not preserved: - One chunking default (DEFAULT_CHUNKING_CONFIG). The agent defaulted minSize to 1 against the API's 100, so identical input produced differently-chunked knowledge bases depending on who created it. The agent path now chunks at 100. - Every successful mutation is audited inside the orchestration function. The copilot tool called recordAudit zero times, so agent-created knowledge bases, document uploads, updates and deletes left no audit trail at all. - Failures classify by class, not by message text. The knowledge service errors are OrchestrationError subclasses and storage-quota rejections throw a shared StorageLimitExceededError, replacing four separate message greps for "already exists" / "does not have permission" / "storage limit". delete_connector reported the opposite of what happened. It reached the route through an internal HTTP self-call that sent no query string, so the route's keep-documents default always applied while the agent told the user the documents had been removed. The self-call is gone — all four connector operations run in-process — and the orchestration returns the real counts. Also: - OrchestrationErrorCode gains 'payload_too_large' (413 / PAYLOAD_TOO_LARGE). Without it, dropping the storage-limit message match would have regressed the documented 413 on knowledge base create and document upload to a 500. - messageForOrchestrationError renders a route's own wording for an unclassified fault, so a driver's message no longer reaches the client on a 500. - v1 and v2 knowledge base update now forward actorUserId, which the service requires for a workspace move; both omitted it. - The connector DELETE route reads deleteDocuments through parseRequest. Its contract declared z.boolean(), which would have rejected the string a query param actually is. - Drop the 409 from POST /api/v2/knowledge/{id}/documents in the OpenAPI spec. Nothing on the upload path throws a conflict; it was only ever reachable by the message match this change removes. Behavior change worth noting: a v1/v2 PUT carrying only the workspaceId scope field and no actual updates now returns 400 rather than 200 with the unchanged knowledge base. Deliberately deferred: document update remains internal-only. Extracting performUpdateKnowledgeDocument makes exposing it on v1/v2 a contract and a route away, but that is a new public surface rather than part of this consolidation. * fix(knowledge): make connector create atomic and stop flattening failures Review round 1 on #6154. - Resolve the billing payer before the connector is committed, not after. A malformed attribution header rejected post-commit left a live connector behind a 500, and a retry created a duplicate plus duplicate sync work. Manual sync resolves before writing its audit for the same reason. - Let the source-config validator carry its own failure class. Collapsing every rejection to `validation` flattened the connector PATCH route's 401 (stale stored credential) and 409 (missing workspace context) into a 400. - Add `unauthorized` to OrchestrationErrorCode. It is the class that 401 was already expressing on this route, and the v2 vocabulary already had UNAUTHORIZED; only the shared union was missing it. - Report a knowledge base that exists but failed to archive as failed, with the reason, rather than as not found. The copilot delete loop folded every non-not-found failure into `notFound`, telling the user it was never there. - Route copilot failures through the same message helper the HTTP surfaces use, so an unclassified fault's raw text (a driver's failed SQL) no longer reaches the agent verbatim while the UI and public APIs get the generic wording. * feat(api): expand the public v2 files surface (#6160) * feat(api): expand the public v2 files surface Adds folder support, rename/restore, move, bulk archive, share, and content replace to /api/v2/files, so managing files by API no longer stops at upload + download + archive-one. Routes are thin: auth -> parse -> perform* -> serialize. Share and content replace get their orchestration extracted first so the session routes and the public ones cannot diverge on the effective-authType resolution, the EE public-sharing gate, or the storage-quota classification. Presigned upload stays session-only: presign does an advisory quota check and the real debit happens in the separate register step, so a caller that never registers leaves unaccounted bytes with no reaper. The buffered multipart path debits inside uploadWorkspaceFile's own transaction. * fix(files): classify folder and content failures instead of 500ing them Bugbot round 1. The v2 routes map errorCode straight to a status, so every manager failure that arrived unclassified became a 500 for what is really a caller-fixable 400 or 404. - Folder manager throws OrchestrationError: missing target/folder -> not_found, reparent cycle / self-parent / restore-into-archived-workspace -> validation. - File manager does the same for the in-transaction 'File not found' paths that the earlier pass missed. - updateWorkspaceFileContent's outer catch re-wrapped everything in a bare Error, which stripped the class off StorageLimitExceededError and the new not_found alike. It now rethrows a classified failure untouched and attaches cause to the generic wrap, so asOrchestrationError can still walk the chain. - Every remaining perform* gained the asOrchestrationError branch. - renameWorkspaceFile returned the pre-update read, so the v2 PATCH reported a stale updatedAt; it now returns the timestamp it actually wrote. Docs: upload auto-suffixes a duplicate name rather than rejecting it, matching the in-app uploader. The description claimed 409 and was simply wrong. * fix(files): surface a failed upload read-back as the real error getWorkspaceFile swallows a query failure and returns null unless throwOnError is set, so a transient blip on the post-upload read reported as 'file could not be read back'. Distinguish the two: a real null after a just-committed write is an invariant break, a query failure is itself. * revert(api): drop the dedicated v2 file-folder routes File folders already live in the shared folder table as resourceType 'file' (#6045 cut them over, #6051 dropped workspace_file_folders), and the remaining file-specific folder machinery is being folded into the generic folder engine. Publishing /api/v2/files/folders/** would pin that transitional split into a public contract we'd then have to keep or break. Files stay folder-aware — folderId/folderPath on the projection, folderId on upload, and the move route — because a folder id is a folder.id and survives the unification untouched. Folder management belongs on /api/v2/folders once that surface serves resourceType 'file'; until then there is no v2 way to enumerate file folders, which is the deliberate gap. The orchestration classification fixes stay: the internal routes and the copilot file-folder tools still call those perform* functions. * fix(files): classify upload failures instead of matching their wording Bugbot round 2. uploadWorkspaceFile had the same outer-catch rewrap that updateWorkspaceFileContent did, so a blown storage quota reached the route as a bare Error and the v2 handler recovered the status by substring-matching the message. Any rewording silently demoted a 413 to a 500. - uploadWorkspaceFile rethrows a classified failure untouched and attaches cause to the generic wrap. - FileConflictError is now an OrchestrationError('conflict'), so a duplicate name classifies like every other conflict. Its 'FILE_EXISTS' discriminator had no readers and is gone; the instanceof checks elsewhere still hold. - The v2 upload handler uses v2CaughtOrchestrationError, dropping all three string matches. Also documents that bulk-archive is best-effort: unknown or already-archived ids are skipped rather than failing the call, and deletedItems is what actually happened. That asymmetry with the single-id DELETE was undocumented. * feat(api): add search, filtering, and sorting to the v2 list endpoints (#6189) * feat(api): add search, filtering, and sorting to the v2 list endpoints One convention across every v2 list, documented on lib/api/contracts/v2/shared.ts: `search` (case-insensitive substring on the resource's natural name field), `sortBy` + `sortOrder` (per-resource enum, never a free string), and enumerated resource-specific filters. Reuses the sortBy/sortOrder pair v2 logs and v2 knowledge-documents already ship rather than inventing a third dialect alongside the Logs filters and the Tables predicate grammar. Every filter and sort is pushed into SQL. GET /api/v2/files previously read the whole scope and sorted/sliced it in JS; it now goes through a new queryWorkspaceFiles that filters, orders, and bounds the page in one query. Cursors are stamped with the sort they were minted under, so replaying one under a different sort is a 400 instead of silently duplicated or skipped rows. * fix(api): validate v2 cursor key values and compare timestamps at ms precision Two review findings, fixed at the root by making a keyset key own its cursor codec instead of hand-writing a decoder per sort. Cursor key values are caller-controlled, and matching the sort stamp and key count was not enough: an unparseable timestamp or a non-numeric size reached the query as an Invalid Date or NaN and surfaced as a 500. Each key now type- checks its own value and rejects a cursor it cannot hold, which both routes render as the documented 400. Timestamp keys now order and compare on date_trunc('milliseconds', col). Postgres keeps microseconds and defaultNow() populates them, but a cursor value round-trips through a millisecond-only JS Date — comparing the raw column against the truncated value re-admitted the page's own last row, duplicating it and stalling pagination outright at a page size of one. Reachable today via workspace_files.updated_at, which insertFileMetadata leaves to defaultNow(). * feat(api): complete the v2 workflows resource with versions and CRUD (#6184) * feat(api): complete the v2 workflows resource with versions and CRUD Adds version listing/detail plus create, update, and delete to the v2 workflows surface, which previously covered only execution and deployment. - GET /api/v2/workflows/[id]/versions — cursor-paginated, newest first - GET /api/v2/workflows/[id]/versions/[version] — version + pinned state - POST /api/v2/workflows, PATCH and DELETE /api/v2/workflows/[id] All six delegate to the existing orchestration and persistence helpers; no new domain logic. * fix(api): check folder containment before lock state; reject malformed version cursors assertFolderMutable walks a folder's ancestor chain without filtering on workspace, so inspecting it before containment let a caller tell a locked folder in someone else's workspace (423) from a nonexistent one (400). Create and update now assert containment first, matching the ordering import-workflow.ts already uses. A version cursor that decodes to JSON without a numeric version filtered every row out and returned an empty page with nextCursor null, which reads as a clean end-of-list. Malformed cursors are now a 400. * refactor(api): page workflow versions in the persistence helper listWorkflowVersions read every version row and the route filtered and sliced the result in memory, so the response was bounded but the query was not. It now takes optional limit/afterVersion, turning the cursor into a real keyset query; the route asks for limit + 1 and only trims the has-more probe. Both params are optional, so the internal, v1 admin, and copilot callers are unchanged. Also restores the untouched GET handler in [id]/route.ts to its original formatting — collapsing its signature had re-indented the whole body and buried the actual additions in whitespace churn. * feat(api): expand v2 tables with stateless multipart transfers (#6188) * feat(api): expand the public v2 tables surface Adds 16 operations so a v2 caller can do what the internal surface can: rename/move/lock a table, restore it, manage saved views, run enrichment columns, look up rows, and import/export with observable job control. Extracts lib/table/orchestration/import.ts (performTableCsvImport, performCreateTableFromCsv) and lib/table/export-stream.ts from the first-party routes, then repoints those routes at them, so v1 and v2 cannot drift on what an import or export actually does. events/stream, metadata and dispatches stay internal — they are editor state, not public API. * fix(api): make v2 table PATCH all-or-nothing and name the lock in every 423 Greptile P1: PATCH applied locks, rename and move as three sequential transactions, so a folder rejected mid-request left the earlier writes persisted while the response reported failure — and the schema-changed signal was skipped, leaving open clients on stale state. Every rejectable condition now runs before the first write, and the signal fires whenever anything did land. Cursor: v2TableLockError dropped the lock kind, so async import, column run, enrichment and table mutations returned a bare LOCKED. A table has four independent locks, so the caller could not tell which to clear. * fix(api): report the lock kind on classified 423s too, not just thrown ones The previous commit named the lock only where the rejection was thrown and caught at the route boundary. Where it instead arrives as a classified `errorCode: 'locked'` outcome — delete table, delete row, update column, and the table mutations — the kind was dropped, so those 423s stayed unactionable while their neighbours improved. The orchestration results now carry `lock`, and a shared `v2TableOrchestrationError` renders both arrival paths into the same `{ code, message, details: { lock } }` body. `details` is omitted rather than sent null when the kind is unknown, so a caller branching on it sees absence instead of a phantom value. * fix(api): make async table imports observable, not just startable `POST /import-async` pointed callers at `GET /api/v2/tables/jobs` to track progress, but that endpoint filters to `type = 'export'` — imports are derived onto the table itself, one write job at a time, and exports get a separate list precisely because they are excluded from that derivation. The public Table shape omitted those derived fields, so an async import could be started and cancelled but never observed to completion, failure, or progress. That is the gap the import/export/job-control set was meant to close. Table now carries `job` — id, type, status, rowsProcessed, error, or null when idle — and the import-async docs point at the table rather than the export list. * feat(api): make v2 table PATCH state which operations landed on failure Greptile held the PR at 4/5 on the residual non-atomicity and named two acceptable resolutions: make PATCH atomic, or have the contract adopt and expose partial-success explicitly. Atomicity would mean threading one transaction through renameTable, moveTableToFolder and updateTableLocks — three shared service functions with four non-test callers including the first-party route and two copilot tools — and deferring their per-operation audits to commit time. That is a refactor of shared write paths well outside this PR. So the contract states it instead. Every rejectable condition is already pre-validated, so a failure here is a genuine fault; when one follows a successful operation the error now carries `details.applied` listing what is live. Absent when nothing applied, so its presence always means "these changes took effect despite the error". Documented on the operation. `v2ErrorForOrchestration` gained the optional `details` this needs. * fix(api): make table lock flags read-only on the public v2 surface The new PATCH /api/v2/tables/[tableId] accepted a `locks` object, gated on workspace admin plus the table-locks feature. That still lets an API key clear the guard placed there to stop it: `write` is the floor for the endpoint, and admin keys are ordinary API keys, so a lock is no longer a boundary the key cannot cross. Locks stay readable on the table resource and enforcement is unchanged (a locked verb still returns 423). Changing one is now a first-party admin action only. The v2 body is declared here rather than reusing the first-party updateTableBodySchema, which keeps its `locks` field so the UI can still toggle them. It is .strict(), so a request carrying `locks` is rejected with a 400 naming the field instead of silently succeeding without applying it. * fix(api): keep reporting applied operations when the PATCH re-read fails The composite table PATCH promises that `error.details.applied` names the operations that are live despite an error, but `applied` was scoped inside the try. A rename or move that committed and was then followed by a throw in the final re-read — or a re-read finding the table archived — returned a bare 500/404 with no details, telling the caller nothing had landed. It would then retry into a duplicate-name conflict or repeat the move. `applied` is now function-scoped so every post-write exit carries it: the 404 on a missing re-read, a thrown lock error, a classified orchestration error, and the generic 500. `v2TableLockError` gains the same `extraDetails` parameter `v2TableOrchestrationError` already had. * feat(api): add workflow group writes to the v2 tables surface v2 exposed GET /groups but none of the writes, so the public API could run an enrichment or workflow column and read its binding, but never create one. A caller could add a plain data column and trigger the machine; wiring the two together still required the UI. Adds POST/PATCH/DELETE on /api/v2/tables/[tableId]/groups. The group is the unit that fills columns — one group feeds several — so creating one creates its output columns in the same call, matching the first-party shape rather than inverting it onto the column endpoint. Four departures from the first-party body, all public-surface concerns: - group.id is optional and server-generated. The UI mints an id to render optimistically; a public caller has no such need and a client-chosen id is a collision waiting to happen. - outputColumns[].workflowGroupId is dropped from the body and stamped from the resolved group, so it cannot disagree with it. - autoRun defaults to false. First-party defaults true so a UI add fills cells immediately; here it would make one POST fan out a metered run across every existing row. - A group naming neither a workflowId (type manual) nor an enrichmentId (type enrichment) is a 400 rather than a half-specified group the route has to guess about. Also rejects an outputColumns entry no group output feeds — the two arrays are joined by column name, and the first-party client builds both from one picker so it cannot desync, but a public caller can. Workspace containment on workflowId is asserted before it is persisted, on create and on any update that re-points the group; without it a table becomes a way to invoke workflows the key cannot otherwise reach. * improvement(api): make v2 table import and export async-only Drops the three synchronous entry points: POST /tables/[tableId]/import, POST /tables/import-csv, and GET /tables/[tableId]/export. Sync import tied a write to the lifetime of an HTTP request. The body *was* the data, so it carried a 10 MB cap that Next silently truncates past — a partial import reporting success. It also had no job, so a timeout mid-write left rows in place with nothing to poll and nothing to cancel. The async path reads the file from storage instead: upload via POST /api/v2/files for a key, start with POST /import-async, watch GET /tables/[tableId] -> job, stop with POST /job/cancel. Sync export carried no such hazard, but one shape per operation beats two: with both removed the surface has exactly one way to move a table in or out, and the CLI wraps the extra calls. This also removes the last multipart handling in v2 tables. Those were the only routes bypassing parseRequest — form fields were parsed by hand against separate form schemas, outside the contract system every other v2 write goes through. Create-a-table-from-CSV is now two calls: POST /tables, then /import-async with createColumns. csvImportModeSchema is append|replace, so there is no single-call create. Route baseline 1064 -> 1061. * docs(api): correct the import-async note about upload size limits The docstring claimed there is no synchronous upload endpoint and so no request-body size cliff. Both are wrong: POST /api/v2/files is a synchronous multipart upload with a 100 MB cap, and it is the only v2 upload path (presigned is deliberately absent). What async-only actually bought: the cap went 10 MB -> 100 MB, it fails on an explicit size check and a bounded body read rather than a proxy cap that silently truncates, authorization completes before any body is buffered, and the table write is a job that can be watched and cancelled. * feat(api): unify file and table transfers * improvement(api): make multipart transfers stateless * fix(api): make table import completion retries idempotent * feat(v2-tables): paginate the table list `GET /api/v2/tables` returned every table in the workspace in one response — it used the cursor envelope but hardcoded `nextCursor: null`, and had no `limit`. That was defensible when tables were only created through the UI; `POST /api/v2/tables` is public now, so a script can create them in bulk and the list has no way to ask for less. Adds `queryTables` alongside `listTables` rather than changing it, so the internal callers that genuinely want the whole scope are untouched — the same split `queryWorkspaceFiles` / `listWorkspaceFiles` already uses. Filter, order and slice all run in the query, so a `search` never costs a full-workspace read. A cursor whose values don't bind raises a validation error instead of being coerced to "no filter", which would have silently served page 1 under a resumed cursor. The keyset closes on `id` so a page boundary inside a run of equal names or timestamps stays stable. The shared `LimitQuery` doc component said "Maximum rows to return"; it now serves the table list too, so the wording is resource-neutral. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(api): add multipart knowledge document uploads * fix(api): keep usage admission at knowledge upload session creation * feat(knowledge): wire knowledge base uploads to multipart sessions * fix(knowledge): refuse to abort an upload once a document is bound * fix(uploads): prevent multipart cleanup races * Unify file creation and signed upload sessions (#6264) * feat(uploads): unify signed upload sessions * fix(uploads): preserve attachment storage semantics * feat(files): add authored file creation * fix(uploads): omit hoisted S3 metadata headers * feat(api): add file metadata endpoint * improvement(api): scope folders to resource paths (#6284) * improvement(api): scope folders to resource paths * fix(files): serialize folder resolution with uploads * fix(files): release folder lock before upload setup * fix(api): normalize folder paths and unblock resource mutations * fix(api): make resource cleanup and metadata consistent * improvement(uploads): persist multipart sessions in postgres * fix(db): store table row trigger timestamps in UTC * improvement(api): default folder deletion to non-recursive * fix(billing): unify chat usage source * improvement(logs): expose trace spans on log detail * fix(logs): parse list trace spans * improvement(api): replace workflow jobs with execution resources (#6294) * improvement(api): replace workflow jobs with execution resources * fix(api): preserve legacy jobs while preferring v2 executions * fix(api): make execution polling resume-aware * fix(ui): hide async examples for public workflows * fix(api): bridge resume queue visibility lag * feat(api): add v2 workflow resume endpoint * fix(api): project pending resume attempts * fix(api): prefer terminal logs over stale resumes * improvement(api): unify v2 resource query layers (#6319) * improvement(api): unify v2 resource query layers * fix(api): address v2 review findings * fix(api): preserve cancelled queue status * fix(api): guard cancelled job transitions * fix(api): close v2 resume and log gaps * feat(api): rename v2 executions to runs * feat(api): split credentials and secrets * feat(api): add workspace metadata and email attribution * improvement(api): consolidate public v2 route handling * improvement(files): centralize operations across APIs and Copilot (#6392) * improvement(files): unify rename authorization * chore(skills): add file operation migration guide * improvement(files): consolidate file operation authorization * improvement(files): extract shared operation foundation * improvement(api): simplify internal route declarations * improvement(files): centralize application authorization * refactor(api): share workspace file name validation * refactor(files): centralize copilot application calls * docs(skills): generalize application operation migration * improvement(api): centralize remaining v2 resource operations (#6412) * improvement(api): centralize v2 resource operations * fix(api): preserve custom tool conflict errors * improvement(api): migrate policy-sensitive v2 reads (#6410) * improvement(workflows): centralize v2 application operations (#6411) * refactor(api): migrate v2 knowledge operations (#6413) * refactor(api): migrate v2 knowledge operations * fix(knowledge): fail upload completion on dispatch errors * fix(knowledge): preserve upload retry and VFS errors * improvement(tables): centralize v2 application operations (#6414) * improvement(tables): centralize v2 application operations * fix(tables): preserve run validation and signals * feat(auth): add scoped internal executor delegation (#6459) * feat(auth): add scoped internal executor delegation * fix(auth): derive delegation lifetime from one timestamp * Include share status in file metadata * feat(auth): centralize delegated identity policy (#6462) * improvement(copilot): consolidate application adapters (#6450) * improvement(api): harden application route boundaries (#6451) * improvement(api): harden application route boundaries * fix(folders): reject creates at workspace cap * fix(knowledge): enforce trusted workspace scope (#6452) * fix(knowledge): enforce trusted workspace scope * refactor(knowledge): declare v2 body lifecycle * finish knowledge application migration * refactor(knowledge): compose copilot batch commands * fix(knowledge): parse connector query flags * fix(knowledge): finalize partial batch effects * fix(knowledge): align merged application boundaries * fix(knowledge): close application boundary review gaps * style(knowledge): satisfy branch biome checks * fix(knowledge): page connector documents in editor * refactor: enforce Copilot table application boundary (#6453) * refactor: enforce copilot table application boundary * fix(tables): finish application boundary migration * fix(tables): restore scoped copilot imports * fix(tables): compose copilot commands atomically * fix(tables): preserve workflow group scheduling * fix(tables): complete fixed copilot composition * fix(tables): reject enrichment output mutation * fix(tables): complete authorized application boundary * fix(workflows): migrate Copilot application boundary (#6455) * fix(workflows): migrate Copilot application boundary * fix(workflows): finish delegated application migration * fix(workflows): encode VFS folder aliases * fix(workflows): close application composition gaps * fix(workflows): preserve VFS validation errors * fix(workflows): complete application boundary migration * test(workflows): format canonical binding coverage * fix(workflows): scope executor metadata reads * fix(workflows): bind executor metadata targets * improvement(skills): align application operation guidance (#6532) * feat(api): expose v2 resource owners * fix(api): distinguish visible resource authorization failures (#6537) * feat(api): generate v2 OpenAPI from contracts (#6509) * feat(api): generate v2 OpenAPI from contracts * fix(api): preserve string boolean wire defaults * fix(api): document file download headers * fix(docs): use TypeScript CLI with Next.js * fix(docs): avoid client-rendered theme script * fix(api): document departed audit default * feat(api): replace legacy core docs with v2 * feat(api): generate v2 OpenAPI from contracts * feat(api): refine generated v2 OpenAPI docs * fix(docs): align localized v2 execution examples * fix(ci): restore Helm diff and sync audit mock * fix CI regressions after staging merge --------- Co-authored-by: Waleed <walif6@gmail.com> Co-authored-by: Theodore Li <theodoreqili@gmail.com> Co-authored-by: Siddharth Ganesan <33737564+Sg312@users.noreply.github.com> Co-authored-by: Theodore Li <theo@sim.ai> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-08-11 05:45:25 -04:00
Waleed	ee0157df4b	feat(tables): add currency column type on a new column-type registry (#6106) ... * feat(tables): add currency column type on a new column-type registry Adds a `currency` column type, and consolidates the per-type knowledge it would otherwise have been scattered across. **Currency.** Stores a plain number and carries an ISO 4217 `currencyCode` as display metadata. That split is what keeps it cheap: filtering, sorting, uniqueness and CSV export all reuse the numeric paths unchanged, changing a column's currency rewrites no rows, and the public row output stays a number rather than a locale-formatted string consumers would have to reparse. Input accepts the shapes an amount actually arrives in — `$1,234.56`, `1 234,56 €`, `(12.00)` — so pastes, CSV imports and tool writes land as numbers instead of being nulled. **The registry.** Adding this type initially required edits in ~40 places: 32 switch arms under `lib/table`, ~26 UI branches, two hand-maintained icon maps, and a coercion implementation duplicated four times. Every one of those failed silently when missed — a missing `jsonbCastForType` arm compares numbers as text; a missing compatibility arm blocks all conversions. `lib/table/column-types/` now holds one file per type carrying its label, icon, badge colour, storage cast, filter operators, coercion, validation, compatibility and formatting. `Record<ColumnType, …>` on both registries is the completeness gate: adding a type to the union is a compile error naming exactly the two files to fill in, and the interface then requires every field. The 32 switch arms are down to 3. Two duplicates collapse as a consequence: - The client no longer mirrors the server's select id-resolution. Those helpers lived in `validation.ts`, which imports drizzle, so anything reaching them became server-only and the grid hand-rolled its own copy. Extracting them to `select-options.ts` lets both sides share one implementation, so the optimistic cache can no longer disagree with what gets persisted. - The two icon maps become one registry read. It also fixes a live inconsistency it surfaced: currency got a numeric keypad in the grid's inline editor but a plain text field in the row modal. Behaviour-neutral by construction: all 1046 tests in the touched areas pass unchanged, with no test edits. * test(tables): guard the column-type registry's invariants Property tests for the registry itself rather than any one type: entries key by their own id, COLUMN_TYPES stays derived, an unknown type degrades to string instead of throwing, only opaque-id types restrict filter operators, only configuration-free types are CSV-inferable, and every type that can reject a draft has a message to show. Plus the metadata-ownership matrix, which pins the generic ownership check to the same answers the hardcoded per-type rules gave. These target the registry's silent-failure class — a wrong jsonbCast or a stray operator whitelist used to be invisible until a filter failed in SQL. Both are verified to fail under mutation. * fix(tables): read exponent-form amounts and reject bad currency PATCHes up front Two P1s from review. Scientific notation lost magnitude. `String()` emits exponent form past 1e21, so a stored amount round-trips through the editor as `1e+21` — and the sanitizer treated the `e` as decoration to strip, reading it back as 121. An untouched cell silently lost 19 orders of magnitude on its next edit. Exponent form is now taken at face value, but only when the string is wholly a numeric literal once symbols are removed, so `12 EUR` (whose `E` survives the strip) still parses through the separator path. A failed currency PATCH left a partial rename. `renameColumn` commits in its own transaction before the currency write, so a `currencyCode` the service would reject — an unsupported code, or any code on a non-currency column — errored only after the rename had stuck. Both are now caught before the first write, matching the guard the route already applies to unique-on-select for exactly this reason. * refactor(tables): finish the registry migration and drop the dead config Audit pass over every consumer, closing the gaps the first cut left. Functional gap: the copilot agent had no currency support at all — it could create a currency column with no code and could never re-denominate one. `add_column` and `update_column` now accept `currencyCode`, with the same up-front validation and the same code-only routing as the HTTP routes. Config that consumers were still restating, now read from the registry: - `supportsUnique` replaces the unique-on-select guard stated in three places (service, both column routes, the copilot tool). - `editor === 'toggle'` replaces seven `type === 'boolean'` checks in the grid and expanded popover, all of which meant the same thing. - `defaultMetadata` replaces the per-type stamping in `addTableColumn` and `updateColumnType`. - `sampleValue` replaces the per-type example values in the LLM prompt scaffolding. - `storesOpaqueIds` replaces the select filter in the find-row matcher. Dead config removed: `getTypeBadgeVariant` had zero callers (already dead on staging), and it was the only reader of `badgeVariant` — so the field, its union, and all seven values went with it. `inferFromCsv` was read by nothing but a comment; CSV inference is an ordered heuristic a boolean cannot express, so it is gone too and `InferredCsvColumnType` is no longer exported. Fixes a latent crash found on the way: unique-constraint checking normalized a cell keyed on its RUNTIME type but reconstructed it keyed on the column's DECLARED type, so a unique `date` column stored a bare `2024-01-01` and then threw `SyntaxError` parsing it back. Both directions now go through JSON unconditionally. Pre-existing, unrelated to currency. Adds the `/add-column-type` skill and a Tables section in CLAUDE.md/AGENTS.md pointing at it, so the next type is one file plus two registry entries. * fix(tables): run the column PATCH guards ahead of the rename, not after it Greptile was right and my previous reply was wrong. The guards were added in the right shape but the wrong place — below `renameColumn`, which is the first write and commits in its own transaction. A PATCH combining a rename with an invalid currency therefore still committed the rename and then returned 400, exactly the counterexample reported. Moved the column lookup and all three pre-flight guards above every write. This also closes the same latent hole for the pre-existing unique-on-select guard, which sat in the same position. Adds route tests that assert `renameColumn` was never called on each rejection path, and that a valid combined rename + currency change still targets the new name. Verified to fail against the previous ordering. * fix(tables): make the retype gate and the write path share one parser A simplify pass over the registry found two real defects and several places the abstraction was being worked around. Silent data loss on conversion. `isCompatibleWith` was hand-written per type and had already drifted from `coerce`, despite the interface promising they could not: `boolean` accepted '1'/'0'/0/1 in the gate but only 'true'/'false' in the write path, so converting a column holding "1" reported zero incompatible rows and then nulled every one of them. `date` drifted the other way. `isCompatibleWith` is now optional and defaults to `coerce(...).ok`, so the two are the same code; only `select` overrides, because its rules are about the column (cleared-vs-required, cardinality) not the value. `isColumnType` used `in`, which matches inherited keys — `isColumnType('toString')` was true and `columnTypeById('toString')` returned `Function.prototype.toString`, which the validator would then call `.validateDefinition()` on. Now `Object.hasOwn`. `defaultMetadata` only ran on the currency arm of a retype, so a future type would get its defaults on create but silently not on conversion. It now runs for every non-select target, carrying forward only metadata the TARGET type declares it owns — a currency→text conversion no longer strands a currencyCode. The index doc claimed the registry is kept out of the `@/lib/table` barrel so 44 server modules don't pull `@sim/emcn/icons`. That was false: `constants.ts` re-exported `COLUMN_TYPES` from the icon-carrying `registry.ts`, and the barrel re-exports `constants`. `COLUMN_TYPES` now lives in the icon-free `types.ts`; verified with an import tracer that both are icon-free again. Also: 5 no-op `validateDefinition`s and 4 duplicated formatters collapsed into registry defaults; `CURRENCY_OPTIONS` was an eager module-load IIFE costing ~8ms of ICU work on every table API route for a list only the config sidebar reads, now built on first call; and the skill's validation grep claimed 'should return nothing' when it returns 8 legitimate hits — it now explains how to tell a leak from a genuine special case. * fix(tables): reject a non-leading sign so dates don't parse as amounts Found by Cursor Bugbot. `parseCurrencyInput` dropped every `-` as decoration, so an ISO date's hyphens vanished and its digit groups joined: `2024-01-01` read as 20240101. With the gate now sharing the write path's parser, a date → currency conversion reported zero incompatible rows and silently turned every cell into a huge number. A sign is only meaningful at the front; an interior one means the string is not a single amount. Leading signs, accounting parentheses, symbols, ISO codes, grouping separators, and exponent form all still parse — covered by the existing cases plus new ones, verified to fail without the fix. * fix(tables): use getErrorMessage in the columns route test mock `check:utils` bans the inline `e instanceof Error ? e.message : fallback` form; the mock for `rootErrorMessage` used it. * fix(tables): rename the column last so a failed write leaves it untouched Greptile's remaining concern: the pre-flight guards read a schema snapshot, so a column-type change landing concurrently can still make a later write fail — and with the rename running first, that failure returned an error with the rename already committed. Guards cannot close that window; each write is its own locked transaction and only the write itself sees the authoritative state. Ordering can. The rename is the one write that is purely cosmetic, so it now runs last: a failed typed write leaves the column entirely untouched, and a failed rename leaves the typed change applied under the old name — the recoverable half. The typed writes target the column's current name, since no rename has happened yet. Tests cover both directions: a typed write rejected mid-flight must not rename, and a successful one must rename strictly after. Verified to fail under the previous ordering. * fix(tables): write back coerced values on every conversion Round 4 findings, all real. A conversion is allowed exactly when the target type's `coerce` accepts the value — and `coerce` frequently TRANSFORMS it. Only `select` and `currency` wrote the transformed value back, so a conversion to any other transforming type left the cell holding its old bytes under the new type. Converting a number column to `date` accepted epoch values, stored them unchanged, and then `(data->>'col')::timestamptz` failed on EVERY query against that column. I opened this myself by defaulting `isCompatibleWith` to `coerce(...).ok`. Fixed at the class rather than the instance: the compatibility scan now records whatever `coerce` produced whenever it differs from what is stored, and one generic write-back applies it. That subsumes the currency-specific migration entirely, so it and its helpers are gone. `select` keeps its own id↔name migrations, which are not coerce-expressible in the outbound direction. The post-conversion column definition is built once, before the scan, so the coercion reads the same metadata the stored value is later validated against. Exponent parsing was ambiguous when followed by text: `1e5 EUR` read as 15. An `e` with a digit on both sides is an exponent marker, so if the string is not a clean numeric literal it is refused rather than guessed — the digit on both sides is what keeps the `E` inside `12 EUR` parsing normally. A failed rename could still leave a typed change committed. The one rename failure a caller can cause — a name already taken — is now rejected up front, leaving only the concurrent-collision race, which no pre-flight check can close without spanning all writes in one transaction. * fix(tables): stop a blank cell blocking an optional type conversion Found by Cursor Bugbot. `''` is incompatible with every numeric type, and the compatibility scan counted it as a hard blocker regardless of whether the target was optional — so a text column with a single empty cell could not be converted to a number at all, and the error said 'to a required ...' either way. An unreadable-but-empty cell is not a conversion failure. The write path already turns an unreadable value into null on an optional column, so the conversion now does the same and records null for it. A required target still reports it, which the existing guard above already does with the message that actually fits. Also pins the two intentional divergences from the pre-registry behavior. A differential run of the registry against the pre-refactor implementations (55 values x 7 column shapes) found ZERO coercion differences and exactly two compatibility differences, both deliberate: boolean now rejects the '1'/'0' conversions the old gate accepted and then nulled, and date now accepts the epoch numbers its write path always accepted. Tests pin both so neither can be silently reverted or widened. * fix(tables): refuse conversions that would invent or destroy values Final adversarial scan found two data-corrupting conversions, both opened by defaulting the retype gate to the write path's parser. number → date destroyed every value. `date.coerce` reads a number as epoch milliseconds, which is right for one deliberate write and catastrophic applied to a whole column: 1, 5, 42 became three timestamps in January 1970, and a Unix-seconds column landed in 1970 rather than the year it meant. Irreversible. `date` now overrides the gate to reject numbers, restoring the pre-refactor behavior, and the contract states the rule the override obeys: a gate may be STRICTER than `coerce`, never looser. Stricter refuses a bulk conversion while single writes still work; looser is the direction that corrupts. string → currency invented values. The parser stripped every non-digit and joined what was left, so `01/02/2024` read as 1022024, `Room 101` as 101, and `0.1.2` as 12 — a column of SKUs or phone numbers converted with zero reported incompatibilities. What remains after removing symbols, spacing and an ISO code must now be only digits and separators, and grouping must be well-formed (a first group of 1-3 digits, the rest exactly 3). Every legitimate form still parses, including all the locale variants. Also generifies the last three metadata leaks: `buildConvertedColumn` strips and carries back by iterating the key list rather than naming keys (naming them meant a future type's metadata rode onto a target that rejects it, failing that column's validation on every later write), `normalizeColumn` forwards metadata through a shared `typeMetadataOf`, and `filterOperatorsFor` moved onto the definition — it was a per-type branch inside the registry's own accessor, the one thing the registry exists to forbid. Skill corrected: it claimed COLUMN_TYPES derives from the registry (backwards), promised exactly two compile errors (four once a type owns metadata), used a grep that missed half the real branches, and never mentioned `import.ts`'s second coercion path, whose silent default arm is the costliest miss available. Differential re-run vs the pre-refactor implementations: 0 coercion differences, 1 intentional compatibility difference (boolean no longer accepts the 0/1 conversions the old gate accepted and then nulled). * fix(tables): let the row modal accept formatted amounts again Found by Cursor Bugbot. I unified the row modal's input type with the grid's `inputMode` last round, but in the wrong direction: mapping `inputMode: 'decimal'` to `<input type="number">` made the modal reject $1,234.56, 1.234,56 and (12.00) — the exact formats `parseCurrencyInput` exists to accept, and which the grid's inline editor takes fine. A native number input and a numeric keypad are different things. Types whose parser accepts formatted text now say so, and get a text field with `inputMode='decimal'` — the shape the grid already uses. A plain number keeps the native input, its spinner, and its validation. * fix(tables): fold a rename into the write it accompanies Closes the last partial-update window, properly rather than by pre-checking around it. A rename is metadata-only — `renameColumn`'s own comment says so: rows, metadata, and workflow-group refs all key on the stable column id, so it is a pure schema write. Nothing forced it to be its own transaction. Running it separately is what created the window: whichever half committed first survived a failure in the other, and no pre-flight guard can close a concurrent collision because only the write itself sees authoritative state. The four column writes now accept an optional `newName` and apply it through one shared `applyPendingRename`, which validates the name shape and checks the collision against the very schema snapshot that write is landing in. A combined request rides the rename on whichever write runs last, so both halves commit together or neither does — a concurrent claim on the name now aborts the whole transaction instead of leaving the other change applied. The routes also address every write by the column's stable id rather than its name, so folding a rename into one write cannot break the next one's lookup. A rename with nothing to ride on still runs standalone. What remains partial is a type write followed by a failing constraints write — two independently locked transactions, pre-existing, and untouched by this PR. * fix(tables): migrate scalar cells when converting a column to select Found by Cursor Bugbot. `resolveSelectOptionId` stringifies a number or boolean before matching, so a `number` column whose values equal option NAMES passes the compatibility gate — but `migrateCellsToSelectIds` only rewrote JSONB `string` and `array` cells. Those cells stayed raw numbers inside a select column, where they render as nothing and fail option membership on the next write. `data->>key` yields the text form for every scalar, so the existing lookup already worked; the predicate was simply too narrow. Widened to cover `number` and `boolean`. The outbound migration is unchanged — cells leaving a select column are option ids, always strings or arrays. Pre-existing on staging (both the resolver's scalar handling and the migration SQL predate this branch), but it lives in a file this PR creates. Tests pin the resolver behavior the predicate depends on, so narrowing either one without the other now fails. * fix(tables): validate a retype's unique against the values it writes Validated the last partial-update seam with a focused investigation rather than assuming. The answer was split. `required` is already safe: `updateColumnType` runs the same `countEmptyCells` against the constraint the request is about to set, which is why that check exists. `unique` was not, and the reachable case commits the unrecoverable half. A text column holding "5" and "5.0", PATCHed with {type: number, unique: true}: the conversion succeeds and coerces both to 5, then the separate constraint write finds duplicates and 400s — with the column already numeric and "5.0" irreversibly rewritten. A pre-scan of the raw text finds nothing; the conversion is what manufactures the duplicate. The retype now carries `unique` and checks it after the write-back, against the values it just wrote. Constraint changes on a workflow-output column were the same shape — rejected by the constraint write, after a type change had committed. Now rejected in the route's pre-flight block, before any write. The duplicate scan is extracted and shared between both paths for the same reason `countEmptyCells` is: two copies of one rule is the drift that produced the original required-check bug. Deliberately NOT merging `updateColumnType` and `updateColumnConstraints`. They assert different lock levels (destructive vs schema-only) and only the retype needs the full row scan, so merging would either force a constraints-only toggle to materialize every row or reintroduce the branching it was meant to remove. With both reachable failures pre-validated, what remains at the seam is concurrent races no in-process check can close. * fix(tables): don't drop a rename when the write it rides on no-ops Found by Cursor Bugbot — a bug I introduced folding the rename in. `updateColumnCurrency` returns early when the code is unchanged, and that return sat ahead of the rename, so PATCH {name, currencyCode} with the column's current code answered 200 with the rename silently discarded. Both early returns now treat a pending rename as work: the currency path only no-ops when the code is unchanged AND no rename is riding along, and the retype path applies a rename-only write when the type is unchanged. `applyPendingRename` signals "nothing to do" by returning the same reference, which is what lets both detect it cleanly. Also extracts `persistColumns` — five sites were repeating the same schema-write-and-return. * fix(tables): make a combined column PATCH a single transaction Finishes the fold-in rather than pre-validating around the seam. A retype now APPLIES the constraints it already validates against — it checks empty cells for `required` and post-conversion duplicates for `unique`, so it was doing the work without persisting the result — and the route skips the separate constraint write when the type changed. A request combining a rename, a retype and constraint changes is now one locked transaction: no half of it can commit while another fails. The separate constraint write remains for requests that do not change type, which is the only case that still needs it. Deliberately still NOT merging the two service functions. They assert different lock levels (destructive vs schema-only) and only the retype needs the full row scan into memory, so a merged function would force a constraints-only toggle to materialize every row or reintroduce the branching it was meant to remove. Folding the payload in gets atomicity without either cost. * fix(tables): reject a flattened list as an amount; fold constraints into every typed write Two findings from round 11. Multi-select converted to nonsense amounts. `selectValueForConversion` flattens a multi cell to its comma-joined option names, and the parser read that as a formatted number: options 12 and 34 became 12.34, and 100 and 200 became 100200. No real amount puts whitespace after a separator, but a delimited list does — so a separator followed by whitespace is now refused. Every legitimate form still parses, including space-grouped locales. Combined options-or-currency + constraints could still commit partially. Those two writes now carry constraints the same way the retype does, through one shared `applyConstraints` that validates (workflow-output, empty cells for required, supportsUnique and duplicates for unique) and applies them. The separate constraint write now runs only when no typed write does. Three copies of those rules is the drift that produced the original required-check bug, so they live in one place. * fix(tables): validate constraints after the migrations that rewrite cells Self-caught while reviewing my own previous commit, which introduced both. `updateColumnOptions` ran the shared `applyConstraints` BEFORE its cell migrations. Those migrations rewrite stored values — a single<->multi toggle changes the shape, removing an option clears cells — so a `unique` scan read the pre-migration values, passed, and the rewrite could then produce the duplicates the scan was meant to prevent. Moved to after the migrations, which is where `updateColumnType` already had it. The same commit also left the options path running `required`'s empty-cell check twice: once in the shared helper and once in its original inline block, whose comment still described a separate constraint write that no longer runs. Removed the duplicate — one query, one rule, which is the whole point of the shared helper. Also routes the options path through `persistColumns` like the others. * fix(tables): stop inventing amounts from identifiers; fix the copilot retype Adversarial pass over the final state, seven real findings. Two destroyed data. The copilot `update_column` still used the two-transaction pattern the HTTP routes were fixed for: `unique` was never forwarded to the typed write, so a retype+unique committed the conversion and then failed the constraint — the same irrecoverable half. It now rides the typed write, and the separate constraint write only runs when no typed write did. And the parser's three-letter strip removed ANY three letters, not an ISO code: `SKU400` parsed as 400, `ABC1234` as 1234. Converting a column of part numbers to currency rewrote every cell with an invented value — while the comment two lines above claimed a SKU was exactly what it prevented. The rule is now that a letter touching a digit means identifier, not amount; a currency marker is always separated by a space or a symbol. That same change fixed a class the review surfaced: the pinned currencies could not parse their own conventional notation. `R$ 1.234,56`, `1 234,56 kr`, `1234,56 zł`, `CHF 1’234.56` and Indian lakh grouping (`₹12,34,567.89`) all work now — these are what Intl emits, so a paste from a spreadsheet was being rejected. `updateColumnConstraints` was a fourth copy of the constraint rules the shared helper exists to unify, and had already drifted: it hardcoded `type === 'select'` where the helper asks the registry, so a future type declaring `supportsUnique: false` would have been ignored on that path. It now uses the helper. `updateColumnType`'s unchanged-type early return silently discarded every field except the rename. Callers gate on the type changing, but from a read taken before the lock — so a concurrent change could land there with real work pending and answer success. It now throws. Also: `UpdateColumnCurrencyData` was missing `required`, which only compiled because the routes pass it through a spread; a missing column returns 404 instead of a 400 reading "of type undefined"; and the comments describing the old two-transaction architecture are gone. Verified NOT a bug: CSV export of a currency column writes the raw number, so export/import round-trips losslessly. * fix(tables): read the negative and RTL forms Intl actually emits An Intl sweep across 24 locales found two forms the parser rejected, both from an ordinary spreadsheet paste. `Intl` emits U+2212 MINUS SIGN rather than the ASCII hyphen for negatives in several locales, so `−12,50 kr` read as null instead of -12.5. And it wraps RTL-locale output in invisible bidi control marks, so `‏1,234.56 ‏₪` carried characters that are not part of the amount. Both are now normalized away. 24 locales x 6 amounts now round-trip, up from 99/100 when the sweep started — and the test generates them from `Intl` rather than listing them by hand, so a parser change cannot quietly regress a locale nobody remembered to write down. Locales that format with their own numeral systems (Arabic-Indic) are still rejected, and now say so in the docstring. That is a safe failure — null rather than a wrong value — and supporting them is a wider decision than this type, since it would also touch `number`, display, and sorting.	2026-07-30 17:21:17 -07:00
Vikhyath Mondreti	91f9dfdaec	improvement(governance): derived access (#5134) ... * improvement(governance): org-ws-credential roles clarity * revert isHosted * improvement(credentials): code cleanup * address comments * make kb cascade delete on user hard delete * revert env flags * chore(db): drop local 0242 migration to regenerate after merging staging Our 0242 collides with staging's 0242. Remove it (and its snapshot + journal entry) so the KB-cascade migration can be regenerated with the correct number on top of the merged staging migrations. * chore(db): regenerate kb→workspace cascade migration as 0243 Regenerated via drizzle-kit generate on top of the merged staging migrations (staging took 0242). Re-applied the safety edits: NOT VALID + separate VALIDATE on the FK re-add, and the -- migration-safe note on the DROP. check:migrations passes. * improve copy * update docs	2026-06-19 12:47:09 -07:00
Emir Karabeg	192f77ba02	improvement(emcn): consolidate chip chrome, enforce ChipModalField, paint real chrome in loading fallbacks (#4935) ... * improvement(emcn): consolidate chip chrome, enforce ChipModalField, paint real chrome in loading fallbacks - Move chip chrome single-source to chip/chip-chrome.ts (surface, typography, and new content tokens); delete chip-input/chip-field-chrome.ts - Rework Chip variants: implicit default replaces ghost, filled reserved for chip fields/triggers and removed from Chip's public API; add ChipChevron - Migrate every labeled modal body field to ChipModalField across knowledge, settings, tables, files, deploy, sidebar, and ee modals - Replace skeleton loading.tsx files with ResourceChromeFallback that paints the page's real header, actions, search/filter/sort chips, and column headers - Rename resource-options-bar to resource-options; simplify resource.tsx and resource-header - Delete legacy Breadcrumb, Callout, and FormField components - Add shared connector-config-fields (knowledge) and ee InfoNote; add useTagUsageQuery; make useInlineRename save async with isSaving - Update AGENTS.md/CLAUDE.md and emcn/styling rules (with .cursor/.agents mirrors) * fix(rename): make inline rename await mutateAsync so isSaving disables the field; recover edits on failure - useInlineRename: catch onSave rejection — log, restore the original name, keep the edit session open, and re-arm doneRef (mirrors useItemRename) - switch rename call sites (files, tables, table header, knowledge base, document) from mutate() to mutateAsync() so isSaving spans the request - table-grid column rename stays fire-and-forget by design (optimistic local update + undo entry) * fix(rename): type onSave as void | Promise<unknown> so fire-and-forget call sites pass the build table-grid's optimistic column rename is a block-bodied callback returning void, which is only assignable when the declared return type is literally void — 'undefined | Promise<unknown>' rejected it and failed the Next.js type check in CI.	2026-06-09 20:04:45 -07:00
10 people	0075ab9cf6	improvement(platform): remove tour, simplify sidebar/header, drop loading skeletons (#4354) ... * improvement(platform): workspace UI/UX overhaul + integrations catalog Rework the workspace around the AI-workspace model: a Mothership home, a top-level Skills route, connected-credential and integration-detail pages, and a polished sidebar/settings surface. Replace the notifications store with a unified toast system (provider-level dismiss/pause, countdown ring). Integrations & catalog: - Add a BlockMeta layer (tags + catalog templates) scoped to catalog-visible integrations; every catalog integration carries >=7 grounded templates. - Rework the taxonomy: each block declares category tools|blocks|triggers. 3rd-party services are 'tools'; first-party primitives (postgres, mysql, knowledge, file, search, stt/tts, image/video generators, thinking, etc.) are 'blocks'. Versioned blocks follow the upgrade paradigm (old hidden, latest in toolbar/docs). - Generate integrations.json + tool docs canonically from block configs. Architecture & cleanup: - Consolidate block data extraction behind a single latest-version strategy (getCanonicalBlocksByCategory; version-consistent getBlockMeta). - Unify version-suffix handling in @sim/utils/string (stripVersionSuffix / isVersionedType, with tests); registry, generate-docs, tools/utils, and integrations all route through it. - Repair latent broken barrels, remove dead code, fix BlockMeta-related type errors and 5 broken docs links. Behavior-preserving for block execution and the toolbar's tool/block listing. * refactor(platform): remove forms, templates, and creators features Remove three standalone features and their supporting code: - Forms: form-deployment pages, API routes, execution path, and docs. - Templates: the template gallery (landing + workspace) and template APIs. - Creators: creator-profile routes and contracts. Add a super-user permissions module (lib/permissions/super-user) and an organizations API contract; update the audit/db/testing packages, billing, and the session/theme providers accordingly. * test(workflows): update archiveWorkflow update count after forms removal The forms feature was removed, dropping the form-table update from archiveWorkflow. Update the stale assertion from 8 to 7 tx.update calls. * upgrade * improvement(knowledge): polish tag filter dropdowns (#4816) * improvement(logs): object storage backed tracespans (#4787) * improvement(logs): obj storage backed tracespans * fix storage write context * fix tests * address comments * address comments * chore(db): remove migration 0219 to regenerate after staging merge Drops the 0219_robust_shard SQL, its snapshot, and the journal entry so the trace-spans/cost schema migration can be regenerated on top of the latest staging migration chain (avoids a number collision with staging's migrations). Co-authored-by: Cursor <cursoragent@cursor.com> * improvement(billing): accurate per-member usage via shared ledger helper Per-member/per-user usage in the org-member routes now adds the usage_log ledger to the currentPeriodCost baseline (which is no longer incremented), via a shared getOrgMemberLedgerByUser helper to avoid repeating the subscription→period→ledger lookup across the admin and member-facing routes. Co-authored-by: Cursor <cursoragent@cursor.com> * regen migrations * update migration * address comments * more code cleanup * incorrect type cast --------- Co-authored-by: Cursor <cursoragent@cursor.com> * improvement(providers): harden OpenAI-compatible providers + add tests (#4796) * improvement(providers): harden OpenAI-compatible providers + add tests * fix(vllm): let tool-loop errors propagate instead of returning silent partial success * fix(litellm): force tool_choice 'none' on final structured-output call The deferred final call used tool_choice 'auto', so the model could emit another tool_calls round instead of the structured answer, leaving content stale. Use 'none' (matching vLLM/Fireworks) on both the streaming and non-streaming final calls so the model must return the structured response. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(providers/ollama): drop tools from post-tool streaming call Ollama ignores tool_choice (not in its supported fields), so vLLM/Fireworks' tool_choice:'none' guard is a no-op here. Omit tools from the final streaming payload instead so the summarization turn can't emit dropped tool calls. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(litellm): spread payload into deferred final call so reasoning_effort carries over The non-streaming deferred finalPayload hand-picked fields and dropped reasoning_effort (and any future payload field), diverging from the streaming path which spreads ...payload. Spread payload here too for consistency. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(providers/ollama): restore enrichment TSDoc block Keeps parity with sibling Chat Completions providers (cerebras/mistral/xai). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(fireworks): restore TSDoc on utils helpers Restore the TSDoc blocks on supportsNativeStructuredOutputs, createReadableStreamFromOpenAIStream, and checkForForcedToolUsage — TSDoc is the codebase documentation standard and should not have been stripped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(litellm): remove inline rationale comments (codebase uses TSDoc) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(providers/ollama): drop orphaned enrichment TSDoc The block documented a function that now lives in trace-enrichment.ts, so it documents nothing in this file. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * chore(copilot): deprecate mcp server (#4797) * chore(copilot): deprecate mcp * update error codes * deprecate copilot api v1 route * feat(integrations): hosted API keys for Findymail, Prospeo, and Wiza (#4777) * feat(integrations): hosted API keys for Findymail, Prospeo, and Wiza Add hosted-key support across all credit-consuming Findymail, Prospeo, and Wiza operations so Sim provides the key when a workspace has not brought its own. Register the three BYOK providers, consolidate Wiza's two-step reveal into a single polling wiza_individual_reveal op, and hide the API key field on hosted Sim for hosted operations. * fix(integrations): harden Wiza reveal polling, soften enrichment getCost guards Address Greptile + Cursor Bugbot review on #4777: return explicit failures from the Wiza individual_reveal poller instead of throwing (thrown errors were swallowed into a false queued success), short-circuit when the initial reveal is already terminal, tolerate transient 5xx/429 during polling, and return 0 (not throw) from Findymail getCost when the contacts/employees array is absent. * chore(integrations): biome formatting after wiza merge resolution * fix(wiza): type isTerminalReveal param structurally for next build typecheck * feat(enrichments): add Findymail, Prospeo, Wiza to work-email waterfall * feat(enrichments): add Wiza + Prospeo phone reveal to phone-number waterfall * feat(enrichments): opportunistic identifiers + LinkedIn URL input across work-email & phone cascades * fix(tables): reduce column header chevron size and fix sidebar shadow bleed (#4800) * feat(slack): add install + privacy section to integration landing page (#4799) * feat(slack): add install + privacy section to integration landing page Adds a hand-authored, slug-keyed landing-content module (separate from the generated integrations.json so it survives regeneration) and renders an install walkthrough + privacy-policy link on integration pages when present. Also refreshes generated docs (data-enrichment entry, icon mappings, tool mdx). * fix(landing): render privacy section independently, align CTA analytics label * docs(landing): clarify the Slack install button is behind sign-in * refactor(landing): bake integration landing content into generated json via docs-gen Moves landing content (install walkthrough + privacy) out of a render-time augment and into the generation pipeline: generate-docs reads the pure-data content map and writes landingContent into integrations.json, so the page reads a single source (integration.landingContent). Canonical types live in integrations/data/types.ts. * improvement(enrichments): align enrichments sidebar with design system (#4801) * improvement(enrichments): align enrichments sidebar with design system * fix(enrichments): consistent close button pattern and fix url link hover * fix(misc): upgrade path change for new better-auth version, billing issue for workflow block agent usage (#4803) * fix(misc): upgrade path change for new better-auth version, double-billing for workflow block agent usage * fail loudly if stripe sub id missing * fix(copilot): seq migration (#4804) * chore(db): drop redundant idx_webhook_on_workflow_id_block_id index (#4809) Removed because (workflow_id, block_id) is a left-prefix of idx_webhook_on_workflow_id_block_id_updated_at_desc, which fully covers it. The dropped index was non-unique and enforced no constraint. * perf(copilot): read chat transcripts from copilot_messages (R+1 cutover) (#4808) * perf(copilot): read chat transcripts from copilot_messages, not JSONB Flip user-facing chat reads from the legacy copilot_chats.messages JSONB array (5.7GB, 99% TOAST) to the normalized copilot_messages table via a new loadCopilotChatMessages helper ordered by seq NULLS LAST, created_at, id — the verified canonical order. Both chat-detail getters (getAccessibleCopilotChat, getAccessibleCopilotChatWithMessages) now drop the messages column from their metadata select (no more whole-array detoast on every load) and assemble the transcript from the table after authorization. This cascades to the copilot + mothership GET endpoints and to resolveOrCreateChat's conversationHistory (the LLM payload). The normalize/effective-transcript pipeline is source-agnostic (copilot_messages.content == a JSONB array element), so transcripts are byte-identical. Dual-write and the JSONB column stay in place as the internal-logic source and fallback; removing JSONB writes is a later step. Prod integrity verified before cutover: 0 messages missing, 0 NULL-seq, 0 dup keys/seq, 0 orphans, order-parity vs JSONB = 0 mismatches. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(copilot): cover auth-deny on a found row skips the messages query Address PR review: exercise the `if (!authorized) return null` contract — when the chat row exists but authorization fails, the getter returns null and never issues the copilot_messages read. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(tables): right-align run/stop in embedded toolbar; workflow cells format like normal cells (#4806) * fix(tables): right-align run/stop in the embedded table toolbar Add a right-aligned `trailing` slot to ResourceOptionsBar and move the embedded mothership table's run/stop control into it, so Filter + Sort stay left-aligned and run/stop sits opposite on the right. No-op for the search-bearing consumers (logs, resource list), which don't pass `trailing`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(tables): workflow-output cells format values like normal cells Workflow-output columns short-circuited in resolveCellRender and rendered their value as plain text, so a sim-resource URL / external URL / JSON / date produced by a workflow never got the chip, favicon link, or typed formatting a normal cell gets. Factor value formatting into a shared `resolveValueKind` helper used by both the workflow-value branch and the plain-cell branch; the workflow branch keeps the typewriter reveal for plain streaming text via a `typewriter` flag. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(tables): detect resource/URL links on workflow output regardless of column type Workflow output columns default to `json` (columnTypeForLeaf), so routing their values through the type-based formatter (a) gated chip/URL promotion behind `column.type === 'string'` — a URL produced by a json-typed output never became a chip — and (b) JSON.stringify'd plain string values, adding quotes and losing the typewriter reveal. Detect links (sim-resource chip / favicon URL) on the value string directly for workflow outputs, falling back to the plain `value` kind; plain cells keep the type-based formatting. Addresses Greptile P2 on #4806. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(icons): repair broken integration icon rendering (#4810) * fix(icons): repair broken integration icon rendering Two distinct bugs left integration icons broken on the /integrations page (visible at 32-40px, hidden at the toolbar's 16px): 1. Corrupted SVG paths (Notion, Greptile, Granola, Calendly, Grafana, Bedrock): over-minified data dropped elliptical-arc flag digits (e.g. `A1 1 0 5.9 7` instead of `A1 1 0 0 0 5.9 7`); Granola's cubic stream was truncated. Browsers abort path parsing at the first invalid arc flag, so each rendered as a fragment or blank. Replaced with correct path data from canonical sources, preserving each icon's existing fill/gradient and bgColor. 2. Invisible glyph (Bright Data): its icon uses fill='currentColor' but bgColor was '#FFFFFF', and every surface forces text-white on the glyph - white-on-white. Changed bgColor to Bright Data's brand blue (#3d7ffc) so the white glyph reads, matching the white-glyph-on-brand-chip convention. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(icons): restore Calendly dual-tone brand colors Addresses review feedback: the previous fix replaced the broken Calendly icon with a monochrome #006BFF path, dropping the cyan #0ae8f0 accent from the original dual-tone mark. Restored the two-tone logo (blue + cyan) using clean, valid path data, cropped to a tight square viewBox so it fills the chip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * improvement(icons): enlarge icons, fix Zoom contrast and Quiver chip - Zoom: glyph was blue-on-blue (#0B5CFF on #2D8CFF chip); switched to currentColor so it renders as a white glyph on the blue chip. - Quiver: chip bgColor #000000 -> #FFFFFF to match the icon's near-white box, and enlarged the mark slightly (viewBox crop). - Enlarged (tightened viewBox, verified no clipping): RevenueCat, Prospeo, Granola, Firecrawl, Enrich.so, and the AWS icons (RDS, DynamoDB, SQS, CloudFormation, Athena, CloudWatch, SES, Bedrock, S3). - ZoomInfo left unchanged: it is a full red rounded-square logo that already fills its frame, so a crop would clip it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(icons): use Bright Data wordmark on white chip; repair Circleback - Bright Data: replaced the flame glyph with the official two-tone 'bright data' wordmark (provided asset), centered in a symmetric viewBox. Reverted the chip bgColor from #3d7ffc to #FFFFFF since the blue wordmark is invisible on a blue chip (the wordmark is designed for a light background). - Circleback: a minifier had rounded the pattern's image scale to scale(0), collapsing the embedded logo to zero size (invisible). Restored the correct scale (1/280 = 0.00357142857) so the C. mark renders. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(docs): sync Quiver block color card to white chip Reflects the Quiver bgColor change (#000000 -> #FFFFFF) in the docs block info card. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * improvement(icons): enlarge AWS/Cloudflare/Dagster icons, fully white Zoom - Enlarged (tighter viewBox, render-verified, no clipping): Cloudflare, Dagster, and the red AWS icons AWS IAM, Identity Center, Secrets Manager, SES, STS. Identity Center was anomalously small (filled ~32% of its frame); the group is now sized consistently (~80% fill). - Zoom: the camera lens triangle was still #0B5CFF (blue-on-blue); switched it to currentColor so the whole camera renders white on the blue chip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(wiza): consolidate individual reveal into a single operation Merges the separate Start/Get Individual Reveal operations into one Individual Reveal operation in the Wiza docs and integrations data (operationCount 5 -> 4). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * improvement(icons): size remaining AWS icons to match the set (~80% fill) Bring RDS, DynamoDB, SQS, CloudFormation, Athena, CloudWatch and S3 up to the same ~80% fill as the AWS IAM/Identity Center/Secrets Manager/SES/STS group, so all AWS icons are visually consistent. Bedrock left as-is (already ~92% fill). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(icons): use Bright Data flame mark, enlarge ZoomInfo - Bright Data: the full 'bright data' wordmark was illegible at chip size. Replaced with just the flame-'i' brand mark (blue #4280f6 on the white chip), centered. - ZoomInfo: cropped the viewBox toward the white 'Zi' so it's larger; the red rounded-square background still fills the chip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * improvement(icons): enlarge CrowdStrike icon The falcon mark sat small in its chip because the icon used a wide 768x500 viewBox (letterboxed in the square chip). Switched to a square viewBox centered on the mark so it fills ~80%, consistent with the other icons. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(tables): serialize schema mutations to prevent parallel column clobber (#4812) * Make workflow description nullable * fix(tables): serialize schema mutations to prevent parallel column clobber * fix(tables): load workflow outside schema lock; use DbOrTx for getTableById * fix(tables): scale idle timeout in updateColumnType to avoid aborting large type changes * fix(tables): skip stale remap types when workflowId changes concurrently * fix(tables): scale idle timeout in updateColumnConstraints for large tables * fix(wait): resume live/draft async waits and preserve cell context on chained waits (#4814) * Make workflow description nullable * fix(wait): resume live/draft async waits and preserve cell context on chained waits * improvement(knowledge): polish tag filter dropdowns * improvement(knowledge): soften filter section labels * improvement(knowledge): soften list filter labels * fix(security): harden SSO domain registration, webhook path isolation, and CSV export (#4813) * fix(security): harden KB file access, SSO domain registration, webhook path isolation, env secrets, and CSV export * fix(sso): scope domain conflict query with indexed lower(domain) filter Address PR review: avoid a full-table scan on every SSO provider registration by filtering candidate rows in SQL with lower(domain) = <normalized>, keeping the in-memory ownership check. Also tighten the normalizeSSODomain TSDoc. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: condense env route security comments Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * icons update * chore(security): tighten inline comments in CSV export and KB file authorization Condense verbose comment blocks to concise TSDoc/single-line form; no behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): validate internal serve origin in KB file authorization Replace the bypassable isInternalFileUrl substring check in resolveInternalKbKey with an origin allow-list (base URL, internal API base URL, TRUSTED_ORIGINS). A crafted external host whose path is /api/files/serve/<victim-key> no longer resolves to the victim key. Relative same-origin URLs are unaffected. * style(sso): use idiomatic sql lower() comparison for domain conflict query Match the repo's prevailing `sql`lower(col) = value`` idiom for the case-insensitive SSO domain conflict lookup. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): align workspace env admin gate with hasWorkspaceAdminAccess Use the same admin check the secrets UI uses (owner, admin permission, or org-admin) so owners and org-admins are not wrongly denied their own decrypted workspace secrets, while read-only members remain restricted to names only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sso): rely on lower(domain) match for conflict detection, drop dead in-memory recheck Address PR review: the SQL `lower(domain) = <normalized>` predicate already excludes rows that the in-memory `normalizeSSODomain(...) === domain` recheck claimed to catch, making that recheck dead/misleading code. Match on the canonical lower-cased domain and filter purely by ownership. Malformed legacy values (wildcards, schemes, ports) never match an email domain at sign-in, so excluding them is not a gap. Test DB mock now applies the lower() predicate so the casing-variant case is genuinely exercised. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): scope webhook deploy path conflict to active webhooks findConflictingWebhookPathOwner omitted the isActive filter that the runtime dispatcher (findAllWebhooksForPath) applies, so an inactive but non-archived webhook from another workflow (e.g. after undeploy or failure auto-disable) would permanently block any new deployment on that path even though it never receives deliveries. Align the guard with the runtime isActive + archivedAt filter; the earliest-owner runtime check remains the authoritative cross-tenant protection. Also trims verbose TSDoc on the webhook path-isolation helpers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): exclude archived workflows from webhook deploy path conflict findConflictingWebhookPathOwner now joins workflow and filters isNull(workflow.archivedAt), matching the runtime dispatcher (findAllWebhooksForPath). A webhook on an archived workflow can never receive deliveries at runtime, so it must not block legitimate path reuse with a 409. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): anchor KB file ownership to earliest document in any state A KB file's owner is now the earliest document referencing its key regardless of state (active/archived/deleted/excluded); access is granted only when that owning document is still active. Closes the residual where an attacker could plant an active document to claim a file whose original document was archived or deleted. * updated greptile icon * revert(security): drop KB file authorization changes Reverts the knowledge-base file-access work (origin-pinning / owner-pinning / origin allow-list in verifyKBFileAccess) and its test. The other hardening fixes (SSO domain registration, webhook path isolation, workspace env secrets, CSV export) are unchanged. apps/sim/app/api/files/authorization.ts is restored to its origin/staging baseline. * fix(sso): treat caller's own user-scoped provider as owned during conflict check Self-hosters often register SSO user-scoped via the CLI script (no SSO_ORGANIZATION_ID). If they later enable organizations and reconfigure the same domain org-scoped through the UI, the conflict check previously treated their own user-scoped row as another tenant's and returned a misleading 409. Recognize the caller's own user-scoped provider as owned so that migration is allowed, while still blocking another user's or another org's domain. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * revert(security): remove workspace-env admin gate Defer to a credential-based access model (separate change). Restores GET /api/workspaces/[id]/environment to main behavior and removes the test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(security): consolidate webhook path-collision check into one helper Extract findConflictingWebhookPathOwner to lib/webhooks/utils.server.ts as the single source of truth for cross-tenant path-collision detection, used by both webhook creation paths (deploy sync and the manual /api/webhooks route). This also repairs two latent issues in the manual route's previous inline check, which queried with limit(1) and only webhook.archivedAt: - limit(1) inspected one arbitrary row, so a same-workflow row could mask a foreign collision (false negative). The shared helper scans all matching rows. - It omitted isActive/workflow.archivedAt, so inactive or archived-workflow webhooks (which never receive deliveries) permanently blocked path reuse. The helper mirrors the runtime dispatcher's filter. Same-workflow webhook reuse for upsert is now a separate, explicit lookup. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): block private/reserved IPs for hosted 1Password Connect SSRF (#4818) * fix(security): block private/reserved IPs for hosted 1Password Connect SSRF * test(security): use real isPrivateOrReservedIP and cover IPv6 edge cases * improvement(integrations): validate and expand devin, cursor, and greptile (#4820) * improvement(integrations): validate and expand devin, cursor, and greptile - devin: fix missing org_id path segment on all session endpoints, add 7 session sub-resource tools (list messages/attachments, get/append/replace tags, archive, terminate), pagination, and is_archived output - cursor: add get_api_key_info, list_models, list_repositories tools - greptile: align block and docs - normalize array outputs to default [] and tighten types * refactor(cursor): simplify list_repositories v2 array normalization Collapse the redundant `?? []` + `Array.isArray` double-guard into a single Array.isArray check, per PR review feedback. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(devin): scope session-tag mapping to tag ops and normalize array tag inputs - Only map sessionTags into the tools tags param for append/replace operations, preventing stale sessionTags state from clobbering create_session tags - Fall back to a wired tags value when sessionTags is empty for tag operations - Normalize tag inputs (string or wired string[]) via normalizeTags so array values from other blocks no longer throw on .split Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(cursor): restore base64 file data in legacy download_artifact metadata The legacy CursorBlock exposes only content + metadata (no v2 file output), so metadata.data was the only way legacy-block workflows could access downloaded artifact bytes. Restore the base64 data field and document it in the outputs/type instead of dropping it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(devin): coerce terminateArchive to archive flag for boolean-wired input * docs(integrations): regenerate tool docs for new devin and cursor operations --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(search-replace): don't auto-navigate when content edits invalidate the active match (#4819) * fix(search-replace): don't auto-navigate when content edits invalidate the active match * fix(search-replace): clear afterReplaceIndexRef on apply failure and zero matches * fix(search-replace): remove duplicate setActiveSearchTarget(null) on close * fix(search-replace): move afterReplaceIndexRef write inside handleApply past the guard * fix(search-replace): auto-navigate when hydration resolves with no prior active match * chore(search-replace): remove inline comments * fix(search-replace): revert !activeMatchId guard that caused immediate re-navigation after deselect * improvement(enrichments): limit company-info to fields both providers return (#4817) Hunter's company dataset returns null industry/foundedYear for many large companies (verified against the live API for Microsoft, Amazon, Google), so under the first-non-empty-wins cascade those columns appeared inconsistently across rows. Limit company-info outputs to employee count and description — the fields Hunter and PDL both reliably return — so every row is consistent. employeeCount is a string so Hunter's range bucket and PDL's exact count share the column. * fix(files): don't reject external URLs containing '..' in file parse validation (#4821) * fix(files): don't reject external URLs containing '..' in file parse validation The file block's file_fetch operation rejected any external URL whose path contained '..' (e.g. Slack files-pri slugs with a literal '...') with 'Access denied: path traversal detected'. Traversal checks only apply to local paths — external http(s) URLs are fetched with SSRF protection downstream and are never resolved against the filesystem, so they now short-circuit as valid. Internal /api/files/serve/ URLs keep full traversal protection. * test(files): fix external-URL assertion to handle undefined error * test(files): assert success explicitly in external-URL traversal test * fix(files): keep traversal protection for https URLs matching internal serve paths * feat(google-sheets): add row filtering to read with numeric operators (#4822) * feat(google-sheets): add row filtering to read with numeric operators Adds client-side row filtering to the Google Sheets read (v2) operation. Filter the returned rows by a header column using text operators (contains, not_contains, exact, not_equals, starts_with, ends_with) and numeric/ordering operators (gt, gte, lt, lte). Filtering lives in a pure, unit-tested helper (filterSheetRows) and runs over the fetched read range; an optional `filter` output reports whether the column was found and how many rows matched. Also hardens the surrounding tools: - trim spreadsheetId in write/update/append URL builders (matches read) - URL-encode the v1 read default range - expose valueInputOption for the update operation in the block Backwards compatible: with no filter requested, read output is byte- identical and the `filter` field is omitted. The filterMatchType union is widened additively (4 -> 10 values). * fix(google-sheets): correct filter metadata for missing column and header-only sheets - matchedRows is now 0 (not totalRows) when the filter column is not found, so it no longer contradicts applied=false / columnFound=false - columnFound now reflects an actual header lookup for empty/header-only sheets instead of being hardcoded true - add tests covering header-only and empty sheets with present/absent columns * fix(selectors): fetch all pages for paginated dropdown list routes (#4823) * fix(selectors): fetch all pages for paginated dropdown list routes Dropdown selectors fetched only the first page of paginated provider APIs, silently hiding results past page one. Add bounded server-side draining to the list routes across Microsoft Graph, Google, Notion, Atlassian, Linear, AWS CloudWatch, and offset/token REST APIs, plus a shared client-side drain cap in the selector hook. Response shapes, stored values, and tool execution are unchanged; CloudWatch list tools still honor a caller-supplied limit. Also fixes the Word file picker that was searching for .xlsx files. * fix(selectors): harden JSM and Monday pagination draining - JSM service-desk/request-type drains advance `start` by the actual row count returned (not the fixed page size) and stop on an empty page, so a short non-final page can't skip items. - Monday boards drain now checks `response.ok` per page, surfacing a mid-drain HTTP failure instead of treating it as an empty final page and returning a partial 200. * docs(selectors): clarify JSM drain advances start by actual row count The offset-advancement fix (advance `start` by the rows returned, not the fixed page size) landed in 7b19788a8; update the TSDoc to match so it no longer reads as advancing by `limit`. * fix(selectors): drain fetchPage in direct fetchList callers Making `fetchList` optional left three direct callers (outside the useSelectorOptions hook) calling it unguarded, which broke the build's type check. Route them through a shared `loadAllSelectorOptions` helper that uses `fetchList` when present and otherwise drains `fetchPage`. This also prevents a regression: `confluence.spaces` / `knowledge.documents` now paginate via `fetchPage` only, and these callers (search/replace, value resolution) would otherwise have silently returned no options. * chore(selectors): rename MAX_PAGE_PAGES to MAX_NOTION_PAGES for readability * fix(sso): re-check domain conflict before write and reject IP-address domains (#4825) * improvement(copilot): make copilot_messages the sole transcript store, remove JSONB dual-write (#4826) Stop writing/reading the legacy copilot_chats.messages JSONB column now that reads are cut over to copilot_messages. Make appendCopilotChatMessages the primary write (throws on failure instead of swallowing), repoint peripheral readers (workspace VFS, chat cleanup, data drains, fork, superuser import) to copilot_messages, and persist the assistant turn inside finalizeAssistantTurn's transaction so it commits atomically with the stream-marker clear. The column itself is dropped in a follow-up migration after this bakes. * feat(tables): expand filter operators (not-contains, starts/ends-with, not-in, empty) (#4827) Add does-not-contain ($ncontains), starts-with ($startsWith), ends-with ($endsWith), not-in-array ($nin, previously executed server-side but unexposed in the UI), and is-empty/is-not-empty ($empty) filter operators end-to-end — SQL builder, condition types, query-builder converters/constants, the filter UI, the Table tools/block descriptions, and docs. Also fix correctness bugs in the filter builder surfaced by the wider operator set: - Same-column AND rules (e.g. age > 18 AND age < 65, or name startsWith 'A' AND name endsWith 'Z') silently overwrote each other because the AND group was keyed by column name. They now merge into one operator object, which also makes Filter -> rules -> Filter round-trip losslessly for multi-operator columns. - $nin values were not split into an array like $in, and textual-match values like "123" were numeric-coerced (breaking the ILIKE path). - A non-boolean $empty operand from the raw API silently inverted the check; it now coerces 'true'/'false' strings and otherwise returns a 400. * improvement(copilot): stop persisting tool-call result outputs in transcripts (#4829) Opening a Mothership task could take many seconds because a single persisted assistant message in copilot_messages.content can reach hundreds of MB, almost entirely inside contentBlocks[].toolCall.result.output (e.g. a get_workflow_logs or run_workflow result). The DB query is ~2ms; the cost is detoasting that payload, shipping it to the browser, and parsing it. These outputs are dead weight on the Sim side: they are never rendered (the thread shows only tool name/title/status) and never replayed to the model (the upstream copilot service owns conversation memory). So drop result.output before it is persisted, keeping result.success/error plus the tool metadata. - add stripToolResultOutput() in persisted-message.ts - apply it in messages-store toRow (covers every write path) and in loadCopilotChatMessages (existing rows render fast on read) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * feat(providers): add Together AI, Baseten, and Ollama Cloud model providers (#4830) * feat(providers): add Together AI, Baseten, and Ollama Cloud model providers * fix(providers): guard Ollama streaming fast-path with hasActiveTools Match Together/Baseten/Fireworks: when tools are supplied but all are filtered out (usageControl 'none'), take the single streaming call instead of an extra non-streaming round-trip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(providers): filter non-chat model types from Together model list * refactor(providers): dedupe Ollama Cloud upstream schema ollamaCloudUpstreamResponseSchema was byte-for-byte identical to ollamaUpstreamResponseSchema (both /api/tags endpoints return the same { models: [{ name }] } shape). Drop the duplicate and reuse the shared schema. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(knowledge): calendar view sync, deduplicate popover animation classes, type-safe filter cast * cleanup(knowledge): remove TRIGGER_BORDER_CLASS duplication, inline displayLabel, drop enabledFilterParam alias --------- Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Waleed <walif6@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Theodore Li <theo@sim.ai> Co-authored-by: andresdjasso <andresdjasso@users.noreply.github.com> * feat(blocks): add BlockMeta to Quiver and Linq; fix invalid block config fields; update skills Block fixes: - Add QuiverBlockMeta (tags + 3 templates: icon generator, diagram creator, vectorizer) - Fix QuiverBlock: remove invalid tags field from BlockConfig, IntegrationType.Design → IntegrationType.AI (Design doesn't exist in the enum) - Fix GreptileBlock: remove invalid tags field from BlockConfig, IntegrationType.DeveloperTools → IntegrationType.DevOps - Fix LinqBlock: remove invalid tags field from BlockConfig (tags belong only in BlockMeta) Skills: - add-block: add dedicated BlockMeta section with structure, rules, and registration pattern; add BlockMeta checklist items - add-integration: add BlockMeta to block structure template, add rules clarifying that tags must NOT appear on BlockConfig and integrationType must be a valid enum value; update registry snippet to include blocksMeta; add checklist items * fix(integrations): fix category dropdown by defining missing LANDING_INTEGRATIONS_DATA_PATH and regenerating integrations.json The staging merge introduced landing-content.ts but forgot to define LANDING_INTEGRATIONS_DATA_PATH in generate-docs.ts, causing the script to crash before writing integrations.json. The stale JSON had integrationTypes (plural array) from an older script version, while the Integration type and workspace UI both read integrationType (singular string) — so ALL_CATEGORY_SECTIONS bucketed to undefined and the category filters never appeared in the dropdown. Fixed by adding the missing path constant and re-running the generator. integrations.json now has 192 entries with the correct integrationType field. * fix(sidebar): restore resize handle on all pages commit 3109104582 wrapped the resize handle in {(isCollapsed || isOnWorkflowPage) && ...} and added a useEffect that resets sidebar width to SIDEBAR_WIDTH.MIN whenever the user navigates away from a workflow page. Together these made the sidebar non-resizable on Tasks, Tables, Knowledge Base, and every other non-workflow page. Restore the staging behavior: always render the resize handle and remove the effect that forced the width reset on page transitions. * fix(sidebar): match staging onKeyDown and tabIndex on resize handle The resize handle was still conditionalizing onKeyDown and tabIndex on isCollapsed, blocking keyboard accessibility of the separator role when expanded. Staging always attaches both unconditionally. onKeyDown={isCollapsed ? handleEdgeKeyDown : undefined} → onKeyDown={handleEdgeKeyDown} tabIndex={isCollapsed ? 0 : undefined} → tabIndex={0} * feat(integrations): show connected credentials on integration detail page When navigating to /integrations/google-docs (or any integration), a Connected section now appears above the templates listing all workspace credentials tied to that provider. Each row links back to the credential detail page (/integrations/connected/${id}) for management actions. Pairs with the earlier change that routes connected items from the integrations list to the provider detail page instead of directly to the credential detail page. * fix(integrations): rename Add in chat to Add to Sim * fix(skills): rename Add button to Add to Sim * fix(platform): restore M1/M2/M3 regressions and LazyMotion on landing page M1 — Invitation guard: re-introduce usePermissionConfig().isInvitationsDisabled alongside the workspace inviteDisabledReason check. The flag now also respects NEXT_PUBLIC_DISABLE_INVITATIONS and EE permission-group disableInvitations, not just billing policy. M2 — Settings redirects: /settings/integrations and /settings/skills now server-redirect to /integrations and /skills respectively so old bookmarks and emails don't silently land on General. M3 — Starter block search exclusion: restore block.type !== 'starter' guard in the search store so users cannot add a duplicate Starter block via the command palette. LazyMotion: restore LazyMotion + domMax/domAnimation wrappers and m.* components in landing-preview-panel and landing-preview-home. The removal was accidental (the full motion bundle was left after an import cleanup), which caused the entire framer-motion feature set to load eagerly on the landing page. * fix(integrations): revert connected list to credential detail; remove settings redirects * feat(sidebar): restore workspace switcher search with updated styling Shows a search input in the workspace dropdown when the user has more than 3 workspaces (WORKSPACE_SEARCH_THRESHOLD). Keyboard navigation: ArrowDown/Up to move through results, Enter to switch, resets on close. Styled to match the current branch (border-1/surface-5 tokens, sm text, 11px Search icon) rather than the old staging styles. Highlight state is wired through chipVariants active prop so it follows the same active appearance as clicked/hovered items. * fix(sidebar): clean up workspace search — layout, memo, and effect guard * fix(sidebar): align workspace rename input selection style with workflow rename * perf(sidebar): eliminate React re-renders during sidebar drag resize Previously, every mousemove during resize called setSidebarWidth(), which both updated the --sidebar-width CSS variable (sync) and set Zustand state (async). This caused: 1. A 1-frame transition flash on mousedown — isResizing state had to round-trip through React before the is-resizing CSS class was applied, so the width transition fired for the first pixel of movement. 2. A React re-render per pixel dragged — components reading sidebarWidth from the store (avatars, usage-indicator) lagged one frame behind the container, making the + and ... buttons appear to jump ahead. New approach: - handleMouseDown adds is-resizing directly to the sidebar DOM node before any React involvement (synchronous, no frame lag). - mousemove writes only to the CSS custom property (zero React renders). - mouseup persists the final width to Zustand exactly once. - isResizing / setIsResizing state removed from the store and hook — they are no longer needed since the class is managed via direct DOM mutation. * perf(sidebar): add requestAnimationFrame throttle to resize mousemove handler * fix(sidebar): fix drag-right lag caused by WorkspaceChrome overflow-hidden transition The sidebar-container's is-resizing class correctly suppressed its own width transition, but the two wrapper divs in WorkspaceChrome both have transition-[width]/transition-transform with a 175ms ease. The outer wrapper also has overflow-hidden, so while the sidebar content was at the correct width instantly, it was visually clipped by the outer wrapper which was still animating — causing the + and ... buttons to appear to lag behind the resize line on drag-right (not on drag-left, since shrinking doesn't clip content). Fix: add sidebar-shell-outer/sidebar-shell-inner class names to both chrome wrappers, and suppress their transitions via html.sidebar-resizing rule when a drag is active. The html.sidebar-resizing class is toggled directly in the resize hook alongside is-resizing, so it takes effect synchronously on mousedown. * fix(icons): redesign Download icon to match Upload style; fix Upload/Download confusion Download icon was missing the tray/shelf line at the bottom that Upload has, making it look like a plain arrow rather than a matched pair. Updated Download to use the same viewBox, stroke weight, and three-path structure as Upload (tray + stem + arrowhead), just pointing down. Also fix 5 places where Upload (↑) was incorrectly used for download/export actions: - files.tsx: two Download action rows in toolbar and context menu - tables/table.tsx: Export CSV toolbar button - table-context-menu.tsx: Export CSV context menu item - logs.tsx: Export toolbar button - landing-preview-logs.tsx: decorative Export button Import CSV and actual upload actions correctly keep the Upload icon. * fix(icons): replace Upload with Download on all remaining export/download actions - panel.tsx: Export workflow dropdown item - context-menu.tsx: Export in sidebar workflow context menu - chat.tsx: Export chat button - output-panel.tsx: Export console CSV button - terminal.tsx: Export console CSV button - resource-content.tsx: Export table as CSV + Download file buttons * fix(icons): fix remaining Upload→Download on download actions in files and logs - action-bar.tsx: download button in files toolbar - file-row-context-menu.tsx: Download item in file context menu - file-download.tsx: both download buttons in log details file viewer * updated block skills, settings pages, modals, buttons -> chips, blocks missing metadata * updated skill modal * improvement(resource-header): refine breadcrumb truncation ux * improvement(resource): add floating overflow text tooltips * wire up credits counter * improvement(resource-header): mute path dropdown title * refactor(resource-header): share floating-tooltip engine, prune dead overlay tooltips (#4844) Clean up the breadcrumb truncation feature for reuse and correctness: - Extract useFloatingTooltip / useIsOverflowing / FloatingTooltip into a shared floating-tooltip module. BreadcrumbSegment and FloatingOverflowText now consume one implementation instead of duplicating ~150 lines of positioning, velocity, overflow-detection, and portal logic. - Replace the hardcoded terminal-label regex in ResourceHeader with a typed `terminal` flag on BreadcrumbItem (set by the document chunk/loading crumbs), decoupling the generic header from knowledge-base copy. - Clear the path-popover close timeout on unmount and reuse the shared POPOVER_ANIMATION_CLASSES constant. - Drop the redundant manual overflow-state writes (fixes a sticky fade mask). - Revert FloatingOverflowText inside Combobox `overlayContent` back to plain truncating spans across files/logs/tables/scheduled-tasks/document: the combobox overlay is pointer-events-none, so the tooltip handlers never fired there. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * refactor(emcn,resource-header): address PR #4844 review feedback - useIsOverflowing now uses a callback ref so the ResizeObserver follows the element across mount/unmount/reassignment instead of capturing it once at mount. Safe for conditionally rendered consumers of the shared hook. (greptile P2) - Move POPOVER_ANIMATION_CLASSES out of chip-date-picker implementation internals into emcn/components/popover/popover-animation.ts, exported from the @/components/emcn barrel. Consumers now import from the module boundary. (greptile P2) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * upgrade table and styling upgrade * fix schema to include integration * fix(files): align delete icon with tables view (Trash → Trash2) Co-Authored-By: waleed <waleed@simstudio.ai> * fix(mothership): preserve blockType for integration contexts in sent messages Integration mention chips were missing their provider icons in sent messages because blockType was dropped when mapping ChatContext to messageContexts. renderIntegrationTile returns null without blockType, silently hiding the icon. * fix(mothership): allow 'integration' resource type in chat resources API The VALID_RESOURCE_TYPES allowlist was missing 'integration', causing a 400 error when adding integrations to the Mothership resource tab — so they never persisted and disappeared on refresh. * fix(ui): Add "File" title next to file resource header * fix(ui): fix resource header columns being bolded * fix(resource): keep the floating tooltip from jumping on click Gate the focus-driven show behind :focus-visible so a mouse click (which focuses the trigger) no longer re-shows the tooltip anchored to the element's bottom edge. On click the tooltip now hides cleanly instead of jumping down; keyboard focus still shows it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * perf(sidebar): eliminate unnecessary re-renders in workspace switcher for non-search users - onMouseEnter: only set highlightedIndex when showSearch is true, preventing a state update + re-render on every workspace row hover for users with ≤ 3 workspaces where the search is never shown - onOpenChange: only reset workspaceSearch and highlightedIndex when showSearch is true, since both values are always already at their defaults for non-search users and setting them triggers a pointless re-render during dropdown close - data-workspace-row-idx: only set when showSearch is true since the scroll effect that reads this attribute is already gated on showSearch * feat(search): context-aware cmd-k results on the integrations page When cmd-k is opened on the integrations page, show two new result groups: connected accounts (visible even with empty input) and catalog integrations (appear once the user types). Selecting an OAuth integration deep-links to its detail page with ?connect=oauth so the connect modal auto-opens. Non-OAuth integrations navigate to the plain detail page. Both groups are gated to the integrations page only and respect the hideIntegrationsTab permission. The credentials fetch shares the same React Query cache key as the integrations page itself (no double fetch). * refactor(emcn): make the floating tooltip the one canonical Tooltip Replace the Radix-based emcn Tooltip with the cursor-following floating tooltip so every tooltip in the app uses one consistent style. Built on the shared floating-tooltip engine (relocated into emcn), not a parallel implementation. - Move the floating-tooltip engine into emcn/components/tooltip and export it from the barrel; re-point its consumers (FloatingOverflowText, resource-header) - Extend the FloatingTooltip bubble to render arbitrary children (+ role/id for a11y) so it can back general tooltips, not just overflow text - Rebuild emcn Tooltip (Root/Trigger/Content/Provider/Shortcut/Preview) on useFloatingTooltip — compound API preserved, ~350 call sites unchanged, legacy side/align props accepted and ignored (the tooltip follows the cursor). Removes @radix-ui/react-tooltip usage (package kept for a later cleanup; react-slot retained for asChild) Note: general tooltips now show instantly (no hover delay) and follow the cursor. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * style(emcn): put tooltip text on the design scale (text-caption) Replace the tooltip's ad-hoc `text-xs` + `leading-[18px]` with the semantic `text-caption` (12px) font-size token so the text styling is fully on the design scale and self-documenting, matching how the rest of the system is set up. The color already used the global `--text-body` token. No visual change (still 12px with a ~18px line height). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(sidebar): add empty task state and inline task creation - Show "No tasks yet" in the Tasks section (expanded and collapsed) when the list is empty - Clicking + now creates a task via the API and navigates directly to it, rather than navigating to home - Add isCreatingTaskRef guard to prevent double-click from spawning multiple tasks - Disable + button while creation is pending - Fall back to home navigation on creation error * invite, billing, home * improvement(seats): auto purchase seats on invitations into workspace (#4857) * improvement(seats): auto purchase seats on invitations into workspace * improve sampling for seat drift reconciler * address comments * feat(knowledge): align connector UI with integrations page styling - ConnectorTypeCard now matches integration rows: brand-colored rounded-xl tile, ArrowRight, title/subtitle hierarchy - ConnectorCard icon upgraded from flat surface-4 to branded tile (white icon on brand bg, graceful fallback) - Connector header badges use chipVariants instead of custom Button classes - Add-connector search input aligned to integrations style (h-[30px], rounded-lg, border-1) * fix(icons): trim Folder SVG viewBox to remove right-side whitespace The folder path only extends to x≈14.33 in a 15-unit viewBox, leaving ~0.5 units of empty space on the right. At 12px rendered size this produces ~0.4px extra gap (visible as ~1px on retina displays) compared to solid icons like the workflow color square. Trimming the viewBox to 14.5 units makes the folder fill its chip slot evenly. * fix(user-input): restore draft text synchronously to preserve contexts on nav The SSR-safe approach (empty useState + effect restore) created a timing window where the sync effect in useContextManagement fired with message='' before the value was set, clearing any restored contexts. Folder and workflow contexts (not re-added by applyAutoMentions) were lost on every nav-back. Revert to the staging approach: initialize value synchronously from the draft store so message is already populated when effects run, matching the behavior on staging. * fix(queue): render context chips in queued messages - Remove plainMentions from queued message rows so context chips render with icons, consistent with sent messages - Fix computeMentionRanges to use '/' prefix for skill contexts (content has the slash trigger restored at submit time, not '@') * fix(mothership): remove integrations from add-resource dropdown * fix(mothership): comment out integrations from add-resource dropdown * chore(db): drop form, templates, template_creators, template_stars tables These tables backed the Forms and Templates platform features which were intentionally removed from this branch. Clean up the DB schema to match. * chore(db): add migration metadata for 0224 drop tables * block icons, sidebar, toolbar * chore: remove remaining dead code for template-profile feature - Remove 'template-profile' from SettingsSection union type - Remove 'template-profile' entry from SECTION_TITLES - Remove now-redundant template-profile guard in settings sidebar - Remove commented-out template-profile nav item * fix(multi-select): preserve anchor on range selection for tasks and folders After a shift+click range, the anchor (lastSelectedTaskId / lastSelectedFolderId) was being updated to the end of the range (toId). This caused subsequent shift+clicks to extend from the wrong point instead of the original click. Standard behavior: anchor stays at the initial click (fromId) so repeated shift+clicks always expand/contract relative to where you started. * feat(emcn): add SearchInput component and unify search bars platform-wide - Add SearchInput to emcn: 30px chip-family filled search input matching the integrations page pattern (border-1, surface-5, leading Search icon) - Migrate all 22 search bars across settings, EE pages, and integrations to SearchInput (only layout classes allowed at callsites) - Rename Sim Keys -> Sim API Keys in nav/title; page copy now says API key - Remove components/ui input, label, and verified-badge; migrate consumers to emcn equivalents or raw inputs (table cell editor, wand prompt bar) - Delete dead EE skeleton files (data-drains, data-retention) - General settings: Home Page chip moves to header left as navigation * fix(files,tables): restore new-file editor autofocus and CSV import error toasts Both were dropped on the staging line and regressed vs production (main): - Files: the new-file editor autofocus chain (files.tsx -> file-viewer -> text-editor) was stripped by the react-doctor dead-code pass in #4544, which misread the prop-drilled `autoFocus` (consumed by an imperative `editor.focus()` effect) as unused. Restored the prop through all three layers and the one-shot focus effect so creating a new file focuses the editor immediately. - Tables: CSV import failures were silently logged with no user feedback. Restored the per-file and generic `toast.error` surfacing. * feat(home): score suggested actions by workspace signals - Derive the suggestion pool from the curated block template catalog (1,343 prompts across 172 blocks) instead of 15 hardcoded entries - Fix inverted relevance: prompts for connected providers are now boosted 4x (instantly runnable) instead of excluded; unconnected discounted 0.4x - Weight by featured (3x), popular category (1.5x), and resource gaps (no tables -> boost table starters; has KBs -> dampen KB-creation prompts) - Weighted sampling without replacement, max one suggestion per block - Connect rows weighted by catalog template count; 2 for fresh workspaces, 1 once something is connected - Key the catalog map by both versioned and base block types so gmail_v2 templates resolve (gmail, github, notion, linear were silently dropped) - Replace derive-in-effect state with a useMemo keyed by a shuffle nonce - Add suggested_action_clicked / suggested_actions_shuffled / suggested_actions_toggled PostHog events * billing, teammates * improvement(credentials): credentials invites, secrets tab wiring up (#4874) * improvement(credentials): move away from invite notion * wire up secrets ui/ux * address comments * get consistent styling by removing emcninput + text area * styling consistency * remove fallback * address comment: * refactor(ui): migrate settings & workspace UI to chip design system Migrate modals to ChipModal (showDivider, hint, resizable, size, leading, ChipModalTabs), standardize Chip variants, add ChipCombobox wrapper, and apply chip inputs/dropdowns across settings, knowledge, logs, tables, inbox, EE tabs. Render ChipModalTabs as a ChipSwitch segmented control. Align /settings/secrets detail with /integrations, refresh whitelabeling, and restore file-editor autofocus and CSV import error toasts. * fix(mothership): restore integrations to useAvailableResources for @ mention Integrations were fully removed from useAvailableResources which broke the @ mention menu since user-input shares the same hook. Now integrations are always included in the hook but excluded at the AddResourceDropdown component level, keeping them out of the sidebar + menu while remaining available for @ mention autocomplete. * refactor(settings): chip design-system consistency pass across all tabs Extract a shared chip-field shell (CHIP_FIELD_SHELL/CHIP_FIELD_INPUT) mirroring Input variant='chip' and route secrets, credential detail, and integrations credential detail through it (30px height, font-medium, focus ring). Add a Discard action to the secrets header when dirty. Group BYOK providers into Models/Search & web/Enrichment sections and align its tiles to the integrations tile. Normalize list-row typography to text-[14px]/text-[12px] and icon tiles to rounded-xl + border across api-keys, copilot, custom-tools, mcp, workflow-mcp-servers, credential-sets, and access-control. Tone the secrets Details chip and per-row affordances to ghost. Fix token correctness: raw tailwind colors to design tokens (data-drains), missing chip variant on the Snowflake role input, ColorInput chip-field reuse (whitelabeling), error token and icon sizes (workflow-mcp-servers), border token (mcp), no-results sizing (secrets), row chrome (recently-deleted), hover token and Button to Chip (access-control), and deduped textarea chrome (sso). * feat(settings): unify filter dropdowns on ChipSelect (integrations style) Add a ChipSelect emcn component — a filled chip trigger + chevron opening a DropdownMenu, matching the integrations category filter — supporting single select, multi-select (checkbox rows), grouped options, and optional in-menu search. Migrate every settings/EE filter dropdown off ChipCombobox to it: audit-logs (resource-type multi + time-range), data-retention, data-drains, general, admin (grouped tool picker), inbox status filter, and the workflow MCP-server pickers. The SSO provider-id field stays an editable combobox since it accepts free-text slugs. Also fix audit findings: chip the MCP client-secret input, normalize an MCP error-text size, and drop now-dead destination-icon code in data-drains. * fix(mentions): require explicit @ for integration mentions; decorate sent messages robustly - Bare integration names in prose (Monday, Notion, Clay) are no longer auto-converted to mentions or chipped — mention treatment is strictly opt-in via a token-starting @ (fixes the scunthorpe problem) - @-prefixed mentions still canonicalize casing (@slack -> @Slack) on both the keystroke fast-path and bulk paths (paste, template, draft, STT) - Sent/queued messages now self-sufficiently decorate @IntegrationName tokens via a text scan, covering messages sent before the input pass ran or authored outside the chat input - Integration contexts missing a resolvable blockType (messages persisted before blockType was saved) are backfilled by label lookup so their mention pills render the brand icon again * refactor(settings): section the API keys page like secrets Wrap Workspace, Personal, and the allow-personal-keys toggle in SettingsSection (muted label + divider) instead of bare bold headers, matching the secrets and BYOK pages. * fix(settings): ChipSelect renders above modals + full-width form mode Raise the ChipSelect menu to --z-popover so it layers above modal surfaces (--z-modal) instead of opening behind them. Add a fullWidth prop that stretches the trigger and right-aligns the chevron for form-field use, and apply it to the workflow MCP-server pickers. * fix(emcn): ChipSelect uses the emcn flat chevron, not lucide's square one The lucide ChevronDown is square; rendering it at the chip's 9x7 footprint stretched it. Switch to the custom emcn ChevronDown (built for that wide aspect), matching the integrations filter and ChipDropdown. * fix(emcn): ChipSelect trigger hugs its content (w-fit) In a stacked form layout the trigger was stretched by align-items: stretch, leaving an empty gap to the right of the value. Add w-fit so the chip sizes to its content (a compact pill) everywhere; fullWidth form selects are unaffected. * fix(emcn): ChipSelect uses a square lucide chevron Revert to lucide's ChevronDown sized square (size-[14px]) so it renders crisp, matching the standard select chevron used by Combobox. * renamed tasks to chats * rename and file change * improvement(billing): wire up billing, org, teammates tabs + remove deprecated subscription tab (#4887) * improvement(billing): wire up billing, org, teammates tabs + remove depr subscription tab * pass exec timeout to tool routes * reuse helper * address comments * address disable comment * chore(db): remove migration 0224 to regenerate on top of staging Co-authored-by: Cursor <cursoragent@cursor.com> * fix type errors and regen migration? * chore(db): drop branch migration 0226 ahead of staging merge; will regenerate * chore(db): regenerate migration 0226 after staging merge * externalize before compaction in fallback' * fix save/discard chips to be consistent * fix(ui): remove smodal tabs in favor of chip modal tabs * fix(ui): skip auto-scrolling on mouse highlight of workspace * fix(platform): restore settings redirects, forgot-password Enter submit, and tag tooltip visibility - Re-add SETTINGS_REDIRECTS so /settings/integrations and /settings/skills deep links redirect to their top-level routes instead of rendering an empty settings panel (accidentally removed in 86da193cc3 one minute after cca5054cf6 added it) - Add opt-in onSubmit to ChipModalField input/email variants and wire it in the forgot-password modal so Enter submits again (lost in the ChipModal conversion) - Knowledge tag tooltip: drop the max-h/overflow-y-auto clamp that the pointer-events-none floating tooltip made unreachable; truncate each tag row instead so all tags stay visible with bounded height * chore(db): remove migration 0226_third_spot before staging merge * chore(db): regenerate migration as 0227 after staging merge * feat(telemetry): add posthog + audit coverage for new platform actions Audit log (compliance/permission-relevant only): - org_seat.provisioned — seat auto-purchased when an invite acceptance grows the org (actor = accepting user, includes seat delta) - org_plan.converted — Pro→Team conversion triggered by invite acceptance - org_seat.drift_reconciled — hourly cron healed a drifted seat count - credential_member.added/removed/role_changed — credential sharing surface was previously fully unaudited - table.created — parity with existing table.updated/deleted - skill updates now record skill.updated instead of mislabeled skill.created PostHog: - seats_provisioned, credential_shared/unshared, environment_updated/deleted (key counts only, never names/values) - table_import_started/completed — background CSV imports previously had zero failure observability - table_exported, file_downloaded, skill_updated - credential_connected now fires for OAuth completions (draft-hooks), credential_deleted for OAuth disconnects; previously only manual credentials were tracked - table_workflow_run gains deployment_mode (live/deployed/mixed) Also: logger.warn on all new credential-admin 403 denials (members + environment routes); invite-created audit enriched with enforcedFixedSeats/plan. Deliberately excluded as noise: per-hour dead-letter audit rows (would re-record the same stuck event every cron run) and a duplicate system-actor org_plan.converted in the Stripe outbox handler. * feat(home): fill textarea on suggested prompt click instead of sending Clicking a prompt action in the Suggested Actions panel now populates the Mothership user-input textarea (via applyAutoMentions) and focuses it with the caret at end, rather than immediately submitting. The user can review, edit, and send manually. * fix(templates): name owning integration in featured block template prompts Four featured prompts omitted their owning integration name, making them unbranded and disconnected from their title. Each prompt now explicitly names GitHub or Google Sheets so mentionifyIntegrations renders the chip and the copy reads as a self-contained agent-building instruction. * fix(templates): rewrite fragment prompts so each names its integration and reads as a complete instruction Featured and non-featured block template prompts that either omitted the owning integration's canonical name or were phrased as marketing fragments rather than natural user instructions have been rewritten. Each updated prompt now starts with an imperative verb ("Build a workflow that…"), names the owning integration explicitly so the @-mention chip renders correctly, and aligns with the entry's title. Files changed: salesforce, hubspot (×2), github, slack, airtable, firecrawl, iam. * fix(templates): name integration in remaining block template prompts - google_docs.ts: replace 'Google Doc' with 'Google Docs document' in 4 prompts; update title 'Meeting notes to Google Doc' to 'Meeting notes to Google Docs' - google_sheets.ts: replace 'Google Sheet' with 'Google Sheets spreadsheet/Google Sheets' in 3 non-featured prompts - slack.ts: replace 'Google Doc' with 'Google Docs document' in 'Daily standup summary' - stripe.ts: replace 'Google Sheet'/'Slacks' with 'Google Sheets'/'Slack' in 'Weekly metrics report' - reddit.ts: add 'Reddit' to 3 prompts that only referenced subreddits - notion.ts: rewrite featured prompt to start with a verb and name Notion - jira.ts: rewrite featured marketing-fragment prompt to start with a verb - linear.ts: rewrite featured marketing-fragment prompt to start with a verb - gmail.ts: rewrite featured marketing-fragment prompt to start with a verb and name Gmail * fix(icons): convert monochrome dark brand icons to currentColor for dark mode LinkupIcon, InfisicalIcon, IntercomIcon, LumaIcon, GranolaIcon, OnePasswordIcon, and RailwayIcon were hardcoded to black/near-black fills/strokes, making them invisible in bare DARK mode. Convert all to currentColor so they follow the theme-aware text color. Add iconColor: '#286efa' to IntercomBlock (Intercom brand blue is a confident mid-tone, safe on both themes). Two-tone icons (StagehandIcon, AgentPhoneIcon, QuiverIcon) are left unchanged because their white details are structural — converting to single-tone would destroy logo legibility. * removed color, user input, sidebar, suggested actions, chips/emcn * removed color migration * improve(blocks): audit block catalog metadata for accuracy and fill gaps - Fix template prompts that claimed capabilities blocks don't have (fabricated triggers, non-existent tools) across ~98 integrations - Normalize integration tags to family conventions and valid union values - Align alsoIntegrations and modules with template prompt content - Add BlockMeta for circleback, imap, and rss trigger integrations - Add templates to clickhouse and greptile metas - Remove duplicate PageSpeed deploy-gate template * chore(telemetry): drop org_seat.drift_reconciled audit System self-heal bookkeeping doesn't belong in the user-facing audit trail — membership and seat-purchase changes are already audited, and the cron's logger output covers ops visibility. * chore(db): consolidate branch migrations into single 0227 * fix(emcn): make ChipModal scroll internally when content exceeds viewport The Modal→ChipModal migration dropped the old ModalBody scroll container: ChipModal renders ModalContent bare (no overflow-hidden) and its wrappers had no min-h-0 chain, so tall modals (e.g. New data drain with an S3 destination) overflowed max-h-[84vh] off-screen with no way to scroll. Complete the flex min-h-0 chain through ChipModal's frame and give ChipModalBody flex-1 min-h-0 overflow-y-auto — header/footer stay pinned, body scrolls only when constrained. Short modals are unaffected (max-h caps, it doesn't stretch), and dropdowns inside the body are Radix-portaled so the new scroll container cannot clip them. Five modals that had locally patched this with max-h-[Nvh] overrides keep working unchanged. * fix(emcn): don't close modal when dismissing a dropdown via outside click Radix dispatches pointer-down-outside to every open dismissable layer at once, so clicking outside an open dropdown/select inside a modal closed both the dropdown and the modal in one jarring step. ModalContent now prevents its own dismissal while a portaled popper layer is open — the first outside click closes just the popper, the next one closes the modal. * docs(skills): de-duplicate and correct agent docs; canonical styling tokens; mirror sim-sandbox rule * docs(skills): broaden boundary-raw-fetch scope note, sync cursor rule mirrors * fix(emcn): harden modal popper guard and exempt caret-anchored dropdowns from body scroll Adversarial review follow-ups to the ChipModal scroll + dismissal fixes: - Popper guard now requires data-state="open" inside the popper wrapper, so a dropdown that is merely animating closed no longer swallows the next outside click on the modal (DropdownMenu has an exit animation that keeps its wrapper mounted briefly) - Port the same guard to SModalContent for consistency - custom-tool-modal: opt its body out of the chrome scroll container (flex-none + overflow-visible); the caret-anchored EnvVar/Tag autocomplete dropdowns are absolute-positioned inside the body and must spill past its bounds rather than clip against a scroll boundary * refactor(billing): drop unnecessary useCallback wrappers from event handlers * fix(data-drains): complete chip migration of destination forms and fill missing placeholders Finishes the in-flight FormField → ChipModalField conversion for all destination form specs and adds the placeholders that several inputs never had (S3 bucket/region/access keys, Azure account key, Datadog API key, webhook signing secret/bearer token) — the cause of the New Data Drain modal showing placeholder-less inputs inconsistently. * fix(emcn): broaden modal popper guard to onInteractOutside Covers the focusOutside dismissal path too: when a popper's focus scope unwinds on close, the transient focus shift could still dismiss the modal (and simultaneous body pointer-events lock teardown could freeze the page). Same data-state="open" scoping as the pointer guard. * fix(emcn): harden modal outside interactions * update audit mock --------- Co-authored-by: andres <k62hc5kjst@privaterelay.appleid.com> Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Waleed <walif6@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Theodore Li <theo@sim.ai> Co-authored-by: andresdjasso <andresdjasso@users.noreply.github.com> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai> Co-authored-by: waleed <waleed@simstudio.ai>	2026-06-06 11:39:48 -07:00
WaleedandClaude Opus 4.7	10f7d36f13	fix(settings): accurate View navigation after restore in recently deleted (#4546) ... * fix(settings): accurate View navigation after restore in recently deleted Files now deep-link to /files/{id} (was /files), and folders expand the restored folder + its parent chain in the sidebar before navigating to /w so the user actually lands on the item they restored. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix: fall back to archived folders for parent-chain lookup The restored folder may not be in the active folders cache yet when View is clicked (the invalidation+refetch fires from onSettled, after onSuccess surfaces the View button). Merge archived folder data — where the restored item still lives — into the lookup map so the expansion loop can always resolve the folder and walk its parent chain. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(styling): canonicalize size-* shorthand for equal height/width Document the size-* shorthand as the canonical pattern across CLAUDE.md, AGENTS.md, .claude rules, .cursor + .agents commands, and the emcn design-review skill. Default icon size is size-[14px]. Treat h-[Npx] w-[Npx] and h-N w-N pairs as refactor targets. Also migrate the remaining occurrences in recently-deleted.tsx. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>	2026-05-10 13:12:08 -07:00
Vikhyath Mondreti	6b0de36238	chore(skills): update checklist for boundary e2e checklist (#4363) ... * chore(skills): update checklist for boundary e2e checklist * fix formatting	2026-04-30 13:12:09 -07:00
Vikhyath Mondreti	b8959eb20d	improvement(repo): zod based client-server boundary (#4355) ... * improvement(repo): centralized zod contracts (#4336) * improvement(repo): zod schema contracts * type checks * fix(notion): correctly register tool (#4337) * fix func blokc * more improvements * fix tests * type check * remove v3 refs * minor type improvements * address comments * update jira contract * remove validateJsonBody * improvement(repo): consolidation of boundary helpers + better unknown usage (#4352) * improvement(repo): consolidation of boundary helpers + better unknown usage * address comments * improve file transfer error messaging * fix docs listing schema drift * fix inocrrect type casting * address council comments * remove prefix	2026-04-30 12:16:28 -07:00
3 people	5f0f0edd63	improvement(repo): separate realtime into separate app (#4262) ... * improvement(repo): restructuring to make realtime image narrower scoped * improvements * chore(repo): rebase fixes and quality improvements for realtime split Addresses merge-time issues and gaps from the realtime app split: - Retarget stale vi.mock paths to @sim/workflow-persistence/subblocks - Restore README branding, fix AGENTS.md script reference - Restore TSDoc on workflow-persistence subblocks helpers - Use toError() from @sim/utils/errors in save.ts - Add vitest config + local mocks so @sim/audit tests run standalone - Move socket.io-client to devDependencies in apps/realtime - Add missing package COPY steps to docker/app.Dockerfile - Add check:boundaries/check:realtime-prune scripts and wire into CI Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(security): consolidate crypto primitives into @sim/security Move general-purpose crypto primitives out of apps/sim into the @sim/security package so both apps/sim and apps/realtime can share them. @sim/security exports (all pure, dependency-free): ./compare safeCompare (constant-time HMAC-wrapped equality) ./encryption encrypt/decrypt (AES-256-GCM, iv:cipher:tag format) ./hash sha256Hex ./tokens generateSecureToken (base64url) Migrate apps/sim call sites to use these + @sim/utils helpers: crypto.randomUUID() -> generateId() from @sim/utils/id createHash('sha256').digest -> sha256Hex timingSafeEqual on hashed hex -> safeCompare new Promise(setTimeout) -> sleep from @sim/utils/helpers No behavior change: encryption format, digest output, and token length are preserved exactly. * refactor(copilot): use toError in remaining otel/finalize sites Replace the last two `error instanceof Error ? error : new Error(String(error))` patterns with toError from @sim/utils/errors. Completes the sweep of clean candidates — no behavior change. * refactor(security): consolidate HMAC-SHA256 primitives into @sim/security Adds hmacSha256Hex and hmacSha256Base64 to @sim/security/hmac and migrates 15 webhook providers plus 5 other hot paths (deployment token signing, outbound webhook requests, workspace notification delivery, notification test route, Shopify OAuth callback) off bare `createHmac` calls. Secret parameter accepts `string | Buffer` to cover base64-decoded Svix-style secrets (Resend) and MS Teams' HMAC scheme. AWS SigV4 signing in S3 and Textract tools intentionally retains direct `createHmac` usage — its multi-step key derivation chain doesn't fit a generic helper. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(packages): post-audit test + packaging polish - Add safeCompare unit tests (identity, length mismatch, hex-nibble diff). - Add Buffer-secret cases to hmac tests to lock in Svix/MS-Teams contract. - Declare `reactflow` as a peerDependency on @sim/workflow-types — only used for type imports. - Add a barrel export to @sim/workflow-persistence for consumers that prefer package-level imports; subpath exports retained. - Document the data-field invariant in load.ts for loop/parallel subflow patching. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(realtime): address PR review feedback - Remove redundant SOCKET_PORT=3002 env from Dockerfile runner stage (env.PORT already defaults to 3002 via zod schema). - Reorder PORT fallback so an explicitly-set SOCKET_PORT wins over the schema default for PORT; keeps SOCKET_PORT functional as an override instead of dead code. - Add dedicated type-check CI step for @sim/realtime so TS errors surface pre-deploy (the Dockerfile runs source TS via Bun and has no implicit build-time type check). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(realtime): remove unused SOCKET_PORT env var SOCKET_PORT has lived in the socket server since the June 2025 refactor but was never actually set in any deploy config — docker-compose.prod, helm values/templates, .env.example, and docs all use PORT or the 3002 default exclusively. No self-hoster was ever pointed at SOCKET_PORT, so removing it is safe. Simplifies realtime port resolution to `env.PORT` (zod-validated with a 3002 default) and drops the orphaned sim-side schema entry. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Waleed Latif <walif6@gmail.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>	2026-04-22 23:06:16 -07:00
Waleed	b5674d9ed4	improvement(codebase): centralize test mocks, extract @sim/utils, remove dead code (#4228) ... * improvement(codebase): centralize test mocks, extract @sim/utils, remove dead code * improvement(codebase): apply @sim/utils conventions to staging-introduced files	2026-04-18 14:39:03 -07:00
WaleedandClaude Opus 4.6	a680cec78f	fix(core): consolidate ID generation to prevent HTTP self-hosted crashes (#3977) ... * fix(core): consolidate ID generation to prevent HTTP self-hosted crashes crypto.randomUUID() requires a secure context (HTTPS) in browsers, causing white-screen crashes on self-hosted HTTP deployments. This replaces all direct usage of crypto.randomUUID(), nanoid, and the uuid package with a central utility that falls back to crypto.getRandomValues() which works in all contexts. - Add generateId(), generateShortId(), isValidUuid() in @/lib/core/utils/uuid - Replace crypto.randomUUID() imports across ~220 server + client files - Replace nanoid imports with generateShortId() - Replace uuid package validate with isValidUuid() - Remove nanoid dependency from apps/sim and packages/testing - Remove browser polyfill script from layout.tsx - Update test mocks to target @/lib/core/utils/uuid - Update CLAUDE.md, AGENTS.md, cursor rules, claude rules Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * update bunlock * fix(core): remove UUID_REGEX shim, use isValidUuid directly * fix(core): remove deprecated uuid mock helpers that use vi.doMock --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-04-05 11:28:54 -07:00
Daniel Gómez Martínez	668b948f0b	feat(agents): generalize repository guidance for coding agents (#3760) ... * feat(agents): generalize repository guidance for coding agents * fix(agents): use repo-root link in sim app guidance	2026-03-25 10:29:47 -07:00