597 Commits

Include renames

Author	SHA1	Message	Date
Waleed	0c38037bf1	fix(knowledge): harden ingestion pipeline (#7077) ... * fix(knowledge): harden ingestion pipeline * fix(knowledge): correct bounded ingestion edge cases * fix(knowledge): tighten redaction and chunk validation * fix(knowledge): close bounded ingestion gaps * fix(files): preserve parser complexity limits	2026-08-25 17:55:56 -07:00
Vikhyath Mondreti	b99411a86f	fix(workflow): highlight references in code previews (#7081) ... * fix(workflow): highlight references in code previews * fix(workflow): harden reference highlighting * fix(workflow): scan references in linear time	2026-08-25 17:32:36 -07:00
Theodore Li	19b8652f35	chore(flags): remove released feature gates (#7087) ... * chore(flags): remove released feature gates * chore(helm): bump chart version	2026-08-25 20:26:40 -04:00
Vikhyath MondretiandClaude Fable 5	a25d993266	improvement(billing): ledger-only usage + period-advance cycle close (#7078) ... * improvement(billing): make usage ledger-only and close cycles off period advance Usage is now the attributed usage_log ledger everywhere: the userStats baselines (currentPeriodCost / currentPeriodCopilotCost), the includeLegacyBaseline compatibility flag, the pro-snapshot join/leave machinery, and departedMemberUsage accrual are removed from all read and write paths. Cycle rollover (final sub-threshold overage collection, billedOverageThisPeriod reset, last-period bookkeeping) moves off the invoice.finalized payload parsing — dead for org subscriptions since May — onto a period-advance sweep with a durable per-subscription close marker (subscription.last_closed_period_start), transaction-enlisted Stripe outbox invoicing, and stamp-matched ledger sums. Enterprise closes are bookkeeping-only; reporting-anchor orgs advance the marker alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(billing): include departed actors in close refresh and gate threshold billing on close currency Cycle close now unions current members with every actor holding org-stamped ledger rows in the closed period, so a departed member's daily-refresh consumption offsets the final overage exactly like their billed usage. Threshold billing defers with a pending-cycle-close no-op while a subscription's close marker lags its current period, so the shared billedOverageThisPeriod tracker can never mix an elapsed period's settlements with the new period's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(billing): seal cycle-close races and align refresh actors with billed usage Threshold settlement revalidates the close marker and period under the tracker lock via the same isSubscriptionCycleCloseCurrent predicate the preflight uses, so a rollover between check and transaction aborts as a concurrent-state retry instead of settling against the wrong cycle. Terminal bookkeeping claims the close marker with its tracker reset, so a deletion racing an in-flight sweep close serializes through the one marker and the loser rolls back rather than re-billing settled overage; anchored enterprise deletions only claim the marker since their windows derive from the anchor, not Stripe bounds. A close with overage due but missing Stripe identifiers now defers loudly instead of claiming the marker and silently forgiving the money, the closed window's start derives from the ledger's own period stamps so anchor-day drift cannot misalign the refresh window, calculateSubscriptionOverage unions departed ledger actors into the org refresh deduction like the close does, and blocked accounts report their real ledger usage while staying blocked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(billing): defer ownerless org closes and make the drift test load-bearing A close with overage due but no owner-role member now defers loudly like the missing-Stripe-identifier case instead of claiming the marker and silently forgiving the money. The stamp-drift test pins the marker before the stamped boundary so only the ledger-stamp lookup can produce the asserted window. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(helm): bump chart to 1.6.0 for the billing-cycle-close cron job Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(billing): revalidate the org roster under close locks and align invoice labels The close re-reads the member roster inside its transaction and defers on any change, mirroring threshold billing — an owner transfer moves the billed-overage tracker between rows, so a pre-lock roster could settle against the wrong tracker. Invoice labels now use the closed period's end month like every other overage path, and the sweep test's rows are shaped like rows the candidate query can actually return. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(billing): hold cycle close for a settlement grace after rollover Billing attribution is frozen at run start, so a run straddling a rollover can insert elapsed-period-stamped rows after the period ends. Closing only once the rollover is older than any possible in-flight run guarantees the close's ledger sums are final; the sweep picks the period up on a later run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(billing): resolve reporting windows through the canonical period resolver The close paths now ask resolveSubscriptionUsagePeriod whether a subscription derives its windows from a reporting anchor instead of re-checking metadata shape locally, so a malformed hand-edited anchor that the resolver rejects (falling back to Stripe bounds) books its Stripe-stamped ledger rows normally instead of skipping bookkeeping. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(billing): union departed ledger actors in org threshold settlement Organization threshold billing now reads the period ledger per user and unions the actors holding org-attributed rows with the current roster before computing refresh deductions — the same actor set calculateSubscriptionOverage and the cycle close use — so a departed member's usage cannot be settled without their daily-refresh offset. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(billing): claim the terminal period before deletion settlement and delete vestigial refresh bounds Subscription deletion now claims the close marker from the fresh subscription row before computing or charging final overage, serializing with the cycle-close sweep so both paths can never bill the same period — an in-flight close fails its guarded claim and rolls back, and the deletion settles against the row's real period instead of a possibly stale webhook payload. The per-user refresh bounds machinery is deleted outright: its only source was proPeriodCostSnapshotAt, which this PR stopped writing, and ledger entity stamps already scope refresh to org-attributed rows — a joiner's pre-join usage is user-stamped and can never enter the org refresh scan, while a departed member's org-stamped rows participate exactly like a current member's. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(billing): scope daily refresh by entity stamps and close lagging periods before deletion settlement The refresh actor-list threading (userIds + departed-actor unions) violated the entity-stamp principle the ledger sums already follow: org-attributed rows from departed members counted in pooled usage but never consumed refresh on the monitor/resolved-usage paths. Daily refresh now scopes rows by the same write-time entity and period stamps as the ledger — no actor list anywhere — which deletes the unions, the rollup memberIds, and the org-specific refresh variant. Deletion settlement now closes any elapsed-but-unclosed period (grace bypassed — no later sweep revisits a canceled sub) before claiming the terminal period, so a deletion racing the sweep can no longer jump the marker past an unsettled period and silently forgive its final overage. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(billing): bucket refresh by clamped day so stamped stragglers stay in the deduction Refresh membership is now the entity/period stamps alone — identical to the ledger sums it offsets. A row written after the rollover but stamped to the elapsed period (attribution frozen at run start) is billed by the stamp-based close, so it must consume refresh too; created-at now only assigns the day bucket, clamped into the period, instead of excluding the row entirely. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(billing): pair the overage tracker with the marker's period and pin test clocks billedOverageThisPeriod only ever holds collections for the period that began at the close marker — the threshold gate blocks settlement whenever the marker lags. Both consumers now honor that pairing: a close that skipped forgiven periods counts nothing from the tracker against the period it bills, and the deletion settlement ignores the tracker when the marker was still lagging at claim time. Ignoring is provably safe in both cases because a lagging marker means no current-period collections exist. The cycle-close and daily-refresh suites pin the system clock: their grace and window checks compare fixed period fixtures against Date.now(), which made them dependent on the host date. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(billing): reject lagging terminal claims and thread billingInterval into deletion bookkeeping claimTerminalPeriod no longer advances the marker over an unclosed elapsed period: a lagging marker is reported without a write so the deletion handler can run the elapsed close once more (healing a rollover that committed between close and claim) and only then seal the marker explicitly, with an error log, when the period is genuinely unclosable. Sealing preserves the in-flight-sweep abort guarantee. Deletion bookkeeping now passes the subscription's billingInterval through, so an enterprise reporting subscription whose interval lives on the row column (not metadata) still resolves as reporting-anchored and keeps its bookkeeping no-op, matching every other resolver call site. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-25 16:34:44 -07:00
Waleed	32a15fde2f	improvement(access-control): wire tool and model permissions into copilot editing (#7080) ... * improvement(access-control): wire tool and model permissions into copilot editing Permission groups already supported a per-tool denylist (deniedTools) and model restrictions, but only the canvas honored them. The copilot edit path gated on block type alone, so Sim could build workflows using tools and models the user was not allowed to run — the executor refused them at run time instead. Enforce both at authoring time, and stop advertising what the viewer cannot use. * fix(access-control): use the path alias for the permission-groups type import * fix(access-control): close two denied-tool leaks in copilot discovery The VFS stamped every integration schema from the shared static map before the per-viewer loop re-authored the permitted subset, so a denied operation's schema stayed published. Skip the shared copy for integration paths; the viewer loop is the only projection that knows the denylist. Block metadata resolved denied operations from the catalog's `operation.toolId`, which the projection fills only from `tools.config.tool` — a block whose operation ids are its tool ids left it undefined and read as fully permitted. Resolve through the shared operation gate instead. * fix(access-control): key the copilot schema cache on permission policy The deferred integration-tool schemas now depend on the viewer's permission group, but the cache key encoded only identity and block visibility, so an admin's change to deniedTools took effect only when the entry expired. Resolve the config before the key and add a gate signature alongside the existing visibility signature, mirroring how block visibility already keys the same cache. The read moves out of the cached section rather than being added: what the entry caches is a user-tool schema per exposed integration tool, which dominates it.	2026-08-25 15:41:09 -07:00
Vikhyath Mondreti	b515fe0681	improvement(workflow): add compact code hover previews (#7074) ... * improvement(workflow): add compact code hover previews * fix(workflow): address code preview review feedback	2026-08-25 13:18:53 -07:00
Waleed	98a453d7e6	feat(integrations): add Microsoft Word (#7069) ... * feat(integrations): add Microsoft Word * fix(microsoft-word): guard document edits against concurrent overwrites * fix(microsoft-word): reject non-Word targets, scope SharePoint access, and tighten input bounds * chore(docs): regenerate docs manifest for the Microsoft Word page * fix(microsoft-word): strip XML-forbidden control characters from generated documents * fix(microsoft-word): fail closed when a document reports no version to compare	2026-08-25 11:38:20 -07:00
Waleed	ed60fbaa4e	feat(integrations): add Semrush (#7068) ... * feat(integrations): add Semrush Adds the Semrush SEO API as a block with 44 operations across overview, domain, subdomain, URL, keyword, comparison, and backlink reports. Reports answer as delimited CSV, so the shared decoder maps each header cell back to the export column it was requested as rather than reading by position: the API may return fewer columns than were asked for, and two columns can render under the same header label. * fix(integrations): sync docs manifest for the Semrush page * fix(integrations): address Semrush review findings - Only the API key stays user-only; report selectors (target scope, limit, offset, date, sort, filter) are user-or-llm so an agent can set them - Hold the row limit at one row: a positive fraction floored to zero and sent display_limit=0 - Locate Domain vs. Domain metric columns by their own headers, so a dropped position column shortens the compared-domain run instead of shifting competition, search volume, and CPC onto the wrong values - Describe competitor and domain-list metrics as belonging to the row's domain, not to the target - Describe paid and URL traffic cost as an estimated cost, matching the Traffic Cost header those reports return, not the organic Traffic Cost (%) - Drop the newest-first claim from history outputs, which order by display_sort - Replace the erased any casts in the request tests with a typed helper	2026-08-25 11:17:58 -07:00
Theodore Li	4c0bd944dc	feat(slack): launch v2 triggers and backfill custom bots (#6873) ... * feat(slack): launch v2 triggers and backfill custom bots * fix(slack): propagate legacy webhook dispatch failures * fix(slack): continue shared legacy webhook fanout * fix(slack): acknowledge filtered webhook deliveries * fix(slack): finalize custom bot migration rollout * fix(slack): dedupe migrated bots per workflow * fix(slack): harden custom bot rollout * fix(slack): acknowledge permanently ignored deliveries * fix(slack): retry failed webhook deliveries	2026-08-25 14:11:02 -04:00
WaleedandTheodore Li	656840a1b1	feat(api): make the platform operable headless over v2 (#6912) ... * feat(v2): download run output files by API key Adds GET /api/v2/workflows/{id}/runs/{runId}/files/{fileId}, closing the async-run loop for headless callers. A run's output carries UserFile URLs pointing at /api/files/serve/..., which rejects x-api-key outright, so an async run that produces a file previously had no byte path out for an API key at all. The file is addressed by the id the run reported and resolved against the run's own recorded execution data, from which the storage key is read. The request never supplies a storage key, so the endpoint cannot be aimed at bytes the run did not produce. Resolution deliberately reads the materialized-but-undisplayed recording, because the display projection strips exactly the `key`/`context` fields a byte read needs. Also hardens normalizeStartFile to derive a file's storage key only from a validated internal serve URL, discarding any caller-supplied `key`/`context`. A workspace API key has no human subject, so the executor resolves its actor to the workspace billing owner (preprocessing.ts -> resolveSystemBillingAttribution); verifyFileAccess then authorizes a workspace-context key as that owner, whose reach is not bounded by the key's workspace. Accepting an attacker-authorable key made that substitution exploitable as a confused deputy. Normalization is all-or-nothing, so a forged file now drops the whole files input. * feat(workflows): one graph-write door, principal-derived audit source, and v2 authoring endpoints Extract replaceWorkflowNormalizedState as the single persistence primitive for a workflow graph replace and route both the internal editor save and the Copilot edit tool through it, so neither can skip state preparation, the row lock, the lastSynced stamp, or custom-tool extraction by choosing a different entry point. Derive the audit source from the acting principal instead of hardcoding 'copilot', then widen workflows.variables.apply_operations and workflows.bulk.move to every principal kind. Add GET/PUT /api/v2/workflows/{id}/state, POST /operations, /duplicate, /restore, PATCH /variables, and POST /api/v2/workflows/move over surface-neutral application use cases; move the edit engine to lib/workflows/editing. * test(workflows): cover the graph-write primitive, audit source, and the v2 authoring surface Pin the two-doors fix (preparation runs, the row is locked, custom-tool extraction is post-commit and best-effort) and the false-audit fix (a session principal writes source: 'session', a delegated one writes its service). Both were verified to fail with the fix reverted. Add the application matrix for replaceWorkflowState, applyWorkflowOperations, readWorkflowGraph, and restoreWorkflow — role floor, principal-kind rejection before canonical load, asserted-scope concealment, lock, validation, atomic conflict, plan gate, and audit-then-notify ordering — plus route tests for every new endpoint. * test(workflows): pin the internal graph-write door and the v2 list scope Characterize saveWorkflowNormalizedState's statuses, messages, and notification after the persistence extraction, and cover the new scope filter on GET /api/v2/workflows including a cursor replayed under a different scope. * feat(api): add v2 block, tool, connector-type, and enrichment catalogs Adds six read endpoints under /api/v2 that publish Sim's code-defined catalogs: GET /blocks, GET /blocks/{blockId}, GET /tools, GET /tools/{toolId}, GET /connector-types, and GET /enrichments. These read like static reference data and are not. What a caller may place is decided per workspace by its permission-group integration allowlist, per organization by which unreleased blocks have been revealed, per deployment by ALLOWED_INTEGRATIONS, and per workspace again by the workflows it has deployed as blocks. So all six are plain defineWorkspaceOperation reads at minimumRole 'read' with workspaceApiKey 'allow' — the exact policy of credentials.providers.list — and every response keeps Cache-Control: private, no-store, because an unrevealed preview block's existence must not leak across organizations through a shared cache. Trigger blocks ride as ?capability=trigger rather than a second endpoint, and workspace custom blocks ride inside /blocks discriminated by `source`, so "what may I place?" stays a one-call question. The block projection is extracted out of the Copilot get_blocks_metadata tool and rewritten onto @/tools/metadata and @/tools/metadata-outputs. That cuts the tool's own @/tools/registry edge as a side effect: its module graph drops from 6,756 to 1,318, and the new routes land at 1,673-1,734, next to the shipped /v2/credentials/providers baseline of 1,668. Supporting changes: - scripts/sync-tool-metadata.ts derives hostedApiKey ('always' | 'conditional' | 'none') from each tool's `hosting`. The config itself stays excluded because it holds closures, but "does Sim host the key" is a first-order authoring question, so the answer is emitted. - getCopilotToolDescription takes hostedApiKey as an option instead of reading `hosting` off the tool, so both an executable ToolConfig and the generated metadata can answer it through one shared derivation. - principalUserId / allowedIntegrationTypes move out of lib/credentials/application/provider-catalog.ts into lib/integrations/principal-scope.server.ts. Two copies of the workspace integration gate would diverge first on the workspace-key path, which has no user for permission groups to key on. - scripts/check-tool-registry-boundary.ts walked page.tsx/layout.tsx under app/workspace only, so a route importing the executable registry passed green. It now walks a list of entry sources, seeded with the four catalog route subtrees and the shared projection barrel. Routes are covered per subtree rather than wholesale because 122 of ~1,130 route files legitimately execute tools. Registry sweeps parse every block, tool, connector type, and enrichment through its published response schema and compare against the wire round-trip. They caught a real drift while being written: an operation's inputs were typed as a union of the tool-param and block-input shapes, and the union resolved to whichever member matched first, silently dropping a block input's `schema`. * docs(api): stop publishing a 413 GET /workflows/{id}/state cannot emit * feat(v2): read upload-session state Adds GET /api/v2/files/uploads/{uploadId}. Only DELETE was exported, so a caller that lost track of a transfer could abort it but could not ask whether the session was still alive, already finalized, or failed — the resume story was missing. Runs on a new files.upload.read operation at minimumRole 'read' rather than reusing uploadCancel, which is a 'write': asking about a session must not require permission to destroy it. The GET is a control leg like every other, so it carries the signed upload token and re-authorizes the caller's present workspace permission through reauthorizeWorkspaceUploadPurpose instead of resolving the session on its id alone. * fix(api): reconcile v2 catalog and workflow-authoring integration Merging the catalog and workflow-authoring branches surfaced four issues that neither produced in isolation. - Route and OpenAPI counters were bumped to the same value on both branches, so git merged them as one change while the merged tree holds the sum. Corrects the route ratchet to 1142 and the workflows document to 29 operations (152 total), then regenerates the OpenAPI documents and the CLI surface from the reconciled contracts. - The seven new workflow operations were published in the spec but absent from the workflow API reference groups, which `check:openapi` rejects. - `route-policies.ts` reached `WorkflowOperationsNotAppliedError` through `apply-workflow-operations`, dragging the edit engine — and its diff and comparison dependencies, which reach a client OAuth hook — into every route that uses the shared workflow error policies. The class moves to its own leaf module, mirroring `WorkflowImportError`, and each importer now takes it from there. - The operations route test shadowed that class inside its module mock, so `instanceof` matched a fake and the assertion pinned a message the production class never emits. It now uses the real class and asserts the real message. * feat(v2): extract ZIP archives over the public API Adds POST /api/v2/files/{fileId}/extract and widens files.extract_archive from principalKinds ['session'] / workspaceApiKey 'deny' to admit personal and workspace API keys at the unchanged 'write' role. The widening is an authorization change, so the justification lives in the operation's TSDoc: extraction grants no capability an API key lacks, since every file it writes could be created one at a time through files.create and files.upload.create, both already 'allow' at the same role. It only collapses many calls into one. The previous ['session'] restriction read as an artifact of the UI having been the only caller. Delegated services stay out — no copilot or executor caller exists and admitting one is a separate decision. The response is counts plus the destination folderPath, never the extracted files: a large archive would otherwise materialize thousands of objects into one body. Callers page GET /api/v2/files?folderPath=... instead. The use case returns the internal display path and the adapter projects it to a v2 path, keeping the use case surface-neutral. * feat(v2): extract file text over the public API Adds GET /api/v2/files/{fileId}/text. Text extraction previously sat behind checkInternalAuth on /api/files/parse, a route that also mixes in external-URL fetching, execution-file upload, and multi-file aggregation, so it could not be reused. The parse call is lifted into a thin application use case instead. Runs on the existing files.read_content operation unchanged — it is already workspaceApiKey 'allow' at the read role, and turning bytes it already authorizes into text grants no further reach. `degraded` is a required, non-optional boolean on the response. The legacy doc and ppt parsers deliberately return best-effort or placeholder content rather than throwing, so an omittable flag would let a client that never checks it treat guessed text as extracted text. It is reported honestly rather than converted into an error, because the parsers' behaviour is deliberate and characterization-tested. The read is bounded on its input at 25 MiB before extraction rather than on its output after, given the parsers' documented DoS history; a caller may lower the ceiling but never raise it. * feat(v2): restore archived folders and list the archived set DELETE /api/v2/files/folders archives recursively, so a recursive delete was unrecoverable over the API: the archived files stayed visible through GET /api/v2/files?scope=archived, but nothing could rebuild the folder structure. Adds POST /api/v2/files/folders/restore, path-addressed like the rest of the v2 folder family, and a `scope` selector on the folder list so a caller can find the archived path to hand it. `scope` extends the files folder-list query rather than the shared v2ListFoldersQuerySchema: only workspace files have an archived folder set, so adding it to the shared schema would give tables, workflows, and knowledge a parameter they ignore. GET /api/v2/files/folders is a FULL_SET_LIST, not paged, so no cursor binding changes — list-pagination.test.ts passes unchanged. Restore resolves the archived folder from its path by scanning the archived set rather than walking the live tree, which by definition does not contain the folder being restored. The folder-restored analytics hook now reports the folder actually restored rather than the requested selector, which carries no id on a path-addressed surface. * feat(v2): bulk-download a file selection as a zip Adds GET /api/v2/files/bulk-download, an adapter over the existing downloadWorkspaceFileItems use case and its internal binary route. Path collision: a static segment beside [fileId] permanently shadows a file whose id equals it, and workspaceFileIdSchema does accept [A-Za-z0-9_-]+. Rather than invent a new shape, this follows the existing bulk-delete sibling: the hyphenated form cannot be produced by either minted id shape (UUID v4 or wf_<shortId>), so the shadowed id is unreachable in practice. Documented on the contract so the reasoning is not lost. Folders are addressed by path, matching the rest of the v2 file surface. The paths resolve against the folder set the selection already loads, so it costs no extra query, and a path matching no folder is rejected rather than silently dropped — a misspelled folder must not yield a zip of whatever else was selected. The empty-selection and folder-count guards now account for folderPaths, which a path-only selection would otherwise have tripped. Selections are comma-separated only: v2 rejects a query parameter sent more than once, so a repeated-parameter form would never reach the schema. Pinned by a test so the contract cannot advertise a form the boundary rejects. * feat(v2): expose run output files and optional inline bytes on the runs read GET /api/v2/workflows/{id}/runs/{runId} now reports the files a run produced, each with the downloadPath that fetches its bytes, and can inline them as base64 on request. Gated by includeOutput, matching `output`'s nullability: a caller that did not ask for output does not receive a file list it did not request. The async execute request's rejection of includeFileBase64 is deliberately left alone — at submit time the run has not happened, so there is nothing to inline; reading a finished run is the first moment the question means anything. Inlining is capped per file at the executor's 16 MiB inline ceiling, which a caller may lower but never raise. A file above it answers 413 naming that file's downloadPath, so the caller is told exactly how to get the bytes rather than being left stuck. The descriptor deliberately omits the storage key — files are addressed by id and the key is re-derived from the run's recording — and omits an expiry, which the recording does not carry and which would be fabricated if published. The route becomes headSafe: false, since inlining reads object storage. The builder enforces that this requires the use case to expose authorize(), so HEAD still answers from a real authorization rather than from authentication alone. * feat(v2): permanently delete an archived file DELETE /api/v2/files/{fileId} only archives — the OpenAPI says its stored bytes are never removed — so there was no way to actually destroy a file over the API. Adds the repository primitive, application use case, operation, and DELETE /api/v2/files/{fileId}/permanent. A distinct path rather than a flag on the ordinary delete: a query parameter that turns a recoverable archive into an irreversible destruction is set by accident, and the two acts carry different minimum roles, which one route declaration cannot express. The file must already be archived; a live file answers 409 naming the archive step, so no single request can turn a live file into lost bytes. minimumRole 'admin', which forces workspaceApiKey 'deny' since the workspace-key ceiling is 'write' — the desired policy anyway: unattended credentials should not destroy bytes. Row first, then object. The two legs commit independently, so one can survive a crash between them: deleting the row first leaves at most an orphaned object for the storage sweep, while the reverse would leave a live row pointing at bytes that no longer exist — a file that lists and opens but can never be read. A failed object delete is therefore reported as objectDeleted: false rather than thrown, because the request has genuinely succeeded once the row is gone. Both directions are pinned by failure-injection tests, verified to fail when the order is reversed. Audited as a distinct FILE_PERMANENTLY_DELETED action, not a reuse of FILE_DELETED, which records the recoverable archive step. * feat(api): v2 log analytics, itemized cost, filters, and sortable query Adds the aggregate and rich-read halves of the public logs surface, and fixes three defects the existing reads carry. Aggregate analytics. `GET /api/v2/logs/stats` returns time-bucketed run counts, success rate, error count, mean latency, and the window bounds, per workflow and for the workspace. The first-party route was a raw handler with inline SQL and inline aggregation, so it is split into a repository (`lib/logs/stats-queries.ts`), a pure aggregator (`lib/logs/stats.ts`), and an application use case. That route keeps its legacy authorization — it answers a caller without workspace access with a zeroed 200, where v2 conceals the workspace as a 404 — and consumes only the two surface-neutral halves. `segmentCount` had no `.int()`, `.min()`, or `.max()`, so `0` divided by zero and `1e9` allocated two billion-element arrays: both caller-reachable 500s. Bounded on both contracts. `workflows` is capped, with the workspace totals still computed from every workflow and the cut reported as `workflowsTruncated`. Detail reads gain the itemized `cost.items` ledger (`null` and `[]` are distinct answers and both reachable) and `workflowInput`, restoring a v1→v2 regression. The list gains `workflowName` and `status` filters, and `includeJobRuns`, which unions Chat and Sim-agent job runs into the sequence behind a new `kind` discriminator — without it a job run is indistinguishable from a run whose workflow was deleted. A filter no job row can answer drops the branch outright rather than meaning two things across the union. `POST /api/v2/logs/query` carries the additional sort columns. `GET /logs` is untouched: its single `order` param rests on there being exactly one sortable column, and both escapes from that are ruled out, so the rich read gets its own endpoint — the split the table surface already ships. It uses the shared keyset scheme with the two nullable sort columns read through a sentinel, since a keyset cannot compare against null. `folderPaths` now covers a folder's whole subtree on the public path, as it already did everywhere else; it previously omitted every nested run with no error. The path strings did not change, so a folder-scope version is stamped into the cursor and in-flight tokens restart rather than silently skipping rows. Also fixes `folderName`, which ILIKEd `workflow.name` — a copy of the clause above it — and so searched workflow names instead of folders. `buildLogSortCursorCondition`'s `IS NULL` disjunct is documented and pinned: under `NULLS LAST` the null block is only reachable through it, so removing it as a duplicate-row fix makes those runs unpageable. Ratchets: route count 1142 -> 1144; logs OpenAPI operations 2 -> 4; total operations 152 -> 154. * feat(api): v2 tables run state, dispatch polling, batch update, bulk, archive Closes the headless gaps on the v2 tables surface. - Per-cell run state is now readable through an opt-in `includeRunState` on `GET /rows`, `POST /query`, and `GET /rows/{rowId}`. The default projection is byte-identical; a page whose sidecar outgrows its byte budget is a 413 rather than a silent truncation. - Run dispatches are addressable: `GET /tables/dispatches/{dispatchId}` publishes the column's full four-state domain so polling a finished run is not a 500, and `GET /tables/{tableId}/dispatches` lists what is in flight. - `POST /rows/batch-update` takes one distinct patch per row. Its transaction moved out of the Copilot-only module into a surface-neutral use case both surfaces now call. - `GET .../enrichment/{groupId}` publishes the provider cascade, cost, and timing behind one enrichment cell. - `POST /tables/bulk-move` and `/bulk-delete` reach the existing bulk use cases, which now accept folders by canonical path and resolve them inside the application layer. - `DELETE` is recoverable: `scope=archived` on the table list plus `POST /tables/{tableId}/restore`. * feat(api): expose knowledge chunks, tag writes, archive/restore on v2 Closes the knowledge cluster's remaining public-surface gaps. Chunks: list/read/create/update/delete/bulk under `/api/v2/knowledge/{id}/documents/{documentId}/chunks`. `queryChunks` gains an `id` tiebreaker on every sort so the list pages on a keyset rather than an offset — `tokenCount` and `enabled` are both non-unique, so a page boundary inside a run of equal values used to repeat or drop the tied rows. The internal offset caller is unchanged; the two positioning schemes share one read. Tag definitions: create, update, delete, next-slot, usage, and the document-scoped save and cleanup. Without them a caller could write a tag value into a slot with no definition and then had no way to name it, so tag-filtered retrieval was unbuildable end-to-end. `v2KnowledgeTagSchema` gains `id`, without which PATCH and DELETE are unaddressable. The document-scoped DELETE is pinned to `action: 'cleanup'`: the domain's `'all'` deletes the whole knowledge base's tag vocabulary from a document path. Archive/restore: `GET /api/v2/knowledge/archived` as a sibling route rather than a `scope` param — the two reads bind different operations and a v2 route declares one — plus `POST /api/v2/knowledge/{id}/restore`. `knowledge.restore` is a new workspace operation carrying `delete`'s policy, since an operation's inverse must not be harder to reach; the internal session route now delegates its workspace branch to the shared use case and keeps only the legacy personal one. Also: `POST .../documents/from-workspace-files` surfaces `addWorkspaceFiles`, so a file already in workspace storage no longer has to be re-uploaded byte-for-byte to be indexed; the `chunkingConfig` write widens to the first-party five-key schema with its refines and separator bounds, while the response stays `.catchall` so a legacy JSONB row cannot 500; and `CONNECTOR_MANAGED_RESOURCE_READ_ONLY` joins `FORBIDDEN_DETAIL_CODES` now that the bare 403 on connector-managed chunk writes is wire-reachable. Document upsert is deliberately not included. * feat(api): add v2 credential rotation and a gate-exempt capabilities endpoint PATCH /api/v2/credentials/{credentialId} rotates service-account secret material or renames a credential in place, preserving the credential id so existing workflow, deployment, paused-run, connector, and webhook references keep working. Re-posting to POST /api/v2/credentials answers 409, and delete-and-recreate mints a new id, so rotation previously had no door. The route is adapter-only: updateWorkspaceCredentialUseCase already owned the rotation, its audit projection, and credentials.update. It gains one additive assertedWorkspaceId field for the v2 workspace assertion, and the per-principal credential-type table that deleteCredentialUseCase already applied is lifted into requireManageableCredentialType so both operations share it. Without it a personal API key could rename an env_workspace row and toV2Credential's throw would surface as a caller-reachable 500. CredentialProviderOperationError now maps to 503 with Retry-After when the provider is unreachable, instead of the 400 its OrchestrationError('validation') base projected. A transient outage rendered as a permanent input error invites a caller to revoke a working credential. GET /api/v2/meta reports the calling key's rollout cohort, type, and expiry. It is the one route declaring the new typed gate: 'exempt' option, because the rollout gate and the unknown-path catch-all answer byte-identical 404s and a gated /api/v2/meta could never resolve that ambiguity. Authentication still runs first, so the only fact disclosed is one about the caller's own credential. * feat(api): publish deployment lifecycle and workflow-MCP v2 surfaces Adds the four deployment-lifecycle operations v2 was missing, and the workflow-as-MCP publishing surface, both as adapters over application use cases that already existed. Deployment lifecycle: - PATCH /api/v2/workflows/{id}/versions/{version} relabels a version. Deliberately not the internal route's body-shape dispatch between "rename" and "promote to live". - POST .../versions/{version}/activate promotes a version. Same use case as rollback under a different transition, on its own path because the two mean opposite things to a caller. - POST .../versions/{version}/revert overwrites the draft. Accepts the literal `active` alongside a version number. - PATCH /api/v2/workflows/{id}/deployment toggles unauthenticated public execution. `workflows.public_api.update` widens from session-only to session plus personal API key: it is an admin-role change the same accountable human may make from a script. Workspace keys stay denied. Its EE refusal now carries PUBLIC_SHARING_NOT_ALLOWED instead of a bare forbidden. Workflow MCP servers: - /api/v2/workflow-mcp-servers list, create, update, delete, plus publish and unpublish of a workflow as a tool. Named apart from /api/v2/mcp-servers, which registers the external servers Sim calls. - The six mcp_servers.workflow_deployments operations widen from ['delegated'] to admit sessions and personal API keys; roles and the workspace-key denial are unchanged. - The server list gains keyset pagination, matching its external sibling, since nothing caps how many a workspace publishes. - Server, tool, and workflow reads move out of the use case into lib/mcp/queries. Route ratchet 1150 -> 1160; OpenAPI operations 161 -> 171. * feat(api): extract chat deployments and publish the v2 surface Chat deployment was a shipped module with no public API and two authorization systems: `lib/workflows/application/chat-deployments.ts` had deploy/undeploy extracted, but only Copilot used them — the REST routes reimplemented workflow authorization inline, and `PATCH /api/chat/manage/[id]` additionally owned password encryption, the auth-type field-clearing matrix, identifier uniqueness, the redeploy-gating protocol with two 409s, a raw db.update, and a manual recordAudit. New `lib/chat-deployments` domain: - `chat_deployments.list/read/update/delete`, keyed on the deployment whose workspace is derived by joining its workflow. Creation stays `workflows.chat.deploy`, which is keyed on the workflow. - The PATCH extraction, including the field-clearing matrix and the asynchronous-cutover invariant the route had hand-mirrored from `performChatDeploy`. - One `buildChatDeploymentUrl`, replacing three constructions that had already drifted onto two different host helpers. There is no chat subdomain, so nothing publishes a host. - Repository reads moved out of the use cases into `lib/chat-deployments/queries`. Internal routes are now adapters over those use cases. `GET /api/chat` is deliberately not migrated: it scopes by `chat.userId` while every other chat operation authorizes by workspace admin, and reconciling the two is a product decision. `PATCH` keeps its 400 for an identifier collision through a typed `ChatIdentifierInUseError`; v2 reports the 409 the condition actually is. v2 surface at `/api/v2/chat-deployments`: list, create, read, update, delete. Workspace-scoped, keyset-paged, and a stored password is never readable — reads carry `hasPassword` only, and the session-only reveal endpoint deliberately has no v2 counterpart. Also: an email- or SSO-gated chat with an empty allow-list is now refused in the use case rather than only at the internal boundary, since it is unenterable; and the doc comment on `processHostedKeyCost` claiming a `usageLog` write is corrected — no such write exists. Route ratchet 1160 -> 1165; OpenAPI operations 171 -> 176. * fix(api): close three review findings, two of them caller-reachable - Run output files are filtered to keys under the run's own execution prefix. The recording they came from is not a trustworthy key source: the start block copies every caller-supplied input field verbatim into its output and `collectUserFilesById` accepts anything carrying the `UserFile` shape, so a caller could name any storage key and have the download and base64 paths — neither of which authorizes per file — serve it back. - `getBlock` reads own keys only. `BLOCK_REGISTRY` is an object literal, so `constructor`, `toString` and friends returned inherited functions that every consumer then treated as a block, turning a path segment into a 500. `getToolMetadata` already guarded this way. - A folder-scoped log page no longer unions in every job run in the workspace. The guard read `filters.folderIds`, which the public surface never sets — it carries the folder filter in `folderScope` — so the page contradicted the contract's promise that job runs are dropped whenever a filter they cannot answer is set. Also: the log cursor stamps `includeJobRuns` only when it is on, so its `.default(false)` no longer puts a constant in every fingerprint and rejects cursors minted before it existed; and the `folderName` subquery is scoped to the workspace and to workflow folders instead of scanning the whole `folder` table. * fix(api): close two more review findings, one an authorization bypass - `workflows.operations.apply` no longer admits a workspace API key. The use case authorizes against three per-user policies — the EE permission config, block visibility, and credential reachability — and all three take a human subject. An actorless key has none, and both substitutes fail open: attributing to the workspace billing owner evaluates the batch as the least-restricted account in the workspace, and passing no user makes `getUserPermissionConfig` return `null`, which every caller reads as unrestricted. Either way a workspace constrained by an allowlist was edited as though it were not. Personal keys keep the capability, so headless editing is unaffected for a credential that names a human. - `GET /workflows/{id}/state` reads its variables through `parseWorkflowVariables`, and the stored variable response schema drops the two assertions the column cannot honour. The column has carried a JSON string and a legacy array as well as the current record, the realtime `variable.add` op types `type` as `z.any()`, and the parser writes `name` through verbatim — so the input bounds on the read turned a stored workflow into a 500 on the endpoint that opens it. The write schema keeps them, which is where they can still be honoured. - `GET /workflows?scope=archived` projects folder paths tolerantly. Archiving a folder cascades onto the workflows inside it but leaves their `folderId` dangling — which is why restore has to null it — so the strict projector threw a bare `Error` and took the whole page down with no cursor able to step past the row. * fix(files): bind Start-block file keys to the executing workspace The Start block derived a file's storage key by parsing the caller's own `url`, which `isInternalFileUrl` matches on any host and `extractStorageKey` returns verbatim — so a request body could name any tenant's bytes. The key is now accepted only when its own layout names the workspace the execution runs in, and every file is dropped when the execution carries no workspace. Also bounds `includeFileBase64` with an aggregate response ceiling and a worker pool instead of an unbounded `Promise.all`, scopes the bulk download's authorization resource to the workspace when folder paths are requested, makes the folder-restore selector mutually exclusive at the type level, and names the bound in the `maxBytes` validation message. * fix(api): close v2 log review findings Cursor scope: `scope` on the workflow and table lists carries `.default('active')`, so it entered every fingerprint as a constant and refused every cursor minted before the param existed — with the "cursor does not match the requested filters" 400, which is actively misleading for a caller that changed nothing. Both now stamp the default as absent, so only a caller who asked for `archived` gets a new sequence. Dashboard stats: `maxWorkflows` capped the response, not the allocation. Segment series are now densified after the cut instead of before, so returning 200 series no longer materializes one `segmentCount`-length array per workflow in the window. The aggregate still sums every workflow, now from the sparse per-workflow maps. Cost keyset: `cost_total` is an unconstrained `numeric`, so its anchor travelled through `Number()` and was compared back at full precision — rows differing beyond float64 collapsed onto one anchor. Adds `decimalKey`, which carries the digit string and binds it `::numeric`. Run detail: `cost_total` is a backfilled projection, so a run predating the backfill reported `cost: null` even with a real ledger, making `items` unreachable for exactly the runs the ledger explains. Falls back to the ledger total. Also caps the log folder-path index reads at MAX_FOLDERS_PER_WORKSPACE like every other reader, publishing the folder-tree 413 on the four log operations; reverts a dead `status` widening in `v2CommaListSchema`; drops an unread `executionData` select; corrects the segment-count and searchLogs prose; and replaces the sort-cursor SQL-text assertions with a two-page walk over a fixture with a null block. * fix(api): close knowledge v2 review findings - widen knowledge.list_archived to the delete/restore policy so a workspace API key can discover what it may restore - escape LIKE wildcards on the now-public chunk search - derive tag slot capacity from TAG_SLOT_CONFIG per field type - type updateKnowledgeBase's chunkingConfig as ChunkingConfig and project every declared field explicitly - attribute a restore to the calling surface instead of a literal 'api' - gate 'knowledge chunks batch-update' behind --yes, since it can delete - present the tag-cleanup action from the parsed request rather than faulting on the domain result after the delete committed - unbind asserted-scope workspaceId from the nested knowledge cursors, matching the table-row lists - add executed-SQL coverage for the chunk keyset * fix(catalog): close the catalog and registry-boundary review findings The module-graph ratchet treated an entry with no baseline row as informational, so the six catalog routes and the projection barrel were unratcheted while the summary still read "within their module-count baseline". An unbaselined entry now fails --check, the summary counts only what was actually compared, and the baseline is re-recorded. The Copilot block-metadata tool — the reason the shared projection exists, 6,756 modules down to 1,321 — was in no guarded subtree. It is now an entry source and a catalog boundary root. Catalog behaviour: - hostedApiKey is gated on the deployment, so a self-hosted install reports none instead of promising 127 tools' keys it will never supply - block detail resolves an unversioned base type to its newest version and projects through the viewer's visibility, so it can no longer 404 a block the list contains or name it differently - offset-cursor ordering compares code units rather than the process locale - projections copy every array they publish instead of handing out the registries' own - an options function returning a thenable throws rather than silently widening the providers-store substitution across the event loop - a throwing block projection costs the Copilot tool one block, not all of them - the trigger-kind log returns to debug: chat/manual/api are entry-point kinds, not authoring defects Also sweeps the custom-block detail branch against its response schema, guards each projection module rather than the dead barrel over them, and drops a provably dead branch in processHostedKeyCost. * fix(workflows): close v2 workflow-authoring review findings - Read a blockless draft back as an empty graph. `PUT /state` of `{ blocks: {}, edges: [] }` — the contract's own published example — deletes every block row, and the loader answers `null` for a blockless workflow, so the following `GET /state` answered 404 while the list endpoint still showed the workflow. Existence is the workflow row's to decide; the null is now projected as an empty graph. - Rewrite the `readWorkflowGraph` authorization test so it can fail. It called `authorize?.()` and asserted only a negative, so deleting `authorize` or replacing it with a no-op both passed — the invariant the head-safe `HEAD` path depends on. - Route `setWorkflowBlockEnabled` through `replaceWorkflowNormalizedState`, the same door the other two graph writes use, instead of writing the normalized tables itself without state preparation or custom-tool extraction. - Count applied operations directly. Enablement refusals landed in the same skipped-item array and were subtracted from the operation count, which `Math.max(applied, 0)` then masked when it went negative. - Give a `disabled_ancestor` refusal its own member of the published skip enum instead of reporting it as `block_locked`. - Refuse an `atomic` batch whose credential or hosted API key would be stripped, and carry the dropped inputs in the 409 details. - Publish the whole lint report — `sources`, `sinks`, `orphanBlocks`, `emptyOutgoingPorts`, `invalidBranchPorts`, `invalidConnectionTargets`, `fieldIssues`, and the `kind` discriminator on unresolved references — rather than only free-text reference prose. - Stop reporting unresolved lint references as `inputValidationErrors`. `collectUnresolvedReferences` is read-only, so those values stay persisted; they were double-reported, and falsely as dropped inputs. - Replace two unfalsifiable negative-principal tests, which used a principal kind `Exclude`d from `PrincipalKind`, with a reachable one. - Nits: drop a stranded TSDoc block; assert the membership predicate in the selector-validator admin test; make the HEAD test assert a representation; assert the sanitized graph is what `replaceWorkflowState` writes; add a route test rejecting `baseGraph` in a v2 body; carry `principalAuditSource` on restore/duplicate/moveBulk audit; unify the two `base64MaxBytes` ceilings on `MAX_INLINE_MATERIALIZATION_BYTES`. * fix(api): close seven v2 review findings, one an authorization bypass Raise mcp_servers.workflow_deployments.update_server to admin: its body carries isPublic, and a public server executes with no Sim credential, so a write member could remove authentication from every workflow it publishes. create_server already grants the same visibility at admin. Pin the widened operations in registry tests — the six workflow-MCP ones, the four workflow widenings, and files.extract_archive. Reject secret fields on a credential that has no rotatable secret instead of dropping them behind a 200, classify a non-transient provider 4xx as caller error rather than a retryable outage, and reconcile a provider outage to 503 with Retry-After on all three surfaces. Give /v2/meta a declarative principal policy through a new defineOperation factory, carry the key expiry on the auth context instead of reading the api_key table from the application layer, and make the impossible principal branch an invariant error rather than a codeless 403. Enforce the rollout-gate exemption at definition time, against the one contract path it is reserved for, and remove the gate parameter from the exported admission helper so the builder is its only door. * fix(tables): bound the run-state sidecar, and close six v2 review findings Enforces the 2 MiB run-state ceiling INSIDE the sidecar drain rather than over its materialized result, refuses the unbounded query form paired with it, and normalizes the two stored blobs the v2 surface publishes from bare `as` casts. - The run-state budget now travels into `loadExecutionsByRow`, which drains row ids in bounded chunks and refuses before fetching the next one. The post-hoc `requireBoundedRunState` walk is gone: it measured a spike that had already happened, and re-serialized every entry to do it. - Both row reads that accept `includeRunState` cap the page at `V2_MAX_RUN_STATE_ROW_LIMIT`, and `POST /tables/{id}/query` additionally refuses the flag paired with `limit: 0`. - `runState.status` and the enrichment cascade blob are projected onto the published shape before presentation; both were caller-reachable 500s on a well-formed read. - `sim tables bulk-delete` now gates behind `--yes`, and the CLI sweep that should have caught it covers destructive non-DELETE forms. - `POST /tables/{id}/restore` is idempotent (200, no audit) like its knowledge sibling, and bulk folder selection deduplicates after resolution. - The batch-update backstop keeps the looser Copilot ceiling and says so in TSDoc: it is a backstop no surface reaches, because the contracts stop a v2 caller at 1000 and the Copilot tool stops itself at 5000. Each caller sees the bound that actually applies to it; neither surface's cap moved. * fix(chat-deployments): close v2 chat review findings Fixes the chat-deployments slice of the v2 review, several of which are regressions the application-operation extraction introduced. - Stop a `500` on schemaless JSONB: the response now declares a stored shape without bounds and `toV2ChatDeployment` projects `customizations`, `outputConfigs`, and `allowedEmails` onto it. The request schemas keep `.strict()` and their bounds. - Restore the specific validation message on `POST /api/chat` and `PATCH /api/chat/manage/[id]`, and the deleted test that pinned it. - Restore `chat_deployments.read` to workspace `admin`; the detail read serves the visitor gate. - Narrow the list projection so `chat_deployments.list` can stay a `read` operation reachable by a workspace API key: `allowedEmails`, `hasPassword`, and `customizations` are gone from the list entry and available only from the admin-gated detail read. Serialized field by field so a field added to the detail shape cannot reach the list by default. - Classify create-path failures: `performChatDeploy` carries an `errorCode`, so an in-flight deployment is a `409` and an invariant failure a `500` instead of every refusal being a `400`. - Delete the callerless `GET /api/chat`, which served the encrypted password column with no response contract. - Propagate undeploy infrastructure failures instead of concealing them as `404`, and return `ChatDeploymentView` from both delete paths. - Name `CHAT_AUTH_MODE_NOT_PERMITTED` on the create path. - Guard `getBaseUrl` inside `buildChatDeploymentUrl`, which otherwise throws on a self-host with no `NEXT_PUBLIC_APP_URL`. - Correct the published allow-list claim: a replacement `allowedEmails` is applied after the auth-type clear, so it does survive. - Assert `workspaceId` on the v2 detail routes and reconcile the concealment TSDoc with what the error policy actually renders. - Move `resolveActiveWorkspaceApplicationContext` to the workspaces domain so chat-deployments no longer imports workflow application code. * test(credentials): pin the reconciled provider-outage status The internal route alone answered 502 where the v2 surface, the shared status helper and `PROVIDER_OUTAGE_CODES`' own TSDoc all say 503. The test pinned the divergence; it now pins the reconciliation, including the `Retry-After` a 503 carries. Corrects a stale comment that still named 502 as the value callers see. * fix(executor): restore cloud-storage Start files, dropped by the key rule The ownership check accepted a key only when it could be parsed out of an internal `/api/files/serve/...` URL. But the server-side uploader for run inputs returns a *presigned cloud* URL whenever object storage is configured, whose path is the bucket key — so every chat-deployment attachment, API `files[]` payload and generic-webhook file field resolved no key, and because normalization is all-or-nothing the entire `files` input was dropped with no error. It passed locally and under vitest only because the uploader falls back to an internal URL when no object storage is configured, which is exactly why no test caught it. The test is ownership, not provenance: a key is accepted when its own layout names the executing workspace, whether it arrives directly or is parsed out of the URL. Neither field has to be trusted, since both are caller-authored and both are held to the same check. A payload whose key and URL disagree is refused rather than resolved in the caller's favour — a genuine uploader writes the two consistently, so only a forged pairing is turned away. `context` is now derived from the accepted key rather than read from the payload or the URL's `?context=`, so an owned key can no longer be labelled with a bucket its bytes do not live in — the hardening the previous comment claimed but did not perform. * fix(api): close four defects the fix pass introduced - `resolveLatest` built a `RegExp` from the caller's block id and read the registry with a bare lookup, so the catalog detail route — moved onto it by the version-alias fix — routed around the `ownBlock` guard added for exactly this. `GET /api/v2/blocks/%5B` was a `SyntaxError` 500 and `.../constructor` an inherited function. Matched by string comparison now, the way `tools/tool-ids.ts` resolves the same convention, and read through `ownBlock`. - The run-state byte budget was applied inside `queryRows` rather than at the callers that publish it, so the first-party table grid — which reads run state at five times the row limit and publishes no ceiling — turned a large page into a hard failure, with an error naming a parameter it does not expose. The budget is now an explicit option the public reads pass and internal callers omit. - Three graph-write CLI commands shipped ungated because the sweep meant to catch them matched only the names already enumerated, so it could never fail. It now forces every non-`GET` operation into a destructive or non-destructive list, and the three carry confirmations. - Unbinding `workspaceId` from the knowledge-documents cursor was right on the merits and wrong in effect: the value is constant per sequence, so removing it changed the fingerprint and refused every cursor already in flight. Restored there; the chunks list is new in the same change and keeps the cleaner reading. * fix(api): resolve a detail read to a version the viewer can see `getLatestBlockForViewer` took the newest version and then hid it, which inverted the contradiction it was written to close: `slack_v2` and `table_v2` are preview-gated while their v1 deliberately stays in the toolbar, so an unrevealed viewer got a `404` on a detail read for a type `GET /api/v2/blocks` was listing in the same breath. It now walks versions newest-first and answers with the first one visible to that viewer. Also: - The chat password guard ran after `performFullDeploy`, so a request that could never succeed burned a real workflow deployment version and then answered 400. Its two sibling gate guards already refuse ahead of the deploy; this one now does too. - The Copilot sub-block serializer published the registry's own `options` and `dependsOn` arrays by reference. Pre-existing, but the catalog projection this parallels copies every array it publishes precisely because they are process-global and shared by every request. * fix(api): restore the locked read-modify-write and the password validator Two findings verified as real regressions against staging, out of ten checked — the rest were pre-existing, latent, or false. `setWorkflowBlockEnabled` read the graph outside the row lock and wrote it back inside a later transaction. The editor's own save takes that same lock, so an autosave committing in the window was silently discarded: this operation writes a whole graph, not a delta. The persistence primitive now accepts a reader that runs after the lock is taken, and the toggle re-reads and re-decides there. Its lock predicate is also scoped to the workspace and to a live row again, so a workflow archived mid-flight is refused rather than written. The v2 chat-deployment contracts inlined their own password rule twice instead of using `chatDeploymentPasswordSchema`, losing the refusal of a whitespace-only password — which the internal contract rejects precisely because it strands the deployment behind a password the visitor form will not submit. Both sites use the canonical validator now. * fix(api): one folder projection, one dynamic-provider list, honest 413s - `toV2Folder` existed twice, and the second copy had been written without the name/path invariant — so a row the list read refuses loudly would have been served with a mismatched pair by the restore read. One definition, guard included. - The catalog projection restated `DYNAMIC_MODEL_PROVIDERS` and had drifted by one member. Derived from the canonical list instead. - The tables reads documented a `413` for run state that they cannot emit — the budget became opt-in, and the row limit is the bound now — so the claim is removed rather than declared. The workflow run read has the opposite problem: it genuinely emits one, on a single file *or* the run's inlined total, and declared neither. Now declared, and the sentence covers both. - Reclassifies the operations staging added into the destructive sweep, so the triage stays exhaustive. * fix(v2): classify storage and uniqueness failures, drop permanent file delete - remove the permanent file-delete endpoint; the platform offers no such action in the UI, and its manager wrote outside a transaction with no storage accounting - extract a generated document's text from its compiled artifact rather than its generation source, matching the download path; a `.pdf` source was a 500 and a `.docx` source returned generator JavaScript as clean content - report a run file whose object retention has already swept as 404 rather than 500, on both the inline base64 read and the download stream - report a knowledge tag that loses at a unique index as 409, naming whether the slot or the display name is taken - gate `workflows versions revert` behind a CLI confirm; it overwrites the draft graph and was classified non-destructive * feat(v2): report lint from both graph writes and add dry-run previews - `PUT /workflows/{id}/state` now returns the same `lint` report as `POST /operations`; an agent authoring a graph from scratch needs the findings at least as much as one editing incrementally - extract the report into one shared builder so the two writes cannot drift, and one shared presenter so the wire shape is identical - skip the credential/tool reference pass when the caller has no human subject, rather than resolving it against the workspace billing owner: that would misreport what the workflow can reach and disclose another person's grants. `lint.notes` says when it was skipped - add `?dryRun=true` to both graph writes: validates and lints, persists nothing, records no audit, notifies nobody. A query param, not a body field, since the body of a PUT is the resource itself - CLI: a dry run no longer demands `--yes`; requiring confirmation to preview a change teaches callers to pass `--yes` reflexively - CLI: name the graph commands for their verbs — `workflows state get`, `workflows state replace`, `workflows operations apply` — instead of the derived `state list` / `state update` / `operations create` - document when to use `rollback` vs `versions/{version}/activate` on both * chore(docs): sync generated docs manifest for the new CLI pages * feat(v2): add the missing workflow-MCP reads and align bulk naming - add `GET /workflow-mcp-servers/{serverId}` and `GET /workflow-mcp-servers/{serverId}/tools`. The resource could be PATCHed and DELETEd but never read, and its tools could be published and unpublished but never listed — the server list reports tool names only, so nothing published the `workflowId` that addresses a tool for deletion. Both mirror `mcp-servers` beside them, and carry that family's workspace-API-key denial rather than the wider `mcp_servers.read` policy - rename `POST /tables/bulk-move` to `POST /tables/move`, so tables matches the shipped `files` resource exactly (`move` + `bulk-delete`) - name the CLI commands for their operations instead of the derived `... create`: `tables move`, `workflows move`, `tables bulk-delete`, and `tables rows update-each` for the per-row batch, which sits beside the existing filter-based `tables rows batch-update` `POST /tables/{id}/rows/batch-update` keeps its name: a distinct payload per resource is precisely AIP-234 BatchUpdate, and `bulk-` would have collided one word away from the filter form. * fix(v2): correct documented statuses and a caller-reachable 500 - duplicating a workflow into a locked destination folder answered 500: `FolderLockedError` is a plain Error carrying `status = 423`, which the v2 error policy does not classify. Converted to OrchestrationError('locked') at the application boundary, matching the bulk-move path - restore workflow promised a 413 for an oversized folder tree that its response list never published; the cap is real, so the status now is too - move workflows and apply variables documented 409/423 they cannot emit: every per-item lock and conflict is reported in `failed`, not thrown - bulk download and delete knowledge tag can both 409 and did not say so; cleanup tag definitions cannot and did say so - apply workflow operations denies workspace API keys but never documented it - the dry-run responses are not byte-identical to a committed write: `needsRedeployment` describes the pre-write state and persistence warnings cannot appear. Reworded rather than overclaimed - read file text and get file upload are head-safe, so the "HEAD skips the effect" sentence did not apply to them * fix(v2): guard tag field-type changes and publish the 415 every body route can return - `PATCH /knowledge/{id}/tags/{tagId}` accepted a `fieldType` incompatible with the slot the tag already occupies. Slots are enumerated per field type, so a text tag could be relabelled `number` and every later read would interpret its values as the wrong type. Create checked this; update now runs the same two checks - derive `415` from the contract the way `413` already is: the JSON builder answers UNSUPPORTED_MEDIA_TYPE for any body under a content type it cannot read, so all 100 body routes could return a status none of them published - give version activation its own result component instead of publishing it as `RollbackResult`; the shipped rollback keeps that name - correct descriptions that promised behaviour the code does not have: a `processingStatus` field never returned, a `gmail_send` resolution example that short-circuits, bucket widths that overflow the window, a bulk tag save that relocates rather than overwrites, and per-server tool names actually gathered under a page-wide budget - drop 409 from three knowledge and upload reads that cannot emit it * docs(v2): correct the upload transfer contract and 16 other published claims The upload transfer step was documented as Sim's own data plane on every deployment: "success is 204" and "a failure is the v2 error envelope". That holds only when Sim stores objects itself. With object storage configured the URL is the provider's presigned URL, so S3 and GCS answer 200 and Azure 201, and a failure is the provider's XML — a client written to the old text reads a successful cloud upload as a failure. Also states that part ETags do not need retaining: completion takes no body because Sim lists the parts from the provider itself. Other corrections, all to shipped descriptions rather than behaviour: - DELETE table and bulk-delete files archive rather than erase, and neither said so; bulk delete also cannot emit the 409 it declared - complete knowledge upload published a 402 only the create leg can raise - billing status conceals a foreign workspace id as 404, not the 403 its TSDoc and description both claimed - audit entries null a folder's resourceId and strip folder ids from metadata at every level; neither redaction was documented - details=full adds the workflow summary to workflow runs only, never to job runs; GET /logs folderPaths covers a subtree like its two siblings; getLog now carries the retention sentence - list secrets returns description too, and the logs and resources documents described only part of what they serve * improvement(api): consolidate the v2 surface and close seven defects Endpoint consolidation: - Fold POST /logs/query into GET /logs; add sortBy/sortOrder, cap the comma lists, and move the list onto the shared keyset codec - Fold GET /knowledge/archived into GET /knowledge?scope=archived, matching files, tables, and workflows - Re-home chat deployments as a singleton under the workflow they belong to; keep the workspace-scoped discovery list - Move the tag-definition writes off the document path onto /knowledge/{id}/tags, where they already acted - Nest the table export and dispatch reads under their parent table Defects: - Publish isPublicApi on the deployment read; it was write-only, so a workflow could be opened to unauthenticated execution unauditably - Stop publishing raw storage keys and an unusable URL in log files - Classify a chat-identifier unique violation as 409 rather than 500 - Fall back to the root path instead of throwing when a knowledge base's folder is archived - Cap bulk-download at the ceiling it actually enforces - Normalize variables through one helper on both graph write paths - Fix a folder-name log filter that matched workflow names Naming and gaps: - Rename /files/{id}/extract to /unarchive, /rows/find to /rows/search, /rows/batch-update to /rows/bulk-update, /columns/run to /dispatches - Type the last six generic [id] path segments - Add table folder restore and id-addressed dispatch cancel * chore(audits): record the v2 catalog routes in the boundary baseline * fix(api): accept a null chat password and correct three published claims - performChatDeploy validated `password: null` as a password, so the replace-shaped chat PUT answered 400 for every mode that owns no password — public (the default), email, and sso. The declared payload type has always allowed null, and the stored value is cleared by authType regardless, so null needs no validation of its own. The route test could not catch it: it mocks the orchestration module and pinned the exact null the real guard refused. - Redirect the two docs slugs this branch retired that were genuinely published: findTableRows and runTableColumns. - The table folder restore described an idempotent no-op for an already active folder; it answers 404. Say so, and say where the path comes from, since the tables folder list cannot yet report archived folders. - Name the customizations exception to the chat PUT's replace semantics. - A cursor-binding case used status=error, which is a level and not a status, so it failed contract validation and never reached the cursor check. Use an accepted value and pin the reason, not just the status. * chore(cli): classify the new v2 chat operation as non-destructive * feat(cli): expose canonical resource URLs * fix(api): close three caller-reachable failures found by the final probe - GET /files/{fileId}/text called parseBuffer unguarded, and parseBuffer signals every failure as a bare Error that no v2 policy classifies. A zero-byte upload or a mislabelled archive was an unhandled 500. Empty bytes now answer empty text — a zero-length file has no text — and unparseable bytes answer 409, matching the rendered-artifact resolver. - GET /workflows/{workflowId}/state asserted write-side bounds over stored data. workflow_blocks.name and .type are bare text() and the realtime rename op accepts z.string(), so a block renamed past 255 characters made the workflow unreadable, and unrepairable, over v2. The read shape now takes the same input/stored split the variable schema already had. Stored subflow conditions are coerced in the loader beside the existing numeric guards. - GET /knowledge stamped scope into the cursor fingerprint unconditionally. scope defaults to active and is new on that list, so every cursor the deployed build handed out would have been refused with a message saying the caller changed a filter they never sent. Its siblings already carry the guard and the comment. Also: POST /workflow-mcp-servers answers 201 like every other v2 create; GET /logs/stats reuses the log list's entry ceilings; the execute and resume routes install the media-type-aware 415 they publish; and a rationale citing an endpoint that never reached the wire is corrected. * fix(tables): carry the dispatch terminal timestamps through the stale sweep Staging's abandoned-dispatch recovery builds its own `DispatchRow`, and this branch had added `completedAt`/`cancelledAt` to that shape for the id-addressed dispatch read and cancel. The merge was textually clean and left the new mapping short two fields. * fix(workflows): deny workspace API keys on the graph replace `PUT /workflows/{workflowId}/state` stores blocks and their tool wiring wholesale, but the policies deciding which of those a member may add — the EE permission config and block visibility — take a human subject. A workspace API key has none, and both substitutes fail open: the billing owner is a different, typically less-constrained person, and passing no user makes the permission lookup return null, which every caller reads as unrestricted. That made the replace a second graph-write door storing what its sibling `POST /workflows/{workflowId}/operations` refuses, which denies workspace keys for exactly this reason. Both doors now agree. Personal keys keep the capability, so headless authoring is unaffected for a credential that names a human. * chore(api): drop the enrichment catalog endpoint GET /api/v2/enrichments listed the code-defined table enrichments. The per-row enrichment run detail stays; only the catalog read goes. Removes the route, contract, response and query schemas, the semantic operation, the use case, the projection module, its registry-boundary entry, and the CLI command. The "not found" and "blank search" cases it covered are repointed at the connector-type sibling so the shared behaviour stays tested rather than deleted with it. * improvement(api): make the workflow operations endpoint self-describing Two things stood between this endpoint and a caller who has only the published spec. The accepted `params` keys existed only in the edit engine's source. The spec said "the accepted keys depend on the target block type", so a caller reading it could create a nameless empty block and nothing more — while the Copilot tool catalog, over the same engine, has always spelled out the envelope. That guidance now lives in the contract, shared by the add, edit, and insert_into_subflow parameter schemas so the two surfaces cannot describe one engine differently: `inputs` keyed by sub-block id, `retry`/`triggerMode`/`advancedMode` beside it rather than inside it, `connections` keyed by source handle, and `removeEdges` for dropping one edge without restating the rest. A `block_id` that is not already a UUID is replaced with a minted one, and the mapping was computed and then dropped. A caller could not reference the block it had just created except by re-reading the graph and matching on name. The engine now returns it and the response publishes it as `mintedBlockIds`, with the in-batch versus cross-request rule stated in the operation description. * fix(api): close the pre-merge scan's blocking findings Docs and public wire, all of it permanent surface once released. - Two published tag groups, Catalog and Meta, had no sidebar entry in any locale, so six operations shipped unbrowsable. Added to all six, and a test now fails when a published tag has no entry. - deleteWorkflowChatDeployment pointed callers at DELETE on /deployment; the undeploy verb is on /deploy. - PUT /state still promised workspace API keys a degraded lint pass after the operation began rejecting them outright. It also lived in a single-quoted string, so the shared clause would not have interpolated. - POST /tables/move took targetFolderPath as nullable-but-required, which rendered the CLI flag as a required `<json|@file>`: `--to /Archive` failed to parse and omitting it failed outright, while the docs said "omit for root". Now optional and a plain string, matching POST /files/move; the route supplies the null the use case wants. - Table dispatch status and row run state published `cancelled` beside imports, exports, and job state publishing `canceled`, and the note explaining the split was wrong about its own sibling. Both new schemas now publish `canceled`; the stored column is unchanged and mapped at the presenter. The shipped `cancelled` count field is left alone. - applyWorkflowVariables can answer 423 and both workflow-MCP deletes can answer 409; none declared it. Hand-assembled error lists replaced with the shared sets. - /files/{fileId}/unarchive became /unzip. `extract` reads as "extract text" and `unarchive` reads as the inverse of restore on a resource where archived means soft-deleted; unzip collides with neither and is what the implementation calls itself. * improvement(cli,docs): finish the naming and clear the enrichment leftovers - Four single-record GETs derived to `list` while returning one thing: meta, a workflow's chat deployment, log stats, and file text. Renamed to `meta status`, `workflows chat status`, `logs stats`, `files read`, matching the `workflows deployment status` correction already in this branch. None of the old spellings shipped. - Dropped five `renamedFrom` aliases pointing at spellings that never existed, each of which built a hidden command and a permanent deprecation warning for argv nobody could have typed. `tables rows find` keeps its alias — that one really shipped. - Removed the enrichment catalog from three prose sites left behind when the endpoint went: the resources spec description, its Catalog tag, and the contract and pagination-test comments. - Documented the six new command groups in the CLI index table. That page is a guide page, so the docs staleness check cannot flag it. - Published the `customizations` exception to the chat replace semantics. It was in the route TSDoc and invisible to every caller reading the spec, which is where the claim "Replace, not merge" is made. * improvement(cli): use batch- for the tables bulk delete, matching its siblings The CLI renames a bulk form only when it would collide with its singular sibling, and uses AWS's `batch-` prefix when it does — `files batch-delete`, `tables rows batch-delete`, `knowledge chunks batch-update`. `tables delete` exists, so `tables bulk-delete` was that same rename reaching for the other word, and the only `bulk-` command on the surface. No `bulk-` CLI command has ever shipped, so this costs nothing now and would be permanent later. `files bulk-download` keeps its name: there is no `files download` to collide with, and it is one archive rather than N operations. Its config block now says why it exists at all, since the command is never built — the builder skips non-JSON response modes, but the contract sweeps still read the entry and require the folder-path field to be marked. * fix(workflows): allow operations on blockless drafts * feat(workflows): add manual and run-from-block execution * fix(cli): update workflow run description test * fix(tests): align fixtures with current contracts * fix(api): derive chat activity from workflow deployment --------- Co-authored-by: Theodore Li <theo@sim.ai>	2026-08-25 04:19:34 -04:00
Theodore Li	7b761ba560	feat(setup): add incremental Chat setup (#7043) ... * feat(setup): add incremental Chat setup * fix(setup): retain existing Compose Chat config	2026-08-24 19:22:42 -04:00
Vikhyath Mondreti	0a5b3801ea	feat(secrets): let workspace secrets opt out of redaction (#7045) ... * feat(secrets): let workspace secrets opt out of redaction * fix(secrets): certify no sandbox exemptions once the registry is incomplete * feat(secrets): carry visible secret values on the v2 list and document visibility * fix(secrets): read visible values by own property so prototype-named secrets cannot poison the list	2026-08-24 13:46:57 -07:00
Waleed	6d313a4a94	refactor(renderer): drop the isWorkflowRunning prop the views never read (#7040) ... Removing only the unused binding and keeping the prop was half a fix. The views declared it, the app passed it, and nothing read it — so the prop was dead, and dead code does not become live by being documented. Its TSDoc claimed it "holds every block's action swell open". That behavior does not exist in either view. Keeping the prop on the chance someone wants it later is the speculative-generality smell: if the toolbar should pin open during a run, that gets implemented deliberately and the prop comes back with logic behind it. Removed from both view interfaces and from both call sites. The store subscription stays — `workflow-block.tsx` and `subflow-node.tsx` each passed the same value twice, once to the dead view prop and once to `ActionBar`, which has 28 real reads and is what the surrounding TSDoc is actually describing when it says the flag "only swaps Run for Stop and disables mutations". `workflow-edge-view` uses it too and is untouched. The renderer test that passed it loses the argument. Worth noting it set the flag to stage a workflow run, and since the view ignored it those two cases were never exercising the run state they name.	2026-08-24 12:04:56 -07:00
Vikhyath Mondreti	04380b79a5	fix(webhooks): requeue deliveries dropped by retryable setup infrastructure failures (#7038) ... * fix(webhooks): requeue deliveries dropped by retryable setup infrastructure failures * fix(webhooks): restore terminal log on failed requeue and make retry backoff abort-aware	2026-08-24 11:28:27 -07:00
Waleed	cae80c5f20	chore(lint): turn on the rules that would have caught the dead code (#7037) ... Three rules were off, so nothing enforced them. Measured, fixed the sites, and enabled them where the cost is bounded. `noAccumulatingSpread` — 2 violations, both real O(n²) reducers, both now `Object.fromEntries`. One duplicates a block's subBlocks on every block duplication; the other rebuilds a Record from every workspace env var. Enabled repo-wide. `noUnusedVariables` / `noUnusedFunctionParameters` — 633 repo-wide, but only 6 under `packages/`. Fixed those 6 and enabled both at error for `packages/**` via an override, which permanently covers 979 files. `apps/sim`'s remaining 627 are left deliberately: that is a sweep of its own, and a rule enabled with 627 outstanding warnings teaches people to ignore it. This is the class of rule whose absence let the dead code in #7019 accumulate — eleven unread loggers, a whole unimported file, write-only locals — none of which any gate could see. Two of the six were in `workflow-renderer`, where the fix is narrower than it looks. `isWorkflowRunning` is destructured-but-unread in both the block and subflow views, and the app passes it from `workflow-block.tsx` and `subflow-node.tsx`. Its TSDoc claimed it "holds every block's action swell open"; nothing reads it, so that behavior does not exist. Removing the prop breaks the callers and implementing it is a UX decision — there is adjacent logic deliberately not pinning the toolbar during a handoff. So only the unused binding goes, and the TSDoc now says what is true. Not enabled: `noDocumentCookie` (3 sites, and its fix is the CookieStore API, which is a browser-support call) and `useExhaustiveDependencies` (384 errors).	2026-08-24 11:10:19 -07:00
3 people	528b34f564	fix(workflow): hide idle nested subflow end handles (#6976) ... * fix(workflow): hide idle nested subflow end handles * perf(workflow): avoid repeated subflow edge scans * perf(workflow): stabilize subflow edge selector --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com>	2026-08-24 00:58:03 -07:00
Waleed	82b02fb4d9	improvement(ui): standardize modal default actions (#7029) ... * improvement(ui): standardize modal default actions * fix(ui): keep aggregate deletes on safe default	2026-08-24 00:43:08 -07:00
Waleed	cc087498ae	refactor(utils): add slugify and adopt it at the eight sites that hand-rolled it (#7018) ... The same three-step derivation — lowercase, collapse each non-alphanumeric run to a hyphen, strip the leading and trailing one — sat in eight files. Two of them carried a TSDoc line whose only job was to warn that they mirrored a third (`instance-org.ts`: "Derives a slug the same way the admin organization API does"; `consolidate-users-into-organization.ts`: "Mirrors the slug derivation used by POST /api/v1/admin/organizations"). A comment asserting two implementations agree is the shape duplication takes when it cannot be checked. All eight were semantically identical. Two anchored the strip with `-+` rather than `-`, and one followed it with a `--+` collapse, but `[^a-z0-9]+` has already collapsed every run by that point, so neither could ever match more than the single-hyphen form. Nothing changes. Truncation stays at the call sites. Four of them bound the result — at 24, 64 and 80 — and only `copy-chats.ts` strips again afterwards, because slicing can land mid-run and leave a trailing hyphen the earlier strip never saw. Folding a `maxLength` into the helper would have had to pick one of those behaviors and silently impose it on the others. `artifact-stylesheet.ts` keeps its copy: it lives inside the `SIM_ARTIFACT_SHELL` template literal and runs in the viewer's browser, where there is no import to resolve.	2026-08-23 18:57:52 -07:00
Waleed	a20a5465ef	perf(db): optimize recurring query paths (#7014) ... * perf(db): optimize recurring query paths * perf(logs): batch keyset export reads * fix(workspaces): type nullable member count targets * fix(db): harden query performance changes * fix(logs): guard export stream cancellation	2026-08-23 17:15:10 -07:00
Vikhyath MondretiandClaude Opus 5	43aba98527	fix(desktop): stop the OAuth connect callback from failing on a bare-path callback URL (#7005) ... * fix(desktop): stop the OAuth connect callback from failing on a bare-path callback URL The desktop connect launcher passed better-auth a same-origin path as its callbackURL. Better Auth stores that value verbatim in the OAuth state, and the callback's credential-draft reader parsed it with a bare `new URL()`, which rejects a path. That throw happened inside the `account.create.before` database hook, which better-auth's OAuth callback does not guard, so the provider redirect landed on a 500 after authorization had already succeeded. Send an absolute URL from the connect page, matching the workspace-scoped branch and every other connect surface, and accept a path-absolute callback URL in the draft reader so the shape can never fail the callback again. Protocol-relative and malformed values still throw, keeping an unreadable binding loud. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(desktop): compose the connect completion URL through the URL API Concatenating `getBaseUrl()` with the completion path leaves the result dependent on how the deployment spelled `NEXT_PUBLIC_APP_URL`: the helper only adds a missing protocol, so a trailing slash produced `//desktop/connect/complete`, a pathname that matches no route. The completion page is what bounces the OAuth result to the desktop app's loopback, so that typo would have stranded the flow just past the callback it was meant to fix. Both callback URLs in the page — the launcher's and the workspace-scoped authorize redirect's — now go through one helper that resolves the path against the base with `new URL`, matching how the same function already builds the authorize URL, with coverage for a trailing-slash base. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(urls): give base URLs the no-trailing-slash form their call sites assume `getBaseUrl()` returned `NEXT_PUBLIC_APP_URL` as the operator spelled it, while almost every consumer builds `${base}/path`. A base configured with a trailing slash therefore produced a `//path` pathname that matches no route, and broke the `startsWith(`${base}/`)` prefix checks that decide whether a redirect target is our own — the OAuth authorize route rejected its own completion callback and fell back to the workspace page, so the desktop handoff never ran on those deployments. The previous commit fixed one such URL; this fixes the reason it was wrong, for the ~30 concatenation sites that share the assumption. `normalizeBaseUrl` now strips trailing slashes alongside the protocol it already added, which is the invariant SITE_URL has always documented. A path-prefixed base keeps its path. `getInternalApiBaseUrl` gets the same treatment, since its callers concatenate identically. `@sim/testing`'s urls mock is a hand-written mirror of this module, so it moves in step. `internal-api-base-url.test.ts` now unmocks the module it names — otherwise it asserts against that mirror and any drift between the two passes unnoticed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(urls): stop claiming path-prefixed base URLs are supported The previous commit's doc and test said a path-prefixed base keeps its path. That reads as support for a deployment shape the app does not have: there is no Next `basePath`, so routes are served at the origin root and such a value could not address them however the base were normalized. Every documented example is origin-only. Says only what is true — trailing slashes are the one spelling absorbed — and reframes the test as pinning the trim's shape rather than asserting a path-prefixed deployment works. No behavior change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-22 23:28:00 -04:00
Theodore Li	bbf408bf30	fix(security): harden public auth rate limits (#6997) ... * fix(security): harden public auth rate limits * fix(security): fail closed without client IP * fix(security): backstop public OTP requests * fix(security): preserve independent rate-limit backstops	2026-08-22 21:51:58 -04:00
Theodore Li	8104c33bba	Fix knowledge connector sync follow-up (#6927) ... * Fix knowledge connector sync follow-up * Fix connector sync pause race * fix(knowledge): surface connector sync dispatch failures * fix(knowledge): make connector sync recovery durable * fix(knowledge): deduplicate connector sync dispatches * fix(knowledge): preserve pending connector syncs * fix(knowledge): lock connector sync snapshot	2026-08-22 20:50:48 -04:00
Justin Blumencranz	bb0792564a	fix(chat): copy workspace resources as portable links (#6956) ... * fix(chat): copy workspace resources as portable links * refactor(chat): simplify portable resource copying	2026-08-22 17:11:37 -07:00
3 people	c26529a82e	feat(bitbucket): add repository webhook triggers (#6934) ... * feat(bitbucket): add repository webhook triggers * fix(bitbucket): harden webhook trigger delivery * fix(bitbucket): address final trigger review * chore(bitbucket): address review conventions * fix(bitbucket): harden triggers and connector sync --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com>	2026-08-22 15:23:35 -07:00
Theodore Li	7173a3fb37	feat(status): surface major service incidents (#6987) ... * feat(status): surface major service incidents * fix(status): isolate status endpoint failures	2026-08-22 18:11:13 -04:00
Vikhyath Mondreti	7ea8692e7d	fix(files): read a file nobody vouched for, as an untracked one already is (#6986) ... * fix(files): read a file nobody vouched for, as an untracked one already is A workspace file whose secret provenance could not be recorded was refused by every model and runtime boundary at once — attachments, mounts, and the tool routes that parse, transcribe or describe a file. A file that was never tracked at all returned exact-empty and worked fine. Both say exactly as much about their contents: nothing. So the refusal was a permanent penalty for having tried to record provenance and failed, with no way back, since nothing rewrites a file's provenance but another content write. Workspace files now do what the other durable surfaces do — proceed and record an audit entry naming the surface and the count, so the people who own the secrets are told. The state is distinguished from the four that really are unknown (missing row, version mismatch, stale binding, malformed), which stay refused; only a current sidecar that says so reads as unrecorded. Absence survives a merge rather than dropping through to exact, so a derived file cannot launder an unrecorded input into a positive claim. Script migration 0007 clears the files already stranded in the old state, taking the parent lock first in id order to match the live writer, with the unknown status re-checked under it so a concurrent write is never undone. Nothing changes for a legacy file: no sidecar row still means untracked, still reads exact-empty. The enforcement flag stays in place. * fix(files): narrow an initialization to a status the column accepts The union carries three states; the sidecar's CHECK constraint accepts two. Only the replace path discriminated — initialize forwarded the status verbatim, so an 'unrecorded' would have reached the database as a value it rejects, aborting the enclosing transaction rather than writing a bad row. Nothing passes one today, which is why it belongs here and not in a caller. Also names the storage mismatch on the type itself: a stored 'unknown' reads back as 'unrecorded', not as the union's 'unknown'. Same word, two layers, different meanings. * fix(provenance): repair only what the reader calls an absence The repair selected on status alone, which is wider than the branch it was meant to undo. The reader answers 'unrecorded' only for a sidecar that is version 1, bound to the file's current bytes, and holding a well-formed entries array; anything else is a fault it refuses and no policy relaxes. Clearing a fault was not a smaller claim but a larger one. Deleting the sidecar sets secret_provenance_version to NULL, and a NULL version reads as exact-empty — so a file refused because its provenance described bytes it no longer holds would have come back positively vouched for, silently, with no audit entry. Four such rows exist in production today. Both the candidate query and the delete now carry every condition, so a concurrent write cannot lose the distinction between an absence and a fault. Stale, unversioned and malformed sidecars stay refused exactly as the surface refuses them, and recover as they always have, on the next content write. * fix(provenance): store an absence apart from a refusal Relaxing a stored `unknown` assumed it always meant "nobody recorded this". It did not. The sidecar collapsed two opposite claims into one value: bytes nobody recorded, and bytes a writer refused on purpose — a child of an archive whose parent provably held secrets, a generated asset whose safety decision came back false, a transcode whose scanner knew secrets were present and could not locate them. Relaxing the second would have walked known-secret-bearing content through every model boundary at once. The decision type already told them apart: a registry that never ran is `safe: true`, while every refusal is `safe: false`. Only storage lost it. Writers now persist `unrecorded` and `unknown` separately, readers relax the first and refuse the second, and the constraint is widened to accept it. Only workspace files store this distinction, deliberately. Every other durable surface produces a non-exact sidecar from one condition — an incomplete incoming bundle or registry — which is always an absence. Files are the only surface that derives one stored object from another, so they are the only one that can refuse on purpose. The shared policy is unchanged and uniform: read an absence and audit it, refuse a taint. Also aligns the metadata batch classifier, which answered `unknown` where the single-file reader answers `unrecorded`, leaving two classifiers describing one policy differently. * test(provenance): assert the enforcement switch on a row it can act on The test that pins "closing the surface refuses an unrecorded file again" used a stored unknown, which the split had just turned into a taint. A taint is refused whatever the flag says, so the assertion held with enforcement off too and proved nothing about the switch this posture rests on. It now uses a recorded absence, and fails if the flag is ignored. * fix(files): copy a recorded absence as an absence The copy path required an exact source, so a stored unrecorded one fell in with the refusals and the target was written as a taint the source never carried. A workspace fork or a chat file copy therefore turned a readable file into a permanently refused one — the pathology this surface exists to undo, reached by copying — and nothing rewrites a file's provenance but another content write. Safe past the scope checks it now precedes: those exist to stop one workspace's secret entries landing in another's file, and a recorded absence has no entries to carry. The legacy Function export marker keeps writing unknown. Its record had resolved secret names in scope and an unreadable bundle, so whether that is an absence is not something this change can establish, and guessing in the readable direction is what caused this round.	2026-08-22 15:05:39 -07:00
Vikhyath Mondreti	b861062370	fix(provenance): clear the rows that went unknown after the first repair (#6982) ... * fix(provenance): clear the rows that went unknown after the first repair 0005 is finished and will not run again — the runner records a name in script_migrations and never offers it back, which is the contract a run-once repair wants. But it cleared the backlog that existed at the instant it ran, and the writer that produced that backlog kept running until the fix in this branch. Nothing heals such a row in place, so each one goes on reporting on every later read; a few dozen of them account for thousands of log lines a week. A second entry rather than deleting the first's tracking row: the registry is append-only, and a repair that ran twice should say so twice. It shares 0005's walk rather than restating it — the parent-first lock ordering and the status re-check under that lock are subtleties worth having once — and is idempotent, so it costs one empty query if there is nothing left to repair. * chore(provenance): record the deploy ordering the second pass depends on promote-images needs migrate, so a script migration runs while the previous image is still serving. A row an old instance creates between this walk and the end of the rollout sits behind the cursor, and the name is recorded on success, so it is never offered again. Widening the walk would not help — the exposure is the minutes after it returns, not the milliseconds during — so the requirement is to ship it in a release after the writer fix is already promoted, which the file now says.	2026-08-22 13:35:06 -07:00
Bill LeoutsakosandBill Leoutsakos	be508b0789	fix(utils): prevent randomInt power-of-two hangs (#6955) ... Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local>	2026-08-22 10:48:38 -07:00
Waleed	8937bb3550	fix(kb): let the server say a connector sync is queued (#6968) ... * fix(kb): let the server say a connector sync is queued The connector chip inferred "a sync is coming" from `createdAt` inside a 2-minute window, because nothing on the row distinguished a queued sync from an idle connector until a worker took the lock. The guess was wrong under queue backlog and under client clock skew, and it forced a pile of client state to stand in for it. Adds `pending`, written as the sync is handed to the queue and cleared when a worker takes the lock or the hand-off is found to have been lost. It is a phase of the same lock `syncing` holds, so it opens the lease and takes an ownership token the same way — the lease is what the scheduler ages a stranded queue entry against (`updatedAt` cannot serve: a pending connector is still editable, so any unrelated write would renew the recovery it should trigger), and the token is what proves a late release belongs to this dispatch. Deletes the 2-minute window, the in-flight id sets, the 5-minute cooldown timers and the forced re-render they needed. The cooldown lived in a ref inside a modal, so it evaporated whenever the modal closed; the disable now comes from durable server state and is shared across tabs. Also fixes, all found while tracing the lifecycle: - An on-demand sync on a paused or disabled connector silently resumed it for good. Nothing could put the pause back: success writes `active`, a lost queue entry writes `error`, and the due-sweep keeps syncing that. Refused. - A failed hand-off no longer advances the connector's auto-disable breaker. A queue outage would otherwise increment every connector in the fleet until they all disabled themselves for a fault that was never theirs. - Manual sync on an established connector gave no feedback at all: the poll only ran while the predicate matched, which it never did. - Four over-broad invalidations that refetched every cached chunk page and chunk search in a base when one connector document was excluded. - The dead-process reporter re-sent a PATCH per stale document on every poll. * fix(kb): refuse to start a queued run on a paused connector The queue outlives the decision to sync. Pausing a connector after its run was queued cleared the queue entry's token but left the task itself alive, and the lock CAS accepted any row that was not already `syncing` — so the worker took the paused row and wrote its own terminal `active` over the pause. Moves the rule to the two points that can enforce it: an explicit `LOCKABLE_CONNECTOR_STATUSES` allowlist on the lock acquisition, and the same allowlist on `markSyncPending`, which closes the mirror race where a dispatch already in flight rewrites a just-paused row back to `pending`. Queueing and starting now agree on one rule, and a skipped hand-off is reported as its own outcome rather than a concurrency conflict. Also patches the connector detail cache alongside the list on an optimistic status write, so an already-expanded card starts its own sync poll instead of showing stale history behind the list's spinner. * fix(kb): make a queued sync prove it is the run that was queued `markSyncPending` minted an ownership token but only `releaseFailedDispatch` checked it, so the worker could consume a queue entry that was not its own. A task delayed past its lease is reclaimed and replaced; the status check alone let that stale task take the replacement's entry and run superseded options — a plain sync where the user had just asked for a full resync — while the replacement was turned away as `sync_in_progress`. Carries the token in the task payload and matches it at lock acquisition, the same discipline `holdsSyncLockToken` already applies to the `syncing` phase, extended to the phase before it. A superseded run is now reported as such rather than as a concurrency conflict. The payload field is optional for the rollout window only: tasks already in the queue carry no token, and stranding them would be worse than letting them fall back to the status check for one deploy. * fix(kb): report a paused connector as paused, not superseded Pausing a queued connector releases its token, so testing ownership before status reported every pause-while-queued — the common case — as a superseded dispatch. The mismatch is the symptom there; the status is the reason. * fix(kb): stop a status update landing on a run that already started The update's guards ran against a row read moments earlier and the write carried no compare-and-set, so a worker taking the lock in between meant the write landed on a `syncing` row — overwriting the run's status and, because leaving `pending` also clears the lock columns, wiping the token its heartbeat and terminal write match on. That stranded a sync that had already begun. The write is now conditional on the status the request was authorized against, and a lost race is reported as a conflict rather than "not found". Also restores the in-flight guard on the pause control. The optimistic status flip relabels it Pause -> Resume immediately, so a second click could send `active` before the first pause settled and resume a connector the user meant to pause. Read from the mutation's own pending state rather than the local id set this PR removed — React Query already knows which row is in flight.	2026-08-21 20:53:16 -07:00
Waleed	dce34932b4	improvement(menus): tighten dropdown spacing and cap the @ mention list (#6972) ... Menu rows composed their own gap (8px) instead of the platform's chipContentGap (6px), so a menu row and the chip it opens over spaced their icon/label pairs differently. Surface padding was 6px against an 8px row radius and a 12px surface radius, so a first or last row's rounding cut across the corner it sat in; 4px makes them concentric. Separators carried a 13px gutter against a 0px gap between rows in the same group, which read as the groups floating apart. DropdownMenuLabel padded to an uncontrolled, font-dependent height and sat two type steps below its rows in a colour heavier than them; it now composes the shared row height at one step down in --text-muted. The @ mention list showed every integration. The per-family preview cap defaulted to uncapped, and integrations are 300+ near-identical rows sorted first, so the unfiltered list was the whole catalogue and no other family was reachable without scrolling past all of it. Capping is now the default, families are labelled, the menu is capped shorter than the action-menu default since it floats over the chat input, and the integrations shown are curated popular ones rather than whatever sorts first alphabetically. Both hand-rolled menus that render plain buttons for Radix focus reasons now compose the exported dropdownMenuRowClass instead of re-deriving row chrome.	2026-08-21 20:22:04 -07:00
Waleed	4c8fec1a41	improvement(tabs): fade the tab strip with a mask, and shrink the tab chip (#6971) ... The scroll-edge fade was a gradient div tinted with the surface colour, laid over the tabs. Tabs paint their own fills, so at an edge that washed a pill toward the surface colour instead of dissolving it, and it was only correct while whatever sat behind the strip was exactly that colour. A mask fades pill and label together to real transparency — the way the command palette fades its results, and the way every other horizontal fade in the app is drawn. The ramp goes to 24px, close to the palette's, since a short one reads as a cut. The tab chip drops to 26px, leaving 7px above and below instead of 5px. The collapse toggle and the action buttons stay 30px: a tab paints a fill so its box is visible and wants air, where those are bare glyphs whose box only shows on hover. The close button now centres itself rather than encoding (band - 24) / 2, which would have put it off-centre the moment the band changed.	2026-08-21 20:21:27 -07:00
Waleed	8177f9abeb	fix(tabs): restore the resource header's spacing and clear the scroll fades (#6967) ... Porting the tabs onto the shared strip had taken the header from 43px to 34px, which moved the overlaid collapse toggle from 6.5px below the panel's top edge to 2px while its right inset stayed at 16px — the corner read lopsided. The header goes to 40px: still shorter than it was, with the toggle back to 5px. The toggle also gets its 8px radius back, dropped in that port for no reason anyone asked for. Selecting a partly-hidden tab scrolled it flush against the container edge, which is exactly where the fade gradient sits, so it arrived half-faded and still looked cut off. Reveal now insets by the fade width and clamps at the scroll extremes, where no gradient is drawn. Floating tabs cap at 160px so no single tab dominates the row.	2026-08-21 19:58:37 -07:00
Justin Blumencranz	3d40003482	improvement(tables): backfill default views for existing tables (#6899)	2026-08-21 19:25:58 -07:00
Waleed	c16883e9dc	improvement(tabs): give the resource tabs a quieter floating look (#6960) ... * improvement(tabs): give the resource tabs a quieter floating look Adds a `floating` variant to the shared TabStrip and uses it for the mothership resource tabs. Only the active tab carries a shape; the rest are bare labels divided by a hairline, sized to their content up to a cap. The browser and terminal strips keep the attached look, which stays the default. * improvement(tabs): align the floating tab ramp and icon size with the platform Each surface token now does the job it is named for: a bare tab hovers to --surface-hover instead of the Button variant's --surface-4, and a selected tab takes the rung between hover and --surface-active. Tab icons match the action icons beside them at 16px.	2026-08-21 19:01:37 -07:00
Vikhyath MondretiandClaude Opus 5	71129cd112	feat(custom-blocks): let a publisher decide whether their block's runs reach consumer traces (#6950) ... * feat(custom-blocks): let a publisher decide whether their block's runs reach consumer traces Joining a custom block's child run into its caller's trace shipped on by default, gated at read time by whether the person reading could already open the source workspace. That gate is doing the wrong job: a custom block's whole point is that consumers need no access to the source, so the check refuses exactly the readers the feature exists for, and it makes the answer depend on who is looking rather than on what the block's owner agreed to publish. The decision moves to the party whose data it is. `custom_block.trace_child_runs` is set by the publisher in Settings, applies org-wide, and is the entire policy — nothing downstream re-checks a caller. `getCustomBlockAuthority` already resolves per invocation and is the one lookup both the canvas handler and the Agent-tool runner pass through, so one column covers both surfaces and no consumer input can assert it. It defaults to FALSE. With the viewer check gone, an opted-in block publishes the source workflow's block names, inputs, outputs, and prompts to anyone who can read a consuming workflow's log. That is the same boundary curated outputs and redacted errors hold, so it opens by an affirmative act of the publisher or not at all — never as the residue of a column default on rows nobody revisited. Closed means the handle is withheld outright rather than persisted behind a flag: with no `childExecutionId` there is nothing for a reader, a migration, or a later refactor to join. What replaces it is a `_childTraceDisabled` marker, because a boundary span with no children renders exactly like a leaf block and an untraced run would otherwise read as one that did nothing. The consumer-facing failure `ref` is untouched either way — it is the only thing that makes an untraced failure reportable. Custom blocks invoked as Agent tools now join too. The child's handle already reached the agent's persisted `toolCalls[].result` (`postProcessToolOutput` strips only `__`-prefixed keys); nothing lifted it onto the tool span. Both span builders lift and strip it, and `hydrateChildTraces` needs no change — its boundary walk already recurses. The same handle is stripped from the model-facing copy of the tool result in `executeProviderTool`, the single point where the raw and model copies diverge: an opaque execution id in a tool result reads to a model like data the tool returned. The live SSE stream keeps one condition beyond the policy: an identified consumer. Not an authorization check — no workspace query — but chat deployments and the public API leave `liveTraceViewerUserId` unset because their consumer may be anonymous, and opting into org-wide tracing is not consent to stream a publisher's raw agent tokens to the internet. Copilot deliberately cannot set the field; exposing a team's internals org-wide is a human decision, not one an agent makes while publishing on their behalf. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(custom-blocks): read the publisher's trace policy at read time, not from the handle's presence Treating a persisted `childExecutionId` as proof of publisher consent is only true for handles this PR's writer produced. Every handle written before it meant something else — "a child ran; authorize the reader" — and the rows carrying them outlive the migration, so removing the reader check turned them into an open door: a consumer could open an old parent log and receive the source workflow's block names, inputs, outputs, and prompts from a block whose publisher never opted in. `hydrateChildTraces` now resolves the policy live, per boundary, from `custom_block.trace_child_runs`. The child log row's `workflowId` is the key — publish enforces one block per workflow — which also covers an Agent-tool boundary, whose span carries no block type to look up. A workflow with no block row (never published, or since deleted) has no publisher left to consent and stays shut, as does a failed policy read. This is not redundant with the write-time withholding. The handler still emits no handle for a block that was closed when the run executed, so such a run stays closed forever even if the block is opened later; this check decides whether the runs that DO carry a handle may still be shown. Turning the policy off therefore also closes what is already recorded, which is what a governance switch has to do to mean anything. Reported by Greptile on #6950. Also drops `any` from the trace-policy tests: outputs read through `Record<string, unknown>` (the handler's declared return does not name these internal keys) and failures narrow through `ChildWorkflowError.isChildWorkflowError`, which pins the failure type as well as its fields. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(logs): sum the child-trace drop counters from the struct, not a hand-listed set `totalDropped` re-listed four of the five counters, so a read whose only drops were policy refusals computed zero and skipped the log entirely. That is the commonest drop there is now — every handle written before the publisher policy existed refuses at that gate — so the one signal telling an operator the live check is closing joins went silent exactly when it started mattering. Summed from the struct instead. A hand-maintained list beside a struct is stale the moment a field is added, which is precisely how `policyClosed` was left out. Reported by Cursor Bugbot on #6950. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(db): renumber the custom-block trace migration around a 0299 collision Staging landed its own 0299 (`table_run_dispatches.heartbeat_at`) while this branch was open. The two migrations are independent — different tables, no shared statement — so only the number and drizzle's snapshot chain collided. Regenerated rather than hand-merged: a drizzle snapshot is a full-schema dump whose `prevId` links it to its parent, so editing one by hand to sit after a migration it was not generated against is how the chain silently stops matching the database. Staging's 0299 and its snapshot are taken verbatim; this is 0300, generated against them, and its SQL is byte-identical to what it replaced. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-21 18:21:08 -07:00
Waleed	f0a0970d2f	improvement(tabs): port resource tabs onto the shared TabStrip (#6953) ... Replace the hand-rolled resource tab bar with the shared TabStrip primitive, and generalize the strip where this caller needed more than it offered.	2026-08-21 16:29:47 -07:00
Waleed	7b6c58113d	fix(knowledge,tables): recover abandoned dispatches, bound the sweep and the workbook preview (#6945) ... * fix(tables,knowledge): recover abandoned dispatches and bound the sweep Three defects measured in production this afternoon. A dispatcher killed by an OOM left `table_run_dispatches` at `dispatching` forever. Every terminal transition on that table is user- or flow-initiated, so nothing reclaimed the row: four dispatches were stranded in one afternoon, pinning each table's "X running" overlay and blocking re-runs, with no way to clear them from the product. The `table_run_dispatches_watchdog_idx` index has existed for this sweep since the table was created, unused. Liveness comes from a new `heartbeat_at`, stamped by the per-window writes that already advance `cursor` and `processed_count`, so a slow-but-live dispatch is spared however long it runs — the in-process path has no duration ceiling, so ageing from `requested_at` would reclaim live self-hosted work. The sweep reads `COALESCE(heartbeat_at, requested_at)` so rows written before the column stay reclaimable rather than NULL-false forever, and runs as the last arm of the existing stale-execution cron at the same 95-minute window its table-job sibling uses. Rows are cancelled, not completed: the scope never finished. The OOM itself is not a leak. Peak RSS is a flat plateau — 457 MB at 20-45s and 461 MB past 200s, so ten times the duration buys four megabytes — that has crept about two percent per release for a month, from 446 MB in late July to 545 MB, past the 512 MiB `small-1x` ceiling. CPU peaks at 0.19, so the larger preset is bought for RAM alone. `maxAttempts` never covered the kill either: Trigger.dev retries `TASK_PROCESS_OOM_KILLED` only when `retry.outOfMemory.machine` names a preset, and all four runs recorded `attempt_count = 1` while the docstring claimed they resumed from the persisted cursor. The connector stuck-document sweep dispatched without a bound. Its chunk size paced the loop but the candidate query had no limit, so one connector enqueued 2,959 documents in fifteen seconds onto the queue every workspace shares. Nothing was double-billed — those documents were genuinely unindexed — but one connector monopolized the queue, and each dispatch mints a fresh requestId, so the idempotency key differs every pass and none of it deduplicates. Candidates are now taken oldest-first and capped per sync; a deeper backlog is deferred to the next sync rather than dropped. * fix(file-parsers): read officeparser's entry point across module systems `officeparser` is CommonJS — `main: officeParser.js`, no `type`, no `exports` map — so what `await import('officeparser')` yields depends on who built the code. Node and webpack synthesize named exports from `module.exports`, so `.parseOfficeAsync` is there. esbuild, which builds the Trigger.dev worker bundle, puts `module.exports` on `.default` and leaves the named export undefined, and the package is in neither `build.external` nor `additionalPackages`, so it is bundled. Reading the named export directly therefore worked everywhere except the worker, where calling it threw `TypeError: parseOfficeAsync is not a function`. All four parsers treat that as "the library failed" and answer with a scrape of the archive, which returns `degraded: true`, and the document pipeline rejects a degraded parse outright. The visible result was every `.pptx` and legacy `.doc` reporting "No text could be extracted from this file — it may be scanned, image-only, or password-protected", naming a cause that had nothing to do with the fault. 118 pptx and 14 doc failures landed in a single burst when one connector's sync first succeeded after ten consecutive crashes. Resolved in one shared loader rather than per bundler: externalizing the package has to be repeated in every build config this code runs under and regresses silently the day one is missed. The shape handling is split into a pure `resolveParseOfficeAsync` because the failing shape cannot be reproduced by mocking the specifier — Vitest's module-namespace proxy throws on a missing export rather than yielding the `undefined` a real bundle produces, so a test going through `import` can only assert the shape that already worked. That is also why the existing parser suites never caught this: each mocks `officeparser` with a fabricated named export, which presupposes the interop being broken here. * fix(knowledge): bound the workbook preview to the rows it emits `sheet_to_json` allocates from a worksheet's DECLARED `!ref` range rather than its populated cells, and Excel routinely writes an inflated range from stray formatting. The 1,000-row preview cap was applied to the result, so it bounded the emitted string while the allocation it was meant to bound had already happened. An 880 KB workbook exhausted an 8 GB worker; the same content exhausted 16 GB when this ran inside the connector sync. No machine size fixes that, because the allocation scales with a number the file declares about itself — fleet p99 for this task is 691 MB against 8 GB, so this is a cliff, not pressure. Passing the window into the conversion is what makes the cap real. `defval` goes with it: defaulting every cell in the range made each row dense, so allocation scaled with columns x declared rows rather than with populated cells, and because no row was left empty it silently defeated the `blankrows: false` beside it. Reported totals still come from the declared range, so bounding the conversion does not change what the metadata says the workbook holds. The eleven documents killed this way recorded `attempt_count = 1`: `maxAttempts` does not cover `TASK_PROCESS_OOM_KILLED`, which Trigger.dev retries only when a larger preset is named. Adding that escalation is a safety net rather than the fix, and the same gap the dispatcher had. Also corrects the machine comment, which claimed `large-1x` was 2 vCPU / 2 GB. It is 4 vCPU / 8 GB, and believing the stale figure makes a resize look like the answer when the parser is what is unbounded. * fix(tables): keep a cancelled dispatch cancelled when a step claims it `dispatcherStep` reads the dispatch, then awaits the table load before writing `dispatching`. Keying that write on the id alone resurrected a dispatch cancelled inside that window — a Stop-all, or now the stale-dispatch sweep — and the fresh heartbeat it writes would then buy the resurrected row another full window before the sweep could reclaim it again. The race predates the sweep, but the sweep is a new writer of `cancelled` that no user action drives, so it is newly reachable without anyone touching Stop. Re-asserting the status the step already read is the whole fix. * fix(tables,knowledge): spare a live window, and restore the truncation notice A lease needs its heartbeat interval to sit well under its TTL. The dispatch heartbeat is stamped between windows, not during them, and `batchTriggerAndWait` checkpoints the loop for the whole window — so the interval is really "one window", which nothing bounds: the window ends when its cells do, and the in-process path has no ceiling at all. A window outliving the stale threshold had its dispatch cancelled while it was plainly alive. Its cells carry the signal the checkpointed parent cannot — `updatedAt` on every in-flight row execution, written by the cell tasks themselves. Both signals must be stale before a dispatch is reclaimed, so a slow window is spared for as long as its cells keep reporting while a run with nothing beating and nothing executing is still collected. The subquery rides the partial `(table_id, status)` index that already covers exactly those three statuses. Bounding the workbook conversion also made its truncation notice unreachable: the converted length can no longer exceed the window it was compared against, so every sheet larger than the preview cap silently stopped reporting that it had been cut. Compared against the declared row count instead, which is what the comparison meant before the conversion was bounded. * fix(tables,knowledge): act on the claim outcome and scope liveness to the dispatch Three defects, two of them created by the previous round's fixes. Guarding the pending-to-dispatching claim without reading its outcome was the worse half of a fix. When a Stop-all or the stale sweep won the race the row correctly stayed `cancelled`, while the step went on to announce `dispatching`, stamp cells and enqueue a window for it — and an empty window would then reach the unguarded `markDispatchComplete` and overwrite `cancelled` with `complete`. The step now ends when it did not claim the row. The cell-liveness probe was table-scoped, and `table_row_executions` carries no dispatch column, so a live dispatch's cells vouched for an abandoned dispatch beside it and the abandoned row was never reclaimed — turning the stuck overlay this sweep exists to clear into a permanent one. Narrowed to the dispatch's own groups, which it already stores. Two active dispatches over the same groups can still mask each other, but that is the state `markActiveDispatchesCancelled` already prevents. Truncation asks whether the window cut the sheet short — a question about the declared range against the cap. Comparing the converted length to the cap made it unreachable once the conversion was bounded; comparing the declared count to the converted length then reported truncation for any sheet merely containing blank rows, which are now skipped rather than defaulted into existence. * fix(tables): scope dispatch liveness to its rows, not just its groups The previous round narrowed the cell-liveness probe to the dispatch's groups on the reasoning that two active dispatches over the same groups cannot coexist, because starting a run cancels prior work on its scope. That reasoning was wrong. `cancelPriorRuns` in `workflow-columns` requires `isManualRun`, so auto-fired runs never cancel anything, and the per-row path is explicitly a no-op for dispatch cancellation. Same-group coexistence is ordinary. A dispatch that names rows now only accepts liveness from those rows, which covers the auto-fired and row-scoped runs that reach this state. What remains is two table-wide dispatches over the same groups, where nothing in the row execution says whose work it is; closing that needs a `dispatch_id` column on `table_row_executions` threaded through six write sites, including the shared cell-write path every cell task uses. That residue is a delay rather than a permanent mask — the live dispatch's cells stop updating when it finishes, and the next sweep after a quiet window reclaims the abandoned row. * refactor(tables): name the dispatch liveness predicate and bound its fan-out Extracts the cell-activity check into `hasRecentCellActivity`, so the stale predicate reads as its two conditions — nothing beating, nothing executing — rather than a twenty-line SQL blob nested inside an `and()`. No behaviour change; this is the code three review rounds found defects in, and being able to read it is what makes those defects findable. Bounds the terminal-event fan-out with `mapWithConcurrency`, matching how the scheduler already fans out. The sibling cancel paths emit over one table's dispatches; this sweep can carry a whole tick's worth across many tables, and each event is its own write. Also repairs the test that covers it. `collectChunks` walks into the `tableRowExecutions` table object the fragment interpolates, so every column name appears in the chunks whether the predicate references it or not — the group, row, table and timestamp assertions all passed with their predicates deleted. Matching the literal SQL instead makes them fail, which mutating each clause now confirms. * fix(tables): make the row bypass NULL-safe and guard the post-wait completion `jsonb_typeof(scope -> 'rowIds') <> 'array'` was the table-wide bypass, but a table-wide dispatch has no `rowIds`: the extraction is SQL NULL, `jsonb_typeof` returns NULL, and `NULL <> 'array'` is UNKNOWN rather than TRUE. The bypass never fired, so no live cell could satisfy the probe and the sweep reclaimed exactly the long-running table-wide dispatches the row filter was added to protect — inverting it. `IS DISTINCT FROM` is the NULL-safe form, and the same pitfall is already handled with `coalesce` in `markActiveDispatchesCancelled`. `completeDispatch` also wrote through the unguarded `markDispatchComplete`. Both its callers run AFTER the window's wait, so a Stop-all or the sweep landing during that wait leaves the row `cancelled` and the write overwrote it with `complete`, publishing a completion event after the cancellation one. The claim guard cannot cover this — the cancel arrives long after the claim. It now goes through `completeDispatchIfActive`, which already exists for exactly this, and emits nothing when the transition does not land. * fix(knowledge): give connector sync logs a retention pass Nothing pruned `knowledge_connector_sync_log`, so it grew by one row per sync run forever — a connector on a fifteen-minute interval writes about 35,000 rows a year by itself. That cost lands on `loadPreviousListingObservation`, which reads the newest `completed` row per connector through an index covering `connector_id` alone, so every retained row makes the sort behind the deletion-safety corroboration slower. Added as another arm of the cleanup cron, batched the same way as its two sibling prunes. Two `exists` guards are load-bearing rather than defensive: the newest row per connector always survives, and so does the newest `completed` one, because that is the row `loadPreviousListingObservation` reconstructs the previous listing from — and that reconstruction decides whether a suspect listing is corroborated, i.e. whether reconciliation may delete documents. Pruning it would silently change deletion behaviour. `started` rows are never eligible; they are in flight or waiting on the scheduler's own sweep. * fix(tables): funnel every post-claim completion through the guarded write The empty-window exit still wrote through the unguarded `markDispatchComplete`, and it runs after the claim like the other two — so a cancel landing during its window query was overwritten with `complete`. Shorter window than the two post-wait exits, same defect, and leaving one of three unguarded is how this came back twice already. All three now route through `completeDispatch`, so the guard lives in one place and covering it once covers every exit. The redundant test for this path went with it: it could not be made to fail against the mock, and a test that cannot fail is worse than none — the guard is held by the test on the shared funnel. * fix(tables): bound how long cell activity may spare a dispatch The liveness probe cannot tell whose cells it is looking at when two table-wide dispatches share a group, because `table_row_executions` carries no dispatch column. On a quiet table that is only a delay — the neighbour finishes and the next sweep reclaims — but a busy table with continuous auto-fired work can keep an abandoned dispatch masked indefinitely, which is the stuck overlay this sweep exists to clear. A ceiling bounds it: past a day without a heartbeat, a dispatch is reclaimed whatever its cells are doing. That is safe because a live dispatch stamps its heartbeat between windows regardless of cell activity, so only a single window outliving the ceiling could be reclaimed wrongly, and no window lasts a day on any path — the Trigger.dev run ceiling is ninety minutes. The real fix is a `dispatch_id` on the executions row. Threading it through the patch layer and the upserts underneath it is a change to the hottest write path in tables and belongs in its own review, not on the sixth round of this one. * refactor(tables): give the stale predicate one definition of "last beat" `COALESCE(heartbeat_at, requested_at)` was written twice — once for the stale threshold and again for the absolute ceiling — so the two could drift into disagreeing about what proof of life means. One `lastBeat` fragment, one `notBeatingSince(cutoff)` helper, both cutoffs expressed through it. Also corrects the ceiling's comment: it triggers a day past the stale threshold, not a day past now. * fix(tables): delete the unguarded completion rather than guard it a fourth time The two pre-claim exits — table missing, no target groups — still wrote through `markDispatchComplete`. Last round I argued they run before the claim, "where forcing a terminal state is the intent". That was wrong twice over: the table lookup is awaited, so a cancel lands in that window like any other, and a dispatch cancelled mid-lookup has not completed its scope any more than one cancelled mid-window has. Routing them through `completeDispatchIfActive` left `markDispatchComplete` with no callers, so it is gone. That is the part worth having: this is the fourth place the same defect appeared, each time because an unguarded writer was sitting there to be reached. With it deleted, `completeDispatchIfActive` is the only way to complete a dispatch and the class cannot recur. * fix(tables): re-read the dispatch before committing a window Several round trips separate the claim from the enqueue — the window query, the executions prefetch, the tombstone filter — and nothing rechecked the dispatch across them. A Stop-all or the stale sweep landing in that gap had the step stamp cells and run a whole window for a dispatch already recorded as cancelled; the existing recheck sits after the window, which is too late to prevent it. Mirrors that existing check on the other side of the enqueue. It narrows the gap to a single statement rather than closing it — a cancel arriving after this read still races the enqueue, and no check can fix that. The cell-level `cancellationGuard` and the `isExecCancelledAfter` tombstone filter are what catch the remainder.	2026-08-21 15:02:12 -07:00
Vikhyath Mondreti	81ebb37563	fix(delivery): stable provider idempotency tokens, and correctness fixes around them (#6943) ... * fix(delivery): stable provider idempotency tokens, and correctness fixes around them Six independent fixes found while investigating a Slack transport failure. None of them are that failure; all of them are live. **Money writes could be delivered twice.** Square (8 tools), Brex (5) and Outlook Calendar minted their provider idempotency token with `generateId()` at request-build time. That is stable inside the transport retry loop — `prepareToolRequest` runs once above it — but a BLOCK-level retry re-enters the handler and mints a fresh one, defeating the provider's dedupe. A builder ticking "retry" on a Square block turned a committed write into a second card charge, and a Brex one into a second money transfer. Tokens now come from `deriveDeliveryKey`, a pure function of execution + block + tool + invocation, so every retry layer derives the same value. Stripe joins them: all ~50 tools previously sent no `Idempotency-Key` at all. The comment on `brex/create_transfer.ts` claimed a fresh key per transfer *prevents* duplicate money movement. That is inverted — fresh per *attempt* is what permits it — and is corrected here. **`invocationId` is required, not optional.** Deriving from `executionId` alone would be worse than the bug: five loop iterations paying five invoices would share one token, the provider would honour the first and silently drop four real payments, and it would look like five successes. Also: - `INTERNAL_API_BASE_URL` is now ignored on Trigger.dev workers. It names a route that resolves only inside the app container, and several modules run in both runtimes — `guardrails/mask-client.ts` says so in its own TSDoc — so setting it produced `PII redaction failed: Unable to connect` on every worker-side redaction. Mirrored into `packages/testing`'s urls mock, which reimplements the function and would otherwise have diverged. - `engines.bun` raised to >=1.3.14. Measured on 1.2.15, which the old floor permitted: a fully-delivered POST is silently replayed and the caller sees 200. - CloudWatch `put-metric-data` pinned to `maxAttempts: 1`. The AWS SDK default of 3 already replayed it, and `PutMetricData` aggregates rather than overwrites, so a duplicate silently corrupts the customer's metric series and alarm thresholds. - `webhookIdempotency` given a bounded in-progress lease. An untimed run held a SEVEN DAY lease while concurrent duplicates polled it once a second. Verified: `tsc --noEmit` clean, 30,100 tests pass. * fix(ci): restore staging files the branch split had reverted, and format Three problems, all from assembling this branch by checking paths out of a WIP branch built on an older `staging`. Anything `staging` changed since that base came back as a revert. - Root `package.json` had lost the `opentype.js` / `@types/opentype.js` dependencies `staging` added, which desynced `bun.lock` and failed `bun audit`. It also carried a `check:outbound-delivery` script belonging to other work. Every `package.json` is now taken from `staging` with only the `engines.bun` line re-applied. - `executor/utils/block-data.test.ts` — a 77-line file `staging` added — was deleted outright. Restored. - `executor/handlers/generic/generic-handler.test.ts` had lost a test `staging` added. Restored, with only the one `blockId` assertion re-applied. Also formats `keyed-invocation-identity.test.ts` and sorts imports in `internal-api-base-url.test.ts`, which is what `lint:check` failed on. * fix(providers): thread the model's tool-call id into keyed tool execution `prepareToolExecution` accepted an `invocationId` but no provider supplied one, so a keyed tool invoked through an agent always hit the incomplete-context fallback and minted a fresh token — leaving Stripe, Square, Brex and Outlook Calendar writes able to double-deliver under the hosted-key retry layer even though the block path was fixed. The id is now a positional parameter rather than another optional field on `request`, so a provider that cannot supply one fails to compile instead of silently falling through. 22 of the 27 call sites already had the OpenAI-shaped `toolCall` in scope; `tsc` identified the other five, of which Anthropic (`toolUse.id`) and Bedrock (`toolUse.toolUseId`) name it differently. Gemini is left deliberately unthreaded and documented: its function-call parts carry no model-supplied identifier — the streaming loop has to synthesize a local one — and a positional index would not survive the model re-emitting the call. A token that only looks stable is worse than the loud fallback, which names the missing fields. * fix(executor): keep execution order monotonic across a resume `executionOrder` is not carried in the pause snapshot, so a resumed run restarted the counter at 0 and a loop or parallel body executing on both sides of a pause could reuse a pre-pause value. That was cosmetic while the number only ordered logs. It stops being cosmetic once identity is derived from it: a `keyed` tool takes its provider idempotency token from this value, so two distinct writes would present the same token and the provider would silently drop the second. Suppressing a real payment is worse than the duplicate the token exists to prevent, because it looks like success. The counter is now seeded from the highest `executionOrder` among the restored block logs rather than from a new snapshot field, so snapshots written before this change are repaired on resume instead of needing a migration. * fix(providers): thread the tool-call id through the streaming loops too The previous commit only reached call sites written as a single-line three-argument call. The streaming loops are formatted across lines, so openai-compat (Groq, DeepSeek and everything else routing through it), Anthropic and Bedrock still omitted the id and kept falling back to a fresh token. Both Gemini paths now pass `part.functionCall?.id` — the RAW model id, not the `ensureToolCallId` value used for stream events. That helper allocates an execution-local id when Gemini supplies none, and it is freshly allocated per attempt: passing it would complete the keyed context, silencing the "could not derive" warning, while leaving the token just as unstable. Gemini frequently omits the id, in which case this is `undefined` and the loud fallback stands. All 27 call sites are now covered. * fix(providers): make the tool-call id argument required, not optional The TSDoc claimed a provider that cannot supply an id would fail to compile, but the parameter was declared `toolCallId?: string` — so a new call site could omit it entirely, typecheck, and silently take the unstable-token path the positional parameter exists to close. The comment promised a guarantee the type did not enforce. It is now `string | undefined`: required in position, nullable in value. A provider with no model-supplied id must pass `undefined` explicitly and take the loud fallback, rather than being able to forget the argument. All 27 existing call sites already pass it, so this is enforcement only. Verified by deleting the argument at one site: `tsc` rejects it.	2026-08-21 13:35:20 -07:00
Siddharth Ganesan	58aa6379e0	feat(cli): add chat command (#6937) ... * feat(cli): add chat command * fix(cli): harden chat command execution	2026-08-21 12:59:08 -07:00
Theodore Li	4c41fc6c3e	fix(setup): bump sim-setup to 1.0.1 (#6944)	2026-08-21 12:24:06 -07:00
Theodore Li	a0a14383f0	feat(enrichments): add LinkedIn profile lookup (#6926) ... * feat(enrichments): add LinkedIn profile lookup * fix(enrichments): request Findymail profile data * fix(enrichments): avoid Findymail profile charge * refactor(enrichments): omit Findymail profile option	2026-08-21 14:05:23 -04:00
Waleed	63569a2459	fix(connectors): count hard-kill failures and cap deletion blast radius (#6909)	2026-08-21 09:41:21 -07:00
Vikhyath MondretiandClaude Opus 5	01795e1ed2	fix(combobox): report every open, not just the dismissals Radix initiates (#6931) ... The Combobox owns its `open` state but renders a controlled Radix `Popover` (`PopoverAnchor` + `open={open}`, no `PopoverTrigger`), and the consumer's `onOpenChange` hung off that Popover alone. A controlled popover reports only transitions it initiates itself, so outside-click and Escape arrived and nothing else did: the trigger, the chevron, focus, Enter/Space/ArrowDown, and select-to-close were all the component's own `setOpen`, invisible from outside. Every consumer refreshed on open or reset on close, so the damage stayed quiet — the credential selectors, MCP tool selector, workspace-file picker, connector modal, and the sub-block dropdown's remote option list simply never refreshed when opened. Then #6881 gated the agent block's `toolGroups` on the same signal to keep the group build off the canvas's hot path, and a picker that could not learn it was open built nothing: the dropdown rendered "No tools found" over the full block registry. Every transition now goes through one `changeOpen`, which Radix's own `onOpenChange` also feeds, so `setOpen` has exactly one caller and the callback cannot be missed. It dedupes through a ref, because several paths both close and let the popover dismiss — a redundancy the raw setState absorbed silently but a consumer callback would not — and reading that ref lets the toggles resolve their next value without re-creating their handlers on every open. Tests cover the transitions Radix never reported (trigger click both ways, keyboard open, Escape) and the consumer shape that made this visible: options supplied only once the dropdown says it opened must render, not the empty state. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-21 08:40:30 -07:00
Vikhyath Mondreti	ab66ce9149	fix(admin): harden dashboard billing operations (#6914) ... * feat(admin): make dashboard billing operations durable * fix(admin): close dashboard recovery gaps * fix(admin): preserve member operation lock order * fix(admin): harden durable operation boundaries * fix(admin): report member operation failures accurately * test(invitations): cover locked seat admission * feat(enterprise): gate owner activation on acceptance * fix(enterprise): keep owner activation recoverable	2026-08-20 22:31:11 -07:00
Vikhyath Mondreti	aca152c7fb	fix(env): put runtime config on <html> so client reads can't outrun it (#6923) ... * fix(env): put runtime config on <html> so client reads can't outrun it The inline script that assigns `window.__ENV` is rendered from the component tree, so it lands ~13KB after the `<script async>` bootstrap tags React emits in the preamble. `appBootstrap` calls `hydrate()` synchronously whenever `self.__next_s` is empty — which it always is now that the script is a plain tag rather than a `beforeInteractive` one, that queue having been the only thing sequencing the assignment ahead of hydration. So module bodies and the first commit could both read env before the assignment landed: the socket URL fell back to the page origin for the life of the document, `getBaseUrl()` threw, and every module-scope flag in `env-flags` froze on nothing. Carry the same snapshot on `<html>`, the document's first tag, and read it in `getEnv` when `window.__ENV` is not yet assigned. Parsing is memoized against the raw attribute rather than against having run once, so the cache can never serve a value the document no longer carries. `window.__ENV` stays the public global and the preferred read, and both transports are built from one function so they cannot drift. Alongside: guard the read-only webhook-URL field so a base URL it cannot resolve is a blank field rather than a dead canvas; report what the workflow error boundary catches, which it previously swallowed entirely; and enable PostHog's native exception capture, since error boundaries only ever see their own subtree and chunk-load failures, rejected promises and throws from event or socket callbacks reached nothing. * fix(realtime): count each failed connect attempt once `manager.reconnect()` calls `open()`, whose error path emits `error` — which the socket re-emits as `connect_error` — and then emits `reconnect_error` itself. A failed reconnect therefore reached both handlers and advanced the counter twice, so the outage report tripped on the second real attempt while claiming three. Count in `connect_error` alone: it is the only handler that fires exactly once for both the initial failure and every retry. `reconnect_error` keeps its log line and states why it deliberately does not count. * fix(workflow): let an unresolvable webhook URL fail loudly Reverts the guard added earlier in this PR. A blank row titled "Webhook URL" is a worse outcome than a crash: it explains nothing, and the value is one a user copies into a third-party provider, so any substitute — a guessed page origin, an empty string — is a URL that provider accepts and then never delivers to. The read can no longer come back empty from the hydration race this PR fixes, so reaching it at all means the deployment has no application base URL, which breaks webhook registration and callbacks regardless. The error boundary now reports what it caught, so the throw names its own cause instead of surfacing as an unexplained fallback.	2026-08-20 22:12:36 -07:00
Waleed	1ced0b6065	fix(knowledge): stop the stuck-document sweep reclaiming still-queued documents (#6921) ... * fix(knowledge): stop the stuck-document sweep reclaiming still-queued documents The sweep gave 'processing' a staleness cutoff but gave 'pending' and 'failed' none, and the only cross-run guard was uploadedAt < syncStartedAt, which scopes within a sync but not across them. That was harmless while document processing awaited inline: documents were terminal by the time a sync ended. Since dispatch became asynchronous they sit 'pending' until a worker picks them up, so the next sync deleted their embeddings, reset them, and re-dispatched while the original runs were still executing — and each re-dispatch mints a fresh pass id, so it billed again. Queued documents now get a grace period derived from queue drain time rather than run duration: the processing queue's concurrency is 20 and global across workspaces, so a 2,600-document corpus takes roughly two hours to drain. The existing 45-minute threshold bounds a run, not a wait, and would still reclaim live documents on any corpus over about 900. No column records dispatch time, so the signal is processingStartedAt falling back to uploadedAt. The sweep now stamps that column when it re-dispatches, and the user-triggered retry stamps it instead of clearing it — without both, only a document's first dispatch was protected and the sweep churned once per sync forever on anything that kept waiting. The grace narrows the duplicate-billing window but cannot close it: a re-dispatch mints a new request id and the Trigger idempotency key is scoped per dispatch by design. Closing it durably needs a document-scoped key or a dispatch-generation column, recorded in TSDoc and deliberately not built here. * fix(knowledge): record dispatch time in its own column instead of overloading the start time Two review findings on this PR traced to the same root: the queue grace was reading processingStartedAt, a column that means something else. Externally, a pending row carrying a dispatch timestamp reported a processing start time for work that had not started. Internally, updateDocument sets a document pending and refreshes uploadedAt while leaving the previous run's processingStartedAt in place, and completion never clears it — so the sweep aged a re-dispatched document from a leftover stamp rather than its actual dispatch, and could reclaim it inside the grace window while the earlier queue entry was still live. That reopens the duplicate-billing race this PR closes. processing_queued_at is written on every re-dispatch and read by the sweep; processingStartedAt goes back to meaning what its name says, so its external contract is byte-identical to before this PR. Not stamped on first dispatch: the row is created and dispatched inside the same sync run, so uploadedAt is already accurate there and a guarded write per upload would buy no behavior. The internal document route returns a full table row through a passthrough schema, so the new column would have shipped as an undeclared raw Date. Declared and serialized explicitly instead. * fix(knowledge): stamp the queue time in the dispatch funnel, not at each call site Giving dispatch its own column was not enough. The connector content-update path updates a row in place with pending status and a fresh uploadedAt while leaving every prior-run processing column intact, so a document dispatched once already carried a stale queue stamp, the refreshed uploadedAt was never consulted, and the row aged from a dead run's timestamp — the same bug one level down. "Written on dispatch and only on dispatch" is now structural rather than a convention three call sites remember: the stamp lives in processDocumentsWithQueue, which every one of the nine dispatch paths already goes through, so none can forget it. The sweep's and the retry's own stamps are kept because they land inside the same transaction as their reset, so a document keeps its grace even if the dispatch that follows throws. The funnel also clears processingStartedAt, guarded on the row still being pending so it cannot disturb a worker's compare-and-set. That closes the API-facing half at its source: a pending row can no longer report a start time from any path, including the content-update path no serializer would have seen.	2026-08-20 18:59:31 -07:00
Theodore Li	5b28da1989	fix(tables): accept plain row query predicates (#6916) ... * fix(tables): accept plain row query predicates * fix(cli): show table predicate group syntax	2026-08-20 20:48:47 -04:00
3 people	42f6287911	feat(byok): add organization-wide key inheritance (#6834) ... * feat(byok): add organization key management * feat(byok): inherit organization keys at runtime * feat(byok): add organization scope to BYOK settings * fix(byok): refresh org key state after mutations * fix(byok): hide stale inherited status badges * chore(db): drop colliding byok migration ahead of staging merge Staging independently claimed 0293. Remove ours so the merge is clean; it is regenerated at the next free index right after. * chore(db): regenerate byok migration at 0296 Staging claimed 0293-0295 during the merge; the regenerated SQL is byte-identical to the dropped 0293. * docs(byok): document organization scope, precedence, and the full provider list The BYOK section described workspace-scoped keys only. Add the organization scope, its Enterprise requirement, the per-provider precedence rule, what an entitlement lapse does, and the Pi sandbox exposure. Refresh the provider table from the settings page, which had drifted from 14 to 34 entries. * feat(byok): open organization keys to every organization plan Organization BYOK was gated on Enterprise, but an organization is the only thing that can hold the keys, so every plan that can own an organization should qualify — Pro for Teams, Max for Teams, and Enterprise. Add checkOrgPlan/resolveOrganizationPlan beside the Enterprise pair rather than widening checkEnterprisePlan, so the Enterprise-only gates (Access Control, whitelabeling) are untouched, and restore resolveOrganizationEnterprisePlan to module-private now that BYOK no longer needs it. * perf(byok): cache the organization entitlement, not the key material getBYOKKey runs once per agent block and once per hosted-capable tool call, so a loop over N items resolved N times — and each organization-inheriting resolution paid three sequential billing queries on top of the two key reads. Split the two reads by staleness tolerance. Key rows stay fresh, because revocation must be immediate. The entitlement is a billing gate that tolerates bounded staleness in the harmless direction (a lapsed organization keeps using its own key for <=60s), so cache it per organization with an in-flight share so concurrent blocks issue one query set. The management surfaces keep reading it fresh, so an organization that just upgraded is never told otherwise. Also run the block check and subscription read in parallel inside resolveOrganizationPlan, and carry the resolved scope on BYOKKeyResult so a log line can say whether a run used the workspace's key or an inherited one. * feat(byok): let workspaces store the Z.ai and Cohere keys the runtime reads Both ids were already in the BYOK contract enum and both are resolved at execution time — getApiKeyWithBYOK reaches 'zai' (GLM models are in the hosted catalog, so the BYOK branch runs), and 'cohere' backs both the Embeddings block and Knowledge Base reranking — but neither appeared in the settings list, so there was no way to store the key either path looks for. Cohere had no icon; add one from the official multi-color mark so it stays legible on a light and a dark page. Cohere's embed-v4.0 is kbEligible:false, so the description says 'Embeddings and Knowledge Base reranking' rather than claiming KB embeddings. * improvement(byok): shorten the workspace scope chip to 'Workspace' It sits beside 'Organization', so the scope reads from the pair; 'This' only added width. * fix(byok): do not cache a billing outage as an unentitled organization resolveOrganizationPlan maps a failed billing read to false, which is indistinguishable from a real plan lapse. The entitlement cache stored that, so one transient outage held the gate shut for the full TTL and every inheriting run silently fell back to a metered hosted key — and the cache's rejection path, which exists to prevent exactly this, was unreachable. Give the resolver the onError option its neighbours already have and let the cached read ask for 'throw', so a failure stays out of the cache and the next resolution retries. Behavior for the call that saw the error is unchanged: getBYOKKey still fails closed. Reported by Cursor Bugbot. * fix(byok): propagate the subscription read's failure too The previous commit threaded onError through resolveOrganizationPlan's own catch, but getOrganizationSubscriptionUsable soft-fails to null on its own, so a failed subscription read still arrived as an ordinary 'no usable subscription' and returned a successful false — which the entitlement cache then stored for the full TTL. Thread the option into that call as well. Test it at the billing layer rather than the cache layer: the entitlement test mocks resolveOrganizationPlan wholesale, so it could never have caught this. Verified the new test fails against the previous commit. Reported by Cursor Bugbot. * refactor(byok): cache the entitlement with LRUCache, like copilot entitlements The hand-rolled version reinvented three things the codebase already has a canonical answer for. lru-cache is a declared dependency of apps/sim and lib/copilot/entitlements.ts already caches an entitlement with it — by storing the in-flight Promise, which is what makes concurrent callers collapse onto one resolution with no in-flight bookkeeping at all. TTL and the size bound come from the library. That removes the second Map, the manual eviction (and its interaction with an in-flight entry), and the dead value-while-refreshing state: 23 executable lines. The one thing the library does not cover is dropping a rejected promise so a billing outage is not cached for the TTL, which is kept and pinned by a test that fails without it. TTL expiry is no longer re-tested — that is the library's behavior, not ours, and lru-cache reads its clock at module load so faking timers never moved it. * refactor(byok): coalesce the entitlement read with the shared singleflight lib/concurrency/singleflight.ts is the codebase's coalescing primitive and oauth/credential-service.ts already pairs it with a read-through cache. Adopting that shape fixes a case caching the promise directly did not: a *hung* billing read wedged every caller for the full 60s TTL, where coalesceLocally evicts and rejects at its settle deadline. It also removes the hand-rolled rejection eviction — the cache is written only on the success path, so an outage leaves no entry by construction. The cache now holds booleans, which introduces the one trap worth a test: a truthiness check would read a cached false as a miss and re-query billing on every resolution for lapsed organizations. Pinned. * fix(byok): keep an abandoned entitlement producer from writing the cache coalesceLocally does not cancel a producer it timed out — its docstring says so explicitly — so writing the cache from inside the producer let a late billing result overwrite a fresher answer a retry had already cached, and hold it for a full TTL. Move the write onto the value the caller actually received. A caller that timed out throws before reaching it, so an abandoned producer now resolves into nothing. The test reproduces the overwrite and fails against the previous shape. Reported by Cursor Bugbot. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>	2026-08-20 17:45:28 -07:00
3 people	ea70f8dcf1	feat(harmonic): add contact workflow integration (#6902) ... * feat(harmonic): add contact workflow integration * fix(harmonic): sync docs manifest * fix(harmonic): address integration review findings * feat(harmonic): add the missing people endpoints and fix two error paths Extends the integration from 4 to 13 tools, covering every non-deprecated people-scoped Harmonic endpoint, and repairs two defects found by validating the existing tools against Harmonic's OpenAPI and API reference. New tools: - Enrich Person (POST /persons) — the only path from a LinkedIn URL or email a workflow already holds to a Harmonic contact. - Get Person, Get Company Employees — account-based sourcing; employees returns URNs that chain into Batch Get People. - Saved-search net-new results and their acknowledgement, so a monitor stops reprocessing the entire result set on every poll. - Bulk email enrichment: submit, poll, and quota, plus Get Enrichment Status. Fixes: - The error extractor dropped Harmonic's string and object `detail` envelopes. A tool that names an extractor gets no fallback chain, so every FastAPI abort surfaced as "Request failed with status 403". The enrichment 404 also carries the scheduled `enrichment_urn`, which was being discarded — that URN is the only handle on the job, so it is now kept in the message. - The saved-search selector failed the whole dropdown instead of degrading: the response cap was half the sibling value on an endpoint that is unpaginated and returns every saved search with its full query object, and the option ceiling threw rather than truncating. Raised to 1MB and switched to truncate-and-warn, matching the other data-driven selectors. Clearing net-new results now requires an explicit scope. Harmonic treats an absent `entity_urns` as "clear everything", so an empty field would have silently discarded the backlog. Scope deliberately excludes company-side, deal, typeahead, network, and Scout streaming endpoints, and every endpoint retiring on 2026-11-05. --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Waleed Latif <walif6@gmail.com>	2026-08-20 17:12:57 -07:00
Waleed	865f8173ab	feat(affinity): add Affinity CRM integration (#6908) ... * feat(affinity): add Affinity CRM integration Adds the Affinity v2 API as a block with 70 tools, covering 86 of the 87 documented endpoints. Only Send Feedback is omitted — it reports product feedback to Affinity rather than doing workflow work. Endpoint families that differ only by an entity segment are one tool with an entityType param, so companies/persons field, list, row, and relationship reads, the company/person merge endpoints, and entity notes each collapse into a single operation. * chore(affinity): regenerate the docs manifest for the new integration page	2026-08-20 16:27:29 -07:00