mirror of
https://github.com/simstudioai/sim.git
synced 2026-09-24 15:45:35 +08:00
d1eb79ecd35efbe917869209de68ae3536145da2
15
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Siddharth GanesanandTheodore Li
|
0b2cfaf7f7 |
feat(mothership): add superuser env selection (#4558)
* feat(table): live cell updates via SSE + per-table event buffer
Replaces the polling-based row refetch with a push-based SSE stream that
patches the React Query cache directly as cell-state events arrive.
Architecture:
- New per-table event buffer in apps/sim/lib/table/events.ts. Redis sorted-set
with monotonic eventId, 1h TTL, 5000-event cap, in-memory fallback. Modeled
after apps/sim/lib/execution/event-buffer.ts but stripped of complexity
tables don't need (no per-execution lifecycle, no id-batching, no write
queue serialization). ~150 lines instead of 700.
- writeWorkflowGroupState appends a fat event after each successful 'wrote'.
Status transitions carry executionId + jobId; terminal/partial transitions
also include the new output values inline so the client can patch row data
without a follow-up refetch.
- New SSE route at /api/table/[tableId]/events/stream?from=<lastEventId>.
Replays from buffer on connect, polls at 500ms (mirrors workflow execution
stream), heartbeat every 15s, signals 'pruned' if the caller fell off the
back of the buffer.
- Client hook useTableEventStream subscribes via EventSource. Reconnect-resume
with last-seen eventId. On 'pruned', invalidates the rows query and resumes
from the new earliest. Cache patches walk every cached query under
rowsRoot(tableId) so filter/sort variants all stay live.
- Removes refetchInterval from useTableRows and the per-page polling effect
from useInfiniteTableRows. React Query's refetchOnWindowFocus +
refetchOnReconnect cover the durability gap if any push is dropped.
Out of scope:
- Bulk-cancel events (cancellation path is being redesigned separately).
- Generalizing the workflow event-buffer module to a shared primitive (defer
until a third use case appears; for now the table buffer is the simpler
cousin of the workflow one).
* fix(table): drop run-mutation refetch so SSE patches aren't overwritten
useRunColumn.onSettled was canceling in-flight queries and invalidating the
rows query — leftover behavior from the polling era. With the SSE stream
now keeping the cache live via incremental patches, this refetch races the
stream and snaps the cache back to whatever DB shows at the refetch moment,
which can lag the just-arrived queued/running events. Cells appeared stuck
on the optimistic 'pending' even though the SSE was delivering the real
transitions.
* chore(table): simplify SSE plumbing — reuse helpers, drop dead polling code
- Reuse snapshotAndMutateRows for SSE cache patches instead of reimplementing
the page-walk + cache-shape detection. Adds a {cancelInFlight: false} opt
for the SSE caller (mutations still cancel as before).
- Drop client-side type duplication in use-table-event-stream — import
TableEvent and TableEventEntry from lib/table/events directly.
- Drop the now-dead mergePagePreservingIdentity + rowEqual from tables.ts;
their only caller was the polling effect that was removed earlier.
- Drop the defensive try/catch around appendTableEvent in cell-write — the
function is documented as never-throwing (returns null on failure).
- Combine INCR + ZADD into one Lua eval in events.ts. Halves Redis RTT per
cell-write. Lua returns the new eventId; the script splices it into the
pre-built entry JSON.
- Trim refs to plain let bindings inside the effect; trim stale
comments referencing the old polling implementation.
* fix(table): address PR review on SSE buffer
- TTL-expiry silent miss: when all keys expire, hgetall(meta) returns empty
so earliestEventId is undefined and the prune branch was skipped. Reconnect
with non-zero afterEventId now checks the seq counter — its absence (TTL
expired) signals pruned so the client refetches. Memory fallback mirrors.
- Unbounded ZRANGEBYSCORE: cap reads at TABLE_EVENT_READ_CHUNK = 500 events
per call. The route's 500ms poll loop drains chunks across ticks instead of
flushing 5000 entries (multi-MB) in one tick after a long disconnect.
- Pruned handler closes EventSource client-side: server-side close was firing
onerror and routing through the 500ms backoff path. Now we close
proactively, reset the reconnect attempt counter, and reconnect immediately
from the new earliest.
* Cross env copilot
* Force deploy
* Run migration
* Updates
* Fix migration
* Redeploy
* Make dev db push
* restore old migs
* Cross env copilot
* Add custom tools, skills, mcps to mothership
* Update migration
* Fix migs
* UPdate
* Fix types
* Fix
---------
Co-authored-by: Theodore Li <theo@sim.ai>
|
||
|
Theodore LiandClaude Opus 4.7
|
eb871fc0de |
feat(table): live cell updates via SSE + per-table event buffer (#4508)
* feat(table): live cell updates via SSE + per-table event buffer
Replaces the polling-based row refetch with a push-based SSE stream that
patches the React Query cache directly as cell-state events arrive.
Architecture:
- New per-table event buffer in apps/sim/lib/table/events.ts. Redis sorted-set
with monotonic eventId, 1h TTL, 5000-event cap, in-memory fallback. Modeled
after apps/sim/lib/execution/event-buffer.ts but stripped of complexity
tables don't need (no per-execution lifecycle, no id-batching, no write
queue serialization). ~150 lines instead of 700.
- writeWorkflowGroupState appends a fat event after each successful 'wrote'.
Status transitions carry executionId + jobId; terminal/partial transitions
also include the new output values inline so the client can patch row data
without a follow-up refetch.
- New SSE route at /api/table/[tableId]/events/stream?from=<lastEventId>.
Replays from buffer on connect, polls at 500ms (mirrors workflow execution
stream), heartbeat every 15s, signals 'pruned' if the caller fell off the
back of the buffer.
- Client hook useTableEventStream subscribes via EventSource. Reconnect-resume
with last-seen eventId. On 'pruned', invalidates the rows query and resumes
from the new earliest. Cache patches walk every cached query under
rowsRoot(tableId) so filter/sort variants all stay live.
- Removes refetchInterval from useTableRows and the per-page polling effect
from useInfiniteTableRows. React Query's refetchOnWindowFocus +
refetchOnReconnect cover the durability gap if any push is dropped.
Out of scope:
- Bulk-cancel events (cancellation path is being redesigned separately).
- Generalizing the workflow event-buffer module to a shared primitive (defer
until a third use case appears; for now the table buffer is the simpler
cousin of the workflow one).
* fix(table): drop run-mutation refetch so SSE patches aren't overwritten
useRunColumn.onSettled was canceling in-flight queries and invalidating the
rows query — leftover behavior from the polling era. With the SSE stream
now keeping the cache live via incremental patches, this refetch races the
stream and snaps the cache back to whatever DB shows at the refetch moment,
which can lag the just-arrived queued/running events. Cells appeared stuck
on the optimistic 'pending' even though the SSE was delivering the real
transitions.
* chore(table): simplify SSE plumbing — reuse helpers, drop dead polling code
- Reuse snapshotAndMutateRows for SSE cache patches instead of reimplementing
the page-walk + cache-shape detection. Adds a {cancelInFlight: false} opt
for the SSE caller (mutations still cancel as before).
- Drop client-side type duplication in use-table-event-stream — import
TableEvent and TableEventEntry from lib/table/events directly.
- Drop the now-dead mergePagePreservingIdentity + rowEqual from tables.ts;
their only caller was the polling effect that was removed earlier.
- Drop the defensive try/catch around appendTableEvent in cell-write — the
function is documented as never-throwing (returns null on failure).
- Combine INCR + ZADD into one Lua eval in events.ts. Halves Redis RTT per
cell-write. Lua returns the new eventId; the script splices it into the
pre-built entry JSON.
- Trim refs to plain let bindings inside the effect; trim stale
comments referencing the old polling implementation.
* fix(table): address PR review on SSE buffer
- TTL-expiry silent miss: when all keys expire, hgetall(meta) returns empty
so earliestEventId is undefined and the prune branch was skipped. Reconnect
with non-zero afterEventId now checks the seq counter — its absence (TTL
expired) signals pruned so the client refetches. Memory fallback mirrors.
- Unbounded ZRANGEBYSCORE: cap reads at TABLE_EVENT_READ_CHUNK = 500 events
per call. The route's 500ms poll loop drains chunks across ticks instead of
flushing 5000 entries (multi-MB) in one tick after a long disconnect.
- Pruned handler closes EventSource client-side: server-side close was firing
onerror and routing through the 500ms backoff path. Now we close
proactively, reset the reconnect attempt counter, and reconnect immediately
from the new earliest.
* improvement(table): persist SSE lastEventId in sessionStorage
Tab refresh / navigate-away-and-back now resume the stream from where the
previous mount left off instead of replaying from from=0. Mirrors the
useExecutionStream pattern (saveExecutionPointer / loadExecutionPointer).
First-ever mounts and new tabs still start at 0 — sessionStorage is
per-tab, so the safe default applies. On a 'pruned' fallback the new
earliestEventId is also persisted so the next reconnect starts there.
* fix(table): include runningBlockIds + blockErrors in SSE event payload
The cell renderer's 'queued' vs 'running' vs 'pending-upstream' decision
reads exec.runningBlockIds + exec.blockErrors. Without those fields the
inFlight branch falls through to 'pending-upstream' (amber Pending pill)
even when the worker has already written status=running. The worker writes
both fields to DB; the SSE event was stripping them. Thread them through
events.ts → cell-write.ts → use-table-event-stream.ts.
* fix(table): show value once column output has landed mid-run
The cell renderer treated any `status: 'running'` event as in-flight,
even when the column's own output had already been written. During a
multi-block group run, partial-write events for a later block carry
the earlier block's outputs but tag only the later block as running
— that flipped the finished column back to the amber Pending pill
until the terminal `completed` event arrived.
Re-order the priority chain so the column's value wins over
`pending-upstream`. Active re-run of the column itself
(`blockRunning`) still wins over the stale value, so a re-run on a
previously-completed cell still surfaces the running pill before the
new value overwrites.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(table): address PR review nits on tables.ts
- Merge duplicate JSDoc on snapshotAndMutateRows into a single block
- Remove unused useQueryClient() calls from useTableRows and
useInfiniteTableRows (leftover from polling-era code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(table): apply biome formatting fixes
CI lint job flagged import order in two files and over-wrapped union
in lib/table/events.ts. No behavior change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Theodore LiandClaude Opus 4.7
|
98f8e854eb |
improvement(tables): extract TablesDetail wrapper, ship trigger followups (#4476)
* ui improvements * Update status pils, make checkbox column sticky * add Run workflow to context menu * Refactor dispatching logic * fix checkbox width to be smaller if csv is small * Add drag behavior for workflows, stop workflow on multi select * fix z index of checkbox to left, add view workflow button * Switch to emcn buttons for Add inputs * Split up workflow sidebar from column sidebar, refactor cells * Lint and add auto run toggle * fix column reordering, add action bar * Create and use emcn square * Reconcile post-merge: drop positionMap, use rowId-based selection Staging refactored Tables UI to decouple from DB position (gutter from array index, checkedRows keyed by rowId, no PositionGapRows). Bring HEAD's action-bar / context-menu helpers in line: contextMenuRowIds, selectedRowIds, actionBarRowIds now key off row.id and walk `rows` directly. Drop the maxPosition / positionMap derived state. Collapse COLUMN_SIDEBAR_WIDTH_CSS to a numeric COLUMN_SIDEBAR_WIDTH used by both the sidebar shell and the table's reserved padding-right. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(table): backfill remapped workflow outputs from execution logs When a workflow column is re-pointed to a different (blockId, path), populate its existing rows with the new output's value pulled from saved execution logs instead of leaving them empty until the next run. Rows where the new mapping has no logged value clear (matching the previous behavior for those rows), but rows where the workflow already has the new output's value surface immediately. Refactor backfillAddedGroupOutputs into a generalized backfillGroupOutputsFromLogs helper with an `overwrite` flag — used in both the added-outputs path (preserves hand-edited values) and the new remapped path (overwrites since the new mapping is the source of truth). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(table): update column type when remapping workflow output A remap that changes the output's leaf type (string → number, json → boolean, etc.) was leaving the column's declared type stale. The clear- then-backfill flow then failed schema validation on every row, so the backfill silently aborted and the column stayed empty. Resolve the new leaf type via flattenWorkflowOutputs + columnTypeForLeaf for each mappingUpdate, and patch schema.columns[i].type before the schema write. The clear-tx then backfill ordering now works end-to-end across type changes. If the workflow or its target output can't be resolved (workflow deleted, block removed), fall back to leaving the column type alone — the backfill will skip rows whose picked value doesn't match, same as before. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(table): stringify objects instead of "[object Object]" in cells If a column's declared type lags its row data (e.g. a workflow column mid-remap, where the schema cache hasn't refetched yet but the row data already has the new mapping's value), formatValueForInput and the cell-render text variant fell through to String(value) and rendered "[object Object]". JSON-stringify objects in both spots so the transient skew shows the actual data. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(table): drop extra left border on workflow group meta header The meta cell had border-r/b/l while regular headers have only border-r/b. With border-separate tables, that extra 1px left border shifted the meta cell's content one pixel right of the columns below it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(table): align workflow meta header without dropping its left border Restore border-l and pull the cell back -1px with -ml-px so the visible left border overlaps the previous cell's right border instead of adding 1px to the meta cell's box. Content lines up with the columns below. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(table): draw meta header left border via ::before pseudo Adding border-l to the meta cell shifted its content right by 1px because table-fixed + border-separate honors the border inside the colspan'd cell's width budget. -ml-px doesn't work on <th>. Render the visible left edge via a ::before at left: -1px instead — paints over the prior cell's right border without consuming any of the meta cell's content area. Content lines up with the columns below. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(tables): add missing barrels, drop doubled-path imports Match the convention used by logs/components: every component folder exposes its public API via index.ts so consumers import from the folder name, not from its internal filenames. - New barrels: column-config-sidebar/, workflow-sidebar/, table-action-bar/, table/cells/, table/headers/. - Rename table-filter/index.tsx → index.ts (barrel is not a component). - Top-level components/index.ts re-exports every sibling folder so external consumers have one import path. - Replace `from '../foo/foo'` doubled paths in table.tsx with the shorter barrel-anchored form. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(tables): introduce TablesDetail wrapper as thin passthrough Phase 1 step 0 of the wrapper extraction (see plan okay-lets-make-a-shimmying-trinket.md). page.tsx now renders TablesDetail, which today is a passthrough to <Table>. Subsequent commits lift surface state out of <Table> into this wrapper one piece at a time. The mothership chat path (<Table embedded>) is untouched — <Table> stays exportable as a lower-level component for embedded contexts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(tables): lift slideout panel state into TablesDetail wrapper The three right-edge slideout panels (column config, workflow config, execution details) move out of <Table> into the wrapper. The wrapper owns a single useReducer that encodes the at-most-one-open invariant as a discriminated union — opening any one panel automatically closes the others. <Table> emits open requests via three new callback props. Also extract <ExecutionDetailsSidebar> from inline-in-table.tsx to its own folder so the wrapper can compose it cleanly. Update the embedded mothership callsite (resource-content.tsx) to render <TablesDetail embedded> instead of <Table embedded>. Phase 1 step 1 of the wrapper extraction. <Table> shrinks from 3849 → 3787 lines; <TablesDetail> grows from 19 → 145 lines. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(tables): lift delete-table modal + mutation into wrapper The delete-table confirmation modal and `useDeleteTable` mutation move out of <Table> into TablesDetail. <Table> exposes a new `onRequestDeleteTable` callback fired by the page-header Delete action. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(tables): lift CSV import dialog into wrapper ImportCsvDialog moves out of <Table>. Grid exposes `onRequestImportCsv` fired by the page-header menu item; wrapper owns the open state and renders the dialog. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(tables): lift RowModal (edit + delete) into wrapper Both RowModal instances move out of <Table> into the wrapper. Grid emits `onOpenRowModal(row)` (Space key) and `onRequestDeleteRows(snapshots)` (context menu). Post-delete cleanup (push undo, clear selection) needs grid-internal state, so the grid populates an `afterDeleteRowsSinkRef` callback that the wrapper's modal `onSuccess` invokes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(tables): lift delete-columns modal into wrapper The destructive delete-columns confirmation modal moves into the wrapper. Grid emits `onRequestDeleteColumns(names)`; the cascade itself (per-column mutation, undo push, columnOrder + columnWidths cleanup) stays in the grid as a sink the wrapper invokes on confirm — too grid-internal to lift cleanly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(tables): lift run/stop mutations + TableActionBar to wrapper useRunGroup and useCancelTableRuns move out of <Table> into the wrapper, along with the <TableActionBar> render. Grid receives onRunGroup, onRunRows, onStopRow, onStopRows, onStopAll, and cancelRunsPending as props — used by the per-row gutter Play/Stop, the workflow-group meta-cell run menu, and the right-click context menu's Run/Stop on selection items. Action-bar selection state (actionBarRowIds, runningInActionBar, hasWorkflowColumns) is derived from grid-internal state, so the grid emits a `SelectionSnapshot` via `onSelectionChange` from a useEffect. Wrapper uses the snapshot to drive the floating <TableActionBar>. Phase 2 step 1 of the wrapper extraction. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(tables): lift queryOptions to wrapper queryOptions (filter + sort) moves out of <Table> into the wrapper, making it a single source of truth that drives one useTable call. The wrapper passes the bundle down to the grid; sort/filter handlers in the grid call onQueryOptionsChange. Eliminates the previous double-useTable pattern (one for the grid's filtered/sorted view, one in the wrapper's hardcoded null/null query for sidebar metadata). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(tables): lift page header (breadcrumbs/options/filter) to wrapper Phase 3 of the wrapper extraction. The full page-header surface moves out of <Table>: - ResourceHeader (breadcrumbs, table-rename UI, headerActions, createTrigger) - ResourceOptionsBar (sort + filter toggle) - TableFilter (filter panel — wrapper owns filterOpen state) - RunStatusControl (in the leading actions when runs are active) useRenameTable + useInlineRename for the breadcrumb name move to the wrapper. The grid populates pushTableRenameUndoSinkRef so the rename is still part of the grid's undo stack. Extract NewColumnDropdown and RunStatusControl from inline-in-table.tsx to their own folders so the wrapper composes them cleanly without reaching into the grid's internals. Hoist generateColumnName from grid-internal useCallback to a shared util so both the page-header and inline-header NewColumnDropdowns use the same logic. After this lift <Table> is the data grid only — no page surface, no modals, no slideouts, no breadcrumbs. The selection snapshot now includes totalRunning so the wrapper can render the page-header RunStatusControl from outside the grid. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(tables): cleanup pass on TablesDetail wrapper extraction Six-pass cleanup against the wrapper extraction diff: - Effects: add content-compare bailout to onSelectionChange emit so unchanged snapshots don't churn wrapper re-renders. - Memos: drop unnecessary activeSortState memo, fold into sortConfig. - Callbacks: remove ~10 useCallbacks with no observed reference (sidebars not memoized, modals not memoized, inline arrows on non-memoized children); keep the ones that feed into <DataRow>/<RunStatusControl>/ <ResourceHeader> (memoized) or grid-side useCallback deps. - Dead props: drop onQueryOptionsChange/onRequestDeleteTable/ onRequestImportCsv from <Table> — the page-header lift made them unused but the props weren't removed. - React Query: drop redundant tableWorkflowGroupsRef (created when onRunRows was useCallback-wrapped; after callback cleanup it can read the query data directly). - emcn: normalize Loader sizing to h-[14px] w-[14px] to match the codebase convention. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(table): re-seed columnOrder when columns change server-side The metadata-seed effect short-circuited after the first seed, so any later schema change (e.g. adding a workflow output column) couldn't push the new column into local columnOrder. The new column would then fall into the "remaining" bucket of `displayColumns` and render at the end of the table — until the user refreshed and the grid re-mounted with the now-current metadata. Drop the `metadataSeededRef.current` short-circuit from the early return so the effect can also reach the after-first-load re-seed branch, which already does the right thing (only re-seeds when the set of columns changes, leaves pure-reorder cases alone). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(tables): rename wrapper to <Table>, grid to <TableGrid> Match the naming convention used elsewhere in the workspace (workflow.tsx → <Workflow>, base.tsx → <Base>, logs.tsx → <Logs>). - tables-detail.tsx → table.tsx (exports <Table>) - components/table/ → components/table-grid/ (exports <TableGrid>) - components/table-grid/table.tsx → table-grid.tsx - Drop <ExecutionDetailsSidebar> — was a 3-line passthrough (executionId → useLogByExecutionId → <LogDetails>); inline directly into table.tsx where it's used. - Flatten components/run-status-control/ folder to a single components/run-status-control.tsx file. 25-line single-use component with no internal subdirs — folder was overhead. Matches knowledge's max-badge.tsx precedent. Net: 1 wrapper rename + grid rename + 2 folder collapses, all imports updated. The mothership chat callsite updates from <TablesDetail embedded> to <Table embedded>. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(table): don't show "Waiting" for autoRun=false workflow groups A workflow group with autoRun=false never fires from the scheduler — the cell stays empty until the user clicks Run manually. Treating empty cells as "Waiting" misleads the user into thinking the group will auto-fire once deps are filled, which it won't. Skip autoRun=false groups when computing the per-row waiting labels so their cells render the empty-dash instead of the Waiting pill. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(copilot): regenerate tool catalog from copilot dev (#247) Pulls in the workflow_group operations on user_table: add_workflow_group / update_workflow_group / delete_workflow_group / add_workflow_group_output / delete_workflow_group_output / run_workflow_group, plus the autoRun / blockId / dependencies / groupId parameters and a tightened mapping description for import_file. Also picks up biome import-order fixes from `bun run lint`. * improvement(table): action bar in mothership + per-execution mode Three related improvements to the table action bar: 1. Reposition from `position: fixed` to `position: absolute` inside the table's container. Fixed-positioning anchored to the viewport, which centered the bar across the whole window instead of the table panel — wrong in mothership embedded view, where the table sits in the right half. Absolute scopes the bar to the table's bounds. 2. Show the bar for single-execution highlights — when the user selects one workflow-output cell, or 1 row × N cols all within the same workflow group. The bar enters per-execution mode with Run / Stop / View execution buttons targeting that one cell or group. 3. Skip View execution for cancelled cells. A cancelled cell may have been cancelled before the worker ever picked the job up, so its executionId can't be relied on. Tighten the gate everywhere (context menu + action bar) to only `completed` / `error` / `running`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(table): backfill on add_workflow_group_output, don't re-run addWorkflowGroupOutput (the one-shot single-output add path used by the copilot user_table tool) was calling triggerWorkflowGroupRun({ mode: 'all' }) after appending the output — that re-fired the workflow on every row. Trace a307ed8fd5fe2d931aa84dedab5a60f0 shows ~75 workflow-group-cell jobs enqueued in the seconds after a single add_workflow_group_output call. Replace with backfillGroupOutputsFromLogs (overwrite: false), the same flow updateWorkflowGroup uses when receiving newOutputColumns. Reads each row's saved trace spans and writes the new output's value back — no compute beyond a JSONB write per row, no double-billing the user for runs they didn't ask for. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(table): drop sql.raw quote-escaping in column-name interpolation Six call sites in lib/table/service.ts built JSON-key string literals at runtime via `sql.raw(\`'\${name.replace(/'/g, "''")}'\`)` for use with PostgreSQL's `data->'key'` / `data->>'key'` operators. Practically safe (NAME_PATTERN gates column names to alphanumeric+underscore at insert time) but a smelly pattern that breaks the moment validation loosens. Both `data->` and `data->>` accept a parameterized text value as the key, so the `sql.raw` is unnecessary. Replace each with a normal `${name}::text` binding. No behavior change; eliminates the manual quote-escaping surface. Affected sites: renameColumn (the data-rewrite UPDATE), upsertRow's match filter, updateColumnType's IS-NOT-NULL gate, updateColumnConstraints' required-check + unique-duplicate-check. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(copilot-tool): forward autoRun + mappingUpdates on update_workflow_group The sim-side service and contracts already accept both fields, but the copilot tool's update_workflow_group handler was dropping them on the floor. Now `args.autoRun` (toggle the persisted auto-fire flag) and `args.mappingUpdates` (per-output (blockId, path) swap) get forwarded through to updateWorkflowGroup. Pairs with the upcoming copilot-side change that exposes these in the tool catalog JSON / Go handler / prompting (see copilot branch redo-workflow-tools). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(table): keep gutter border visible when hovering Run-row button The per-row Run button sat flush against the row-gutter cell's right border. Its hover background (rounded-rect surface-2) painted over the border line for the 20px height of the button, making the gutter divider appear to disappear at the hovered row. Add mr-px to the button so the hover bg stops 1px short of the cell's right edge, leaving the divider intact. * fix(table): unify auto-fire and manual run paths in scheduler scheduleWorkflowGroupRuns now owns eligibility, autoRun semantics, dep evaluation, and enqueue for both paths. Auto-fire callers omit opts; manual callers (triggerWorkflowGroupRun) pass { groupId, isManualRun: true } to bypass the autoRun=false skip and (for autoRun=false groups) the dep check. Per-row /run-workflow-group route delegates to triggerWorkflowGroupRun with rowIds=[rowId]. Single server-side path for both manual entry points. Also: optimisticallyScheduleNewlyEligibleGroups skips autoRun=false groups so editing a row's data doesn't phantom-mark autoRun=false output cells as Queued. * fix(table): render empty cells as blank, not em-dash Empty cells (any column type) showed an em-dash placeholder. Drop it so empty cells render blank — matches what the user expects when nothing's there. * fix(table): per-row Run fires autoRun=false groups regardless of deps handleRunRow filtered out every group whose deps weren't satisfied, which silently dropped autoRun=false groups (since their deps usually aren't satisfied — that's the whole point of autoRun=false). Click Run row, the autoRun=false group's cells stayed empty. Mirror the scheduler's semantics: autoRun=false bypasses the dep check, autoRun=true still requires deps. * fix ui shape * improvement(table): collapse run ops into run_column, derive action-bar buttons from selection The action bar now reflects what's actually selected: - Selection-driven scope (cells the user highlighted, not their full rows) - Play visible when there's anything empty/failed; Refresh when there's anything completed; both for mixed - run_cell / run_row deleted; everything funnels through run_column - Per-row gutter Play, right-click "Run workflows on N rows", and column-header menu all share the canonical run path - Shared RunMode type from the contract; cleanup pass via /simplify (readExecution / isExecInFlight reuse, runScope helper, flat onViewExecution prop) * chore(copilot): regen tool catalog after dropping run_cell / run_row + dependencies.workflowGroups Mirror the copilot-side catalog change so the generated TS catalog matches the deployed copilot tool surface. * fix(table): atomic per-key writes for executions, plus run-op race fixes The executions blob on user_table_rows was read-modify-written wholesale on every update. Concurrent writers (a column edit and a manual-retry stamp, two pickup calls, a cancel and a cascade) each computed a merge from their own snapshot, and the last writer clobbered keys it never touched — producing stuck "queued" cells, vanished stamps, and stale completed exec records reappearing after retries. Fixes: - updateRow / batchUpdateRows now apply executionsPatch via a SQL jsonb merge expression. Each writer only mutates the keys it explicitly patches; other keys are preserved. Eliminates the cross-key clobber. - writeWorkflowGroupState bypasses the stale-worker guard for `queued` (new scheduler stamp) and `cancelled` (authoritative cancel) writes — those ARE the new authority for the cell. Previously the new run's stamp was being rejected by the same guard meant to block the OLD worker's writes. - skipScheduler flag on UpdateRowData / BatchUpdateByIdData lets the cancel path and runWorkflowGroupsInternal opt out of the implicit auto-fire pass (cancel was waking up siblings; manual-run was racing its own scheduler). - CELL_CONTENT pinned to h-[22px] so status badges don't grow rows. * chore(table): remove table-row sockets, both sides Tables don't use realtime sockets in prod — strip the dead path so we stop paying the per-row HTTP forward + socket emit on every cell write. Polling on running execs already covers reconciliation. Sim side: - service.ts: drop notifyTableRowUpdated/Deleted, notifyTableDeleted, the postRealtimeBridge helper, and all callsites. - hooks/queries/tables.ts: drop the socket subscription block in useTableRows; poll-on-running stays. Remove useEffect / useSocket imports. - app/.../tables/[tableId]/hooks/use-table.ts: drop the merge-on-event useEffect and unused imports. - app/workspace/providers/socket-provider.tsx: drop joinTable/leaveTable, onTableRowUpdated/Deleted/onTableDeleted, currentTableId state, related events + types. Realtime side: - handlers/tables.ts deleted; index.ts no longer wires it. - routes/http.ts: drop /api/table-row-updated, /api/table-row-deleted, /api/table-deleted endpoints. - rooms/{memory,redis}-manager.ts: drop emitToTable, handleTableRowUpdated/ Deleted, handleTableDeleted, related imports. - rooms/types.ts: drop method declarations, TableRowUpdatedPayload type, tableRoomName helper. - middleware/permissions.ts: drop unused verifyTableAccess. Bonus from parallel work: - cell-content typewriter trigger refinement. * fix(table): clearing a workflow output cell also clears its exec record When the user wipes a workflow output column value, the auto-fire reactor needs to be re-armed for that group. Previously, a stale cancelled / error exec record blocked the eligibility predicate (gate at line 79 hard-rejects those statuses on auto-fire) and the cell stayed stuck in its old terminal state — visible as "Cancelled" cells that wouldn't re-run no matter what. Both updateRow and batchUpdateRows now derive an `executionsPatch[gid] = null` for any output column the patch sets to empty. The data clear and the exec clear ride the same SQL transaction, so the row never lands in a stale- status-with-empty-data state. Symmetric to how `completed` already worked via `areOutputsFilled` in the predicate — clearing the cell wins over the prior exec status, regardless of what that status was. (Also revert typewriter-trigger experiment from a parallel session that was in-progress on this branch.) * fix(table): waiting state, optimistic UX, schema-mutation polling, exec cleanup A bundle of small UX + correctness fixes around workflow-cell run state. cell-render.tsx - In-flight (queued/running/pending) now wins over the existing value, so re-runs surface immediately instead of looking like nothing happened until the worker writes the new value. - "Waiting on X" wins over a stale `cancelled` / `error` exec when deps are unmet — clearing a dep now reads as actionable instead of stuck. useRunColumn (hooks/queries/tables.ts) - onSettled now cancels in-flight polls before invalidating. Stops a poll that landed mid-mutation from clobbering the optimistic state with stale data, which produced the queued → cancelled → queued flicker. addWorkflowGroup / updateWorkflowGroup (autoRun toggle on) - Awaits scheduleRunsForTable instead of fire-and-forget. The route returned before the queued exec stamps committed, so the post-mutation refetch saw no in-flight cells and polling never started — cells looked stuck even though the server eventually stamped them. deleteColumn / deleteColumns - Strip orphaned executions[gid] keys when deleting a column orphans its parent group. Without this, stale running/queued exec records lingered on every row forever and inflated the page-header "N running" counter even on tables with no actually-running cells. UI - Action-bar leading label: "Selected N workflow cell(s)". - Context menu: Run / Refresh items mirror the action bar's Play / Refresh split, gated on the same selection-status flags so both surfaces show the actions that match the current state. * refactor(table): consolidate exec-status helpers + fix N-running counter Cleanup pass on the recent table changes — pulls duplicated predicates and SQL snippets into shared helpers and fixes one drift bug along the way. - isExecInFlight: now single export from lib/table/deps.ts. Removed the duplicate in components/table-grid/utils.ts. Used by isGroupEligible (server eligibility) and runningByRowId (client counter). - isOptimisticInFlight: kept local to hooks/queries/tables.ts — renamed from isInFlight to disambiguate from the stricter isExecInFlight. The two predicates differ on `pending` without a jobId: optimistic patches and poll-trigger want the broader version, eligibility wants the strict one. - areOutputsFilled: single export from lib/table/deps.ts, dropped duplicate from workflow-columns.ts. - classifyExecStatusMix: shared row × group walker in table-grid/utils.ts. Replaces two copies of the same loop in table-grid.tsx (selectionStats + contextMenuStats). Both surfaces now have the same short-circuit semantics, including the seen-all-selected-rows early break that contextMenuStats was missing. - stripGroupExecutions: SQL helper in service.ts. Replaces three copies of the `UPDATE user_table_rows SET executions = executions - $gid::text` pattern across deleteColumn / deleteColumns / deleteWorkflowGroup. Drift bug: - runningByRowId / totalRunning counted only `running` and `queued`. Every other in-flight check in the codebase treats post-stamp `pending` as in-flight too, so the page-header "N running" badge briefly dropped to 0 between scheduler stamp and worker pickup. Now uses isExecInFlight. * fix(table): address pr review (drop dead workflowNameById prop, reset didDragRef on dragend, align sidebar width) * fix(table): scope post-clear schedule to targeted groups, forward mode Multi-group manual runs (Run row, gutter Play, action-bar Play across mixed completed + cancelled cells) re-fired completed-and-filled siblings. runWorkflowGroupsInternal cleared only the groups it filtered, then called scheduleRunsForRows with isManualRun: true and no group / mode filter — so the post-clear pass walked every group on the table with default mode 'all', and any autoRun=true completed sibling whose deps were satisfied got queued again. Scope the post-clear call to targetGroups and forward mode. * fix(table): meta-cell drag-leave flicker guard + plumb unique on create * fix(table): strip sibling deps when removing workflow output via updateWorkflowGroup deleteWorkflowGroup already stripped removed-column deps from sibling groups, but updateWorkflowGroup (the path the UI takes when deleting one output of a multi-output group) didn't — schema validation then rejected the update with 'Group X depends on missing column Y'. * improvement(table): debug logs at every cascade decision branch * improvement(table): parallelize queued-stamp writes within concurrency-cap chunks * Simplify stripping column names * fix lint, ci --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
WaleedandClaude Opus 4.7
|
a251e45400 |
feat(sap): add SAP Concur integration block and SAP S/4HANA validation fixes (#4483)
* feat(sap): add SAP Concur integration block and SAP S/4HANA validation fixes
* added
* fix(sap_s4hana): preserve raw Set-Cookie array for CSRF cookie join
SecureFetchHeaders previously collapsed multi-value Set-Cookie headers
with ", ", forcing consumers to re-split via a fragile regex. Cookie
values containing "=" or "," (e.g., Base64 session tokens) could be
misparsed and produce malformed Cookie strings on CSRF-protected
mutations.
Add SecureFetchHeaders.getSetCookie() that returns the raw array, and
update the S/4HANA OData proxy's joinSetCookies to consume it directly.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(sap-concur): rename misleading exchange-rate tool, drop unusable refresh_token grant, validate geolocation host
- Rename sap_concur_get_exchange_rate to sap_concur_upload_exchange_rates (POST bulk upload, not GET)
- Remove refresh_token from SapConcurGrantType / Zod enum / block dropdown / docs (no implementation)
- Validate Concur geolocation hostname against SAP_CONCUR_ALLOWED_DATACENTERS
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* finished
* docs
* fix(docs): escape braces in tool/trigger description prose for MDX
Tool and trigger descriptions can contain URL path placeholders like
{reportId} or JSON-shape hints like { Items, NextPage }. When rendered
as MDX prose (not table cells), these were emitted unescaped and MDX
parsed them as JSX expressions, failing prerender with
"ReferenceError: reportId is not defined".
Escape { and } in the operation-level description and trigger
description renderers, matching the existing escaping in table-cell
descriptions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(sap-concur): align with live API on travel-profile, itineraries, and context types
- list_travel_profiles_summary: rename Status query to Active with 1/0 values, tighten LastModifiedDate format hint
- list_itineraries / get_itinerary: use documented userid_type / userid_value / ItemsPerPage / Page query keys
- create_report_comment: contextType allows MANAGER (move to EXPENSE_READ_CONTEXT_TYPE_OPS)
- get_list_item: drop unused listId from block (tool only needs itemId)
- Tighten description copy on list_expenses/get_itemizations/associate_attendees/remove_all_attendees
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(sap-concur): correct Cash Advance v4.1 paths, add SCIM filter param
- Update Cash Advance create/get/issue tools from /cashadvance/v4/ to /cashadvance/v4.1/ to match the live API
- Add filter query param to list_users (SCIM v4.1 supports filtering by userName, employeeNumber, externalId)
- Regenerate docs MDX
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(sap-concur): drop SCIM list_users filter param (not supported on v4.1 GET)
SCIM Identity v4.1 GET /Users does not accept a filter query parameter — filtering
is only supported via POST /Users/.search (already exposed by sap_concur_search_users).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(sap-concur): final live-API alignment
Verified against live SAP Concur docs (concur/developer.concur.com preview branch):
- Revert Cash Advance paths to /cashadvance/v4/ (v4.1 endpoints do not exist; live spec is v4)
- Travel Profile v2 summary has no Active/Status query param — drop the filter from tool, types, and block
- Report Comments v4 contextType is TRAVELER or PROXY only (NOT MANAGER) — move create_report_comment + list_report_comments into the TRAVELER/PROXY context group
- Trip v1.1 query keys: userid_type / userid_value / ItemsPerPage / Page (snake/Pascal per docs) — already correct, kept
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs
* fix(sap-concur): restore Cash Advance v4.1 paths
Re-verified against live developer.concur.com docs at /api-reference/cash-advance/v4-1.cash-advance.html — only v4.1 endpoints are documented:
- POST /cashadvance/v4.1/cashadvances
- GET /cashadvance/v4.1/cashadvances/{cashAdvanceId}
- POST /cashadvance/v4.1/cashadvances/{cashAdvanceId}/issue
The /cashadvance/v4/ docs page returns 404. Reverts the prior local rollback in
|
||
|
WaleedandClaude Opus 4.7
|
d721dc3358 |
feat(enterprise): add data drains for continuous export to S3 / webhook (#4440)
* feat(enterprise): add data drains for continuous export to S3 / webhook * chore(data-drains): regenerate migration on top of staging + bump route baseline Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(data-drains): clarify retention pairing is user-coupled, not enforced Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(data-drains): preserve explicit forcePathStyle=false + reserve x-sim-signature Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(data-drains): drift guard ensures every webhook header is reserved Asserts that any header buildHeaders writes is rejected when reused as a custom signatureHeader. Adding a new metadata header without mirroring it into RESERVED_SIGNATURE_HEADER_NAMES now fails CI. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
|
09f4c94b3c |
feat(block): Allow wait block to wait up to 30 days (#4331)
* v0.6.29: login improvements, posthog telemetry (#4026) * feat(posthog): Add tracking on mothership abort (#4023) Co-authored-by: Theodore Li <theo@sim.ai> * fix(login): fix captcha headers for manual login (#4025) * fix(signup): fix turnstile key loading * fix(login): fix captcha header passing * Catch user already exists, remove login form captcha * feat(block): Allow wait block to wait up to 30 days * restore ff * Filter out waits from hitl endpoints * Use correct count, filtering out wait blocks * improvement(wait): tighten poll route and pause-manager helpers - Parallelize per-row dispatch with Promise.all - Add status='paused' guard on nextResumeAt rewrite to prevent clobbering concurrent resumes - Extract computeEarliestResumeAt + PauseResumeManager.setNextResumeAt helpers - Use canonical PausePoint type in poll route (drop StoredPausePoint) - Narrow UNIT_TO_MS via as const + WaitUnit guard - Bump LOCK_TTL_SECONDS above route maxDuration - Clearer error when allowedPauseKinds rejects a resume --------- Co-authored-by: Waleed <walif6@gmail.com> Co-authored-by: Siddharth Ganesan <33737564+Sg312@users.noreply.github.com> Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com> |
||
|
WaleedandClaude Opus 4.7
|
029ac9fb05 |
fix(logs): split summary/detail contracts to make trace tab gate type-safe (#4431)
* fix(logs): split summary/detail contracts to make trace tab gate type-safe The Trace tab was silently missing from the Log Details sidepanel because list and detail rows shared one WorkflowLog type with executionData: z.unknown(). The UI couldn't distinguish a summary row (no spans) from a detail row (with spans), so the tab gate read undefined and hid itself. Splits into WorkflowLogSummary (list) and WorkflowLogDetail (typed executionData with optional traceSpans). Detail and by-execution routes both write through to the same logKeys.detail(id) cache, eliminating the two-key fragmentation that caused the merge memo workaround. List route moves to cursor pagination on (sortValue, id) with proper NULLS LAST handling and SQL-side sort across workflow + job execution tables. Detail route now requires and asserts workspaceId. Deep-link path uses useLogByExecutionId instead of auto-paginating the entire workspace. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(logs): audit follow-ups — render side-effect, stats invalidation, enhanced spread order - Move onActiveTabChange call from render into useEffect to avoid side-effects during render (StrictMode safety). - Re-add logKeys.stats() invalidation to cancel/retry mutations so the dashboard reflects status flips immediately. - Reorder enhanced: true after ...execData spread in detail and by-execution routes so the literal discriminator is never overwritten by stale execData. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(logs): mirror SQL NULLS LAST in JS merge for cursor consistency The in-memory merge of workflow + job pages negated the comparator for DESC, which placed null sort values at the start. SQL orders both ASC and DESC with NULLS LAST, so DESC pages emitted a cursor {v: <last non-null>, id: ...} while null rows still satisfied the cursor predicate (OR sort_expr IS NULL) on the next page — producing duplicate null rows across pages on cost/duration sorts. Handle nulls explicitly in the JS comparator so they always sort last regardless of direction, matching the SQL ordering. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(logs): final-audit follow-ups — stable tab callback, byExecution invalidation, optimistic detail patch, trace loading state - Wrap LogDetails -> LogDetailsContent onActiveTabChange in useCallback so the child useEffect doesn't refire on every parent render. - Add logKeys.byExecutionAll() to cancel + retry invalidation so the table-embedded sidebar picks up status changes immediately. - Optimistic write-through to logKeys.detail in useCancelExecution so the open sidebar reflects 'cancelling' instantly; rolls back on error. - Distinguish trace loading from trace-empty: when log.executionData is not yet fetched, render "Loading trace…" instead of the empty state. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(logs): migrate stores/components to contract types Replace the legacy `WorkflowLog` / `LogsResponse` / `WorkflowData` / `CostMetadata` / `ToolCallMetadata` shapes in `stores/logs/filters/types.ts` with direct use of the contract types `WorkflowLogSummary`, `WorkflowLogDetail`, and a new `WorkflowLogRow` alias for surfaces that render either form. Removes the `summaryToWorkflowLog` / `detailToWorkflowLog` bridge in the React Query layer along with their double-cast annotations. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(logs): address PR review feedback - Whitelist sort columns against logSortBy enum to prevent client crash when non-sortable headers (workflow, trigger) reach the contract parser. - Extract fetchLogDetail helper shared by /api/logs/[id] and /api/logs/by-execution/[executionId] — collapses ~360 duplicated lines to a single source of truth keyed on lookup column. * fix(logs): exclude job logs when level filter is workflow-only When level=running or level=pending (workflow-only states involving endedAt/pausedExecutions semantics), jobLevelConditions stayed empty so no level constraint reached jobConditions — every job log in the workspace leaked into the result. Skip the job side entirely when the level filter has no job-applicable values (error/info). * chore(logs): drop dead utils — mapToExecutionLog and friends Remove ExecutionLog/RawLogResponse/ExecutionCost/LogWithExecutionData/ TraceSpan/BlockExecution interfaces and the mapToExecutionLog, mapToExecutionLogAlt, extractOutput functions — all unreferenced after the contract split. -212 lines. * chore(logs): drop unused LOG_COLUMN_ORDER and LogColumnKey * fix(logs): hydrate filters from URL synchronously on mount The previous useEffect-based initializeFromURL caused useLogsList and useDashboardStats to fire once with default store filters, then refetch after the effect updated filters from the URL. Move the initial hydrate into a useState lazy initializer so the first render already reads URL-derived filters; the popstate handler keeps the existing effect for back/forward navigation. * chore(logs): trim verbose comments added during PR Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(logs): guard navigation arrows when selected log is off-page Deep-linked logs resolved via useLogByExecutionId may not be in the current page list, leaving selectedLogIndex at -1. The hasNext prop was evaluating -1 < logs.length - 1 (true for any non-empty list), which enabled the next arrow and jumped to the first item on click. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(logs): sync active-tab callback before paint to keep keyboard guards aligned Run the resolvedTab → onActiveTabChange propagation in useLayoutEffect so the parent's activeTabRef updates synchronously before the next paint. This closes the brief window where window keydown handlers in the logs page would still see activeTabRef.current === 'trace' and short-circuit arrow-key navigation immediately after switching to a log without a Trace tab. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Waleed
|
a479e88718 |
feat(files): export markdown as zip with embedded images (#4413)
* feat(files): export markdown as zip with embedded images in assets/ folder * fix(files): sanitize zip filenames, fix storage context cast, cap embedded image count * fix(files): fix race condition in asset filename deduplication * chore(files): remove extraneous comments from export route * fix(files): sanitize markdown zip entry name, full uuid fallback for filename dedup * fix(files): extract userId const to satisfy TypeScript narrowing in async callback * fix(files): use replacer function to prevent $ special-char corruption in markdown URL rewrite * fix(files): wrap zip buffer in Uint8Array for NextResponse BodyInit compatibility * lock behavior * fix(workflow): track resolved isAdmin in prevIsAdminRef to prevent stale lock notification When workspacePermissions loads asynchronously, prevCanAdminRef (which only tracked effectivePermissions.canAdmin) would not detect the change, causing the early-return guard to skip rebuilding the notification with the correct unlock-button visibility. Track the same resolved value (workspacePermissions ?.viewer?.isAdmin ?? effectivePermissions.canAdmin) that is actually used to build the notification. * refactor(uploads): rename server fn to fetchWorkspaceFileBuffer, move client download to uploads/client/download.ts as triggerFileDownload * more lock updates * fix(tests): update mocks for fetchWorkspaceFileBuffer rename |
||
|
Waleed
|
af55bad491 |
fix(uploads): direct-to-upload workspace files + shared transport (#4407)
* fix(uploads): direct-to-S3 workspace files + shared transport
* chore(testing): centralize posthog and storage-service mocks
* fix(uploads): address PR review — abort propagation, orphan cleanup, error handling
- Throw immediately on AbortError in KB retry loop (no useless 14s backoff)
- Cleanup S3/Blob object on quota or size-cap rejection in registerUploadedWorkspaceFile
- Enforce MAX_WORKSPACE_FILE_SIZE at registration (defense vs presigned PUT lying about size)
- Handle non-OK / non-JSON responses in workspace-files upload paths
* fix(uploads): add Zod contracts for workspace presigned/register routes
* fix(uploads): correct BlobServiceClient type name in headBlobObject
* fix(uploads): address PR review — typo, complete-failure cleanup, double-increment
* fix(uploads): preserve fallback size and reuse existing display name on re-register
* fix(uploads): surface server error message and bypass quota for local-storage fallback
* fix(uploads): align register response schema with UserFile; skip presigned for KB large files
- registerWorkspaceFileResponseSchema now matches the UserFile shape the route actually returns; previous schema required workspace DB-row fields that were never populated, causing requestJson validation to reject successful uploads.
- KB batch presigned fetch now skips files >= LARGE_FILE_THRESHOLD since multipart bypasses the per-file presigned URL anyway.
* fix(uploads): idempotent register skips duplicate audit/posthog; add edge-case tests
- registerUploadedWorkspaceFile now returns { file, created } so the route can skip captureServerEvent and recordAudit on idempotent re-register (existing metadata reused). Previously a re-register fired duplicate analytics + audit log entries.
- Add tests covering: idempotent re-register skips audit/analytics, isNetworkError matches econnreset/timeout/etc keywords, multipart complete failure fires action=abort cleanup.
* fix(uploads): include 50MiB boundary in batch presigned fetch
* fix(uploads): trust HEAD size to prevent quota inflation
The head.size > 0 fallback let a client PUT 0 bytes and register
with an inflated size, debiting quota without storing data. HEAD
on an existing object always returns the true byte count, so trust
it directly — a genuine 0-byte file correctly contributes 0.
* fix(uploads): audit verified file size, not client-supplied
* fix(uploads): handle register retries and name-collision races
Two bugs in registerUploadedWorkspaceFile:
1. Register retry could orphan storage. When a successful response
was lost on the wire and the client retried, the quota check saw
the bytes already counted, failed, and cleanupOrphan deleted the
already-registered storage object — leaving the DB row pointing
to nothing. Fix: check getFileMetadataByKey before quota guard
and short-circuit on existing record.
2. Concurrent same-named uploads could lose data. allocateUniqueWorkspaceFileName
is best-effort; two racing uploads can pass it and both attempt
the same display name. The loser's insert hits 23505, the catch
block called cleanupOrphan, and successfully-uploaded bytes
were deleted. Fix: retry on 23505 with a fresh allocateUniqueWorkspaceFileName,
matching the pattern in uploadWorkspaceFile. Throw FileConflictError
after exhaustion.
* fix(uploads): retry transient DirectUploadErrors at outer KB level
The KB outer retry only triggered on isNetworkError, missing
transient 5xx from S3/Azure (DirectUploadError code
DIRECT_UPLOAD_ERROR or MULTIPART_ERROR). Adds isTransientUploadError
and retries on it, restoring resilience for small-file presigned
PUTs against flaky cloud storage.
* fix(uploads): only retry transient 5xx, not deterministic 4xx
DirectUploadError now carries the HTTP status. isTransientUploadError
gates on 5xx so callers don't loop on 400/403/404 (e.g., malformed
request, expired signature). Multipart per-part retry also short-circuits
on 4xx — same reasoning.
* refactor(uploads): collapse getFileContentType into resolveFileType
The two helpers differed only in whether application/octet-stream
falls back to the extension map. Add an option flag to resolveFileType
and keep getFileContentType as a thin wrapper for direct-PUT callers
that need to preserve the exact browser-reported content-type.
* chore(uploads): trim verbose comments
Drop inline comments that restate code ("Use the full storageKey as fileName"),
collapse a multi-line block comment into a tighter TSDoc on the existence
check, and prune verbose vitest file headers — describe blocks already
document what's tested.
* fix(uploads): regenerate fileId per insert retry; require cloud storage for register
* fix(uploads): cap formdata fallback at 100MB; drop unused size param
* fix(uploads): abort multipart on get-part-urls failure; retry register on transient errors
* fix(uploads): drop vestigial size field from register contract
* fix(uploads): abort multipart on complete-fetch throw
* fix(uploads): set kb presignedEndpoint fallback; race-safe blob HEAD
* fix(uploads): include ?type=knowledge-base on kb presigned fallback
* fix(uploads): remove abort listener on xhr timeout
* fix(uploads): add timeout/abort to kb api fallback upload
|
||
|
Theodore Li
|
31cfb74dc2 |
feat(table): add workflow execution column type (#4338)
* Add table triggers for columns and row added * Add async batching job for running column * Add ui improvements, stop mechanism * Use trigger dev for workflow runs * Use unified column sidebar for table * Add socket waits for tables, multi column workflow support * change back to cell based trigger jobs * Reuse code, add view log inline in table view * reorganize db to treat each column separately * adjust column naming strategy * Column ui improvements * fix live update on table * Change new column behavior * fixed errored workfows not showing as stopped * Column sidebar improvements * fix table column swapping behavior * fix bugs * add prompting, fix lint * fix ui stuff * flip feature flag * Add zod contracts fix initial auto-run of columns * ui improvements * Use db filter to query unran rows * Use live workflow run * Change wording for deleting workflow column * Update tools * Add tool to run selected rows * Add mothership tools * adjust col width * Restore ff * fix drizzle migration * fix test |
||
|
Waleed
|
66bab935db |
fix(chat): close SSO auth bypass via checkSSOAccess body flag (#4408)
* fix(chat): close SSO auth bypass via checkSSOAccess body flag - Remove checkSSOAccess short-circuit; SSO branch always validates via getSession() - Skip chat_auth cookie issuance/validation for SSO deployments to prevent replay - Split eligibility pre-flight into dedicated POST /api/chat/[identifier]/sso route - Drop .passthrough() and checkSSOAccess from deployed chat contracts - Add SSO branch test coverage in chat utils * fix(chat): cast allowedEmails to string[] for SSO eligibility check * fix(chat): close SSO GET cookie replay and add eligibility rate limit - Skip chat_auth cookie validation for SSO in GET handler (replay vector for pre-fix cookies) - Route SSO GET through getSession() instead of always returning auth_required_sso so post-IdP config fetch works - Add per-IP rate limiting to /api/chat/[identifier]/sso to prevent allowlist enumeration |
||
|
Vikhyath Mondreti
|
b10b4479fe |
improvement(repo): update ship skills, flatten internal tools contracts dir (#4379)
* improvement(repo): update ship skills, flatten internal tools contracts dir * update baselines * address comments |
||
|
Waleed
|
8b6307aea4 |
feat(gmail): add edit draft and update label tools (#4374)
* feat(gmail): add edit draft and update label tools * fix(gmail): correct legacy block access list and docs heading for edit_draft_v2 * fix(gmail): use shared contract for edit-draft route * regen docs * fix(knowledge): inline reranker model list in description for doc generator * resolve |
||
|
Vikhyath Mondreti
|
be9c959f1a |
improvement(types): enforce patterns outside just hooks directory and fix CI check + fix tracing billing issue (#4367)
* improvement(types): enforce outside just hooks dir and update CI checks * fix billing account details for kb embeddings * more fixes * fix byok issue * address comments * fix * more comments * address bugbot |
||
|
Vikhyath Mondreti
|
b8959eb20d |
improvement(repo): zod based client-server boundary (#4355)
* improvement(repo): centralized zod contracts (#4336) * improvement(repo): zod schema contracts * type checks * fix(notion): correctly register tool (#4337) * fix func blokc * more improvements * fix tests * type check * remove v3 refs * minor type improvements * address comments * update jira contract * remove validateJsonBody * improvement(repo): consolidation of boundary helpers + better unknown usage (#4352) * improvement(repo): consolidation of boundary helpers + better unknown usage * address comments * improve file transfer error messaging * fix docs listing schema drift * fix inocrrect type casting * address council comments * remove prefix |