mirror of
https://github.com/simstudioai/sim.git
synced 2026-09-24 15:45:35 +08:00
06506bbd385aa052d14e9b5014fc083045ce34d9
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Waleed
|
ee0157df4b |
feat(tables): add currency column type on a new column-type registry (#6106)
* feat(tables): add currency column type on a new column-type registry
Adds a `currency` column type, and consolidates the per-type knowledge it
would otherwise have been scattered across.
**Currency.** Stores a plain number and carries an ISO 4217 `currencyCode`
as display metadata. That split is what keeps it cheap: filtering, sorting,
uniqueness and CSV export all reuse the numeric paths unchanged, changing a
column's currency rewrites no rows, and the public row output stays a number
rather than a locale-formatted string consumers would have to reparse.
Input accepts the shapes an amount actually arrives in — `$1,234.56`,
`1 234,56 €`, `(12.00)` — so pastes, CSV imports and tool writes land as
numbers instead of being nulled.
**The registry.** Adding this type initially required edits in ~40 places:
32 switch arms under `lib/table`, ~26 UI branches, two hand-maintained icon
maps, and a coercion implementation duplicated four times. Every one of those
failed silently when missed — a missing `jsonbCastForType` arm compares
numbers as text; a missing compatibility arm blocks all conversions.
`lib/table/column-types/` now holds one file per type carrying its label,
icon, badge colour, storage cast, filter operators, coercion, validation,
compatibility and formatting. `Record<ColumnType, …>` on both registries is
the completeness gate: adding a type to the union is a compile error naming
exactly the two files to fill in, and the interface then requires every
field. The 32 switch arms are down to 3.
Two duplicates collapse as a consequence:
- The client no longer mirrors the server's select id-resolution. Those
helpers lived in `validation.ts`, which imports drizzle, so anything
reaching them became server-only and the grid hand-rolled its own copy.
Extracting them to `select-options.ts` lets both sides share one
implementation, so the optimistic cache can no longer disagree with what
gets persisted.
- The two icon maps become one registry read.
It also fixes a live inconsistency it surfaced: currency got a numeric
keypad in the grid's inline editor but a plain text field in the row modal.
Behaviour-neutral by construction: all 1046 tests in the touched areas pass
unchanged, with no test edits.
* test(tables): guard the column-type registry's invariants
Property tests for the registry itself rather than any one type: entries key
by their own id, COLUMN_TYPES stays derived, an unknown type degrades to
string instead of throwing, only opaque-id types restrict filter operators,
only configuration-free types are CSV-inferable, and every type that can
reject a draft has a message to show.
Plus the metadata-ownership matrix, which pins the generic ownership check to
the same answers the hardcoded per-type rules gave.
These target the registry's silent-failure class — a wrong jsonbCast or a
stray operator whitelist used to be invisible until a filter failed in SQL.
Both are verified to fail under mutation.
* fix(tables): read exponent-form amounts and reject bad currency PATCHes up front
Two P1s from review.
Scientific notation lost magnitude. `String()` emits exponent form past 1e21,
so a stored amount round-trips through the editor as `1e+21` — and the
sanitizer treated the `e` as decoration to strip, reading it back as 121. An
untouched cell silently lost 19 orders of magnitude on its next edit. Exponent
form is now taken at face value, but only when the string is wholly a numeric
literal once symbols are removed, so `12 EUR` (whose `E` survives the strip)
still parses through the separator path.
A failed currency PATCH left a partial rename. `renameColumn` commits in its
own transaction before the currency write, so a `currencyCode` the service
would reject — an unsupported code, or any code on a non-currency column —
errored only after the rename had stuck. Both are now caught before the first
write, matching the guard the route already applies to unique-on-select for
exactly this reason.
* refactor(tables): finish the registry migration and drop the dead config
Audit pass over every consumer, closing the gaps the first cut left.
Functional gap: the copilot agent had no currency support at all — it could
create a currency column with no code and could never re-denominate one.
`add_column` and `update_column` now accept `currencyCode`, with the same
up-front validation and the same code-only routing as the HTTP routes.
Config that consumers were still restating, now read from the registry:
- `supportsUnique` replaces the unique-on-select guard stated in three places
(service, both column routes, the copilot tool).
- `editor === 'toggle'` replaces seven `type === 'boolean'` checks in the grid
and expanded popover, all of which meant the same thing.
- `defaultMetadata` replaces the per-type stamping in `addTableColumn` and
`updateColumnType`.
- `sampleValue` replaces the per-type example values in the LLM prompt
scaffolding.
- `storesOpaqueIds` replaces the select filter in the find-row matcher.
Dead config removed: `getTypeBadgeVariant` had zero callers (already dead on
staging), and it was the only reader of `badgeVariant` — so the field, its
union, and all seven values went with it. `inferFromCsv` was read by nothing
but a comment; CSV inference is an ordered heuristic a boolean cannot express,
so it is gone too and `InferredCsvColumnType` is no longer exported.
Fixes a latent crash found on the way: unique-constraint checking normalized a
cell keyed on its RUNTIME type but reconstructed it keyed on the column's
DECLARED type, so a unique `date` column stored a bare `2024-01-01` and then
threw `SyntaxError` parsing it back. Both directions now go through JSON
unconditionally. Pre-existing, unrelated to currency.
Adds the `/add-column-type` skill and a Tables section in CLAUDE.md/AGENTS.md
pointing at it, so the next type is one file plus two registry entries.
* fix(tables): run the column PATCH guards ahead of the rename, not after it
Greptile was right and my previous reply was wrong. The guards were added in
the right shape but the wrong place — below `renameColumn`, which is the
first write and commits in its own transaction. A PATCH combining a rename
with an invalid currency therefore still committed the rename and then
returned 400, exactly the counterexample reported.
Moved the column lookup and all three pre-flight guards above every write.
This also closes the same latent hole for the pre-existing unique-on-select
guard, which sat in the same position.
Adds route tests that assert `renameColumn` was never called on each
rejection path, and that a valid combined rename + currency change still
targets the new name. Verified to fail against the previous ordering.
* fix(tables): make the retype gate and the write path share one parser
A simplify pass over the registry found two real defects and several places
the abstraction was being worked around.
Silent data loss on conversion. `isCompatibleWith` was hand-written per type
and had already drifted from `coerce`, despite the interface promising they
could not: `boolean` accepted '1'/'0'/0/1 in the gate but only 'true'/'false'
in the write path, so converting a column holding "1" reported zero
incompatible rows and then nulled every one of them. `date` drifted the other
way. `isCompatibleWith` is now optional and defaults to `coerce(...).ok`, so
the two are the same code; only `select` overrides, because its rules are
about the column (cleared-vs-required, cardinality) not the value.
`isColumnType` used `in`, which matches inherited keys — `isColumnType('toString')`
was true and `columnTypeById('toString')` returned `Function.prototype.toString`,
which the validator would then call `.validateDefinition()` on. Now `Object.hasOwn`.
`defaultMetadata` only ran on the currency arm of a retype, so a future type
would get its defaults on create but silently not on conversion. It now runs
for every non-select target, carrying forward only metadata the TARGET type
declares it owns — a currency→text conversion no longer strands a currencyCode.
The index doc claimed the registry is kept out of the `@/lib/table` barrel so
44 server modules don't pull `@sim/emcn/icons`. That was false: `constants.ts`
re-exported `COLUMN_TYPES` from the icon-carrying `registry.ts`, and the barrel
re-exports `constants`. `COLUMN_TYPES` now lives in the icon-free `types.ts`;
verified with an import tracer that both are icon-free again.
Also: 5 no-op `validateDefinition`s and 4 duplicated formatters collapsed into
registry defaults; `CURRENCY_OPTIONS` was an eager module-load IIFE costing
~8ms of ICU work on every table API route for a list only the config sidebar
reads, now built on first call; and the skill's validation grep claimed 'should
return nothing' when it returns 8 legitimate hits — it now explains how to tell
a leak from a genuine special case.
* fix(tables): reject a non-leading sign so dates don't parse as amounts
Found by Cursor Bugbot. `parseCurrencyInput` dropped every `-` as decoration,
so an ISO date's hyphens vanished and its digit groups joined: `2024-01-01`
read as 20240101. With the gate now sharing the write path's parser, a
date → currency conversion reported zero incompatible rows and silently
turned every cell into a huge number.
A sign is only meaningful at the front; an interior one means the string is
not a single amount. Leading signs, accounting parentheses, symbols, ISO
codes, grouping separators, and exponent form all still parse — covered by
the existing cases plus new ones, verified to fail without the fix.
* fix(tables): use getErrorMessage in the columns route test mock
`check:utils` bans the inline `e instanceof Error ? e.message : fallback`
form; the mock for `rootErrorMessage` used it.
* fix(tables): rename the column last so a failed write leaves it untouched
Greptile's remaining concern: the pre-flight guards read a schema snapshot, so
a column-type change landing concurrently can still make a later write fail —
and with the rename running first, that failure returned an error with the
rename already committed.
Guards cannot close that window; each write is its own locked transaction and
only the write itself sees the authoritative state. Ordering can. The rename is
the one write that is purely cosmetic, so it now runs last: a failed typed
write leaves the column entirely untouched, and a failed rename leaves the
typed change applied under the old name — the recoverable half. The typed
writes target the column's current name, since no rename has happened yet.
Tests cover both directions: a typed write rejected mid-flight must not rename,
and a successful one must rename strictly after. Verified to fail under the
previous ordering.
* fix(tables): write back coerced values on every conversion
Round 4 findings, all real.
A conversion is allowed exactly when the target type's `coerce` accepts the
value — and `coerce` frequently TRANSFORMS it. Only `select` and `currency`
wrote the transformed value back, so a conversion to any other transforming
type left the cell holding its old bytes under the new type. Converting a
number column to `date` accepted epoch values, stored them unchanged, and then
`(data->>'col')::timestamptz` failed on EVERY query against that column. I
opened this myself by defaulting `isCompatibleWith` to `coerce(...).ok`.
Fixed at the class rather than the instance: the compatibility scan now records
whatever `coerce` produced whenever it differs from what is stored, and one
generic write-back applies it. That subsumes the currency-specific migration
entirely, so it and its helpers are gone. `select` keeps its own id↔name
migrations, which are not coerce-expressible in the outbound direction. The
post-conversion column definition is built once, before the scan, so the
coercion reads the same metadata the stored value is later validated against.
Exponent parsing was ambiguous when followed by text: `1e5 EUR` read as 15.
An `e` with a digit on both sides is an exponent marker, so if the string is
not a clean numeric literal it is refused rather than guessed — the digit on
both sides is what keeps the `E` inside `12 EUR` parsing normally.
A failed rename could still leave a typed change committed. The one rename
failure a caller can cause — a name already taken — is now rejected up front,
leaving only the concurrent-collision race, which no pre-flight check can close
without spanning all writes in one transaction.
* fix(tables): stop a blank cell blocking an optional type conversion
Found by Cursor Bugbot. `''` is incompatible with every numeric type, and the
compatibility scan counted it as a hard blocker regardless of whether the
target was optional — so a text column with a single empty cell could not be
converted to a number at all, and the error said 'to a required ...' either way.
An unreadable-but-empty cell is not a conversion failure. The write path
already turns an unreadable value into null on an optional column, so the
conversion now does the same and records null for it. A required target still
reports it, which the existing guard above already does with the message that
actually fits.
Also pins the two intentional divergences from the pre-registry behavior. A
differential run of the registry against the pre-refactor implementations (55
values x 7 column shapes) found ZERO coercion differences and exactly two
compatibility differences, both deliberate: boolean now rejects the '1'/'0'
conversions the old gate accepted and then nulled, and date now accepts the
epoch numbers its write path always accepted. Tests pin both so neither can be
silently reverted or widened.
* fix(tables): refuse conversions that would invent or destroy values
Final adversarial scan found two data-corrupting conversions, both opened by
defaulting the retype gate to the write path's parser.
number → date destroyed every value. `date.coerce` reads a number as epoch
milliseconds, which is right for one deliberate write and catastrophic applied
to a whole column: 1, 5, 42 became three timestamps in January 1970, and a
Unix-seconds column landed in 1970 rather than the year it meant. Irreversible.
`date` now overrides the gate to reject numbers, restoring the pre-refactor
behavior, and the contract states the rule the override obeys: a gate may be
STRICTER than `coerce`, never looser. Stricter refuses a bulk conversion while
single writes still work; looser is the direction that corrupts.
string → currency invented values. The parser stripped every non-digit and
joined what was left, so `01/02/2024` read as 1022024, `Room 101` as 101, and
`0.1.2` as 12 — a column of SKUs or phone numbers converted with zero reported
incompatibilities. What remains after removing symbols, spacing and an ISO code
must now be only digits and separators, and grouping must be well-formed (a
first group of 1-3 digits, the rest exactly 3). Every legitimate form still
parses, including all the locale variants.
Also generifies the last three metadata leaks: `buildConvertedColumn` strips
and carries back by iterating the key list rather than naming keys (naming them
meant a future type's metadata rode onto a target that rejects it, failing that
column's validation on every later write), `normalizeColumn` forwards metadata
through a shared `typeMetadataOf`, and `filterOperatorsFor` moved onto the
definition — it was a per-type branch inside the registry's own accessor, the
one thing the registry exists to forbid.
Skill corrected: it claimed COLUMN_TYPES derives from the registry (backwards),
promised exactly two compile errors (four once a type owns metadata), used a
grep that missed half the real branches, and never mentioned `import.ts`'s
second coercion path, whose silent default arm is the costliest miss available.
Differential re-run vs the pre-refactor implementations: 0 coercion
differences, 1 intentional compatibility difference (boolean no longer accepts
the 0/1 conversions the old gate accepted and then nulled).
* fix(tables): let the row modal accept formatted amounts again
Found by Cursor Bugbot. I unified the row modal's input type with the grid's
`inputMode` last round, but in the wrong direction: mapping `inputMode:
'decimal'` to `<input type="number">` made the modal reject $1,234.56,
1.234,56 and (12.00) — the exact formats `parseCurrencyInput` exists to accept,
and which the grid's inline editor takes fine.
A native number input and a numeric keypad are different things. Types whose
parser accepts formatted text now say so, and get a text field with
`inputMode='decimal'` — the shape the grid already uses. A plain number keeps
the native input, its spinner, and its validation.
* fix(tables): fold a rename into the write it accompanies
Closes the last partial-update window, properly rather than by pre-checking
around it.
A rename is metadata-only — `renameColumn`'s own comment says so: rows,
metadata, and workflow-group refs all key on the stable column id, so it is a
pure schema write. Nothing forced it to be its own transaction. Running it
separately is what created the window: whichever half committed first survived
a failure in the other, and no pre-flight guard can close a concurrent
collision because only the write itself sees authoritative state.
The four column writes now accept an optional `newName` and apply it through
one shared `applyPendingRename`, which validates the name shape and checks the
collision against the very schema snapshot that write is landing in. A combined
request rides the rename on whichever write runs last, so both halves commit
together or neither does — a concurrent claim on the name now aborts the whole
transaction instead of leaving the other change applied.
The routes also address every write by the column's stable id rather than its
name, so folding a rename into one write cannot break the next one's lookup.
A rename with nothing to ride on still runs standalone.
What remains partial is a type write followed by a failing constraints write —
two independently locked transactions, pre-existing, and untouched by this PR.
* fix(tables): migrate scalar cells when converting a column to select
Found by Cursor Bugbot. `resolveSelectOptionId` stringifies a number or
boolean before matching, so a `number` column whose values equal option NAMES
passes the compatibility gate — but `migrateCellsToSelectIds` only rewrote
JSONB `string` and `array` cells. Those cells stayed raw numbers inside a
select column, where they render as nothing and fail option membership on the
next write.
`data->>key` yields the text form for every scalar, so the existing lookup
already worked; the predicate was simply too narrow. Widened to cover
`number` and `boolean`. The outbound migration is unchanged — cells leaving a
select column are option ids, always strings or arrays.
Pre-existing on staging (both the resolver's scalar handling and the migration
SQL predate this branch), but it lives in a file this PR creates.
Tests pin the resolver behavior the predicate depends on, so narrowing either
one without the other now fails.
* fix(tables): validate a retype's unique against the values it writes
Validated the last partial-update seam with a focused investigation rather
than assuming. The answer was split.
`required` is already safe: `updateColumnType` runs the same `countEmptyCells`
against the constraint the request is about to set, which is why that check
exists.
`unique` was not, and the reachable case commits the unrecoverable half. A
text column holding "5" and "5.0", PATCHed with {type: number, unique: true}:
the conversion succeeds and coerces both to 5, then the separate constraint
write finds duplicates and 400s — with the column already numeric and "5.0"
irreversibly rewritten. A pre-scan of the raw text finds nothing; the
conversion is what manufactures the duplicate. The retype now carries `unique`
and checks it after the write-back, against the values it just wrote.
Constraint changes on a workflow-output column were the same shape — rejected
by the constraint write, after a type change had committed. Now rejected in the
route's pre-flight block, before any write.
The duplicate scan is extracted and shared between both paths for the same
reason `countEmptyCells` is: two copies of one rule is the drift that produced
the original required-check bug.
Deliberately NOT merging `updateColumnType` and `updateColumnConstraints`. They
assert different lock levels (destructive vs schema-only) and only the retype
needs the full row scan, so merging would either force a constraints-only
toggle to materialize every row or reintroduce the branching it was meant to
remove. With both reachable failures pre-validated, what remains at the seam is
concurrent races no in-process check can close.
* fix(tables): don't drop a rename when the write it rides on no-ops
Found by Cursor Bugbot — a bug I introduced folding the rename in.
`updateColumnCurrency` returns early when the code is unchanged, and that
return sat ahead of the rename, so PATCH {name, currencyCode} with the column's
current code answered 200 with the rename silently discarded.
Both early returns now treat a pending rename as work: the currency path only
no-ops when the code is unchanged AND no rename is riding along, and the retype
path applies a rename-only write when the type is unchanged. `applyPendingRename`
signals "nothing to do" by returning the same reference, which is what lets
both detect it cleanly.
Also extracts `persistColumns` — five sites were repeating the same
schema-write-and-return.
* fix(tables): make a combined column PATCH a single transaction
Finishes the fold-in rather than pre-validating around the seam. A retype now
APPLIES the constraints it already validates against — it checks empty cells for
`required` and post-conversion duplicates for `unique`, so it was doing the work
without persisting the result — and the route skips the separate constraint
write when the type changed.
A request combining a rename, a retype and constraint changes is now one
locked transaction: no half of it can commit while another fails. The separate
constraint write remains for requests that do not change type, which is the
only case that still needs it.
Deliberately still NOT merging the two service functions. They assert different
lock levels (destructive vs schema-only) and only the retype needs the full row
scan into memory, so a merged function would force a constraints-only toggle to
materialize every row or reintroduce the branching it was meant to remove.
Folding the payload in gets atomicity without either cost.
* fix(tables): reject a flattened list as an amount; fold constraints into every typed write
Two findings from round 11.
Multi-select converted to nonsense amounts. `selectValueForConversion`
flattens a multi cell to its comma-joined option names, and the parser read
that as a formatted number: options 12 and 34 became 12.34, and 100 and 200
became 100200. No real amount puts whitespace after a separator, but a
delimited list does — so a separator followed by whitespace is now refused.
Every legitimate form still parses, including space-grouped locales.
Combined options-or-currency + constraints could still commit partially. Those
two writes now carry constraints the same way the retype does, through one
shared `applyConstraints` that validates (workflow-output, empty cells for
required, supportsUnique and duplicates for unique) and applies them. The
separate constraint write now runs only when no typed write does. Three copies
of those rules is the drift that produced the original required-check bug, so
they live in one place.
* fix(tables): validate constraints after the migrations that rewrite cells
Self-caught while reviewing my own previous commit, which introduced both.
`updateColumnOptions` ran the shared `applyConstraints` BEFORE its cell
migrations. Those migrations rewrite stored values — a single<->multi toggle
changes the shape, removing an option clears cells — so a `unique` scan read
the pre-migration values, passed, and the rewrite could then produce the
duplicates the scan was meant to prevent. Moved to after the migrations, which
is where `updateColumnType` already had it.
The same commit also left the options path running `required`'s empty-cell
check twice: once in the shared helper and once in its original inline block,
whose comment still described a separate constraint write that no longer runs.
Removed the duplicate — one query, one rule, which is the whole point of the
shared helper.
Also routes the options path through `persistColumns` like the others.
* fix(tables): stop inventing amounts from identifiers; fix the copilot retype
Adversarial pass over the final state, seven real findings.
Two destroyed data. The copilot `update_column` still used the two-transaction
pattern the HTTP routes were fixed for: `unique` was never forwarded to the
typed write, so a retype+unique committed the conversion and then failed the
constraint — the same irrecoverable half. It now rides the typed write, and the
separate constraint write only runs when no typed write did.
And the parser's three-letter strip removed ANY three letters, not an ISO code:
`SKU400` parsed as 400, `ABC1234` as 1234. Converting a column of part numbers
to currency rewrote every cell with an invented value — while the comment two
lines above claimed a SKU was exactly what it prevented. The rule is now that a
letter touching a digit means identifier, not amount; a currency marker is
always separated by a space or a symbol.
That same change fixed a class the review surfaced: the pinned currencies could
not parse their own conventional notation. `R$ 1.234,56`, `1 234,56 kr`,
`1234,56 zł`, `CHF 1’234.56` and Indian lakh grouping (`₹12,34,567.89`) all
work now — these are what Intl emits, so a paste from a spreadsheet was being
rejected.
`updateColumnConstraints` was a fourth copy of the constraint rules the shared
helper exists to unify, and had already drifted: it hardcoded `type ===
'select'` where the helper asks the registry, so a future type declaring
`supportsUnique: false` would have been ignored on that path. It now uses the
helper.
`updateColumnType`'s unchanged-type early return silently discarded every
field except the rename. Callers gate on the type changing, but from a read
taken before the lock — so a concurrent change could land there with real work
pending and answer success. It now throws.
Also: `UpdateColumnCurrencyData` was missing `required`, which only compiled
because the routes pass it through a spread; a missing column returns 404
instead of a 400 reading "of type undefined"; and the comments describing the
old two-transaction architecture are gone.
Verified NOT a bug: CSV export of a currency column writes the raw number, so
export/import round-trips losslessly.
* fix(tables): read the negative and RTL forms Intl actually emits
An Intl sweep across 24 locales found two forms the parser rejected, both from
an ordinary spreadsheet paste.
`Intl` emits U+2212 MINUS SIGN rather than the ASCII hyphen for negatives in
several locales, so `−12,50 kr` read as null instead of -12.5. And it wraps
RTL-locale output in invisible bidi control marks, so `1,234.56 ₪` carried
characters that are not part of the amount. Both are now normalized away.
24 locales x 6 amounts now round-trip, up from 99/100 when the sweep started —
and the test generates them from `Intl` rather than listing them by hand, so a
parser change cannot quietly regress a locale nobody remembered to write down.
Locales that format with their own numeral systems (Arabic-Indic) are still
rejected, and now say so in the docstring. That is a safe failure — null rather
than a wrong value — and supporting them is a wider decision than this type,
since it would also touch `number`, display, and sorting.
|
||
|
Vikhyath Mondreti
|
91f9dfdaec |
improvement(governance): derived access (#5134)
* improvement(governance): org-ws-credential roles clarity * revert isHosted * improvement(credentials): code cleanup * address comments * make kb cascade delete on user hard delete * revert env flags * chore(db): drop local 0242 migration to regenerate after merging staging Our 0242 collides with staging's 0242. Remove it (and its snapshot + journal entry) so the KB-cascade migration can be regenerated with the correct number on top of the merged staging migrations. * chore(db): regenerate kb→workspace cascade migration as 0243 Regenerated via drizzle-kit generate on top of the merged staging migrations (staging took 0242). Re-applied the safety edits: NOT VALID + separate VALIDATE on the FK re-add, and the -- migration-safe note on the DROP. check:migrations passes. * improve copy * update docs |
||
|
Emir Karabeg
|
192f77ba02 |
improvement(emcn): consolidate chip chrome, enforce ChipModalField, paint real chrome in loading fallbacks (#4935)
* improvement(emcn): consolidate chip chrome, enforce ChipModalField, paint real chrome in loading fallbacks - Move chip chrome single-source to chip/chip-chrome.ts (surface, typography, and new content tokens); delete chip-input/chip-field-chrome.ts - Rework Chip variants: implicit default replaces ghost, filled reserved for chip fields/triggers and removed from Chip's public API; add ChipChevron - Migrate every labeled modal body field to ChipModalField across knowledge, settings, tables, files, deploy, sidebar, and ee modals - Replace skeleton loading.tsx files with ResourceChromeFallback that paints the page's real header, actions, search/filter/sort chips, and column headers - Rename resource-options-bar to resource-options; simplify resource.tsx and resource-header - Delete legacy Breadcrumb, Callout, and FormField components - Add shared connector-config-fields (knowledge) and ee InfoNote; add useTagUsageQuery; make useInlineRename save async with isSaving - Update AGENTS.md/CLAUDE.md and emcn/styling rules (with .cursor/.agents mirrors) * fix(rename): make inline rename await mutateAsync so isSaving disables the field; recover edits on failure - useInlineRename: catch onSave rejection — log, restore the original name, keep the edit session open, and re-arm doneRef (mirrors useItemRename) - switch rename call sites (files, tables, table header, knowledge base, document) from mutate() to mutateAsync() so isSaving spans the request - table-grid column rename stays fire-and-forget by design (optimistic local update + undo entry) * fix(rename): type onSave as void | Promise<unknown> so fire-and-forget call sites pass the build table-grid's optimistic column rename is a block-bodied callback returning void, which is only assignable when the declared return type is literally void — 'undefined | Promise<unknown>' rejected it and failed the Next.js type check in CI. |
||
|
|
0075ab9cf6 |
improvement(platform): remove tour, simplify sidebar/header, drop loading skeletons (#4354)
* improvement(platform): workspace UI/UX overhaul + integrations catalog Rework the workspace around the AI-workspace model: a Mothership home, a top-level Skills route, connected-credential and integration-detail pages, and a polished sidebar/settings surface. Replace the notifications store with a unified toast system (provider-level dismiss/pause, countdown ring). Integrations & catalog: - Add a BlockMeta layer (tags + catalog templates) scoped to catalog-visible integrations; every catalog integration carries >=7 grounded templates. - Rework the taxonomy: each block declares category tools|blocks|triggers. 3rd-party services are 'tools'; first-party primitives (postgres, mysql, knowledge, file, search, stt/tts, image/video generators, thinking, etc.) are 'blocks'. Versioned blocks follow the upgrade paradigm (old hidden, latest in toolbar/docs). - Generate integrations.json + tool docs canonically from block configs. Architecture & cleanup: - Consolidate block data extraction behind a single latest-version strategy (getCanonicalBlocksByCategory; version-consistent getBlockMeta). - Unify version-suffix handling in @sim/utils/string (stripVersionSuffix / isVersionedType, with tests); registry, generate-docs, tools/utils, and integrations all route through it. - Repair latent broken barrels, remove dead code, fix BlockMeta-related type errors and 5 broken docs links. Behavior-preserving for block execution and the toolbar's tool/block listing. * refactor(platform): remove forms, templates, and creators features Remove three standalone features and their supporting code: - Forms: form-deployment pages, API routes, execution path, and docs. - Templates: the template gallery (landing + workspace) and template APIs. - Creators: creator-profile routes and contracts. Add a super-user permissions module (lib/permissions/super-user) and an organizations API contract; update the audit/db/testing packages, billing, and the session/theme providers accordingly. * test(workflows): update archiveWorkflow update count after forms removal The forms feature was removed, dropping the form-table update from archiveWorkflow. Update the stale assertion from 8 to 7 tx.update calls. * upgrade * improvement(knowledge): polish tag filter dropdowns (#4816) * improvement(logs): object storage backed tracespans (#4787) * improvement(logs): obj storage backed tracespans * fix storage write context * fix tests * address comments * address comments * chore(db): remove migration 0219 to regenerate after staging merge Drops the 0219_robust_shard SQL, its snapshot, and the journal entry so the trace-spans/cost schema migration can be regenerated on top of the latest staging migration chain (avoids a number collision with staging's migrations). Co-authored-by: Cursor <cursoragent@cursor.com> * improvement(billing): accurate per-member usage via shared ledger helper Per-member/per-user usage in the org-member routes now adds the usage_log ledger to the currentPeriodCost baseline (which is no longer incremented), via a shared getOrgMemberLedgerByUser helper to avoid repeating the subscription→period→ledger lookup across the admin and member-facing routes. Co-authored-by: Cursor <cursoragent@cursor.com> * regen migrations * update migration * address comments * more code cleanup * incorrect type cast --------- Co-authored-by: Cursor <cursoragent@cursor.com> * improvement(providers): harden OpenAI-compatible providers + add tests (#4796) * improvement(providers): harden OpenAI-compatible providers + add tests * fix(vllm): let tool-loop errors propagate instead of returning silent partial success * fix(litellm): force tool_choice 'none' on final structured-output call The deferred final call used tool_choice 'auto', so the model could emit another tool_calls round instead of the structured answer, leaving content stale. Use 'none' (matching vLLM/Fireworks) on both the streaming and non-streaming final calls so the model must return the structured response. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(providers/ollama): drop tools from post-tool streaming call Ollama ignores tool_choice (not in its supported fields), so vLLM/Fireworks' tool_choice:'none' guard is a no-op here. Omit tools from the final streaming payload instead so the summarization turn can't emit dropped tool calls. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(litellm): spread payload into deferred final call so reasoning_effort carries over The non-streaming deferred finalPayload hand-picked fields and dropped reasoning_effort (and any future payload field), diverging from the streaming path which spreads ...payload. Spread payload here too for consistency. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(providers/ollama): restore enrichment TSDoc block Keeps parity with sibling Chat Completions providers (cerebras/mistral/xai). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(fireworks): restore TSDoc on utils helpers Restore the TSDoc blocks on supportsNativeStructuredOutputs, createReadableStreamFromOpenAIStream, and checkForForcedToolUsage — TSDoc is the codebase documentation standard and should not have been stripped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(litellm): remove inline rationale comments (codebase uses TSDoc) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(providers/ollama): drop orphaned enrichment TSDoc The block documented a function that now lives in trace-enrichment.ts, so it documents nothing in this file. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * chore(copilot): deprecate mcp server (#4797) * chore(copilot): deprecate mcp * update error codes * deprecate copilot api v1 route * feat(integrations): hosted API keys for Findymail, Prospeo, and Wiza (#4777) * feat(integrations): hosted API keys for Findymail, Prospeo, and Wiza Add hosted-key support across all credit-consuming Findymail, Prospeo, and Wiza operations so Sim provides the key when a workspace has not brought its own. Register the three BYOK providers, consolidate Wiza's two-step reveal into a single polling wiza_individual_reveal op, and hide the API key field on hosted Sim for hosted operations. * fix(integrations): harden Wiza reveal polling, soften enrichment getCost guards Address Greptile + Cursor Bugbot review on #4777: return explicit failures from the Wiza individual_reveal poller instead of throwing (thrown errors were swallowed into a false queued success), short-circuit when the initial reveal is already terminal, tolerate transient 5xx/429 during polling, and return 0 (not throw) from Findymail getCost when the contacts/employees array is absent. * chore(integrations): biome formatting after wiza merge resolution * fix(wiza): type isTerminalReveal param structurally for next build typecheck * feat(enrichments): add Findymail, Prospeo, Wiza to work-email waterfall * feat(enrichments): add Wiza + Prospeo phone reveal to phone-number waterfall * feat(enrichments): opportunistic identifiers + LinkedIn URL input across work-email & phone cascades * fix(tables): reduce column header chevron size and fix sidebar shadow bleed (#4800) * feat(slack): add install + privacy section to integration landing page (#4799) * feat(slack): add install + privacy section to integration landing page Adds a hand-authored, slug-keyed landing-content module (separate from the generated integrations.json so it survives regeneration) and renders an install walkthrough + privacy-policy link on integration pages when present. Also refreshes generated docs (data-enrichment entry, icon mappings, tool mdx). * fix(landing): render privacy section independently, align CTA analytics label * docs(landing): clarify the Slack install button is behind sign-in * refactor(landing): bake integration landing content into generated json via docs-gen Moves landing content (install walkthrough + privacy) out of a render-time augment and into the generation pipeline: generate-docs reads the pure-data content map and writes landingContent into integrations.json, so the page reads a single source (integration.landingContent). Canonical types live in integrations/data/types.ts. * improvement(enrichments): align enrichments sidebar with design system (#4801) * improvement(enrichments): align enrichments sidebar with design system * fix(enrichments): consistent close button pattern and fix url link hover * fix(misc): upgrade path change for new better-auth version, billing issue for workflow block agent usage (#4803) * fix(misc): upgrade path change for new better-auth version, double-billing for workflow block agent usage * fail loudly if stripe sub id missing * fix(copilot): seq migration (#4804) * chore(db): drop redundant idx_webhook_on_workflow_id_block_id index (#4809) Removed because (workflow_id, block_id) is a left-prefix of idx_webhook_on_workflow_id_block_id_updated_at_desc, which fully covers it. The dropped index was non-unique and enforced no constraint. * perf(copilot): read chat transcripts from copilot_messages (R+1 cutover) (#4808) * perf(copilot): read chat transcripts from copilot_messages, not JSONB Flip user-facing chat reads from the legacy copilot_chats.messages JSONB array (5.7GB, 99% TOAST) to the normalized copilot_messages table via a new loadCopilotChatMessages helper ordered by seq NULLS LAST, created_at, id — the verified canonical order. Both chat-detail getters (getAccessibleCopilotChat, getAccessibleCopilotChatWithMessages) now drop the messages column from their metadata select (no more whole-array detoast on every load) and assemble the transcript from the table after authorization. This cascades to the copilot + mothership GET endpoints and to resolveOrCreateChat's conversationHistory (the LLM payload). The normalize/effective-transcript pipeline is source-agnostic (copilot_messages.content == a JSONB array element), so transcripts are byte-identical. Dual-write and the JSONB column stay in place as the internal-logic source and fallback; removing JSONB writes is a later step. Prod integrity verified before cutover: 0 messages missing, 0 NULL-seq, 0 dup keys/seq, 0 orphans, order-parity vs JSONB = 0 mismatches. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(copilot): cover auth-deny on a found row skips the messages query Address PR review: exercise the `if (!authorized) return null` contract — when the chat row exists but authorization fails, the getter returns null and never issues the copilot_messages read. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(tables): right-align run/stop in embedded toolbar; workflow cells format like normal cells (#4806) * fix(tables): right-align run/stop in the embedded table toolbar Add a right-aligned `trailing` slot to ResourceOptionsBar and move the embedded mothership table's run/stop control into it, so Filter + Sort stay left-aligned and run/stop sits opposite on the right. No-op for the search-bearing consumers (logs, resource list), which don't pass `trailing`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(tables): workflow-output cells format values like normal cells Workflow-output columns short-circuited in resolveCellRender and rendered their value as plain text, so a sim-resource URL / external URL / JSON / date produced by a workflow never got the chip, favicon link, or typed formatting a normal cell gets. Factor value formatting into a shared `resolveValueKind` helper used by both the workflow-value branch and the plain-cell branch; the workflow branch keeps the typewriter reveal for plain streaming text via a `typewriter` flag. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(tables): detect resource/URL links on workflow output regardless of column type Workflow output columns default to `json` (columnTypeForLeaf), so routing their values through the type-based formatter (a) gated chip/URL promotion behind `column.type === 'string'` — a URL produced by a json-typed output never became a chip — and (b) JSON.stringify'd plain string values, adding quotes and losing the typewriter reveal. Detect links (sim-resource chip / favicon URL) on the value string directly for workflow outputs, falling back to the plain `value` kind; plain cells keep the type-based formatting. Addresses Greptile P2 on #4806. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(icons): repair broken integration icon rendering (#4810) * fix(icons): repair broken integration icon rendering Two distinct bugs left integration icons broken on the /integrations page (visible at 32-40px, hidden at the toolbar's 16px): 1. Corrupted SVG paths (Notion, Greptile, Granola, Calendly, Grafana, Bedrock): over-minified data dropped elliptical-arc flag digits (e.g. `A1 1 0 5.9 7` instead of `A1 1 0 0 0 5.9 7`); Granola's cubic stream was truncated. Browsers abort path parsing at the first invalid arc flag, so each rendered as a fragment or blank. Replaced with correct path data from canonical sources, preserving each icon's existing fill/gradient and bgColor. 2. Invisible glyph (Bright Data): its icon uses fill='currentColor' but bgColor was '#FFFFFF', and every surface forces text-white on the glyph - white-on-white. Changed bgColor to Bright Data's brand blue (#3d7ffc) so the white glyph reads, matching the white-glyph-on-brand-chip convention. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(icons): restore Calendly dual-tone brand colors Addresses review feedback: the previous fix replaced the broken Calendly icon with a monochrome #006BFF path, dropping the cyan #0ae8f0 accent from the original dual-tone mark. Restored the two-tone logo (blue + cyan) using clean, valid path data, cropped to a tight square viewBox so it fills the chip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * improvement(icons): enlarge icons, fix Zoom contrast and Quiver chip - Zoom: glyph was blue-on-blue (#0B5CFF on #2D8CFF chip); switched to currentColor so it renders as a white glyph on the blue chip. - Quiver: chip bgColor #000000 -> #FFFFFF to match the icon's near-white box, and enlarged the mark slightly (viewBox crop). - Enlarged (tightened viewBox, verified no clipping): RevenueCat, Prospeo, Granola, Firecrawl, Enrich.so, and the AWS icons (RDS, DynamoDB, SQS, CloudFormation, Athena, CloudWatch, SES, Bedrock, S3). - ZoomInfo left unchanged: it is a full red rounded-square logo that already fills its frame, so a crop would clip it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(icons): use Bright Data wordmark on white chip; repair Circleback - Bright Data: replaced the flame glyph with the official two-tone 'bright data' wordmark (provided asset), centered in a symmetric viewBox. Reverted the chip bgColor from #3d7ffc to #FFFFFF since the blue wordmark is invisible on a blue chip (the wordmark is designed for a light background). - Circleback: a minifier had rounded the pattern's image scale to scale(0), collapsing the embedded logo to zero size (invisible). Restored the correct scale (1/280 = 0.00357142857) so the C. mark renders. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(docs): sync Quiver block color card to white chip Reflects the Quiver bgColor change (#000000 -> #FFFFFF) in the docs block info card. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * improvement(icons): enlarge AWS/Cloudflare/Dagster icons, fully white Zoom - Enlarged (tighter viewBox, render-verified, no clipping): Cloudflare, Dagster, and the red AWS icons AWS IAM, Identity Center, Secrets Manager, SES, STS. Identity Center was anomalously small (filled ~32% of its frame); the group is now sized consistently (~80% fill). - Zoom: the camera lens triangle was still #0B5CFF (blue-on-blue); switched it to currentColor so the whole camera renders white on the blue chip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(wiza): consolidate individual reveal into a single operation Merges the separate Start/Get Individual Reveal operations into one Individual Reveal operation in the Wiza docs and integrations data (operationCount 5 -> 4). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * improvement(icons): size remaining AWS icons to match the set (~80% fill) Bring RDS, DynamoDB, SQS, CloudFormation, Athena, CloudWatch and S3 up to the same ~80% fill as the AWS IAM/Identity Center/Secrets Manager/SES/STS group, so all AWS icons are visually consistent. Bedrock left as-is (already ~92% fill). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(icons): use Bright Data flame mark, enlarge ZoomInfo - Bright Data: the full 'bright data' wordmark was illegible at chip size. Replaced with just the flame-'i' brand mark (blue #4280f6 on the white chip), centered. - ZoomInfo: cropped the viewBox toward the white 'Zi' so it's larger; the red rounded-square background still fills the chip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * improvement(icons): enlarge CrowdStrike icon The falcon mark sat small in its chip because the icon used a wide 768x500 viewBox (letterboxed in the square chip). Switched to a square viewBox centered on the mark so it fills ~80%, consistent with the other icons. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(tables): serialize schema mutations to prevent parallel column clobber (#4812) * Make workflow description nullable * fix(tables): serialize schema mutations to prevent parallel column clobber * fix(tables): load workflow outside schema lock; use DbOrTx for getTableById * fix(tables): scale idle timeout in updateColumnType to avoid aborting large type changes * fix(tables): skip stale remap types when workflowId changes concurrently * fix(tables): scale idle timeout in updateColumnConstraints for large tables * fix(wait): resume live/draft async waits and preserve cell context on chained waits (#4814) * Make workflow description nullable * fix(wait): resume live/draft async waits and preserve cell context on chained waits * improvement(knowledge): polish tag filter dropdowns * improvement(knowledge): soften filter section labels * improvement(knowledge): soften list filter labels * fix(security): harden SSO domain registration, webhook path isolation, and CSV export (#4813) * fix(security): harden KB file access, SSO domain registration, webhook path isolation, env secrets, and CSV export * fix(sso): scope domain conflict query with indexed lower(domain) filter Address PR review: avoid a full-table scan on every SSO provider registration by filtering candidate rows in SQL with lower(domain) = <normalized>, keeping the in-memory ownership check. Also tighten the normalizeSSODomain TSDoc. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: condense env route security comments Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * icons update * chore(security): tighten inline comments in CSV export and KB file authorization Condense verbose comment blocks to concise TSDoc/single-line form; no behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): validate internal serve origin in KB file authorization Replace the bypassable isInternalFileUrl substring check in resolveInternalKbKey with an origin allow-list (base URL, internal API base URL, TRUSTED_ORIGINS). A crafted external host whose path is /api/files/serve/<victim-key> no longer resolves to the victim key. Relative same-origin URLs are unaffected. * style(sso): use idiomatic sql lower() comparison for domain conflict query Match the repo's prevailing `sql`lower(col) = value`` idiom for the case-insensitive SSO domain conflict lookup. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): align workspace env admin gate with hasWorkspaceAdminAccess Use the same admin check the secrets UI uses (owner, admin permission, or org-admin) so owners and org-admins are not wrongly denied their own decrypted workspace secrets, while read-only members remain restricted to names only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sso): rely on lower(domain) match for conflict detection, drop dead in-memory recheck Address PR review: the SQL `lower(domain) = <normalized>` predicate already excludes rows that the in-memory `normalizeSSODomain(...) === domain` recheck claimed to catch, making that recheck dead/misleading code. Match on the canonical lower-cased domain and filter purely by ownership. Malformed legacy values (wildcards, schemes, ports) never match an email domain at sign-in, so excluding them is not a gap. Test DB mock now applies the lower() predicate so the casing-variant case is genuinely exercised. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): scope webhook deploy path conflict to active webhooks findConflictingWebhookPathOwner omitted the isActive filter that the runtime dispatcher (findAllWebhooksForPath) applies, so an inactive but non-archived webhook from another workflow (e.g. after undeploy or failure auto-disable) would permanently block any new deployment on that path even though it never receives deliveries. Align the guard with the runtime isActive + archivedAt filter; the earliest-owner runtime check remains the authoritative cross-tenant protection. Also trims verbose TSDoc on the webhook path-isolation helpers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): exclude archived workflows from webhook deploy path conflict findConflictingWebhookPathOwner now joins workflow and filters isNull(workflow.archivedAt), matching the runtime dispatcher (findAllWebhooksForPath). A webhook on an archived workflow can never receive deliveries at runtime, so it must not block legitimate path reuse with a 409. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): anchor KB file ownership to earliest document in any state A KB file's owner is now the earliest document referencing its key regardless of state (active/archived/deleted/excluded); access is granted only when that owning document is still active. Closes the residual where an attacker could plant an active document to claim a file whose original document was archived or deleted. * updated greptile icon * revert(security): drop KB file authorization changes Reverts the knowledge-base file-access work (origin-pinning / owner-pinning / origin allow-list in verifyKBFileAccess) and its test. The other hardening fixes (SSO domain registration, webhook path isolation, workspace env secrets, CSV export) are unchanged. apps/sim/app/api/files/authorization.ts is restored to its origin/staging baseline. * fix(sso): treat caller's own user-scoped provider as owned during conflict check Self-hosters often register SSO user-scoped via the CLI script (no SSO_ORGANIZATION_ID). If they later enable organizations and reconfigure the same domain org-scoped through the UI, the conflict check previously treated their own user-scoped row as another tenant's and returned a misleading 409. Recognize the caller's own user-scoped provider as owned so that migration is allowed, while still blocking another user's or another org's domain. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * revert(security): remove workspace-env admin gate Defer to a credential-based access model (separate change). Restores GET /api/workspaces/[id]/environment to main behavior and removes the test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(security): consolidate webhook path-collision check into one helper Extract findConflictingWebhookPathOwner to lib/webhooks/utils.server.ts as the single source of truth for cross-tenant path-collision detection, used by both webhook creation paths (deploy sync and the manual /api/webhooks route). This also repairs two latent issues in the manual route's previous inline check, which queried with limit(1) and only webhook.archivedAt: - limit(1) inspected one arbitrary row, so a same-workflow row could mask a foreign collision (false negative). The shared helper scans all matching rows. - It omitted isActive/workflow.archivedAt, so inactive or archived-workflow webhooks (which never receive deliveries) permanently blocked path reuse. The helper mirrors the runtime dispatcher's filter. Same-workflow webhook reuse for upsert is now a separate, explicit lookup. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): block private/reserved IPs for hosted 1Password Connect SSRF (#4818) * fix(security): block private/reserved IPs for hosted 1Password Connect SSRF * test(security): use real isPrivateOrReservedIP and cover IPv6 edge cases * improvement(integrations): validate and expand devin, cursor, and greptile (#4820) * improvement(integrations): validate and expand devin, cursor, and greptile - devin: fix missing org_id path segment on all session endpoints, add 7 session sub-resource tools (list messages/attachments, get/append/replace tags, archive, terminate), pagination, and is_archived output - cursor: add get_api_key_info, list_models, list_repositories tools - greptile: align block and docs - normalize array outputs to default [] and tighten types * refactor(cursor): simplify list_repositories v2 array normalization Collapse the redundant `?? []` + `Array.isArray` double-guard into a single Array.isArray check, per PR review feedback. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(devin): scope session-tag mapping to tag ops and normalize array tag inputs - Only map sessionTags into the tools tags param for append/replace operations, preventing stale sessionTags state from clobbering create_session tags - Fall back to a wired tags value when sessionTags is empty for tag operations - Normalize tag inputs (string or wired string[]) via normalizeTags so array values from other blocks no longer throw on .split Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(cursor): restore base64 file data in legacy download_artifact metadata The legacy CursorBlock exposes only content + metadata (no v2 file output), so metadata.data was the only way legacy-block workflows could access downloaded artifact bytes. Restore the base64 data field and document it in the outputs/type instead of dropping it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(devin): coerce terminateArchive to archive flag for boolean-wired input * docs(integrations): regenerate tool docs for new devin and cursor operations --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(search-replace): don't auto-navigate when content edits invalidate the active match (#4819) * fix(search-replace): don't auto-navigate when content edits invalidate the active match * fix(search-replace): clear afterReplaceIndexRef on apply failure and zero matches * fix(search-replace): remove duplicate setActiveSearchTarget(null) on close * fix(search-replace): move afterReplaceIndexRef write inside handleApply past the guard * fix(search-replace): auto-navigate when hydration resolves with no prior active match * chore(search-replace): remove inline comments * fix(search-replace): revert !activeMatchId guard that caused immediate re-navigation after deselect * improvement(enrichments): limit company-info to fields both providers return (#4817) Hunter's company dataset returns null industry/foundedYear for many large companies (verified against the live API for Microsoft, Amazon, Google), so under the first-non-empty-wins cascade those columns appeared inconsistently across rows. Limit company-info outputs to employee count and description — the fields Hunter and PDL both reliably return — so every row is consistent. employeeCount is a string so Hunter's range bucket and PDL's exact count share the column. * fix(files): don't reject external URLs containing '..' in file parse validation (#4821) * fix(files): don't reject external URLs containing '..' in file parse validation The file block's file_fetch operation rejected any external URL whose path contained '..' (e.g. Slack files-pri slugs with a literal '...') with 'Access denied: path traversal detected'. Traversal checks only apply to local paths — external http(s) URLs are fetched with SSRF protection downstream and are never resolved against the filesystem, so they now short-circuit as valid. Internal /api/files/serve/ URLs keep full traversal protection. * test(files): fix external-URL assertion to handle undefined error * test(files): assert success explicitly in external-URL traversal test * fix(files): keep traversal protection for https URLs matching internal serve paths * feat(google-sheets): add row filtering to read with numeric operators (#4822) * feat(google-sheets): add row filtering to read with numeric operators Adds client-side row filtering to the Google Sheets read (v2) operation. Filter the returned rows by a header column using text operators (contains, not_contains, exact, not_equals, starts_with, ends_with) and numeric/ordering operators (gt, gte, lt, lte). Filtering lives in a pure, unit-tested helper (filterSheetRows) and runs over the fetched read range; an optional `filter` output reports whether the column was found and how many rows matched. Also hardens the surrounding tools: - trim spreadsheetId in write/update/append URL builders (matches read) - URL-encode the v1 read default range - expose valueInputOption for the update operation in the block Backwards compatible: with no filter requested, read output is byte- identical and the `filter` field is omitted. The filterMatchType union is widened additively (4 -> 10 values). * fix(google-sheets): correct filter metadata for missing column and header-only sheets - matchedRows is now 0 (not totalRows) when the filter column is not found, so it no longer contradicts applied=false / columnFound=false - columnFound now reflects an actual header lookup for empty/header-only sheets instead of being hardcoded true - add tests covering header-only and empty sheets with present/absent columns * fix(selectors): fetch all pages for paginated dropdown list routes (#4823) * fix(selectors): fetch all pages for paginated dropdown list routes Dropdown selectors fetched only the first page of paginated provider APIs, silently hiding results past page one. Add bounded server-side draining to the list routes across Microsoft Graph, Google, Notion, Atlassian, Linear, AWS CloudWatch, and offset/token REST APIs, plus a shared client-side drain cap in the selector hook. Response shapes, stored values, and tool execution are unchanged; CloudWatch list tools still honor a caller-supplied limit. Also fixes the Word file picker that was searching for .xlsx files. * fix(selectors): harden JSM and Monday pagination draining - JSM service-desk/request-type drains advance `start` by the actual row count returned (not the fixed page size) and stop on an empty page, so a short non-final page can't skip items. - Monday boards drain now checks `response.ok` per page, surfacing a mid-drain HTTP failure instead of treating it as an empty final page and returning a partial 200. * docs(selectors): clarify JSM drain advances start by actual row count The offset-advancement fix (advance `start` by the rows returned, not the fixed page size) landed in 7b19788a8; update the TSDoc to match so it no longer reads as advancing by `limit`. * fix(selectors): drain fetchPage in direct fetchList callers Making `fetchList` optional left three direct callers (outside the useSelectorOptions hook) calling it unguarded, which broke the build's type check. Route them through a shared `loadAllSelectorOptions` helper that uses `fetchList` when present and otherwise drains `fetchPage`. This also prevents a regression: `confluence.spaces` / `knowledge.documents` now paginate via `fetchPage` only, and these callers (search/replace, value resolution) would otherwise have silently returned no options. * chore(selectors): rename MAX_PAGE_PAGES to MAX_NOTION_PAGES for readability * fix(sso): re-check domain conflict before write and reject IP-address domains (#4825) * improvement(copilot): make copilot_messages the sole transcript store, remove JSONB dual-write (#4826) Stop writing/reading the legacy copilot_chats.messages JSONB column now that reads are cut over to copilot_messages. Make appendCopilotChatMessages the primary write (throws on failure instead of swallowing), repoint peripheral readers (workspace VFS, chat cleanup, data drains, fork, superuser import) to copilot_messages, and persist the assistant turn inside finalizeAssistantTurn's transaction so it commits atomically with the stream-marker clear. The column itself is dropped in a follow-up migration after this bakes. * feat(tables): expand filter operators (not-contains, starts/ends-with, not-in, empty) (#4827) Add does-not-contain ($ncontains), starts-with ($startsWith), ends-with ($endsWith), not-in-array ($nin, previously executed server-side but unexposed in the UI), and is-empty/is-not-empty ($empty) filter operators end-to-end — SQL builder, condition types, query-builder converters/constants, the filter UI, the Table tools/block descriptions, and docs. Also fix correctness bugs in the filter builder surfaced by the wider operator set: - Same-column AND rules (e.g. age > 18 AND age < 65, or name startsWith 'A' AND name endsWith 'Z') silently overwrote each other because the AND group was keyed by column name. They now merge into one operator object, which also makes Filter -> rules -> Filter round-trip losslessly for multi-operator columns. - $nin values were not split into an array like $in, and textual-match values like "123" were numeric-coerced (breaking the ILIKE path). - A non-boolean $empty operand from the raw API silently inverted the check; it now coerces 'true'/'false' strings and otherwise returns a 400. * improvement(copilot): stop persisting tool-call result outputs in transcripts (#4829) Opening a Mothership task could take many seconds because a single persisted assistant message in copilot_messages.content can reach hundreds of MB, almost entirely inside contentBlocks[].toolCall.result.output (e.g. a get_workflow_logs or run_workflow result). The DB query is ~2ms; the cost is detoasting that payload, shipping it to the browser, and parsing it. These outputs are dead weight on the Sim side: they are never rendered (the thread shows only tool name/title/status) and never replayed to the model (the upstream copilot service owns conversation memory). So drop result.output before it is persisted, keeping result.success/error plus the tool metadata. - add stripToolResultOutput() in persisted-message.ts - apply it in messages-store toRow (covers every write path) and in loadCopilotChatMessages (existing rows render fast on read) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * feat(providers): add Together AI, Baseten, and Ollama Cloud model providers (#4830) * feat(providers): add Together AI, Baseten, and Ollama Cloud model providers * fix(providers): guard Ollama streaming fast-path with hasActiveTools Match Together/Baseten/Fireworks: when tools are supplied but all are filtered out (usageControl 'none'), take the single streaming call instead of an extra non-streaming round-trip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(providers): filter non-chat model types from Together model list * refactor(providers): dedupe Ollama Cloud upstream schema ollamaCloudUpstreamResponseSchema was byte-for-byte identical to ollamaUpstreamResponseSchema (both /api/tags endpoints return the same { models: [{ name }] } shape). Drop the duplicate and reuse the shared schema. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(knowledge): calendar view sync, deduplicate popover animation classes, type-safe filter cast * cleanup(knowledge): remove TRIGGER_BORDER_CLASS duplication, inline displayLabel, drop enabledFilterParam alias --------- Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Waleed <walif6@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Theodore Li <theo@sim.ai> Co-authored-by: andresdjasso <andresdjasso@users.noreply.github.com> * feat(blocks): add BlockMeta to Quiver and Linq; fix invalid block config fields; update skills Block fixes: - Add QuiverBlockMeta (tags + 3 templates: icon generator, diagram creator, vectorizer) - Fix QuiverBlock: remove invalid tags field from BlockConfig, IntegrationType.Design → IntegrationType.AI (Design doesn't exist in the enum) - Fix GreptileBlock: remove invalid tags field from BlockConfig, IntegrationType.DeveloperTools → IntegrationType.DevOps - Fix LinqBlock: remove invalid tags field from BlockConfig (tags belong only in BlockMeta) Skills: - add-block: add dedicated BlockMeta section with structure, rules, and registration pattern; add BlockMeta checklist items - add-integration: add BlockMeta to block structure template, add rules clarifying that tags must NOT appear on BlockConfig and integrationType must be a valid enum value; update registry snippet to include blocksMeta; add checklist items * fix(integrations): fix category dropdown by defining missing LANDING_INTEGRATIONS_DATA_PATH and regenerating integrations.json The staging merge introduced landing-content.ts but forgot to define LANDING_INTEGRATIONS_DATA_PATH in generate-docs.ts, causing the script to crash before writing integrations.json. The stale JSON had integrationTypes (plural array) from an older script version, while the Integration type and workspace UI both read integrationType (singular string) — so ALL_CATEGORY_SECTIONS bucketed to undefined and the category filters never appeared in the dropdown. Fixed by adding the missing path constant and re-running the generator. integrations.json now has 192 entries with the correct integrationType field. * fix(sidebar): restore resize handle on all pages commit |
||
|
WaleedandClaude Opus 4.7
|
10f7d36f13 |
fix(settings): accurate View navigation after restore in recently deleted (#4546)
* fix(settings): accurate View navigation after restore in recently deleted
Files now deep-link to /files/{id} (was /files), and folders expand
the restored folder + its parent chain in the sidebar before navigating
to /w so the user actually lands on the item they restored.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix: fall back to archived folders for parent-chain lookup
The restored folder may not be in the active folders cache yet when
View is clicked (the invalidation+refetch fires from onSettled, after
onSuccess surfaces the View button). Merge archived folder data — where
the restored item still lives — into the lookup map so the expansion
loop can always resolve the folder and walk its parent chain.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* chore(styling): canonicalize size-* shorthand for equal height/width
Document the size-* shorthand as the canonical pattern across CLAUDE.md,
AGENTS.md, .claude rules, .cursor + .agents commands, and the emcn
design-review skill. Default icon size is size-[14px]. Treat
h-[Npx] w-[Npx] and h-N w-N pairs as refactor targets.
Also migrate the remaining occurrences in recently-deleted.tsx.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
Vikhyath Mondreti
|
6b0de36238 |
chore(skills): update checklist for boundary e2e checklist (#4363)
* chore(skills): update checklist for boundary e2e checklist * fix formatting |
||
|
Vikhyath Mondreti
|
b8959eb20d |
improvement(repo): zod based client-server boundary (#4355)
* improvement(repo): centralized zod contracts (#4336) * improvement(repo): zod schema contracts * type checks * fix(notion): correctly register tool (#4337) * fix func blokc * more improvements * fix tests * type check * remove v3 refs * minor type improvements * address comments * update jira contract * remove validateJsonBody * improvement(repo): consolidation of boundary helpers + better unknown usage (#4352) * improvement(repo): consolidation of boundary helpers + better unknown usage * address comments * improve file transfer error messaging * fix docs listing schema drift * fix inocrrect type casting * address council comments * remove prefix |
||
|
|
5f0f0edd63 |
improvement(repo): separate realtime into separate app (#4262)
* improvement(repo): restructuring to make realtime image narrower scoped * improvements * chore(repo): rebase fixes and quality improvements for realtime split Addresses merge-time issues and gaps from the realtime app split: - Retarget stale vi.mock paths to @sim/workflow-persistence/subblocks - Restore README branding, fix AGENTS.md script reference - Restore TSDoc on workflow-persistence subblocks helpers - Use toError() from @sim/utils/errors in save.ts - Add vitest config + local mocks so @sim/audit tests run standalone - Move socket.io-client to devDependencies in apps/realtime - Add missing package COPY steps to docker/app.Dockerfile - Add check:boundaries/check:realtime-prune scripts and wire into CI Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(security): consolidate crypto primitives into @sim/security Move general-purpose crypto primitives out of apps/sim into the @sim/security package so both apps/sim and apps/realtime can share them. @sim/security exports (all pure, dependency-free): ./compare safeCompare (constant-time HMAC-wrapped equality) ./encryption encrypt/decrypt (AES-256-GCM, iv:cipher:tag format) ./hash sha256Hex ./tokens generateSecureToken (base64url) Migrate apps/sim call sites to use these + @sim/utils helpers: crypto.randomUUID() -> generateId() from @sim/utils/id createHash('sha256').digest -> sha256Hex timingSafeEqual on hashed hex -> safeCompare new Promise(setTimeout) -> sleep from @sim/utils/helpers No behavior change: encryption format, digest output, and token length are preserved exactly. * refactor(copilot): use toError in remaining otel/finalize sites Replace the last two `error instanceof Error ? error : new Error(String(error))` patterns with toError from @sim/utils/errors. Completes the sweep of clean candidates — no behavior change. * refactor(security): consolidate HMAC-SHA256 primitives into @sim/security Adds hmacSha256Hex and hmacSha256Base64 to @sim/security/hmac and migrates 15 webhook providers plus 5 other hot paths (deployment token signing, outbound webhook requests, workspace notification delivery, notification test route, Shopify OAuth callback) off bare `createHmac` calls. Secret parameter accepts `string | Buffer` to cover base64-decoded Svix-style secrets (Resend) and MS Teams' HMAC scheme. AWS SigV4 signing in S3 and Textract tools intentionally retains direct `createHmac` usage — its multi-step key derivation chain doesn't fit a generic helper. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(packages): post-audit test + packaging polish - Add safeCompare unit tests (identity, length mismatch, hex-nibble diff). - Add Buffer-secret cases to hmac tests to lock in Svix/MS-Teams contract. - Declare `reactflow` as a peerDependency on @sim/workflow-types — only used for type imports. - Add a barrel export to @sim/workflow-persistence for consumers that prefer package-level imports; subpath exports retained. - Document the data-field invariant in load.ts for loop/parallel subflow patching. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(realtime): address PR review feedback - Remove redundant SOCKET_PORT=3002 env from Dockerfile runner stage (env.PORT already defaults to 3002 via zod schema). - Reorder PORT fallback so an explicitly-set SOCKET_PORT wins over the schema default for PORT; keeps SOCKET_PORT functional as an override instead of dead code. - Add dedicated type-check CI step for @sim/realtime so TS errors surface pre-deploy (the Dockerfile runs source TS via Bun and has no implicit build-time type check). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore(realtime): remove unused SOCKET_PORT env var SOCKET_PORT has lived in the socket server since the June 2025 refactor but was never actually set in any deploy config — docker-compose.prod, helm values/templates, .env.example, and docs all use PORT or the 3002 default exclusively. No self-hoster was ever pointed at SOCKET_PORT, so removing it is safe. Simplifies realtime port resolution to `env.PORT` (zod-validated with a 3002 default) and drops the orphaned sim-side schema entry. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Waleed Latif <walif6@gmail.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Waleed
|
b5674d9ed4 |
improvement(codebase): centralize test mocks, extract @sim/utils, remove dead code (#4228)
* improvement(codebase): centralize test mocks, extract @sim/utils, remove dead code * improvement(codebase): apply @sim/utils conventions to staging-introduced files |
||
|
WaleedandClaude Opus 4.6
|
a680cec78f |
fix(core): consolidate ID generation to prevent HTTP self-hosted crashes (#3977)
* fix(core): consolidate ID generation to prevent HTTP self-hosted crashes crypto.randomUUID() requires a secure context (HTTPS) in browsers, causing white-screen crashes on self-hosted HTTP deployments. This replaces all direct usage of crypto.randomUUID(), nanoid, and the uuid package with a central utility that falls back to crypto.getRandomValues() which works in all contexts. - Add generateId(), generateShortId(), isValidUuid() in @/lib/core/utils/uuid - Replace crypto.randomUUID() imports across ~220 server + client files - Replace nanoid imports with generateShortId() - Replace uuid package validate with isValidUuid() - Remove nanoid dependency from apps/sim and packages/testing - Remove browser polyfill script from layout.tsx - Update test mocks to target @/lib/core/utils/uuid - Update CLAUDE.md, AGENTS.md, cursor rules, claude rules Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * update bunlock * fix(core): remove UUID_REGEX shim, use isValidUuid directly * fix(core): remove deprecated uuid mock helpers that use vi.doMock --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Daniel Gómez Martínez
|
668b948f0b |
feat(agents): generalize repository guidance for coding agents (#3760)
* feat(agents): generalize repository guidance for coding agents * fix(agents): use repo-root link in sim app guidance |