Commit Graph
4975 Commits
Author SHA1 Message Date
Vikhyath Mondreti 8fe090a3a1 fix(input-format): field not editable race condition (#5102)
* fix(input-format): field not editable race condition

* remove dead code

* simplify
2026-06-16 19:03:10 -07:00
Waleed 2ffc004a0a improvement(models): add DeepSeek V4 + Mistral Medium 3.5, fix Codestral context window (#5103) 2026-06-16 18:01:04 -07:00
Theodore LiandClaude Opus 4.8 15a970d805 feat(integrations): hosted email-enrichment providers + cascade wiring (#5087)
* feat(integrations): hosted email-enrichment providers + cascade wiring

Add Datagma, Dropcontact, LeadMagic, Icypeas, and Enrow integrations —
tools, blocks, brand icons, and BYOK + metered hosted-key support — and
register each in the tool/block registries and BYOK provider list.

Wire the new finders/verifiers into the enrichment cascades:
- work-email: Datagma, LeadMagic, Dropcontact, Icypeas, Enrow
- phone-number: LeadMagic, Datagma, Dropcontact
- email-verification: Icypeas, Enrow
- company-info: Datagma, LeadMagic
- company-domain: Datagma

Add hosting tests for all five providers and cascade tests covering the
new providers (incl. new test files for email-verification, company-info,
and company-domain).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(enrichment): address PR review on Icypeas success + Datagma billing

- Icypeas find_email/verify_email postProcess return success:true for all
  terminal statuses (NOT_FOUND/DEBITED_NOT_FOUND included) so the cascade
  runner calls mapOutput and records invalid/not-found verdicts instead of
  throwing and inflating the error count
- Bill Icypeas verify FOUND (not just DEBITED*) per the documented 0.1-credit
  charge
- Datagma enrich_person only applies the 30-credit phone surcharge when a
  phone lookup (phoneFull) was requested
- Note Datagma's URL-param (apiId) auth in the hosted-key doc comment
- Update hosting tests to match

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(enrichment): only bill Enrow verify on a completed verification

getCost returned a flat 0.25 credits regardless of output, so a job that
fell back to the initial submit response (poll never completed, no
qualification) was still metered. Charge 0.25 only when a qualification is
present; 0 otherwise. Add a no-qualification test case.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore(enrichment): peg hosted credit cost to each provider's lowest paid plan

Align *_CREDIT_USD to the entry tier Sim will provision:
- Datagma: Regular $49/3,000 emails → $0.0163 (was Popular $0.0132)
- LeadMagic: Basic $49/2,000 → $0.0245 (was Growth $0.0104)

Icypeas (Basic $0.019), Enrow (Starter $0.012), and Dropcontact (Starter
~$0.17) already reflect their lowest plan. Tests derive from the constants,
so values stay consistent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(enrichment): address PR review on mononyms + Icypeas verify email map

- work-email LeadMagic: pass full_name + domain so single-token (mononym)
  names are no longer skipped
- work-email Icypeas: firstname/lastname are optional on the API, so run a
  mononym with firstname alone instead of self-skipping
- icypeas_verify_email mapItem reads item.email (verify payload shape) with a
  fallback to the nested results.emails[0].email

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(enrichment): case-insensitive Enrow find billing

getCost compared qualification to exactly 'valid' while the cascade
normalizes with toLowerCase(), so a differently-cased API qualifier could
zero out billing on a valid email. Lowercase before comparing; add a test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(enrichment): drop Dropcontact from the phone-number cascade

Dropcontact is an email/company-data enrichment service, not a phone-discovery
provider — its phone/mobile_phone fields are unreliable and were surfacing
firmographic data (an employee-count range like "5000-20077") as the phone.
Keep the two purpose-built phone finders (LeadMagic find_mobile, Datagma
find_phone); Dropcontact stays in the work-email and company cascades where
its data is reliable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(enrichment): only accept valid-qualified Enrow emails in work-email

Enrow's finder qualifies each email valid/invalid. The work-email mapOutput
accepted any non-empty email, so an invalid-qualified address could fill the
cell while hosted billing (which only charges on valid) charged zero. Gate the
cell on qualification === 'valid', consistent with billing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 19:12:59 -04:00
Vikhyath Mondreti feca5fa6f6 improvement(execution, connectors): offload large function inputs, increase connector limits + better error propagation (#5089)
* fix(execution,connectors): offload large function inputs; harden KB connector size limits

Addresses a class of 10 MB limit failures:

- executor/variables: offload over-budget function block-output context values to
  durable large-value refs (lazy `sim.values.read`) so JS function blocks can merge
  medium files without exceeding the 10 MB inter-block request-body cap.
- connectors: stream downloads via `readBodyWithLimit` (memory-safe), and surface
  oversized files as visible `failed` KB documents instead of silently dropping them
  — listing-time for github/s3/dropbox/onedrive/sharepoint, fetch-time for
  gitlab/azure/google-drive via a shared `ConnectorFileTooLargeError`. Raise the
  per-file cap from a hardcoded 10 MB to the canonical 100 MB KB document limit
  (`CONNECTOR_MAX_FILE_BYTES`), except Google Drive's export path (Google's hard
  10 MB export-API limit).
- sync-engine: `classifyExternalDoc` + bulk `skipDocuments` (failed rows with a
  reason, excluded from retry), byte-bounded batch concurrency to cap peak worker
  memory at the raised cap, and a `metadata.fileSize ?? size` fallback.

* fix zoom

* update skill

* address comments + fix terminal event in sse stream

* fix accounting issue
2026-06-16 14:22:59 -07:00
Waleed cc56408be3 perf(execution): parallelize preflight gates, cache deployed state, memoize Anthropic client (#5098)
* perf(execution): parallelize preflight gates, cache deployed state, memoize Anthropic client

- Memoize Anthropic + Azure-Anthropic SDK clients (new client-cache.ts) keyed
  by apiKey (+beta header; +baseURL/version/pinnedIP for Azure) so HTTP
  keep-alive connections are reused instead of a fresh TLS handshake per call.
  apiKey is the tenant boundary.
- Parallelize the read-only preflight gates in preprocessing.ts (ban +
  subscription, then usage + org-member + rate-limit) while preserving exact
  error precedence (ban 403 -> usage 402 -> rate 429) and keeping the sole
  write (admission reservation) last.
- Parallelize the independent workflow-state and env-var loads in execution-core.
- Cache deployed workflow state by immutable deploymentVersionId with
  deep-clone-on-read, oldest-first eviction, and a 5-min TTL bounding the
  credential-mapping edge across ECS tasks.
- Parallelize the independent personal-subscription + membership queries in
  getHighestPrioritySubscription.
- BYOK: drop the redundant getWorkspaceById existence check (auth already
  validates the workspace); read the key list fresh every call for zero
  cross-instance staleness.

Billing/usage/ban/permission reads stay fresh on the primary (no cache, no
replica). Adds tests for every new mechanism and fixes a pre-existing vitest
class-mock incompatibility that had execution-core.test.ts fully red on staging.

* fix(execution): run rate-limit gate only after ban/usage pass

The rate-limit gate is not read-only — checkRateLimitWithSubscription consumes
a token — so running it in parallel with the read-only gates debited rate-limit
quota for requests that the ban (403) or usage (402) gates reject, which the
original sequential flow never did.

Move the rate-limit gate to run sequentially after the ban and usage gates pass,
preserving the read-only gates' parallelism (ban + subscription + usage) and the
exact ban -> usage -> rate precedence. Add regression tests asserting the rate
limiter is not consumed when an earlier gate rejects, and is consumed once when
they pass.

Caught by Cursor Bugbot review.

* chore(execution): trim redundant preflight comments

Tighten the gate overview to match the sequential rate-limit gate and drop
inline notes that duplicated it or the runRateLimitGate doc.

* refactor(cache): address review — idle TTL for client cache, LRUCache for deployed state

- client-cache: add updateAgeOnGet so the TTL is genuinely idle-based (active
  clients keep their warm keep-alive connections; the JSDoc now matches behavior).
- deployed-state: replace the hand-rolled Map + manual FIFO eviction/TTL with
  LRUCache (real LRU eviction, built-in TTL), matching the effectiveDecryptedEnv
  and integration-tool-schema caches. TTL stays absolute (not reset on read) so
  the credential-migration remap still propagates across ECS tasks.

Both per review feedback from Greptile.

* test(execution): isolate rate-limit gate test from STEP 7 reservation

The 'consumes the rate-limit gate once' test reached the STEP 7 admission
reservation, which depends on Redis — it passed locally (reserve throws and is
swallowed) but failed in CI (reserve returns not-reserved -> 429). Pass
skipConcurrencyReservation so the test isolates the rate gate deterministically.

* perf(providers): memoize SDK clients where the pool is per-client (bedrock, vllm)

Generalize the Anthropic client cache into one shared memoizer
(providers/client-cache.ts) and apply it only where each new client owns its own
connection pool — so reuse actually keeps connections warm:

- bedrock: AWS SDK clients hold a per-client connection pool (reuse is the AWS
  best practice). Keyed by region + credential identity.
- vllm: a pinned endpoint creates its own undici Agent per call; key by the
  resolved IP so DNS re-validation still runs each request.
- anthropic + azure-anthropic: migrated onto the shared memoizer.

Deliberately NOT applied to the OpenAI-compatible providers, groq, cerebras, or
google: their SDKs share a process-global keep-alive pool (Node openai-sdk module
singleton agent; anthropic/global undici), so a fresh client per request already
reuses connections and memoization would add complexity with ~no benefit. litellm
uses a plain shared-agent client (no pinning) and is likewise skipped.

Bounded LRU (max 1000, 30m idle TTL) with no close-on-eviction, avoiding the
unbounded-growth and eviction-closes-in-use-client failure modes seen in similar
client caches.

* chore(perf): trim verbose comments to terse why-notes

* chore(perf): drop obvious inline comments, keep nuance as TSDoc

* fix(bedrock): key client cache on full credential, not just access key id

A corrected secret under the same access key id would otherwise keep serving the
stale cached client until TTL/eviction. Caught by Cursor Bugbot.

* test(execution,providers): fix preflight mock reset + isolate provider client cache in tests

- preprocessing.test: re-establish the checkOrgMemberUsageLimit mock in beforeEach
  (the only gate mock not re-set). In the full suite its implementation was reset
  so the success-path test got undefined -> threw -> 500 -> success:false. Mirrors
  how checkServerSideUsageLimits is handled.
- client-cache: add clearProviderClientCacheForTests; call it in the bedrock and
  vllm test beforeEach so construction assertions always start from a cache miss
  now that those providers memoize their client.

* test(execution): make RateLimiter mock constructable under vitest 4.x

The RateLimiter mock used an arrow factory (vi.fn(() => ({...}))). vitest 4.x
(CI) rejects `new` on an arrow-implemented mock ("not a constructor"); 3.2.4
allowed it. The new rate-gate test is the first to actually `new RateLimiter()`,
so it surfaced the failure only in CI. Switch the mock to a regular function and
drop the speculative beforeEach re-establishments that didn't address it.
2026-06-16 13:49:03 -07:00
Waleed f238184fa0 feat(file): add Compress and Decompress operations to the File block (#5100)
* feat(file): add Compress operation to bundle files into a .zip archive

* feat(file): add Decompress operation to extract .zip archives

Adds the inbound half of the archive pair: extracts a .zip back into the
workspace with zip-slip path sanitization, symlink skipping, and entry/
size caps to bound zip-bomb expansion. Extracted files are returned in the
files output, ready to chain downstream.

* fix(file): align archive ops with v5 output surface and zip mime

- Drop the single 'file' output reintroduced for compress/decompress; v5
  intentionally exposes only 'files' (plus id/name/size/url scalars), so
  compress/decompress reuse the existing surface with no new block output
- Add zip/gz to EXTENSION_TO_MIME (previously only in the reverse map), so
  archive extensions resolve to a real mime instead of octet-stream
- Update File v5 block test for the two new operations

* fix(file): harden compress naming per review

- Flatten zip entry names to a safe basename so untrusted fileInput names
  with .. or / cannot produce zip-slip entry paths (cursor)
- Treat archiveName as a flat name landing at the workspace root instead of
  passing it through splitWorkspaceFilePath, which silently created folders
  for names with separators (greptile)
- Add the upfront empty-input guard before any DB calls, matching the read
  and content operations (greptile)

* fix(file): make decompress extraction atomic and bound per-entry size

- Read and validate every entry before writing any file, so hitting a size
  cap no longer leaves partially-extracted files in the workspace (cursor)
- Enforce the per-entry cap on the materialized buffer in addition to the
  declared size, covering entries that omit an uncompressed size (cursor)
- Pre-check declared sizes up front to reject standard zip bombs before
  materializing, and return 422 when no files could be extracted (cursor)

* fix(file): exclude skipped entries from caps and reject multi-archive decompress

- Resolve safe (sanitized) zip entries up front so unsafe/skipped entries
  no longer count toward the per-entry and total uncompressed-size caps (cursor)
- Reject decompress input that resolves to more than one archive with a clear
  error instead of silently extracting only the first (cursor)

* fix(file): enforce single-archive decompress at the API boundary

The block already rejects multiple archives, but the manage route is the
real boundary (callable directly and by the LLM tool) and still took the
first of multiple resolved inputs. Add the empty-input and >1-archive guards
in the route so extra archives are rejected with a clear error rather than
silently ignored (cursor).

* docs(file): correct compress description and stale file-output references

- Drop the misleading 'under provider upload limits' claim from the compress
  tool description (models cannot read zip archives)
- Fix bestPractices to reference the 'files' output, not a non-existent 'file'
- Remove the stale 'file' property from the compress test fixture so it
  matches the real API response (greptile)
2026-06-16 12:25:33 -07:00
Waleed c864a928bf improvement(models): sort model dropdown by latest release date within each provider (#5099)
* improvement(models): sort model dropdown by latest release date within each provider

* fix(models): preserve input provider order and build catalog index once
2026-06-16 11:21:21 -07:00
Vikhyath Mondreti 73c73ff0f2 fix(kb): canonicalize knowledge-base upload keys (#5096)
* fix(kb): canonicalize knowledge-base upload keys

* fix tests
2026-06-16 10:58:48 -07:00
Vikhyath Mondreti d14bc78d03 fix(realtime): re-check workspace role on mutating socket events (#5080)
* fix(realtime): re-check workspace role on mutating socket events

* address comments
2026-06-16 10:50:36 -07:00
Waleed 2b7f57a788 improvement(providers): tighten Gemini and vLLM agent-attachment ceilings (#5095)
A live-doc audit of the merged large-file feature found two ceilings that were
higher than the provider actually accepts:
- Gemini: 100MB -> 50MB. Gemini hard-caps PDFs at 50MB, so a 50-100MB PDF passed
  our gate, got uploaded + polled, then failed at generateContent. 50MB respects
  the documented limit and is more memory-safe.
- vLLM: 50MB -> 25MB. vLLM's default image-fetch timeout is 5s; a 50MB remote
  fetch routinely exceeds it. 25MB aligns with that reality and matches Baseten
  (the other vLLM-backed provider).
2026-06-16 10:27:07 -07:00
Waleed 2c1392e3a0 fix(chat): autoscroll follow-ups — re-engage threshold + keep end-of-turn options in view (#5094)
* fix(chat): align scrollbar/keyboard detach with wheel/touch re-engage threshold

The onScroll detach branch set only stickyRef.current = false, leaving
userDetachedRef false, so a scrollbar-drag or keyboard detach kept the lenient
30px (STICK_THRESHOLD) re-engage threshold instead of the strict 5px
(REATTACH_THRESHOLD) used after wheel/touch. A programmatic virtualizer re-pin
landing within 30px could then snap autoscroll back on right after the user
deliberately scrolled away. Reuse the detach() helper so all detach paths set
userDetachedRef consistently.

* fix(chat): keep end-of-turn options in view after streaming

When a stream ends, the suggested-follow-up options and the actions row (gated
on !isStreaming) mount, but the virtualizer's getTotalSize — which drives the
scroll container's scrollHeight — only catches up a frame or two later via its
ResizeObserver. The single scrollToBottom() on effect teardown therefore landed
on a stale, too-short bottom and the options were clipped behind the input.
(Pre-virtualization this worked because scrollHeight reflected the new rows
immediately.)

Extract the rAF follow loop already used for CSS height animations into a shared
followToBottom(window) helper and run it for a short settle window on teardown,
so the bottom is chased until the virtualizer re-measures. The follow is
self-interrupting — height growth leaves scrollTop where we put it, while a user
scroll moves it up, so it bails the instant the user scrolls and never fights a
real gesture even with listeners torn down.
2026-06-16 09:51:58 -07:00
Waleed e1f22bdac9 feat(providers): support large agent-block attachments via Files APIs and remote URLs (#5092)
* feat(providers): support large agent-block attachments via Files APIs and remote URLs

Agent-block file uploads were inlined as base64 with a hard 10MB cap. Files
above the threshold now use each provider's native large-file path:

- OpenAI / Gemini: upload to the provider Files API, reference by file_id/uri
- Anthropic: GA url content-block source (no Files API beta, no upload)
- OpenRouter/Groq/Together/Baseten/xAI/vLLM: remote signed URL in image_url/file
- Limits live per-provider in models.ts; the agent block + /models page reflect them

Files <=10MB keep the identical base64 path (zero regression). Server-only file
handles are stripped from untrusted input to prevent SSRF.

* fix(providers): clear forged file handles for inline providers too

attachLargeFileRemoteUrls early-returned for inline-strategy providers before
clearing server-only handle fields, so a forged remoteUrl on an inline-provider
file could still reach a builder (e.g. buildOpenAICompatibleChatContent for
mistral/ollama). Clear the handles for every provider before the strategy check.

* fix(providers): correct OpenAI expiry serialization and Anthropic large-text-doc handling

- OpenAI upload now uses the SDK (client.files.create) so expires_after is
  serialized as a real nested object; the prior expires_after[anchor] bracket
  FormData keys were ignored by OpenAI's server, leaving files un-expiring.
- Anthropic url document source only supports PDFs/images; large non-PDF text
  docs now throw a clear error instead of emitting an unsupported url source.
- Warn when an oversized file can't be sent because cloud storage is unavailable.

* fix(providers): harden large-file path (SSRF fetch, ceiling gate, per-file UI limit)

- Download files for OpenAI/Gemini uploads via validateUrlWithDNS + IP-pinned
  fetch so a forged URL can't reach internal addresses (covers all callers).
- Reject files above the provider ceiling before downloading/uploading.
- UI now validates each file against the provider's per-file ceiling instead of
  summing all files against it, matching server-side per-file validation.
- Lower Anthropic ceiling to 50MB (documented 32MB request cap / page limits).

* refactor(providers): read files-api upload bytes via storage SDK

Read OpenAI/Gemini upload bytes through downloadFileFromStorage instead of
HTTP-fetching the presigned URL. Removes any server-side URL fetch (no SSRF
vector) and works with internal object storage (e.g. self-hosted MinIO), which
an IP-pinned URL fetch would have blocked.

* docs(providers): clarify files-api bytes are read from storage at upload time

* fix(providers): enforce access checks and strip forged ids in the upload path

uploadLargeFilesToProvider runs on raw request messages for every caller (incl.
the internal providers passthrough), so harden it independently of the agent path:
- verifyFileAccess on each file's storage key before reading its bytes, so a forged
  key can't exfiltrate another user's file.
- clear any inbound providerFileId/providerFileUri up front (legit ids are only set
  by the upload itself), so a forged id can't reference a file in a hosted account.

* fix(providers): resolve UI attachment limit with the same model->provider helper as execution

The file-upload control imported getProviderFromModel from @/providers/models, but
the execution path and every other consumer use the one in @/providers/utils (runtime
registry + reseller patterns). Align the UI so its size cap can't disagree with
server-side validation for reseller or dynamically-listed models.

* test(providers): add new models.ts exports to provider mocks

attachments.ts now reads getProviderFileAttachment / INLINE_ATTACHMENT_MAX_BYTES
from @/providers/models; the provider unit tests that fully mock that module need
both exports or attachments.ts fails to load.

* fix(providers): guard Gemini upload response name before polling

ai.files.upload returns name as string | undefined; guard it (instead of an
as-string cast) so a missing name surfaces a clear error at the upload site
rather than an opaque files.get failure on the first poll.

* fix(uploads): type the file-handle key list so omit preserves UserFile fields

The 'as const' readonly tuple widened omit's K to all keys, collapsing
Omit<UserFile, K> to {} and failing the production build's type check. Declare
the array as Array<keyof handle fields> so K is the precise literal union.

* refactor(providers): run handle-clear + URL-mint in executeProviderRequest for all callers

Move attachLargeFileRemoteUrls out of the agent handler and into
executeProviderRequest (right before uploadLargeFilesToProvider), so every entry
point — including the internal providers passthrough — clears forged handles and
mints/access-checks large-file URLs uniformly. The agent handler now only hydrates
base64; its missing-file guard exempts large files (resolved downstream).

* fix(azure-openai): guard optional attachment dataUrl in inline image part

PreparedProviderAttachment.dataUrl is now optional (large files carry a handle
instead); azure-openai builds chat content inline and assigned it directly to a
required url field, failing the production build's type check.

* fix(providers): upload OpenAI files via multipart and fix Buffer Blob part

The installed openai SDK (4.104) does not type expires_after on files.create, so
upload via POST /v1/files directly with the documented expires_after[...] form
fields (gives the file an auto-expiry). Also wrap the storage Buffer in a
Uint8Array for the Blob, which the production build's stricter lib types require.

These two type errors were masked locally because tsc was OOMing silently without
the type-check script's --max-old-space-size flag.

* fix(providers): forward userId from the providers API to executeProviderRequest

Large-attachment prep now needs request.userId for presigned URLs and access
checks; the authenticated providers proxy has auth.userId but wasn't passing it,
so oversized attachments failed for logged-in callers. Forwarding it makes large
files work there and keeps the access check (verifyFileAccess) intact.

* fix(providers): fail clearly when a large attachment has no cloud storage

The doc claimed a base64 fallback that doesn't exist — above the inline cap there
is no base64, so without cloud storage the file previously reached the builder and
died with a generic read error. Throw a clear 'requires cloud file storage' error
at the point of detection and correct the doc.
2026-06-16 09:16:10 -07:00
Waleed e48c960fe8 fix(chat): keep autoscroll pinned when the virtualizer re-scrolls during streaming (#5093)
* fix(chat): keep autoscroll pinned when the virtualizer re-scrolls during streaming

The sticky-scroll detach heuristic (scrollTop drops while scrollHeight
doesn't grow) could not distinguish a user scrollbar drag from a
programmatic scroll. react-virtual re-pins content by moving scrollTop
whenever a measured row's size changes — including the transient height
shrinks streamdown emits as it re-parses each streaming token — so the
hook misread those upward programmatic scrolls as the user scrolling
away and detached mid-stream.

Gate the scroll-delta detach branch behind a genuine recent user gesture
(pointerdown/up tracking + wheel/touch/keydown stamp). Programmatic
scrolls have no preceding gesture, so they no longer detach; scrollbar
drag, wheel, and keyboard detach are preserved.

* fix(chat): address review — reset pointer ref on teardown, stop wheel/touch opening detach window

- Reset pointerDownRef in effect cleanup so a pointer held through teardown
  (e.g. dragging the scrollbar as a stream finishes) can't leak a stuck-true
  ref into the next session and detach on the first programmatic re-pin.
- Wheel-up and touch-drag already detach directly, so the onScroll delta
  heuristic only needs to authorize scrollbar drag (pointerDownRef) and
  keyboard. Stop stamping the gesture window on wheel/touch, which otherwise
  let a harmless downward wheel open a 250ms window where a virtualizer
  shrink could falsely detach.

* fix(chat): scope detach authorization to real scroll gestures; TSDoc comments

- onPointerDown only marks an active drag when the press targets the scroll
  container itself (the scrollbar), not its content, so a text-selection drag
  on a message can't authorize a detach during a programmatic re-pin.
- Reset lastUserGestureAtRef on teardown alongside pointerDownRef so neither a
  held pointer nor a late keydown can leak across streaming sessions.
- Convert the hook's inline comments to TSDoc on the relevant declarations per
  codebase conventions.

* fix(chat): only upward scroll keys authorize a keyboard detach

onKeyDown stamped the gesture window on any bubbling key, so an unrelated
keypress within USER_GESTURE_WINDOW of a programmatic virtualizer re-pin
could satisfy userDriven and detach mid-stream. Filter to the upward scroll
keys (ArrowUp, PageUp, Home, Shift+Space), mirroring the wheel handler's
upward-only rule, so only a genuine upward keyboard scroll authorizes detach.
2026-06-15 23:30:30 -07:00
Siddharth Ganesan 6cbaf4282d fix(scheduled-tasks): fix scheduled tasks schema validation (#5091)
* fix(scheduled-tasks): fix schema rejection

* fix(db): fix duplicate db query
2026-06-15 22:11:19 -07:00
Siddharth Ganesan 91666b585f feat(scheduled-tasks): migrate jobs agent to scheduled tasks agent (#5090) 2026-06-15 21:00:08 -07:00
Vikhyath Mondreti 7b4626e547 improvement(perm-groups): allow workspace filter for permission groups (#5070)
* improvement(perm-groups): allow workspace filter for permission groups

* show errors correctly

* address comments

* address concurrent edit concern

* address locks

* address comments"

* index migration safety

* address at route level
2026-06-15 20:52:01 -07:00
Waleed b7d30c89f9 feat(google-calendar): wire freebusy, align tools with API v3, add calendar + sharing tools (#5084)
* feat(google-calendar): wire freebusy, align tools with API v3, add calendar + sharing tools

* fix(google-calendar): address review — trust offset timezones, make list_acl showDeleted usable, harden unshare error parse, clarify update attendees

* fix(google-calendar): wire list q/pageToken into block, harden invite PUT error parse

* fix(google-calendar): make list orderBy user-selectable, clarify update timeZone applies to start/end

* fix(google-calendar): require timeZone for recurring timed events, clarify recurrence replace semantics

* fix(google-calendar): validate grantee before building share ACL body

Throw a clear error when scopeType is user/group/domain but scopeValue is
missing or blank, instead of POSTing a scope-type-only body that the Calendar
ACL API rejects with an opaque error.
2026-06-15 20:32:58 -07:00
Theodore Li 06f1e72d6a feat(feature-flags): migrate 3 env-flags to AppConfig-backed runtime flags (#5086)
* feat(feature-flags): migrate 3 env-flags to AppConfig-backed runtime flags

* fix(feature-flags): hardcode workflow-columns on, fix feature-flags tests

* chore(feature-flags): document mothership-beta userId targeting limitation
2026-06-15 23:08:50 -04:00
Waleed 05e8c7cd71 refactor(connectors): split client metadata from server runtime (#5076)
* refactor(connectors): split client metadata from server runtime + cover node:net in client bundle

The browser build broke with `Cannot find module 'node:net'`. Server-only
SSRF code in `input-validation.server.ts` (`dns/promises`, and since PR #5060
`undici` → `node:net`/`node:tls`) is statically reachable from the client
bundle via the tool/connector registries, which the workflow editor imports
for metadata. Node networking builtins have no browser shim, so Turbopack
cannot compile them for the client.

Two changes:

1. Split each connector's client-safe declarative metadata into a sibling
   `meta.ts` (`<name>ConnectorMeta`), mirroring the `XBlockMeta` /
   `BLOCK_META_REGISTRY` pattern. `connectors/registry.ts` is now the
   client-safe `CONNECTOR_META_REGISTRY` (+ `getConnectorMeta` /
   `getAllConnectorMeta`); the full registry with runtime fns moves to
   `connectors/registry.server.ts`. Client components consume the meta
   registry; the sync engine and knowledge API routes consume the server
   registry. This removes connectors from the client's server-only graph.
   Connector metadata is byte-for-byte identical before/after; runtime fns
   are untouched.

2. Extend the existing #4899 `turbopack.resolveAlias` browser stub — which
   already mapped `dns`/`dns/promises` to an empty module for the browser —
   to also cover `net`/`tls` (+ `node:` variants), since `undici` now pulls
   those in. The remaining tool/provider definitions still reach
   `input-validation.server` server-side; the browser-only stub keeps those
   Node builtins out of the client bundle while the real modules stay on the
   server, so SSRF validation and IP pinning are unaffected.

Connector authoring/validation skills updated to teach the meta.ts split.

* fix(icons): use Square logo glyph only, drop wordmark

* fix(connectors): share Discord max-messages default across meta and runtime

Discord defined DEFAULT_MAX_MESSAGES separately in meta.ts (config placeholder)
and discord.ts (sync behavior), which could drift. Export it from meta.ts and
import it in the runtime, matching the single-source pattern used by the other
connectors (e.g. gmail, intercom).

* refactor(tools): route grafana/agiloft egress server-side, drop SSRF browser shim

Move the server-only SSRF-pinned fetch out of the grafana (update_dashboard,
update_alert_rule) and agiloft (11 record/search tools) definitions and into
internal API routes, the same pattern the rest of the server-side tools (and
agiloft's own attach/retrieve) already use. The tool definitions are now purely
declarative (request → internal route), so they no longer import
`input-validation.server` and the tools registry is fully client-safe.

With connectors (meta split) and these tools no longer reaching server-only code
from the client bundle, the browser no longer pulls in `dns`/`net`/`tls`:

- Add `import 'server-only'` to `input-validation.server.ts` so any future client
  import fails loudly at build time instead of silently bloating the bundle.
- Remove the `turbopack.resolveAlias` browser stub and delete
  `empty-node-fallback.browser.ts` — the root cause is fixed, the shim is gone.

Behavior is unchanged: each route runs the exact merge/validation/fetch logic the
tool ran before (every header, param branch, JSON-parse guard, error string, and
SSRF pinning preserved); only the location of execution moved from the client-
bundled definition to a server route.

* fix(connectors): move onedrive tagDefinitions into meta; drop server-only guard

- onedrive's tagDefinitions lived in the runtime file, so the client meta
  registry returned undefined for it and the add-connector tag opt-out section
  stopped rendering for onedrive. Move it into meta.ts like the other connectors
  so client and server see identical metadata (verified across all 50).
- Remove the 'server-only' import from input-validation.server.ts: the meta/route
  split already keeps it out of the client bundle, and blocks/tools registries
  don't use the guard either.

* fix(grafana): surface upstream error when the prefetch GET fails

Check response.ok on the existing-resource GET in both update routes and return
the upstream status/body, matching how the tool framework surfaced GET errors
before the move to internal routes (the framework checks response.ok before
transformResponse). Without this, a failed prefetch produced a generic
'Failed to fetch existing ...' message and dropped Grafana's error detail.

* fix(grafana): reject invalid panels JSON instead of silently ignoring it

Grafana's dashboard API treats panels as a required array and returns 400 on
invalid JSON; this route already errors on every other JSON param. Return
'Invalid JSON for panels parameter' instead of swallowing the parse error and
proceeding with a misleading success.

* fix(grafana): trim dashboard/alert-rule UID in route URLs (carry over #5082)

PR #5082 added .trim() on dashboardUid/alertRuleUid in the original tool URL
builders. Those tools now build their URLs in the internal routes, so apply the
same trim there to preserve that behavior.

* fix(grafana): route update_folder egress server-side (carry over #5082)

#5082 added a grafana update_folder tool that does SSRF-pinned fetch in its
postProcess, re-introducing the client-bundle leak. Convert it to the internal
API route pattern like the other update tools so the def is declarative and
input-validation.server stays out of the client bundle.

* fix(grafana): surface route failures in transformResponse instead of masking them

The grafana update tools' transformResponse hardcoded success: true and dropped
the route's error, so an upstream/validation failure (HTTP 200 with
{ success: false, error }) was reported to the workflow as a success. Forward
data.success and data.error (matching the agiloft tools) so failures propagate
as before the move to internal routes.
2026-06-15 19:53:37 -07:00
Waleed 324189ed39 feat(grafana): validate integration and add folder, health, and contact-point tools (#5082)
* feat(grafana): validate integration and add folder, health, and contact-point tools

- Require alert-rule title/ruleGroup/data in the block (create would 400 without them)
- Trim UID path params across dashboard and alert-rule tools to avoid copy-paste 404s
- Use Grafana brand color for the block background
- Surface previously-unsettable list params (limit, starred, annotation type)
- Add get/update/delete folder, check data source health, get health, and create contact point tools
- Strip non-TSDoc comments; regenerate docs and integrations.json

* fix(grafana): scope block param remaps per operation to prevent cross-operation leaks
2026-06-15 19:09:16 -07:00
Siddharth Ganesan ed31edf068 improvement(ci): fix companion regex (#5083) 2026-06-15 19:02:43 -07:00
Theodore Li 3fe061e3b3 feat(feature-flags): AppConfig-backed gated feature flags (#5059)
* feat(feature-flags): AppConfig-backed gated feature flags

* fix(ci): repoint 'Validate feature flags' step to env-flags.ts after rename

* improvement(feature-flags): drop in-code defaults; fallback resolves a per-flag secret, gating is AppConfig-only

* improvement(feature-flags): make flag names a closed set so every flag requires a fallback secret

* improvement(feature-flags): single FEATURE_FLAGS registry — each entry defines name, description, and fallback in one place

* improvement(feature-flags): fallback is the env secret key (keyof typeof env), resolved to a boolean
2026-06-15 21:56:12 -04:00
Siddharth Ganesan aaeef82e22 improvement(ci): rename companion tags to be more descriptive (#5081) 2026-06-15 18:48:51 -07:00
Waleed 0c226b9cbe fix(providers): allow HTTP for self-hosted vLLM endpoints (#5078)
Pass allowHttp to validateUrlWithDNS so plain-HTTP self-hosted vLLM
endpoints are permitted. This only relaxes the protocol check; the
private/reserved-IP blocklist and blocked-port checks still apply, so
SSRF protection is unchanged.
2026-06-15 18:29:10 -07:00
Siddharth Ganesan cbd3d2220f feat(ci): mship companion pr check (#5079) 2026-06-15 17:47:58 -07:00
Waleed 4a7c2ef656 fix(providers): pin vLLM provider endpoint to validated IP (#5077)
Validate the user-supplied vLLM endpoint (request.azureEndpoint) against
the central SSRF guard and pin the connection to the resolved IP before
issuing any request, mirroring the Azure OpenAI/Anthropic providers. The
operator-configured VLLM_BASE_URL stays trusted and unvalidated.
2026-06-15 17:10:42 -07:00
Waleed a09e3939f2 fix(webhooks): cap request body size on public webhook receivers (#5075)
* fix(webhooks): cap request body size on public webhook receivers

Public, unauthenticated webhook endpoints read the entire request body
into memory before any lookup or signature verification, letting a caller
exhaust pod memory with arbitrarily large bodies.

Bound the body via the existing size-limited stream reader (content-length
guard + streamed cap) and return 413 on oversize. Applies to
parseWebhookBody (trigger receiver) and the agentmail route. Cap defaults
to 10 MB, overridable via WEBHOOK_MAX_REQUEST_BYTES.

* refactor(webhooks): extract shared body-size cap to constants module

Address review feedback: hoist WEBHOOK_MAX_BODY_BYTES into a single
lib/webhooks/constants.ts so the trigger receiver and AgentMail route share
one source of truth instead of recomputing the env-derived cap (prevents
drift). Also drop the redundant request clone when the body stream is null.

* refactor(webhooks): drop redundant null-body branch in capped readers

Both capped body readers had an `if (!stream)` fallback to an uncapped
`.text()`/empty string. `readStreamToBufferWithLimit` already returns an
empty buffer for a null stream, so the branch is redundant and the
`.text()` fallback was a theoretical bypass (chunked request, no
content-length, null body). Collapse both to a single capped read.

* chore(webhooks): drop inline comments from capped body readers
2026-06-15 17:03:52 -07:00
Theodore LiandClaude Opus 4.8 f277f5fba5 feat(db): zero-downtime migration safety lint + db-migrate skill (#5041)
* feat(db): zero-downtime migration safety lint + db-migrate skill

Add scripts/check-migrations-safety.ts (check:migrations), a CI gate that
classifies statements in newly-added migrations into hard errors (rewrite),
annotate-to-acknowledge contract ops (`-- migration-safe: <reason>`), and
backfill warnings. Wire it into test-build.yml. Add the /db-migrate skill as
the judgment half (expand/contract phasing, app-code cross-ref, annotation
authoring).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(skills): run cleanup and db-migrate safety checks in /ship

* fix(db): address review — DROP INDEX lock symmetry, RENAME CONSTRAINT false-positive, alter-type literal match

- Non-concurrent DROP INDEX is now a hard error (ACCESS EXCLUSIVE lock), symmetric with CREATE INDEX; DROP INDEX CONCURRENTLY after a COMMIT passes clean. Removes the false-confidence annotate path.
- RENAME rule narrowed to RENAME COLUMN / table RENAME TO; RENAME CONSTRAINT and ALTER INDEX ... RENAME (metadata-only) no longer flagged.
- alter-type regex now requires TYPE to follow the column identifier, so it no longer matches TYPE inside a string default.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(db): enforce IF EXISTS on DROP INDEX CONCURRENTLY for replay idempotency

Symmetric with the CREATE INDEX CONCURRENTLY rule: a post-COMMIT DROP INDEX
CONCURRENTLY replays from the top on failure, so without IF EXISTS it aborts
re-dropping an already-gone index.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* improvement(skills): gate /ship cleanup on UI changes; default migration base to staging

- /ship runs /cleanup only when the diff touches UI code (.tsx or apps/sim/components|hooks|stores); the six passes are React-only.
- /ship runs check:migrations against origin/staging (the PR base).
- check:migrations default baseRef is now origin/staging instead of origin/main.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(db-migrate): add contract-pending TODO convention for deferred drops

Establishes a durable, greppable marker (`contract-pending(<precondition>): ...`)
left on the legacy column in schema.ts when an expand defers a drop, so the
contract phase doesn't rot. The outstanding-work list is `grep -rn contract-pending`;
the contract PR's `-- migration-safe:` annotation references the expand and deletes
the marker.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 20:00:50 -04:00
Theodore Li a49e755418 feat(auth): OAuth-only signup with Microsoft provider (#5073)
* feat(auth): OAuth-only signup with Microsoft provider

- Remove email/password form from /signup — Google, Microsoft, GitHub OAuth only
- Add Microsoft as a social provider (MICROSOFT_CLIENT_ID / MICROSOFT_CLIENT_SECRET / DISABLE_MICROSOFT_AUTH)
- Wire microsoftAvailable through provider checker, API contract, providers route, and all auth UI
- Hide "Continue with email" in auth modal signup view; login view unchanged
- Fix MicrosoftIcon SVG to use official brand colors and proportions

* fix(auth): remove unused useSession, guard invalid-callback warn with ref

* feat(auth): gate email signup via DISABLE_EMAIL_SIGNUP flag

* feat(auth): restore signup email form gated by NEXT_PUBLIC_DISABLE_EMAIL_SIGNUP

* refactor(auth): single DISABLE_EMAIL_SIGNUP env var controls both ui and backend

* fix(config): restore isHosted hostname check
2026-06-15 19:47:44 -04:00
WaleedandVikhyath Mondreti 4f4ff53411 fix(uploads): authorize internal file URLs before download (#5049)
* fix(uploads): authorize internal file URLs before download

downloadFileFromUrl treated any URL containing /api/files/serve/ as
trusted-internal and read the object straight from storage by key with
no access check, while every other resolution path in the file calls
verifyFileAccess. Reachable during workflow execution via file[] inputs
(type: 'url'), letting an authenticated user read arbitrary storage
objects across tenants by supplying a storage key.

Thread the caller's userId into downloadFileFromUrl and run
verifyFileAccess(key, userId, undefined, context, false) on the resolved
key before downloadFile; fail closed when no userId is present. Update
all callers (execution file inputs, tool file outputs, KB ingestion);
webhook and chat inputs already thread userId via processExecutionFiles.

* chore(uploads): log denied internal file downloads for rollout telemetry

* fix(uploads): derive internal file context from key, not query param

Cursor Bugbot flagged a context-spoofing bypass: downloadFileFromUrl
resolved context via parseInternalFileUrl, which honors a caller-controlled
?context= query param. An attacker could label a private storage key with a
world-readable context (profile-pictures/og-images/workspace-logos) so
verifyFileAccess short-circuits to granted while downloadFile still reads the
private object.

Infer context from the key only (inferContextFromKey), mirroring how
/api/files/serve resolves it; ignore the query param. Also move the userId
guard ahead of key resolution so auth failures surface first.

* docs(uploads): move context-derivation rationale into TSDoc

* fix(uploads): match internal file marker in URL path only

isInternalFileUrl matched the /api/files/serve/ substring anywhere in the
string, so a crafted URL could carry it in a query string or fragment and
skip DNS/SSRF validation. Match it in the path component only.

The raw path is checked without URL normalization on purpose: the files
parse route relies on traversal sequences surviving this check (an absolute
https://host/api/files/serve/../.. URL must classify as internal so the '..'
check rejects it, rather than being normalized to /etc/... and waved through
as external). Host is intentionally not gated — cross-tenant reads are
prevented at the storage sink by verifyFileAccess, and host-allowlisting
would break self-hosted/multi-domain deployments. Adds unit tests.

* consolidate access, billing principals

---------

Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>
2026-06-15 16:45:55 -07:00
Waleed dd32abef1e feat(jsm): add Atlassian Assets (Insight/CMDB) tools for asset management (#5072)
* feat(jsm): add Atlassian Assets (Insight/CMDB) tools for asset management

Add nine JSM Assets tools so workflows can read and write Atlassian Assets
(Insight/CMDB) objects — the foundation for keeping JSM asset tables in sync
for software/hardware asset management.

Tools (wired into the Jira Service Management block):
- jsm_list_object_schemas, jsm_get_object_schema
- jsm_list_object_types, jsm_get_object_type_attributes
- jsm_search_objects_aql (AQL search with pagination)
- jsm_get_object, jsm_create_object, jsm_update_object, jsm_delete_object

Each tool proxies through an internal route that resolves the Jira cloudId and
the Assets workspaceId, then calls the Assets API via the OAuth 2.0 (3LO)
gateway form (/ex/jira/{cloudId}/jsm/assets/workspace/{workspaceId}/v1).

Adds the CMDB OAuth scopes to the jira provider (read/write/delete cmdb-object,
read cmdb-schema/type/attribute) with descriptions, contract schemas for each
route, and block operations/subBlocks/outputs. Bumps the API-validation route
baseline for the nine new routes.

* refactor(jsm): harden Assets param coercion and response typing

- Add toOptionalInt helper so non-numeric pagination inputs never emit NaN
  into the Assets query string (startAt/maxResults/page/resultsPerPage)
- Replace Record<string, any> in mapAssetObject with typed Raw* interfaces

* fix(jsm): validate Assets workspaceId and honor `last` pagination flag

Address review findings on the Assets tools:
- Add validateAssetsWorkspaceId and guard the workspaceId in every Assets
  route before it is interpolated into the API path (mirrors the existing
  cloudId guard) — prevents a crafted workspaceId from escaping the
  workspace-scoped path
- Object schema list now falls back to the `last` flag when `isLast` is
  absent, so pagination doesn't stop early

* feat(jsm): allow overriding the auto-resolved Assets workspace

Atlassian provisions one Assets workspace per site, so workspace discovery
uses values[0] by design. For the rare multi-workspace site, expose an
advanced "Assets Workspace ID" override on the block that flows through to
every Assets operation, and document the single-workspace assumption.

* refactor(jsm): include Assets responses in the JsmResponse union

Append the nine Assets tool response types to JsmResponse for completeness
and consistency with the rest of the JSM tool surface.
2026-06-15 16:40:24 -07:00
Siddharth Ganesan d538b76eda feat(copilot): server-side mothership tool/vfs/file metrics (#5071) 2026-06-15 16:32:35 -07:00
Waleed 3e2b641e9d fix(credential-sets): stop leaking open-invite tokens to all users (#5074)
GET /api/credential-sets/invitations returned every pending, unexpired
link-only (null-email) invitation across all organizations, including the
bearer token. Any authenticated user could enumerate and accept another
org's invitation, joining its credential set (cross-tenant access).

Scope the listing strictly to invitations addressed to the caller's own
email. Open-link invites remain redeemable only via the out-of-band
/credential-account/[token] URL.
2026-06-15 16:24:15 -07:00
Waleed 39d0b56e91 refactor(table): split the 5.3k-line service.ts god-file into per-concern modules (#5069)
* refactor(table): extract row ordering, executions, and tx helpers from service.ts

Move the row position/fractional-ordering internals to rows/ordering.ts,
the row-execution (workflow-group result) internals to rows/executions.ts,
and the shared tx-timeout helpers to tx.ts. Pure code-motion — verbatim
bodies, identical behavior. service.ts: 5324 -> 4442 lines.

* refactor(table): extract row CRUD and query into rows/service.ts

Move insert/update/upsert/delete/replace/batch row writes, queryRows/
getRowById reads, and findRowMatches into rows/service.ts. Verbatim
bodies; consumers repointed; @/lib/table barrel re-exports the new module
so callers are unchanged. service.ts: 4442 -> 2788 lines.

* refactor(table): extract column/schema management into columns/service.ts

Move add/rename/delete column ops and column-type/constraint updates into
columns/service.ts. addTableColumnsWithTx stays in service.ts (table-creation
primitive) to avoid a cycle. Verbatim bodies; barrel re-exports the module.
service.ts: 2788 -> 2149 lines.

* refactor(table): extract job state machine and export jobs into jobs/service.ts

Move tableJobs reads/mapping, the job lifecycle state machine, and export-job
queries into jobs/service.ts. service.ts imports latestJob* one-way for table
metadata enrichment (no cycle). Import-data orchestration helpers stay in
service.ts for now. Verbatim bodies. service.ts: 2149 -> 1791 lines.

* refactor(table): extract workflow-group management into workflow-groups/service.ts

Move add/update/delete workflow groups + outputs and pruneStale into
workflow-groups/service.ts, preserving the dynamic backfill-runner import
(cycle-breaker). Verbatim bodies; no cycle. service.ts: 1791 -> 851 lines.

* refactor(table): extract import-job data ops into import-data.ts

Move bulk insert, schema setup, and append/replace import operations into
import-data.ts (consumed by import-runner + import route). service.ts is now
the pure table-entity module (root CRUD + shared lock/column primitives).
Verbatim bodies; no cycle. service.ts: 851 -> 664 lines (5324 at start).

* refactor(table): restore private visibility and drop dead helpers post-split

Un-export DerivedJobFields/JOB_PROJECTION/mapJobRow (file-local in jobs/service.ts,
were private pre-split) so they no longer leak into the @/lib/table barrel. Remove
dead code carried into the new modules: countTables (createTable does its own inline
count check) and the unused buildOrderedRowValues/OrderedRowValue pair.

* refactor(table): use absolute imports throughout and fix an orphaned TSDoc

Normalize all relative imports under lib/table to absolute @/lib/table/...
per the project import rule (the split modules and a few pre-existing
holdouts), and relocate the getTableById TSDoc that had drifted above
applyColumnOrderToSchema. Comment/import-only; zero behavior change.
2026-06-15 15:16:41 -07:00
Waleed 02022e9ab2 fix(providers): pin Azure OpenAI/Anthropic endpoints to validated IP (#5060)
* fix(providers): pin Azure OpenAI/Anthropic endpoints to validated IP (TOCTOU SSRF)

* fix(providers): fail closed when Azure endpoint validates without a pinnable IP

* refactor(security): drop pinned-fetch agent pool, keep per-call dispatcher

* refactor(security): make createPinnedFetch the single pinned-fetch source

* refactor(security): drop pinned-fetch agent pool; keep per-call dispatcher
2026-06-15 13:59:47 -07:00
Theodore Li 2cf51725c9 fix(execute): reject only cross-site session execution (CSRF guard) (#5068) 2026-06-15 16:19:43 -04:00
Waleed 318cb5e414 chore(deps): bump js-yaml to 4.2.0 and nodemailer to 8.0.9 in apps/sim (#5067) 2026-06-15 13:12:56 -07:00
Theodore Li d89824cbd3 Revert "fix(execute): block cross-origin session-authenticated workflow runs (#5062)" (#5065)
This reverts commit 67e02fab3a.
2026-06-15 15:55:25 -04:00
Vikhyath Mondreti 324299eef0 fix(access-control): exempt legacy blocks (#5063) 2026-06-15 12:50:16 -07:00
Theodore Li 67e02fab3a fix(execute): block cross-origin session-authenticated workflow runs (#5062)
* fix(execute): block cross-origin session-authenticated workflow runs

* fix(execute): scope session origin guard to provable cross-origin

Address review on #5062:
- Reject session-cookie execution only when provably cross-origin (Sec-Fetch-Site
  cross-site/same-site/none, or a mismatched Origin) instead of failing closed on
  absent headers. Fixes route tests that 403'd on header-less session requests, and
  reflects that this is CSRF protection, not anti-cookie-replay.
- Drop same-site from the trusted set: only same-origin is our front-end.
- Guard the Origin fallback in try/catch so a getBaseUrl() throw can't escape.
- Add a route-level cross-origin rejection test.
2026-06-15 15:39:43 -04:00
Theodore Li 18edc94b2c fix(billing): deploy modal gates on workspace entitlement, not viewer plan (#5055)
* fix(billing): deploy modal gates on workspace entitlement, not viewer plan

The deploy modal showed the upgrade wall to a free user in a PAID workspace,
because it gated on the viewer's individual plan (useSubscriptionData) while the
server gates on the workspace billed account (rolled-up plan). Add a workspace
api-execution-entitlement endpoint that mirrors isWorkspaceApiExecutionEntitled,
and gate the API/MCP/A2A tabs on it so the UI matches the server exactly.

* fix(billing): key deploy gate on URL workspaceId + refetch entitlement on open

Address review findings:
- key useWorkspaceApiExecutionEntitlement on the URL workspaceId (available on
  mount) instead of workflowWorkspaceId (null until the workflow map resolves),
  so the gate fires immediately instead of leaving the tabs ungated until then
- staleTime 0 so reopening the deploy modal refetches entitlement; a plan upgrade
  happens outside this query's invalidation graph, so the gate self-heals on open

* refactor(billing): workspace owner access state instead of bespoke entitlement endpoint

Replace the single-purpose api-execution-entitlement endpoint with a reusable
workspace-owner billing/access concept — the workspace-scoped counterpart to the
viewer-scoped useSubscriptionData:

- getWorkspaceOwnerSubscriptionAccess(workspaceId): the billed account's rolled-up
  subscription access fields (mirrors getSimplifiedBillingSummary's flag derivation)
- GET /api/workspaces/[id]/owner-billing + useWorkspaceOwnerBilling hook
- deploy modal derives its gate via the existing getSubscriptionAccessState
  (hasUsablePaidAccess) on the owner data, exactly like every other paid feature

Audited the rest of the app: no other UI gates on the viewer's plan where the
server gates on the workspace owner — programmatic execution is the only
workspace-owner-scoped feature; inbox/KB-live-sync/credential-sets all gate
consistently on both sides.

* fix(billing): deploy gate on owner isPaid, not hasUsablePaidAccess

hasUsablePaidAccess rejects past_due and billing-blocked, but the server gate
(isWorkspaceApiExecutionEntitled) allows any paid plan in an entitled status
(active or past_due). Gate on the owner's isPaid so a past_due paid workspace
isn't shown the upgrade wall while the API still works.
2026-06-15 15:36:56 -04:00
Waleed 0673e3c0f7 refactor(sim): consolidate record guards + pure utils into @sim/utils (#5061)
* feat(utils): add record guards and pure helpers to @sim/utils

Add isRecordLike (loose, non-prototype-checked record guard) and
sortObjectKeysDeep; relocate isPlainRecord (strict) and normalizeEmail
into @sim/utils so they are reusable across apps and packages. Unit tests
cover the loose-vs-strict distinction, deep key sorting, and email
normalization.

* refactor(sim): consolidate record guards and normalize helpers onto @sim/utils

Replace ~55 re-implemented loose record guards with the canonical
@sim/utils isRecordLike (and one strict site with isPlainRecord), and
dedupe three normalize clusters: sortObjectKeysDeep (sanitization +
copilot builders), normalizeToken (salesforce + servicenow triggers),
and normalizeEmail. Array-allowing guards and domain-specific normalizers
are intentionally left untouched. Pure refactor — identical predicates
and transforms, no behavior change.
2026-06-15 12:32:19 -07:00
Siddharth Ganesan cefb2dc239 improvement(mship): add enrichment tool, clean up dead tools (#5058)
* feat(mothership): add enrichment_run server tool for one-off lookups

Implement the Sim-side handler for the copilot enrichment_run tool: runs the
enrichment provider cascade for a single entity and returns the result inline,
surfacing the hosted-key cost as _serviceCost for per-round billing (matching
the media tools). Registered in the server-tool router.

Regenerate the copilot tool catalog/schemas to include enrichment_run.

* fix(mothership): remove leftover touch_plan tool references

touch_plan was removed from the copilot tool catalog earlier, but the Sim side
still referenced it. Regenerating the generated tool catalog (for enrichment_run)
synced it to the current contract and dropped the stale TouchPlan export, which
broke the build where router.ts still imported it.

Remove the dead touch_plan server tool and its test, drop its router
registration and WRITE_ACTIONS entry, simplify getServerToolRegistry (no more
beta-gated server tools), and clean up stale "use touch_plan" guidance strings.

* chore(mothership): trigger dev redeploy

* improvement(mothership): log billed cost on enrichment_run lookups

* fix(contracts): regenerate mship contracts

* fix(contracts): fix mship contracts
2026-06-15 10:57:29 -07:00
Siddharth Ganesan 17500432ef Revert "improvement(mship): clean up dead tools, add enrichments (#5056)" (#5057)
This reverts commit 6f4189f015.
2026-06-15 10:43:31 -07:00
Siddharth Ganesan 6f4189f015 improvement(mship): clean up dead tools, add enrichments (#5056)
* feat(mothership): add enrichment_run server tool for one-off lookups

Implement the Sim-side handler for the copilot enrichment_run tool: runs the
enrichment provider cascade for a single entity and returns the result inline,
surfacing the hosted-key cost as _serviceCost for per-round billing (matching
the media tools). Registered in the server-tool router.

Regenerate the copilot tool catalog/schemas to include enrichment_run.

* fix(mothership): remove leftover touch_plan tool references

touch_plan was removed from the copilot tool catalog earlier, but the Sim side
still referenced it. Regenerating the generated tool catalog (for enrichment_run)
synced it to the current contract and dropped the stale TouchPlan export, which
broke the build where router.ts still imported it.

Remove the dead touch_plan server tool and its test, drop its router
registration and WRITE_ACTIONS entry, simplify getServerToolRegistry (no more
beta-gated server tools), and clean up stale "use touch_plan" guidance strings.

* chore(mothership): trigger dev redeploy

* improvement(mothership): log billed cost on enrichment_run lookups
2026-06-15 10:41:33 -07:00
Emir Karabegandwaleed 1c8ac05808 improvement(scheduled-tasks): move recurrence into modal body as a section (#5054)
* improvement(scheduled-tasks): move recurrence into modal body as a section

Replace the footer RecurrenceControl (a row of chip dropdowns) with a
RecurrenceSection rendered between the prompt body and footer: a "Recurring"
Switch toggles one-time vs repeat, and — once on — frequency and end (never,
on a date, after N runs) are labeled ChipModalField rows aligned to the modal
header/footer gutter.

Toggling Recurring off now preserves the recurrence shape (cadence, end, and a
passed-through custom cron) and only sets frequency: 'once', so toggling back
on restores a conversationally-authored custom schedule instead of silently
rewriting it to daily.

Also restore the prompt editor's native scale (text-[15px], -0.015em tracking)
so the editor reads the same in the chat input and the task modal body.

* fix(scheduled-tasks): clear custom cron when switching frequency away from custom

The Recurring toggle restores `frequency: 'custom'` when `recurrence.cron` is
truthy, but switching the frequency dropdown away from custom kept the stale
cron on the object — so editing a custom-cron task to Daily, then toggling
Recurring off and back on, snapped it back to Custom and persisted the old cron.
Clear `cron` in the non-custom frequency branches so it is present only while
the cadence is genuinely custom (matching the type's "custom only" invariant),
making the toggle's restore signal accurate.

Also document the unreachable `once` branch in frequencyOptionFor as a
type-exhaustiveness fallback (keeps the return type without a cast).

* fix(scheduled-tasks): restore the prior cadence when re-enabling recurrence

Toggling Recurring off collapsed frequency to 'once' but toggling back on
forced 'daily' and cleared weekdays, so pausing a weekly/weekdays/monthly task
and re-enabling it silently reset it to daily. Cache the last recurring cadence
in a ref (written during render) and reinstate it on toggle-on, so a paused
"Weekly on Mon" returns as weekly. This also subsumes the custom-cron restore —
the ref remembers 'custom' across the one-time interval — so the toggle no
longer special-cases cron.

* improvement(scheduled-tasks): compose canonical modal separator, tidy imports

Replace the recurrence section's hand-rolled `h-px bg-[var(--border)]`
divider with the canonical ChipModalSeparator (now exported from the
chip-modal barrel) so the modal's hairline has a single source of truth.
Also unify loading.tsx icon imports onto the @/components/emcn barrel.

---------

Co-authored-by: waleed <walif6@gmail.com>
2026-06-15 10:25:40 -07:00
Waleed 940506ad09 feat(square): add Square integration with 34 commerce operations (#5053)
* feat(square): add Square integration with 34 commerce operations

Add a Square integration (API-key auth via personal access token) covering
payments, refunds, customers, locations, orders, invoices, catalog, and
inventory. Catalog image upload routes through an internal API endpoint using
the shared UserFile handling pattern. Adds a dedicated square-errors extractor.

* fix(square): correct catalog image part name and address review feedback

- Fix catalog image upload: Square's multipart part for the binary is `file`,
  not `image_file` (per the live API cURL examples); this would have caused
  upload failures
- Catalog image route: check response.ok before parsing, drop the unreachable
  legacy base64 path, derive MIME from the uploaded file
- Block: split the search query field per operation so placeholders match each
  endpoint's schema; parse each JSON field individually so errors name the field
- Round out coverage: complete_payment version_token; customer nickname/birthday;
  batch inventory states/updated_after/limit

* fix(square): correct canonical file param usage and revert query split

- Read the catalog image file from the canonical `params.file` (the basic/advanced
  inputs are collapsed before the params function runs) instead of the raw
  uploadFile/fileRef ids, which no longer exist at that point — fixes the
  Canonical Param Validation test and a latent upload bug
- Revert the per-operation query split: canonicalParamId is only valid for
  basic/advanced pairs under one condition. Use a single query field with a
  schema-neutral placeholder and a wand prompt that covers each search operation

* chore(square): trigger fresh review

* fix(square): single-location invoice search and guard numeric coercion

- SearchInvoices: Square's invoice filter accepts only one location, so take a
  single locationId (string) instead of an array and wrap it as
  query.filter.location_ids: [locationId]
- Block: fail locally with a clear "<field> must be a valid number" error when
  amount/limit/version/orderVersion are non-numeric instead of forwarding NaN

* fix(square): accept real booleans for autocomplete/includeRelatedObjects

Coerce these from both the dropdown's string values and actual booleans
(which can arrive via connected blocks or templated inputs), so true is not
silently flipped to false.

* fix(square): validate parsed JSON field shapes (array vs object)

parseJsonField now enforces the expected shape so a valid-but-wrong-type value
(e.g. a JSON string where an array is expected for locationIds/objectTypes/
paymentIds/catalogObjectIds/states, or a non-object for order/invoice/etc.)
fails locally with a clear message instead of a confusing Square API error.
2026-06-15 09:55:14 -07:00
Waleed 06191a7fb9 refactor(providers,executor): deepen three shallow modules (#5052)
* refactor(executor): collapse subflow node-ID logic behind a codec

Extract the ~20 scattered subflow node-ID parsing/building helpers and
their regex patterns into a single SubflowNodeIdCodec. All patterns now
live in one place; subflow-utils and execution state delegate to it.
Pure refactor — byte-identical output, ordering, and error modes.

* refactor(providers): extract StreamingExecution assembly into a factory

The near-identical streaming-response assembly (timing segments, cost,
token counts, onComplete wiring, success/logs/metadata envelope) that was
copy-pasted across ~16 providers now goes through createStreamingExecution.
Each provider injects only its stream iterable and delta extractors. Gemini
is intentionally left as-is (its assembly is structurally divergent).
Pure refactor — identical StreamingExecution shape, cost, tokens, timing,
and callback order per provider.

* refactor(providers): dedupe tool-schema wrapping into adapters

Replace the identical inline tool-schema literals repeated across ~16
providers with shared adaptOpenAIChatToolSchema / adaptAnthropicToolSchema
helpers. Content building reverts to calling the attachments builders
directly (the per-provider content seam was pure pass-through indirection).
Pure refactor — byte-identical tool-schema and content output.

* chore(skills): point add-model provider edits at shared response/tool-schema helpers

When editing provider code, reuse createStreamingExecution and the
tool-schema-adapter helpers instead of hand-rolling.

* refactor(providers): drop placeholder stream double-cast in streaming factory

Build the output object before creating the stream so the factory returns
a fully-typed StreamingExecution without 'undefined as unknown as
ReadableStream'. Keeps the strict API-validation boundary ratchet green
(double-cast count back to baseline). No behavior change: the same output
reference is mutated on drain.
2026-06-15 01:03:12 -07:00
Waleed ae075f871f Revert "fix(realtime): re-validate socket role and evict revoked collaborator…" (#5051)
This reverts commit 4ab8760ae0.
2026-06-14 22:20:12 -07:00
Waleed 4ab8760ae0 fix(realtime): re-validate socket role and evict revoked collaborators (#5050)
Socket.io authorized workflow access only at join and cached the workspace
role in presence, so a removed or downgraded collaborator kept live read/
write access until they disconnected.

- Re-validate the cached role against the permissions table on mutating
  events, bounded by a short TTL; refresh or evict on change
- Add /api/permissions-updated so the app reconciles active rooms, evicting
  revoked users (cross-pod) and refreshing downgraded roles
- Notify realtime on workspace member removal and permission changes
2026-06-14 21:19:32 -07:00