* azure devops logo on white
* generated ADO tool docs
* generated ADO tool docs
* added ADO to registries
* ADO workflow triggers
* ADO workflow triggers
* tool layer for ADO, checks passed and manual verified
* ADO workflow triggers
* block layer for ADO
* ADO icon svg
* generated docs for ADO triggers
* committing the tests for azure devops tools and blocks
* Update apps/sim/triggers/azure_devops/utils.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update apps/sim/tools/azure_devops/update_work_item.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* Update apps/sim/triggers/azure_devops/utils.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* comma syntax error patched
* azure devops: validate-integration fixes + manual description
- bgColor switched from white to Azure DevOps brand color #0078D4 (block + mdx)
- WIQL query_work_items: hydrate ALL matched IDs by chunking through batches
of 200 instead of silently truncating; check response.ok on the follow-up
fetch and surface a clear error on 4xx/5xx; trim org/project; expose
totalMatched in metadata so users can see pre-hydration count
- Add MANUAL-CONTENT-START:intro section to the azure_devops.mdx docs page
- Update unit tests for new chunking behavior and update-work-item validation
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* azure_devops: second-pass audit fixes + formatter cleanup
- Add types barrel export to tools/azure_devops/index.ts
- Normalize comment endpoint path casing (/workItems/ -> /workitems/)
- Update test assertions to match normalized path
- Biome formatter reflow across tools, triggers, registry, and docs icon
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* azure_devops: address PR review comments
- Fix bgColor #FFFFFF -> #0078D4 in integrations.json and triggers/azure_devops.mdx
- Bump File tool operationCount from 4 to 5 (Read, Fetch, Get, Write, Append)
- Apply .trim() to org/project across all 15 remaining tools (consistency with query_work_items)
- Fix Found ${data.count} -> Found ${data.count ?? items.length} fallback in list_builds, list_pipelines, list_pipeline_runs content strings
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* idemtpotency
* azure_devops: address bugbot review comments
- triggers/utils: match build.complete result case-insensitively, accept stopped/cancelled in addition to failed/canceled/partiallySucceeded so PascalCase and legacy Azure DevOps payloads aren't dropped
- get_work_items_batch: chunk comma-separated IDs into 200-batch loops with proper status checks (was failing or returning incomplete data on >200 IDs)
- Add tests for both behaviors
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* azure_devops: address additional bugbot comments
- Block update_work_item now forwards areaPath; the Area Path subblock condition expanded to include update operation
- get_build_timeline.failedRecords now also flags partiallySucceeded and succeededWithIssues, normalized case-insensitively. Output description and added a focused test
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* azure_devops: address more bugbot comments
- Webhook provider extractIdempotencyId returns null when subscriptionId or notificationId is missing/empty, preventing the literal "azure_devops:undefined:undefined" key from collapsing unrelated deliveries into duplicates
- Get Work Items Batch validates that at least one non-empty ID is supplied before issuing the API request, throwing a clear error instead of hitting an empty ids= query
- Tests cover both behaviors
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* azure_devops: pin add_comment to documented api-version 7.0-preview.3
Microsoft's Add Comments docs only publish 7.0-preview.3 (the 7.2 view falls back to the 7.0 page). Get Comments stays on the documented 7.2-preview.4. Matches what's strictly in the Azure DevOps REST API reference rather than relying on undocumented version behavior.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Marcus Chandra <mzxchandra@gmail.com>
Co-authored-by: mzxchandra <129460234+mzxchandra@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* fix(docker): restore NEXT_PUBLIC_APP_URL build arg with dummy fallback
getBaseUrl() in lib/core/utils/urls is evaluated at module load during
next build's page-data collection and throws if NEXT_PUBLIC_APP_URL is
unset. PR #4658 removed the build arg, breaking the Docker build at the
"/_not-found" page-data collection step.
Restore the dummy localhost fallback (mirroring DATABASE_URL). The CORS
fix from #4658 is preserved: next.config.ts no longer reads
NEXT_PUBLIC_APP_URL at build time, and no module-level expression
captures getBaseUrl() — every caller invokes it at request time, where
getEnv() reads the deployed container env. The dummy localhost value
cannot leak into runtime CORS response headers.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* chore(docker): trim verbose comment on build-time env args
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* feat(wiza): add Wiza integration for B2B prospect enrichment and search
* fix(wiza): coerce reveal id to string, skip empty filters in prospect search
* fix(wiza): throw on invalid JSON in advanced filter fields instead of silently dropping
* fix(security): remove localhost CORS origin, consolidate CORS in proxy
Move all /api/* CORS handling from next.config.ts to proxy.ts so the
runtime can resolve allowed origin per-request instead of baking it at
build time (which produced "Access-Control-Allow-Origin: http://localhost:3000"
with credentials:true in production).
- proxy.ts: per-route CORS policy table covering auth, MCP, form, and
workflow execute endpoints; OPTIONS preflight short-circuit; Vary:
Origin when origin is not '*'; form routes defer to route handler's
addCorsHeaders to avoid double-setting
- next.config.ts: drop all /api/* Access-Control-Allow-* headers; keep
COEP/COOP/CSP
- deployment.ts: addCorsHeaders sets Vary: Origin alongside reflected
Allow-Origin
- Dockerfile: drop NEXT_PUBLIC_APP_URL build placeholder (Zod has
skipValidation:true; build path doesn't read it)
- Remove 8 dead OPTIONS handlers and their preflight tests now that the
proxy handles preflight uniformly
* refactor(cors): consolidate API CORS into proxy as single source of truth
Move CORS for /api/chat/* and /api/form/* into the proxy policy table with
reflected-origin + credentials:false, and delete the per-route addCorsHeaders
helper. Routes no longer set CORS headers — the proxy is the only writer.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(cors): convert proxy CORS policy chain to a rule table + add tests
Replace the if/else chain in resolveApiCorsPolicy with a CORS_RULES table
so each route's policy lives in one place and is trivially scannable.
Add proxy.test.ts covering each rule and the wildcard-with-credentials
invariant.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(cors): scope embed CORS rule to /api/{chat,form}/[identifier] only
The embed policy (reflected origin, credentials:false) was matching
workspace-internal session-authed routes — /api/chat, /api/chat/manage/*,
/api/chat/validate, and the form equivalents — which need the default
credentialed policy. Tighten the matcher to the embed paths only and add
tests covering the exclusion.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor(cors): replace embed-path regex with explicit segment check
The regex form `^/api/(chat|form)/(?!manage|validate)[^/]+(/(otp|sso))?$`
was opaque on review and would silently exclude any future identifier
subroute outside the hard-coded (otp|sso) group from the embed policy.
Replace it with an imperative segment check and a named
EMBED_RESERVED_SEGMENTS Set, so the policy boundary is visible at the
top of the function and adding a reserved subpath is a one-line diff.
Add a test asserting that future identifier subroutes also get the
embed policy.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(cors): allow PUT in embed CORS policy for OTP verification
Both /api/chat/[identifier]/otp and /api/form/[identifier]/otp export
PUT for OTP code verification. The embed policy advertised only
GET/POST/OPTIONS, so cross-origin embed clients failed preflight on
verify. Add PUT and assert it in the embed policy test.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* fix(tables): type-aware SQL casts for range filters on date columns
* improvement(table): tighten filter-cast types & workspace guards
- Drop redundant tableId/workspaceId from BulkUpdateData and BulkDeleteData; service uses table.id / table.workspaceId so column metadata and DB scope can't drift apart.
- Add missing workspace-id guards to copilot user-table cases (insert_row, batch_insert_rows, update_row, batch_update_rows, rename); collapse duplicated rename check.
- Add service-level integration tests that buildFilterClause/buildSortClause receive table.schema.columns from queryRows, updateRowsByFilter, deleteRowsByFilter.
* improvement(table): cast jsonb date filters/sorts to timestamptz
::timestamp strips timezone offsets from ISO strings, making comparisons
depend on the server TimeZone setting. ::timestamptz preserves the offset
so chronological comparisons are correct regardless of server config.
* improvement(table): correct JSDoc examples for required columns arg
* improvement(table): validate range operator value types at SQL builder
* feat(findymail): add Findymail B2B contact data integration
Adds 11 tools covering verified email lookup (by name, LinkedIn, domain
roles), email verification, reverse email lookup with profile enrichment,
company info, employee discovery, phone lookup, technology stack
detection, and credit checks. Single API-key block with operation
dropdown, gradient-rendered icon, and generated docs.
* fix(findymail): handle HTTP errors and surface last_detected_at
- All 11 tools now check response.ok and return success:false with the API error message on non-2xx responses
- search_technologies now maps last_detected_at to match lookup_technologies and the shared output schema
- Restore file_v3 in docs icon-mapping (translated docs still reference it)
* improvement(findymail): exclude operation from params transform
Match the convention used by enrich/apify/box/calendly — destructure out
operation before forwarding the rest to the tool call, so the operation
key doesn't leak into the tool payload.
* fix(memory): prune toolSchemaCache and semaphores to prevent heap growth
toolSchemaCache (lib/copilot/chat/payload.ts): module-level Map keyed by
userId:workspaceId never deleted expired entries, only checked TTL on read.
With 100K+ unique user/workspace pairs each holding 50-200KB of tool schemas,
this was the primary driver of the 24MB -> 25GB heap growth observed in
CloudWatch. Add a setInterval sweep every 30s (matching the TTL) with .unref()
so it does not prevent graceful shutdown.
semaphores (lib/core/async-jobs/backends/database.ts): acquireSlot created
Semaphore entries that releaseSlot never deleted. With per-execution UUID keys
(e.g. scheduleJobId), each scheduled workflow run would add a permanent entry.
Store the concurrency limit on the Semaphore struct and delete the entry from
the Map when all slots are free and no waiters remain.
validatorCache (lib/copilot/tools/server/generated-schema.ts): validated as
bounded (93 tools x 2 schema kinds = 186 max entries, ~2-9MB). No fix needed.
isolated-vm nativeContexts: validated as deferred GC, self-healed by worker
rotation at MAX_EXECUTIONS_PER_WORKER=200. externalMB spikes trace to
concurrent isolate heaps at peak load (128MB limit x active isolates), not a
reference leak. No fix needed.
* fix(memory): prune effectiveEnvCache and instrument cache sizes in telemetry
effectiveEnvCache (lib/environment/utils.ts): same unbounded accumulation
pattern as toolSchemaCache — module-level Map keyed by userId:workspaceId
with a 15s TTL that is only checked on read, never proactively evicted.
Adds a periodic sweep matching the TTL interval with .unref().
cache-registry (lib/monitoring/cache-registry.ts): lightweight registry
so modules can expose their cache sizes to telemetry without coupling.
toolSchemaCache and effectiveEnvCache both register on module load.
memory-telemetry: emits cacheSizes in every Memory snapshot log so
CloudWatch can confirm the caches stay bounded post-deploy.
* improvement(memory): replace manual TTL Maps with lru-cache for toolSchemaCache and effectiveEnvCache
Replaces the homegrown Map + setInterval sweep pattern with LRUCache from
the lru-cache npm package, which is the standard Node.js solution for
bounded in-process caching with TTL.
Changes per cache:
- Removes manual ToolSchemaCacheEntry / EffectiveEnvCacheEntry types
- Removes setInterval sweep timers (and the .unref() boilerplate)
- Removes the two-phase promise->value entry update inside the IIFE
- Stores Promise<T> directly — in-flight and resolved states share one type
- max: 200 (toolSchemaCache) / max: 500 (effectiveEnvCache) as hard ceilings
- TTL behaviour and concurrent-request deduplication are preserved exactly
- cache-registry .size reporting works unchanged via lru-cache's .size prop
* fix(memory): remove redundant waiters guard in releaseSlot
* fix(knowledge): preserve scroll position when toggling tokenizer in chunk viewer
* fix(knowledge): skip scroll restore on initial mount of chunk editor
* chore(dev): add dev:clean script to purge Turbopack cache
* fix(security): KB fileUrl LFI, MCP/Agiloft SSRF pinning, form OTP, KB authz
* fix(otp): don't leak caught error.message; fail-closed on DB retry exhaust
- Chat/form OTP routes: replace `error.message || fallback` with generic
`Failed to process request` in 500 responses (logger still captures detail).
- otp.ts incrementOTPAttempts DB path: on MAX_RETRIES exhaustion, delete the
verification row and return `'locked'` instead of trusting a possibly-
undercounted final read.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(mcp): use undici fetch directly in pinned-fetch for typed dispatcher
Replace `globalThis.fetch` + double-cast with `undici.fetch` so the
`dispatcher` option is part of the real type contract. This guarantees
pinning won't silently break if a future runtime swaps the underlying
fetch implementation.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(build): keep agiloft/grafana tool configs client-safe
Tool config files are statically reachable from the client bundle (via
tools/registry.ts → tools/{service}/index.ts). Importing
`@/lib/core/security/input-validation.server` from these files pulled
`node:dns/promises` into the Turbopack client bundle and broke the build.
Split agiloft utils into client-safe (`utils.ts`, plain fetch + sync
`validateExternalUrl`) and server-only (`utils.server.ts`, DNS-pinned
variants). Routes that need TOCTOU protection import the pinned helpers;
the executor-side tool path falls back to sync URL validation (matches
the supabase precedent and pre-PR baseline).
Grafana update tools likewise switch from `secureFetchWithValidation`
(server-only) to inline sync `validateExternalUrl` + plain fetch.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(knowledge): case-insensitive scheme checks for fileUrl
Boundary schema accepted uppercase schemes (e.g. HTTPS://, DATA:) via the
case-insensitive http regex, but the processor's case-sensitive
startsWith('data:') / startsWith('http') / startsWith('https://') checks
rejected them with a confusing "Unsupported fileUrl scheme" error.
Aligns processor checks to the schema using case-insensitive regex per
RFC 3986 §3.1.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(mcp): annotate undici/DOM type-bridge double-casts in pinned-fetch
Strict audit was failing on two new `as unknown as` casts in pinned-fetch.ts.
They bridge DOM `RequestInit`/`Response` ↔ undici equivalents (structurally
compatible at runtime since Node's global fetch is undici) and are required
to satisfy the FetchLike contract. Annotate so they count as documented
exemptions instead of new violations.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* fix(redis): apply TLS SNI override to pub/sub clients too
Pub/sub clients in lib/events/pubsub.ts build their own ioredis instances
directly via new Redis(redisUrl, ...) because pub/sub needs dedicated
connections (can't multiplex on the shared client from getRedisClient).
That path skipped the resolveTlsOptions helper added for trigger.dev's
PrivateLink VPCE IP, so every pub/sub channel hit
'Hostname/IP does not match certificate's altnames' on connect.
Export the helper as resolveRedisTlsOptions and use it from pubsub.ts.
* refactor(redis): share connection defaults via one helper
Extract keepAlive/connectTimeout/enableOfflineQueue + TLS SNI into a
single getRedisConnectionDefaults helper. Main client and pub/sub
clients both spread it; caller-specific retry/timeout policy stays
per-caller (pub/sub still needs maxRetriesPerRequest: null and a
different retry strategy for SUBSCRIBE).
* fix(pubsub): surface TLS config errors instead of silently degrading
resolveRedisTlsOptions (via getRedisConnectionDefaults) throws if
REDIS_TLS_SERVERNAME is missing for an IP-based rediss:// URL. Calling
it inside the constructor let createPubSubChannel's try/catch swallow
the error and fall back to in-process EventEmitter — silent
cross-replica pub/sub breakage in prod. Resolve defaults before the
try so config errors propagate; only catch genuine runtime construction
failures.
* feat(redis): allow TLS SNI override for IP-based REDIS_URL
When trigger.dev's hosted workers reach our ElastiCache via PrivateLink,
their REDIS_URL contains the VPCE-assigned IP, not a DNS name. Default
ioredis TLS verification fails because the ElastiCache cert is issued for
the cluster's DNS, not the IP.
Add REDIS_TLS_SERVERNAME env var; when REDIS_URL is rediss:// + IP host,
pass `tls: { servername }` to ioredis so cert hostname verification
matches against the DNS name instead. Throws at client construction if
REDIS_TLS_SERVERNAME is unset in this scenario (fail fast — no silent
TLS bypass).
No-op for in-VPC connections (DNS host), so the always-on Sim app keeps
using default verification.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(confluence-schemas): expose extendable bases before .superRefine
confluenceCommentScopedSchema and confluenceBlogPostScopedSchema were
built with .extend(...).superRefine(...). superRefine returns a
ZodEffects which has no .extend method, so the three downstream
.extend() calls (confluenceUpdateCommentBodySchema,
confluenceGetBlogPostBodySchema, confluenceUpdateBlogPostBodySchema)
threw at module-init time.
Next.js lazy-loads route code per-request and never executed this
top-level chain, hiding the issue. Trigger.dev's bundler eagerly
evaluates all task-reachable modules at startup, which is why the
trigger.dev deploy surfaced it as "confluenceCommentScopedSchema.extend
is not a function" across every background task that transitively
imports this file.
Fix: introduce un-superRefined base schemas and use them as the .extend
target downstream; apply superRefine after each .extend so validation
behavior is preserved for every consumer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(storage-transfer): use z.string().url() for Zod 3 compat
z.url() is Zod 4 top-level syntax. The hoisted node_modules/zod
resolves to v3.25.76 (despite apps/sim/package.json declaring 4.3.6 —
a workspace resolution conflict), so z.url is undefined at runtime.
Trigger.dev's bundler eagerly evaluates all task-reachable modules at
startup and hits this with `external_exports.url is not a function`.
Next.js dev only evaluates routes per-request so the call site never
fires.
Quick fix: revert to the chained .string().url() form which works on
both Zod 3 and Zod 4 (deprecated in 4 but still supported). The
underlying version-resolution conflict is a separate cleanup.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(mongodb-schemas): expose extendable base before .refine
mongoConnectionBodySchema was built with z.object(...).refine(...). Five
downstream schemas (mongodbQueryBodySchema, mongodbExecuteBodySchema,
mongodbInsertBodySchema, mongodbUpdateBodySchema, mongodbDeleteBodySchema)
.extend() that result, which threw at module-init in the trigger.dev
bundle (same root cause as the confluence and storage-transfer fixes:
.refine returns ZodEffects with no .extend method, and the resolved
zod is v3 even though package.json declares v4).
Fix: keep the un-refined mongoConnectionBaseSchema for downstream
.extend() targets. The pairing-validation refine isn't reattached
because the downstream extensions were never actually evaluating it
(module init threw before they could).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(redis,mongodb): address PR review comments
- redis.ts: move resolveTlsOptions call outside the try/catch in
getRedisClient so config errors surface instead of being swallowed
into a silent null return.
- mongodb.ts: re-attach mongoUsernamePasswordPaired .refine after each
of the five downstream .extend()s. Mirrors the confluence pattern
and restores the pairing constraint that the original chain dropped.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Toggling a block's locked state is UI metadata and should not register
as a workflow drift/diff. Strip locked from hasBlockChanged,
computeFieldDiff, the compare.ts blockFields list, and from
extractBlockFieldsForComparison so it's also excluded from the
normalized stringify-based block equality check used by drift detection
and hashing.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* fix(wait): poll partially_resumed rows so chained waits resume
The chained-pause flow leaves a row in 'partially_resumed' status (wait1 done, wait2 still waiting). The poll's WHERE filter only matched 'paused', so wait2 was never picked up. Include 'partially_resumed' in the filter.
* feat(wait): make in-process threshold env-overridable for local testing
Adds WAIT_INPROCESS_MAX_MS env var (default 300000ms = 5 min). Lower it locally (e.g. 5000) to exercise the suspend/cron-resume path with short waits.
* feat(wait): add Suspend Workflow toggle, restore 5-min in-process default
* fix(wait): address bot review — setNextResumeAt + suspend unit default
- setNextResumeAt now matches paused OR partially_resumed; otherwise the
cron poller can't null nextResumeAt after dispatching a chained-wait
row, so it keeps reappearing in every poll batch until execution ends
(flagged by both greptile and bugbot)
- Suspend mode now defaults missing timeUnitLong to 'minutes' instead of
falling back to 'seconds' and immediately erroring (flagged by bugbot)
* improvement(wait): hint the wait-amount cap on the input
Restores the pre-#4331 description on the Wait Amount field so the limit is visible before submit instead of only at runtime. Mentions both the 5 min default and the 30 day cap with Suspend Workflow.
* refactor(wait): rename Suspend Workflow toggle to Async
* fix(wait): reword cap errors to say 'async mode'
* feat(workflows): add GET /workflows/[id]/executions/[executionId] status endpoint
Normalized status (pending|running|paused|completed|failed|cancelled)
across workflowExecutionLogs and pausedExecutions in a single response.
Surfaces paused-state details (resumeAt, pauseKind, blockedOnBlockId)
when a row exists in pausedExecutions, and the error string for failed
runs. finalOutput is opt-in via ?includeOutput=true.
* feat(workflows): support ?selectedOutputs= on execution status endpoint
Returns per-block outputs filtered by selectedOutputs paths (same
shape as the execute endpoint). Reads from executionData.traceSpans,
walks children recursively, and resolves dot-paths into each block's
output. Bare blockId returns the full output.
* fix(wait): drop dead tooltip prop on Async switch
Switch sub-blocks return null from renderLabel (sub-block.tsx:238), so
the tooltip never reached the user. The trade-off explanation already
lives in longDescription and bestPractices. Flagged by bugbot.
* docs(api): document GET /workflows/[id]/executions/[executionId]
Adds the WorkflowExecutionStatus schema and the getWorkflowExecution
operation to the OpenAPI spec, including completed/paused/failed
response examples and the includeOutput + selectedOutputs query params.
Registers the page in the Workflows section of the API reference.
* docs(api): tighten getWorkflowExecution descriptions
* improvement(redis): strip idempotency body and cap mothership stream zsets
* chore(redis): trim verbose comments on idempotency body-strip
* test(buffer): pin exact ZREMRANGEBYRANK stop arg
Pinning -5_001 (= -(DEFAULT_EVENT_LIMIT) - 1) so the off-by-one
boundary is directly validated; expect.any(Number) would have passed
a wrong formula like -eventLimit.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* chore(utils): migrate to shared random/ID utilities and add enforcement linting
- Replace all Math.random(), crypto.randomUUID(), crypto.randomBytes(), nanoid, and uuid usages with shared @sim/utils/random and @sim/utils/id helpers across 72 files
- Add new @sim/utils exports: deepClone, omit, filterUndefined (object), truncate (string), backoffWithJitter, parseRetryAfter (retry), getErrorMessage (errors)
- Sweep all getErrorMessage, sleep, deepClone callsites across 500+ files to use shared utilities
- Add Biome noRestrictedImports rule to catch nanoid, uuid, and crypto named imports at lint time
- Add scripts/check-utils-enforcement.ts to catch Math.random and crypto.* global property access
- Add check:utils script to package.json
* chore(utils): replace deepClone wrapper with structuredClone built-in
deepClone() was a one-line wrapper around structuredClone(), which is
universally available in Node 17+ and all modern browsers. Removing the
abstraction reduces indirection and means contributors don't need to
learn a project-specific name for a well-known built-in.
- Remove deepClone from packages/utils/src/object.ts and index.ts
- Replace all 17 call sites with structuredClone() directly
- Update check:utils script suggestion text
- Update CLAUDE.md and global.md docs
* fix(utils): add missing biome noRestrictedImports rule and correct truncate docs
- Add noRestrictedImports to biome.json under style — bans nanoid and uuid
package imports at lint time (crypto.randomUUID/randomBytes are caught by
the check:utils grep script which handles global property access)
- Correct truncate() TSDoc and parameter name: sliceLength makes it clear
that total output length is sliceLength + suffix.length, matching the
behavior all callers were already written to expect
* fix(utils): add missing getErrorMessage imports at 4 call sites
The sweep agents added getErrorMessage calls without the corresponding
import in 4 files, causing test failures. Added the missing imports.
* fix(utils): fix build errors from getErrorMessage sweep and retry.ts Turbopack issue
- Fix retry.ts cross-file import: Turbopack cannot resolve './random.js' for
internal package imports; inline the jitter crypto call directly
- Add missing getErrorMessage imports to 32 files where the sweep added calls
without the corresponding import (caught by type-check and test runs)
- Remove accidental getErrorMessage import from crowdstrike/query/route.ts
which has its own domain-specific getErrorMessage for parsing CrowdStrike's
JSON error format
- Fix use-sub-block-value.ts type error from structuredClone narrowing:
add 'as T' cast at emitValue callsite (safe — valueCopy is always a
structural copy of newValue)
* fix(tools): use toError in crowdstrike catch block instead of local getErrorMessage
The catch block was calling the local getErrorMessage function which
parses CrowdStrike API JSON responses, not JavaScript Error objects.
Use toError(error).message to correctly extract the message from a
caught value in this context.
* improvement(providers): align attachment dispatch to vendor SDK types
Post-merge audit of #4610 surfaced three follow-ups:
1. xAI Grok vision was blocked. Grok runs through the OpenAI-compatible
chat-completions endpoint, so removing xAI from UNSUPPORTED_FILE_PROVIDERS
and routing it through the image-only branch restores image attachments
on vision models.
2. Azure OpenAI chat-completions deployments blocked any file attachment.
Added a per-message image_url parts path; documents still require the
Responses API endpoint and throw a clear, actionable error.
3. Wire shapes were loosely typed (Record<string, unknown> arrays).
Replaced with `satisfies` clauses against each vendor SDK union at every
push site: OpenAI Responses/Chat, Anthropic ContentBlockParam, Gemini
Part, Bedrock ContentBlock members. AnthropicImageMediaType now derives
from Base64ImageSource['media_type'] so it tracks SDK updates.
Also collapsed the validation cascade into an exhaustive switch with
`never` enforcement, and dropped the redundant per-provider
formatMessagesForProvider call from xai/index.ts (providers/index.ts
already runs the dispatcher centrally).
* fix(providers): restore getProviderAttachmentMaxBytes export and xAI message dispatch
- Restore `getProviderAttachmentMaxBytes` — still consumed by agent-handler.ts
for per-provider attachment size limits in file hydration
- Restore `formatMessagesForProvider(allMessages, 'xai')` — providers/index.ts
does NOT dispatch centrally on this branch; each OpenAI-compat provider
formats its own messages. Without it, xAI Grok vision drops image attachments
* fix(providers): tighten ResponsesInputItem content type to SDK ResponseInputContent
Build fix: buildOpenAIMessageContent returns ResponseInputContent[] which
isn't assignable to Record<string, unknown>[] (ResponseInputText lacks an
index signature). Align the type to the SDK shape.
- Guard completeWithError against overwriting a cancelled execution status — cancel route writes cancelled to DB optimistically, but a block error racing the 500ms Redis check could finalize with failed before the engine detects cancellation
- Add tests covering the guard: cancelled DB status skips the write, non-cancelled proceeds normally, DB failure falls through to cost-only fallback, and subsequent attempts are deduped after guard marks session complete
- Move ImpersonationBanner from workspace root into components/ folder
* v0.6.29: login improvements, posthog telemetry (#4026)
* feat(posthog): Add tracking on mothership abort (#4023)
Co-authored-by: Theodore Li <theo@sim.ai>
* fix(login): fix captcha headers for manual login (#4025)
* fix(signup): fix turnstile key loading
* fix(login): fix captcha header passing
* Catch user already exists, remove login form captcha
* feat(files): folders + vfs update
* address comments
* address comments
* cleanup unnused code
* address comments
* perf improvements
* address next set
* cycle detect
* error handling
* path improvements
* cleanup, best practices
* react query best practices: targeted invalidation, optimistic updates, key factory hierarchy
- Add workspaceLists(workspaceId) intermediate key level to both workspaceFilesKeys
and workspaceFileFolderKeys so invalidation targets only the affected workspace
instead of all workspaces
- Replace all lists() invalidation calls with workspaceLists(workspaceId) across
every mutation (upload, rename, delete, restore, update content, folder mutations)
- Add optimistic updates to useRenameWorkspaceFile and useUpdateWorkspaceFileFolder
with onMutate snapshot, onError rollback, onSettled reconciliation
- Move storage key into the content() factory as optional param so query keys are
always built through the factory (useWorkspaceFileContent, useWorkspaceFileBinary)
- Fix AnimatePresence wrapping in FilesActionBar so exit animation fires on deselect
- Fix ResourceColGroup to use percentage weights instead of pixel widths to prevent
horizontal scroll on narrow viewports
* add shift-click range selection and selection-aware context menu for files
- Extend SelectableConfig.onSelectRow with optional shiftKey param; DataRow captures shiftKey before onCheckedChange fires via a ref so the Radix Checkbox interaction chain stays intact
- Implement shift-click range selection in files.tsx using lastSelectedIndexRef; tracks last-selected index in visibleRowIds to compute the range
- Reset lastSelectedIndexRef on deselect and select-all
- Add selectedCount prop to FileRowContextMenu; hide Open and Rename when multiple items are selected, show "Delete N items" / "Download N items" labels in multi-select mode
* add Move submenu to file context menu and fix shift-click anchor update
- Add nested Move submenu to FileRowContextMenu using DropdownMenuSub/SubTrigger/SubContent; shows available folders filtered by selection, converts '__root__' -> null for moving to the root level
- Add handleContextMenuMove in files.tsx that calls moveItems.mutateAsync directly (no modal) and clears selection on success
- Fix shift-click range selection: update lastSelectedIndexRef after range select so chained shift-clicks extend from the new anchor point correctly
* fix move submenu: use folder names with tree-ordered indentation instead of stale paths
- Compute folder depth from parentId chain client-side (avoids stale server-computed path field)
- Tree-order folders so parents appear before their children, sorted by sortOrder then name
- Show folder.name instead of folder.path so optimistic renames are reflected immediately
- Indent each folder by depth * 12px in the submenu so po/shit renders as 'shit' indented under 'po'
- MoveOption gains optional depth field; contextMenuMoveOptions is a separate memo from moveFolderOptions (modal keeps its existing path-label behavior)
* fix shift-click anchor drift and remove dead stopPropagation constant
- Remove dead stopPropagation const in resource.tsx (replaced by handleSelectRowClick)
- Reset lastSelectedIndexRef when visibleRowIds changes so search/filter/folder navigation doesn't leave a stale anchor that produces wrong ranges on the next shift-click
- Update lastSelectedIndexRef in handleRowContextMenu when right-clicking resets selection to a single item, so the anchor matches the newly-selected row
- Add visibleRowIds to handleRowContextMenu deps (now reads it to compute anchor index)
- Remove moveItems.mutateAsync from handleContextMenuMove deps per project convention (.mutateAsync is stable in TanStack v5)
* complete workspace files feature: audit logs, posthog events, folder restore, empty state, keyboard shortcuts, storage indicator, breadcrumb rename
- Audit + PostHog: wire file_renamed, file_deleted, file_moved, file_bulk_deleted, folder_created, folder_renamed, folder_deleted, folder_moved events to all file/folder API routes
- Add AuditAction.FOLDER_UPDATED, FILE_MOVED, FOLDER_MOVED to audit types
- Folder restore: server function, contract, API route (POST /files/folders/[folderId]/restore), hook (useRestoreWorkspaceFileFolder), Recently Deleted integration with new File Folders tab
- Empty state: contextual emptyMessage passed to <Resource> based on search/filters/folder context
- Keyboard shortcuts: Delete/Backspace deletes selection, Escape deselects, Cmd+A selects all (list view only, input-aware guard)
- Storage indicator: useStorageInfo drives compact "used / limit" display in file list header via leadingActions
- Breadcrumb rename: current folder breadcrumb gains Rename dropdown + inline editing via breadcrumbRename (useInlineRename)
- Resource: thread leadingActions prop from ResourceProps to ResourceHeader
* cleanup: accessibility, emcn design tokens, react best practices across workspace UI
- Add sr-only ModalDescription to dialogs/modals for accessibility
- Replace hardcoded colors and z-indices with design token CSS variables
- Apply emcn design review fixes across tables, knowledge, logs, settings, workflows
* fix audit and posthog: FOLDER_RESTORED action on restore, fire folder_moved event separately from file_moved
* sidebar: add Files section with nested folder tree; polish move UX and cleanup
- Files section in sidebar shows folder/file tree with expand/collapse,
matching Workflows section structure; collapsed sidebar shows flyout menu
- Move action bar now uses nested DropdownMenuSub tree instead of flat modal
- Context menu and action bar share renderMoveOption from move-options.tsx
- FolderInput added to emcn icons barrel; all FolderInput imports migrated
- Drag ghost uses CSS vars (--border, --shadow-medium, --z-toast)
- Selection pruning converted from useEffect to render-time comparison
- Keyboard listener stabilized with handleBulkDeleteRef pattern
- toError() used consistently in restore and move route handlers
* remove Files section from sidebar
* restore Files nav item in sidebar workspace section
* fix infinite re-render on files page - revert selection pruning to useEffect
* add filefolder resource type for ingesting workspace file folders
* export filefolder tree types; add toast feedback for file/folder mutations
* regenerate migration as 0208 after rebase onto staging
* add workspaceFileFolder to schema mock
* add FILE_MOVED, FOLDER_MOVED, FOLDER_UPDATED to audit mock
* add filefolder ChatContext kind and wire through schema and resolver
* add filefolder to AgentContextType
* add filefolder to chat context kind registry; fix resolver to use workspaceFiles table
* add .deepsec to gitignore
* cleanup: effect, emcn tokens, mutation error handling
- Replace selection-pruning useEffect with inline state adjustment during render
- Fix drag overlay using invalid --accent HSL token → --brand-secondary; z-50 → z-[var(--z-dropdown)]
- Move static inline styles on context menu trigger div to className
- Add missing onError toast to useUpdateWorkspaceFileFolder, useRestoreWorkspaceFileFolder, useRestoreWorkspaceFile
* lint
* fix: remove duplicate handleCopilotStopGeneration from rebase
* feat(copilot): folder-aware file context in WORKSPACE.md
* feat(copilot): add move operation to file manage API
* fix(files): make targetFolder optional in move file contract
* perf(files): parallelize buffer fetches, fix N+1 folder queries, stabilize drag useMemo
- download route: fan out all fetchWorkspaceFileBuffer calls with Promise.all
before zip assembly so 100 files resolve in one round-trip instead of sequentially
- getWorkspaceFileFolder: replace per-ancestor SELECTs with a single workspace-wide
folder load + buildWorkspaceFileFolderPathMap, making depth irrelevant to query count
- ensureWorkspaceFileFolderPath: pre-load all workspace folders in one SELECT before
the segment loop; resolve existing segments from an in-memory map; only hit the DB
to CREATE missing segments; conflict retry path preserved and also updates the map
- files.tsx rowDragDropConfig: move activeDropTargetId into a ref so the useMemo
does not recompute on every drag-over event
* fix(files): remove files/ path stripping, fix stale path in optimistic update
- splitWorkspaceFilePath: remove the unconditional .replace(/^files\//, '')
that clobbered paths for files inside a folder literally named "files"
- useUpdateWorkspaceFileFolder: when a name update is in flight, recompute
the path field for the renamed folder (replace last segment) and propagate
the new prefix to all descendant folders so breadcrumbs stay correct
during the optimistic window
* fix(files): revert broken ref opt, clean 409 on restore, null parentId on orphaned restore
- files.tsx: revert the activeDropTargetId ref optimization — the ref doesn't
trigger re-renders so the drop-target highlight never updated during drag;
activeDropTargetId is back in state and in the rowDragDropConfig deps
- restore/route.ts: catch Postgres 23505 unique-constraint violation and
return a clean 409 instead of leaking the raw error as 400
- restoreWorkspaceFileFolder: check if the parent folder is still archived
before restoring; if it is, restore to root (parentId: null) so the folder
is never orphaned under an archived parent
* feat(search): show folder path for files in cmd-k modal, strip extraneous comments
- FileItem interface with folderPath?: string[] added to search modal utils
- MemoizedFileItem component renders folder breadcrumb identically to
MemoizedWorkflowItem — truncated path segments on the right with / separators
- FilesGroup rewritten as a dedicated memo component (was createIconGroup factory)
so it accepts FileItem[] and includes folderPath segments in the search value
- searchModalFiles in sidebar splits f.folderPath string into string[] segments
- search-modal.tsx typed to FileItem and includes folderPath in filterAndSort
- Remove self-explanatory "Phase 1" section label from download route
- Remove redundant TSDoc on the unique index in db schema
* fix(workspace-files): audit fixes — transaction, status codes, contract refinements, guards
* fix(vfs): pass folderPath separately so buildWorkspaceMd groups files correctly
* fix(types): narrow unknown fileInput with Record cast after object guard
* fix(routes): replace instanceof Error with toError() across new workspace file routes
* improvement(files): cleanup pass — remove unnecessary useCallbacks, consolidate emcn icon imports
- Remove useCallback from 5 drag-event handlers in DataRow (passed to native <tr> elements, no observer)
- Remove stable useCallback fns from 3 useMemo deps arrays in files.tsx (editingId/editValue remain)
- Merge all @/components/emcn/icons subpath imports into barrel (files.tsx, action-bar, file-row-context-menu)
* fix(files): apply activeSort to folders, reject drop onto current parent folder
- visibleFolders now respects activeSort column (name/updated/created) and direction
so folder ordering stays consistent with file ordering
- isInvalidDropTarget now returns true when all dragged items are already direct children
of the target folder, preventing a no-op move mutation
* fix breadcrumb
* add new tools to rename, create, delete folders
* move more ui actions into orchestration dir
* address comments
* fix params
* fix tests
* address comments
* improve error codes
* address comments
* address more nits
* fix mcp server error code
---------
Co-authored-by: Theodore Li <theodoreqili@gmail.com>
Co-authored-by: waleed <walif6@gmail.com>
* improvement(gmail): replace custom html-to-text regex with html-to-text library
Resolves 4 CodeQL alerts on htmlToPlainText (incomplete tag/entity handling,
unsafe regex backtracking). Delegates to the html-to-text npm package already
used by the outlook polling trigger and the mail/send route.
* improvement(gmail): match outlook selectors config, add nbsp/anchor tests
Aligns html-to-text options with apps/sim/lib/webhooks/polling/outlook.ts:
suppress anchor hrefs when identical to text, drop bare # anchors, skip
img/script/style content. Adds tests for nbsp preservation and anchor
behavior.
* fix(gmail): send emails as multipart/alternative so they render full-width
* fix(gmail): decode & last in htmlToPlainText to avoid double-decoding compound entities
* fix(gmail): encode body parts as base64 and decode numeric HTML entities
* docs(uploads): clarify QUOTA_EXEMPT_STORAGE_CONTEXTS logs entry in JSDoc
* fix(date-picker): eliminate infinite re-render on re-open with existing selection
The useEffect that syncs picker state on open had initialStart and
initialEnd — Date objects computed on every render — in its dependency
array. Because Object.is returns false for any two distinct Date
instances, the effect fired on every render when open=true, calling
setRangeStart/setRangeEnd and triggering another render, producing an
infinite loop that crashed the page.
Fix: compute start and end as local variables inside the effect and
use the stable string props (props.startDate, props.endDate) as deps
instead.
Also removes the redundant typeof fileSize === 'number' guard in the
multipart quota check — fileSize is z.number() (required) in the
contract so it can never be undefined at that point.
* refactor(date-picker): comprehensive cleanup and reliable crash fix
The previous fix still had derived Date objects in useEffect deps.
Object.is(new Date(), new Date()) === false, so any Date in deps causes
the effect to run every render, reproducing the infinite loop on
re-open with existing time selection.
Key changes:
- useEffect deps now use only stable primitives (startDate, endDate strings)
and compute Date values inside the effect — eliminating the loop
- Replace `rest as any` with a FlatDatePickerProps merged type for safe,
typed destructuring across the discriminated union
- Remove initialStart/initialEnd render-scope variables; compute inline or
inside effects to keep derivation local to each use site
- Callbacks use destructured props (onChange, onRangeChange, etc.) instead
of props.x references
- Remove verbose TSDoc on internal callbacks — names are self-documenting
- Preserve all existing JSX structure and CalendarMonth logic unchanged
* fix(security): supabase rpc path validation, ssh stream byte cap, storage quota coverage
* fix(security): scope execution log writes to owning workflow; add env-var workspace membership guard
Closes two cross-tenant vulnerabilities:
1. Workflow log cross-tenant write (route.ts + logging-session.ts):
- Route: SELECT before creating LoggingSession to verify executionId belongs
to the claimed workflowId; reject with 404 if owned by a different workflow.
- LoggingSession: add workflow_id to all UPDATE/SELECT WHERE clauses
(raw SQL marker queries, flushAccumulatedCost, loadExistingCost) so
writes are a no-op if executionId was somehow injected.
2. Env-var workspace membership guard (environment/utils.ts):
- getPersonalAndWorkspaceEnv now calls checkWorkspaceAccess when workspaceId
is provided; throws if the userId is not a member, preventing any future
caller from reading another workspace's decrypted secrets without
explicit membership verification at the call site.
* fix(security): remove fileSize > 0 quota bypass gate; exempt logs context from quota
* chore: remove extraneous inline comments
* fix(security): scope markExecutionAsFailed UPDATE by workflowId; thread workflowId through HITL callers
* fix(security): add personal credential ownership check in sharepoint site route; scope markExecutionAsFailed by workflowId
* fix: remove logs from user-accessible upload contexts; restore distinct biome .next glob
* fix(sharepoint): migrate site route to authorizeCredentialUse
The previous fix only checked userId equality for personal credentials and
workspace membership (via getUserEntityPermissions) for workspace credentials.
authorizeCredentialUse additionally enforces credentialMember access for
workspace-scoped credentials, matching the standard pattern used by all
other tool selector routes.
* fix(logging): make workflowId required in markExecutionAsFailed
Making workflowId optional left a footgun — future callers could silently
omit it and the WHERE clause would degrade to executionId-only, losing the
cross-tenant scoping guarantee. All callers already supply workflowId, so
making it required (with string | undefined for the middle params to keep
call sites unchanged) closes the gap without touching any caller.
* test(security): add tests for cross-tenant log guard, quota bypass fix, and workflowId scoping
- log/route.test.ts: verifies cross-tenant executionId guard returns 404
when the execution belongs to a different workflow, and passes for same
workflow or fresh executions
- multipart/route.test.ts: verifies fileSize:0 no longer bypasses quota
check and that the logs context is rejected at the endpoint level
- logging-session.test.ts: verifies markExecutionAsFailed scopes by both
executionId and workflowId, and that the instance method forwards workflowId
* fix(lint): move IconComponent outside ToolInput to fix noNestedComponentDefinitions
* fix(logging): scope completeWithCancellation and completeWithPause reads by workflowId
Both SELECT queries that check execution status before writing a
terminal result were only filtering on executionId. Adds workflowId
to the WHERE clause so all seven reads and writes in LoggingSession
consistently scope by (workflowId, executionId).
* fix(integrations): gdrive trashed search, slack blocks-with-file, slack get_message ts
- Google Drive search/list: skip default `trashed = false` when user query
already specifies a `trashed = ...` predicate, so trashed-file searches work.
- Slack send-message with files: forward `blocks` through to
`files.completeUploadExternal` so Block Kit renders when files are attached.
- Slack get_message: switch from `conversations.history` (oldest lower-bound
returned the next message after) to `conversations.replies` with `ts=`
for exact-match lookup, plus a defensive ts-equality guard and clearer error.
* fix(google_drive): revert list.ts trashed guard — query is plain text, not gdrive syntax
* fix(slack): omit initial_comment when blocks present so Block Kit actually renders on file uploads
* improvement(scheduler): drain in chunks instead of a single capped claim
Replaces the fixed MAX_CRON_CLAIMS (200) with a chunked drain loop:
claim WORKFLOW_CHUNK_SIZE + JOB_CHUNK_SIZE per iteration, process via
Promise.allSettled, repeat until both claim queries return empty or
MAX_TICK_DURATION_MS elapses. Throughput is no longer bounded by a
static per-tick ceiling; it scales until DB or trigger.dev is the
limit. Per-iteration chunk size still bounds row-lock set and fan-out
concurrency.
Extracts processScheduleItem and processJobItem so the loop body stays
readable. Existing claim semantics (FOR UPDATE SKIP LOCKED, lastQueuedAt
as the claim signal, staleness reclaim) are unchanged.
* improvement(scheduler): skip claim once a queue is exhausted and drop workflowUtils non-null assertion
Addresses Greptile review on PR #4578:
- track per-queue exhaustion when a claim returns fewer than CHUNK_SIZE
rows; subsequent iterations skip the claim query for that queue. Saves
one DB round-trip per iteration once one queue drains while the other
is still working.
- narrow workflowUtils to a local const inside the loop body so the
schedule processing branch only runs when the import has completed.
Removes the misleading non-null assertion.
* v0.6.29: login improvements, posthog telemetry (#4026)
* feat(posthog): Add tracking on mothership abort (#4023)
Co-authored-by: Theodore Li <theo@sim.ai>
* fix(login): fix captcha headers for manual login (#4025)
* fix(signup): fix turnstile key loading
* fix(login): fix captcha header passing
* Catch user already exists, remove login form captcha
* improvement(db): add session statement/lock timeouts; simplify KB doc tx
* fix(knowledge): close soft-delete TOCTOU on KB document insert
Fix the race the bots flagged: KB delete is soft (`deletedAt = now`) so
the FK can't catch a concurrent KB delete between the existence check
and the document insert.
- Add `insertDocumentsIfKbAlive` helper that gates the insert on
`EXISTS(SELECT 1 FROM knowledge_base WHERE id=$kb AND deleted_at IS NULL)`
in the same statement via INSERT...SELECT...WHERE EXISTS. Atomic at the
MVCC snapshot — no transaction, no row lock.
- Use jsonb_to_recordset to declare column types once, avoiding per-param
casts for nullable columns.
- Wire into both `createDocumentRecords` (bulk) and `createSingleDocument`.
- Keep the upfront KB existence check as a fast-path early-out for the
common case; the atomic insert is the race guard.
---------
Co-authored-by: Waleed <walif6@gmail.com>
Co-authored-by: Siddharth Ganesan <33737564+Sg312@users.noreply.github.com>
Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com>
