mirror of
https://github.com/simstudioai/sim.git
synced 2026-09-24 15:45:35 +08:00
d5c2ead5d4a2c66bee5de7f33a01b70ab2f30abc
4565
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Waleed
|
d5c2ead5d4 |
fix(console): match child-workflow inner blocks by instanceId when reconciling dropped SSE events (#4575)
* fix(console): match child-workflow inner blocks by instanceId when reconciling dropped SSE events * fix(console): drop noisy warn when reconcile finds no matching entry |
||
|
Waleed
|
6503671102 |
fix(security): harden findings — path traversal, SSRF, IDOR, file auth, credential access (#4571)
* fix(security): harden HIGH deepsec findings across multiple attack surfaces
- Supabase tools (get_row, delete, update): validate table name with strict
identifier regex and encodeURIComponent to prevent LLM-controlled path
traversal to admin endpoints; add missing empty-filter guard to update
matching the delete.ts pattern
- SFTP/SMTP/SharePoint upload routes: add verifyFileAccess ownership check
before downloadFileFromStorage, matching the WordPress reference pattern;
rejects files the requesting user does not own with 404
- Gmail labels, OneDrive folders, Wealthbox items (×2): replace bare
resolveOAuthAccountId + workspace-only membership check with
authorizeCredentialUse which enforces credentialMember table; use
credentialOwnerUserId for token refresh instead of bare accountRow.userId
- A2A utils: thread pre-resolved IP from validateUrlWithDNS into A2A SDK
via pinnedFetch (secureFetchWithPinnedIP) for JsonRpcTransportFactory,
RestTransportFactory, and DefaultAgentCardResolver, closing the TOCTOU
DNS rebinding window
- SSH utils: cap stdout/stderr accumulation at 16 MB with truncation marker
to prevent OOM from unbounded command output
- Form DELETE route: replace db.delete() with db.update({archivedAt}) for
true soft delete matching the schema's archivedAt column
- Workflow admin import: fix Array.isArray() guard that silently dropped
all variables (export format is Record, not Array)
- Multipart upload: apply checkStorageQuota and MAX_WORKSPACE_FILE_SIZE to
mothership context, closing the quota bypass for workspace-scoped storage
* fix(security): eliminate workspace env lost-update race with atomic JSONB ops
PUT: use `variables || excluded.variables` in onConflictDoUpdate so
concurrent writes merge atomically in the DB instead of last-writer-wins
at the application layer.
DELETE: replace the read-modify-write upsert with a single UPDATE that
removes keys via the JSONB `-` operator, preventing concurrent deletes
from resurrecting previously-removed secrets.
* fix(security): address audit findings from security fix review
- SMTP send: restructure attachment loop from Promise.all to sequential
for...of so verifyFileAccess denial returns 404 instead of propagating
as a generic 500 via the SMTP error classifier
- Supabase tools: extend table-name validation and encodeURIComponent to
the five previously missed tools — insert, upsert, count, query,
text_search — completing coverage across all nine Supabase tools
- Credential routes: remove unnecessary `request as any` casts in Gmail,
OneDrive, and Wealthbox routes; authorizeCredentialUse already accepts
NextRequest directly
- Form soft delete: also set isActive=false alongside archivedAt so that
any future code paths querying by isActive see a consistent state
- SSH utils: fix exit code fallback from 0 to -1 so an abnormally closed
connection that supplies no exit code is not reported as success
- Workspace env: capitalize EXCLUDED.variables in the onConflictDoUpdate
set clause to make the pseudo-table reference unambiguous
* fix(security): address PR review comments and harden deepsec fixes
- fix(env): replace jsonb operators with transaction+FOR UPDATE read-modify-write
- PUT: uses db.transaction + SELECT FOR UPDATE + JS merge to avoid lost-update race
- DELETE: same pattern; fixes variable scope bug where current was referenced outside tx
- removes broken || and - jsonb operators that fail on json-typed column
- fix(ssh): trim truncated output consistently with non-truncated path
- fix(gmail): remove redundant resolveOAuthAccountId call
- adds credentialType field to CredentialAccessResult
- authorizeCredentialUse now returns credentialType in all success paths
- gmail/labels route uses authz.credentialType and authz.resolvedCredentialId directly
- fix(supabase): centralize table identifier validation
- adds validateDatabaseIdentifier() to input-validation.ts
- all 8 supabase tools use the shared util instead of inline regex
* fix(workflows): fix VariableType assignment in admin workflow import route
The intermediate Record cast used 'string' for the type field which TypeScript
correctly rejected — WorkflowVariable.type is 'VariableType', not string.
Changed the cast to use VariableType so both branches typecheck correctly.
* fix(a2a): handle Request objects in pinnedFetch URL extraction
* fix(security): extract shared file-access guard; merge workspace/mothership branch
* fix(security): advisory lock for env first-insert race; handle all BodyInit types in pinnedFetch
* chore: remove inline comment from advisory lock
* fix(security): remove stray comment; narrow credentialType to literal union
* fix(security): add credentialId validation to wealthbox oauth route; fix null body override in pinnedFetch
* fix(security): stream A2A response body to unblock SSE; keep text/json/arrayBuffer for non-streaming callers
* fix(security): resolve credentialId guard on OneDrive, use assertToolFileAccess in WordPress, memoize body buffer to prevent silent empty reads, fix ArrayBuffer type cast
* fix(security): handle string[][] HeadersInit format in pinnedFetch
* fix(security): keep abort listener alive during body streaming; clean up in stream end/error/cancel
* chore: remove extraneous inline comment
* fix(security): cleanup abort listener when maxResponseBytes limit is exceeded
|
||
|
Waleed
|
ec936be5ca |
improvement(workflow-block): support manual workflow ID via advanced mode (#4573)
* improvement(workflow-block): support manual workflow ID via advanced mode * fix(input-mapping): resolve workflowId via canonical hook for advanced mode * fix(input-mapping): fall back to manualWorkflowId in preview context * refactor(input-mapping): resolve workflowId via useDependsOnGate canonical pattern |
||
|
Vikhyath Mondreti
|
43f53bb7d3 |
feat(execution): payload size bottlenecks with lazy execution value hydration, safer materialization, and batched parallel execution (#4560)
* improvement(resolver): lazy resolution for underlying fields greater than 10MB * progress * feat(parallel): batching * codegen to allow inline substitution * address comments * ui inconsistencies * cleanup redundant code * address more comments * address comments * replace helper * fix tests |
||
|
Waleed
|
f6b246ba44 |
fix(docs): restore media centering and full-width intro image (#4570)
* fix(docs): restore media centering and full-width intro image * fix(docs): drop overflow-hidden from intro media wrappers so focus ring is not clipped * fix(docs): use inset focus ring on lightbox media so parent overflow-hidden cannot clip it * fix(docs): drop focus ring on lightbox media to match original UI |
||
|
Waleed
|
d1eb79ecd3 |
fix(helm): preserve STS serviceName + networkPolicy.egress back-compat (#4569)
* fix(helm): preserve STS serviceName + networkPolicy.egress back-compat
Greptile flagged two real upgrade-breaking changes vs the prior chart:
1. statefulset-postgresql spec.serviceName flipped from <name>-postgresql
to <name>-postgresql-headless. spec.serviceName is immutable, so any
existing install would hit 'Forbidden: updates to statefulset spec ...'
on helm upgrade. Revert to the original name (the headless Service in
services.yaml is added alongside, not as a swap).
2. networkPolicy.egress changed from a list to a map ({extraRules, exceptCidrs}),
silently dropping any custom egress list set by existing users. Restore
the original list semantics for networkPolicy.egress and move cloud-metadata
blocking to a sibling top-level field networkPolicy.egressExceptCidrs.
Adds NOTES.txt upgrade-notes entry covering both + the ESO v1→v1beta1 default
flip (functionally a no-op, but worth surfacing).
* docs(helm): update README egress reference to new key name
* fix(helm): revert copilot-postgresql STS serviceName too (same immutability issue)
Audit caught that the main fix in
|
||
|
Theodore Li
|
05892f74f2 |
improvement(scheduler): raise per-tick claim budget to drain backlog (#4567)
* improvement(scheduler): raise per-tick claim budget to drain backlog MAX_CRON_CLAIMS 20 -> 100; reserved workflow/job slots 10/10 -> 50/50. Throughput was capped at 20 schedules/tick which created a 20+ hour backlog when due work exceeded ~1 item per cron-second. * improvement(scheduler): raise per-tick claim budget to 200 Bumps MAX_CRON_CLAIMS 100 -> 200 (workflow/job split 100/100). Pairs with the fire-and-forget cron Lambda change so per-tick processing time is no longer bounded by the Lambda's 50s HTTP timeout. |
||
|
Waleed
|
9d2dd8f550 |
improvement(helm): helm chart updates with security, ESO, and docs overhaul (#4565)
* improvement(helm): production-ready chart with security, ESO, and docs overhaul
Comprehensive Helm chart improvements bringing the chart up to industry
standards for security, secret management, and documentation.
Security
- Pod Security Standards "restricted" defaults on every pod and container
(runAsNonRoot, allowPrivilegeEscalation=false, capabilities.drop=[ALL],
seccompProfile=RuntimeDefault)
- automountServiceAccountToken=false on ServiceAccount and every pod
- NetworkPolicy egress blocks cloud metadata endpoints by default
- Sensitive app/realtime env keys auto-partitioned into chart-managed Secret
via envFrom; no more plaintext secrets on container specs
Secret management
- Three modes: inline, existingSecret, ExternalSecrets Operator (ESO)
- ESO sync supports arbitrary sensitive keys
- Fail-fast template rendering when ESO enabled but sensitive key unmapped
- AWS/Azure/GCP example files document all three modes
Reliability
- Headless Services for both Postgres StatefulSets
- HPA-aware replicas (omits spec.replicas when autoscaling.enabled)
- PodDisruptionBudget auto-activates when replicaCount > 1
- Startup / liveness / readiness probes with distinct timings
- CronJob ttlSecondsAfterFinished for automatic cleanup
Chart hygiene
- Image tags default to Chart.AppVersion; pullPolicy IfNotPresent
- Optional image.digest pin for content-addressed deploys
- kubeVersion >=1.25.0-0 enforced
- Ollama pinned to 0.23.2; mount moved to /data
Documentation
- README rewritten in cert-manager / Bitnami style
- NOTES.txt with post-install guidance
- Example values files annotated with usage and secret-strategy guidance
* fix(helm): correct resource names in README (sim-sim-* → sim-*)
The sim.fullname helper collapses to the release name when the release
name contains the chart name. With the documented release name 'sim',
actual resources are 'sim-app', 'sim-postgresql', etc. — not the
'sim-sim-*' form previously documented. Fixes copy-paste commands in the
pre-1.0.0 upgrade walkthrough and several troubleshooting snippets.
Also expands the cronjobs component description to reflect the full set
of 13 scheduled jobs (was understated as just Gmail/Outlook polling).
* improvement(helm): split app/realtime env into Secret-bound + inline defaults
- Add app.envDefaults / realtime.envDefaults for chart-shipped operational
tunables (rate limits, timeouts, IVM, feature-flag defaults, localhost URL
fallbacks). Rendered inline on the container, not into the Secret
- Remove operational defaults from app.env / realtime.env so the chart-managed
Secret stays minimal and External Secrets Operator users only map keys they
actually set, not every chart default
- Skip an envDefaults key when the user explicitly sets it in env (K8s `env`
overrides `envFrom`, so an inline default would otherwise mask a Secret
value at runtime)
- Relax values.schema.json to allow empty strings on NEXT_PUBLIC_APP_URL,
BETTER_AUTH_URL, NEXT_PUBLIC_SUPPORT_EMAIL (defaults supplied via envDefaults)
* fix(helm): address PR review — cronjob validation, ESO apiVersion, secret merge order, image guard
- CronJobs reference CRON_SECRET via secretKeyRef; fail-fast at template
time when cronjobs.enabled=true and app.env.CRON_SECRET is empty so users
get a clear error instead of a CreateContainerConfigError loop
- Default externalSecrets.apiVersion to "v1beta1" (supported by every ESO
release since v0.7). The previous "v1" default targets only ESO v0.17+
- Swap merge order in secrets-app.yaml so app.env wins over realtime.env
for shared keys (BETTER_AUTH_SECRET, BETTER_AUTH_URL, …) — both pods
consume the same Secret via envFrom, so the app value must be canonical
- Add `required` guard on sim.image so an empty tag + empty digest +
empty Chart.AppVersion surfaces as a clear template-time error instead
of rendering an invalid `repo:` reference
* fix(helm): require critical secrets to be mapped when ESO is enabled
Previously, enabling externalSecrets without mapping BETTER_AUTH_SECRET /
ENCRYPTION_KEY / INTERNAL_API_SECRET (and CRON_SECRET when cronjobs are
on) rendered cleanly but produced CrashLoopBackOff at runtime with
cryptic missing-env errors. Fail at template time instead.
* fix(helm): auto-enable PDB when HPA minReplicas > 1
Previously the auto-enable predicate only checked the static
app.replicaCount, which defaults to 1 even when autoscaling is on
(HPA owns spec.replicas). PDB now also activates when
autoscaling.enabled=true and minReplicas > 1.
* fix(helm): prevent realtime envDefaults from masking app.env Secret values; add StatefulSet upgrade NOTES
- Realtime override-skip now considers keys set in either app.env or
realtime.env. The shared app Secret is mounted via envFrom on the
realtime pod, so a key set in app.env (e.g. NEXT_PUBLIC_APP_URL) would
previously be masked by the realtime envDefault (inline env overrides
envFrom in K8s).
- NOTES.txt now prints a StatefulSet orphan-delete reminder on upgrade,
surfacing the immutable serviceName issue documented in the README.
* feat(helm): add Claude Skill for chart deployment
Adds a skill at helm/sim/.claude/skills/sim-helm/ that teaches agents how
to deploy and troubleshoot the Sim Helm chart: install path selection
(inline / existingSecret / ESO), secret generation, the values.yaml
four-layer mental model, common-failure troubleshooting, and the
pre-1.0.0 StatefulSet orphan-delete upgrade procedure.
Skill is loadable by Claude Code, Codex, and OpenCode via the standard
skills convention (directory name matches frontmatter name).
* docs(helm): add CRON_SECRET to TL;DR, dry-run, and example install headers
The validateSecrets guard requires CRON_SECRET when cronjobs.enabled=true
(the default), but the quickstart and example file install commands
omitted it — users following the docs hit a hard template-render failure.
Adds CRON_SECRET to README TL;DR, validate-the-install dry-run snippet,
and the install command headers in all example values files.
* fix(helm): require INTERNAL_API_SECRET in inline secret mode
The ESO coverage validator already required INTERNAL_API_SECRET, but the
inline validateSecrets path only checked BETTER_AUTH_SECRET, ENCRYPTION_KEY,
and CRON_SECRET — letting inline installs render successfully and then
crash at runtime when the realtime↔app shared auth secret was missing.
Adds the same fail-fast check to the inline path.
* docs(helm): surface INTERNAL_API_SECRET upgrade requirement in NOTES.txt
The new validateSecrets check makes app.env.INTERNAL_API_SECRET mandatory
on upgrade. Existing installs that never set it would hit a template
render failure with no in-context guidance. Adds an upgrade-only note
with the generation snippet and storage guidance alongside the existing
StatefulSet orphan-delete instructions.
* fix(helm): NetworkPolicy egress to OTEL collector + external-db example format
- Add app/realtime NetworkPolicy egress rules for the OpenTelemetry
collector pod on ports 4317 (OTLP gRPC) and 4318 (OTLP HTTP) when
telemetry.enabled=true. Without these, traces and metrics were silently
dropped with connection-refused errors when both telemetry and
networkPolicy were enabled.
- Migrate values-external-db.yaml from the legacy list-shaped egress
format to the new {exceptCidrs, extraRules} object. The list form would
replace the default object on merge and crash template rendering when
the chart tried to access .exceptCidrs on a list.
* fix(helm): NOTES.txt no longer prints false secret warning for ESO users
The secrets-empty warning only checked app.secrets.existingSecret.enabled
before scanning app.env. ESO users intentionally leave app.env empty —
secrets come from the ESO-synced Secret — so every ESO install/upgrade
printed a misleading 'pods will fail to start' warning.
Reorders the branches so externalSecrets.enabled takes precedence: ESO
users now see a confirmation message with kubectl commands to verify the
ExternalSecret has synced. The empty-app.env warning only fires when
both ESO and existingSecret are disabled.
* fix(helm): existingSecret mode no longer drops app.env / realtime.env values
In existingSecret mode the chart-managed Secret is not rendered, so non-empty
values in app.env / realtime.env had nowhere to land — yet the envDefaults
skip logic still suppressed the matching defaults. Result: keys like
NEXT_PUBLIC_APP_URL, BETTER_AUTH_URL, and NODE_ENV silently went missing
on both pods (the example values-existing-secret.yaml hit this directly).
Both app and realtime deployments now inline non-empty values from app.env
(plus realtime.env on the realtime container) when existingSecret is enabled
and ESO is not. Inline / ESO modes are unchanged: inline still flows through
the chart-managed Secret, ESO still owns the synced Secret.
* fix(helm): correct realtime env overlay + filter chart-computed keys in existingSecret mode
Realtime: Sprig merge gives the first source precedence and treats "" as a
real value, so realtime.env empty defaults for shared keys shadowed
non-empty app.env values. Replace with deepCopy($appEnv) base + manual
non-empty overlay of $rtEnv.
Both deployments: exclude DATABASE_URL/SOCKET_SERVER_URL/OLLAMA_URL from
the existingSecret inline path so user-supplied values can't override
chart-computed ones via last-wins env semantics.
* fix(helm): skip envDefaults in existingSecret mode + document egress rename
In existingSecret mode the user's pre-existing Secret is the source of
truth (loaded via envFrom). Inlining localhost envDefaults for URL keys
(BETTER_AUTH_URL, NEXT_PUBLIC_APP_URL, ALLOWED_ORIGINS) silently shadowed
the Secret-bound values because K8s env always wins over envFrom. Skip
envDefaults entirely on both deployments when existingSecret is enabled.
Also call out the networkPolicy.egress shape change (list -> map with
exceptCidrs + extraRules) in the NOTES.txt upgrade block so operators
migrate their custom rules rather than silently losing them.
* fix(helm): copy-pasteable install commands in copilot + ESO examples
values-copilot.yaml: the install header was missing every required
copilot.server.env.* secret (AGENT_API_DB_ENCRYPTION_KEY, INTERNAL_API_SECRET,
LICENSE_KEY, SIM_BASE_URL, SIM_AGENT_API_KEY, REDIS_URL, one model key) plus
copilot.postgresql.auth.password. Pasting it as-is failed at template render.
values-external-secrets.yaml: NEXT_PUBLIC_APP_URL, BETTER_AUTH_URL, etc. were
declared under app.env / realtime.env. In ESO mode the chart-managed Secret
isn't rendered, so the validator (rightly) rejects keys in app.env that
aren't mapped under externalSecrets.remoteRefs. Moved non-secret URL/config
to envDefaults, which is inlined and not subject to the ESO mapping rule.
* polish(helm): configurable NetworkPolicy ingress peers + clearer API_ENCRYPTION_KEY comment
- networkPolicy.ingressFrom lets operators scope the ingress-controller
rule to a specific namespace/podSelector. Defaults to a single empty
peer (`- {}`), which is the explicit form of "any source" — same
effective behavior as the old `from: []` but unambiguous across CNIs.
To restrict, override with e.g.:
networkPolicy:
ingressFrom:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: ingress-nginx
- API_ENCRYPTION_KEY comment: drop the "must be exactly 64 hex
characters" phrasing that sat awkwardly next to `openssl rand -hex 32`.
The generation command already produces the required length.
* test(helm): add helm-unittest suites + CI workflow + ci values matrix
- 7 helm-unittest suites covering smoke, validators, secret modes,
envDefaults secret-mode-aware inlining (round-9 regression net),
chart-computed env keys (round-8 regression net), NetworkPolicy
shape, and PDB/HPA conditional rendering (38 tests, ~265ms).
- ci/*.yaml render fixtures for default, production, existingSecret,
ESO, and external-db install modes.
- GitHub Actions workflow runs helm lint --strict, helm unittest,
helm template across the ci matrix, and kubeconform validation
against Kubernetes 1.30 schemas.
- CONTRIBUTING.md documents how to run the same gates locally.
* test(helm): add helm test hook + kind apiserver dry-run in CI
- New templates/tests/test-connection.yaml renders a Pod with
helm.sh/hook=test that wgets the app Service (and realtime when
enabled). Lets users run `helm test <release>` after install for
a real in-cluster connectivity check. Restricted PSS context.
- tests.* values block (image, timeoutSeconds, resources) is the
knob to disable or tune the probe; documented in values.schema.json.
- 3 helm-unittest tests cover the hook annotations, PSS context,
and tests.enabled=false skip path (41 tests total).
- New CI job spins up a kind v1.30 cluster and runs
`kubectl apply --dry-run=server` against the rendered manifests
for the CRD-free ci fixtures (default / existing-secret /
external-db). Catches admission and validation issues the static
kubeconform schema check can't see.
* chore(helm): remove pre-1.0.0 upgrade fluff + tighten .helmignore
This is the 1.0.0 release of the chart — there is no pre-1.0.0
predecessor for users to upgrade from, so all of the dedicated upgrade
narration was hypothetical.
- Drop the 'Upgrading from a pre-1.0.0 build' README section and the
matching troubleshooting entry.
- Drop the .Release.IsUpgrade block from NOTES.txt: items 5 (StatefulSet
orphan-delete), 6 (INTERNAL_API_SECRET 'new in 1.0.0'), 7
(networkPolicy.egress shape change). Each described a migration off a
chart version that never shipped.
- Delete references/upgrade-pre-1.0.0.md and remove the corresponding
pointers from SKILL.md.
- Anchor .helmignore patterns to chart root so /tests/ (unit suites)
and /examples/ are dropped from the packaged tarball without also
dropping templates/tests/ (the helm test hook).
* chore(helm): drop CI workflow + ci/ fixtures + CONTRIBUTING.md
The helm-unittest suites in helm/sim/tests/ and the helm test hook
in helm/sim/templates/tests/ stay — those are chart-internal quality
scaffolding, not CI. Removed:
- .github/workflows/helm-chart.yml
- helm/sim/ci/*.yaml (5 render fixtures used only by the workflow)
- helm/sim/CONTRIBUTING.md (mostly documented those gates)
- dead /ci/ and /CONTRIBUTING.md entries in .helmignore
* feat(helm): pod rollout on Secret change + topologySpreadConstraints
- Add checksum/secret pod annotations on app, realtime, and copilot
Deployments (plus checksum/config on app when branding ConfigMap is
enabled). Closes the long-standing footgun where 'helm upgrade' with
a changed Secret would silently leave pods running the old values
until a manual rollout restart.
- New top-level topologySpreadConstraints value (and sim.topologySpreadConstraints
helper) applied to app and realtime Deployments. Mirrors how affinity
and tolerations are plumbed; users supply their own labelSelector
to mirror Bitnami convention.
- 5 helm-unittest cases cover the checksum annotations and topology
spread rendering (46 tests total).
* fix(helm): drop empty-string shadowing in app/realtime env merge
Sprig 'merge' treats "" as a real value, so a default-empty
app.env.BETTER_AUTH_URL would shadow a non-empty realtime.env override
and the URL would never reach the rendered Secret. Replace 'merge'
with an explicit two-pass overlay that filters empties before writing,
mirroring the same pattern already used in deployment-realtime.yaml's
existingSecret block.
Adds two regression tests: realtime.env-only value reaches the Secret
when app.env is empty, and app.env still wins on collision when both
are non-empty (48 tests total).
* fix(helm): make topologySpreadConstraints per-component to match docstring
Greptile flagged that sim.topologySpreadConstraints helper docstring promised
per-component config (.Values.app, .Values.realtime, ...) but call sites
passed .Values, so any app.topologySpreadConstraints / realtime.topologySpreadConstraints
set by the user was silently dropped. The single global key also prevented
distinct app-vs-realtime spread rules.
Pass .Values.app / .Values.realtime to the helper at each call site; move
the top-level topologySpreadConstraints key into both component sections in
values.yaml. Adds a regression test that app constraints don't leak onto
the realtime pod.
* fix(helm): allow cron pods through app NetworkPolicy
Cursor flagged that when networkPolicy.enabled=true and cronjobs.enabled=true
(the recommended production config), the app NetworkPolicy only allowed
ingress from realtime and the ingress controller — silently blocking every
cron pod's HTTP call to /api/schedules/execute, webhook polls, etc. All 13
default cronjobs would fail.
Tag cron pods with a stable simstudio.ai/component-group: cronjob label so
the app NetworkPolicy can allow them with a single rule (no per-job
enumeration). Rule is conditional on cronjobs.enabled. Adds positive and
negative regression tests.
|
||
|
Waleed
|
1b94424b4b |
improvement(mothership): align markdown blockquote, img, em, del with design tokens (#4566)
* improvement(mothership): align markdown blockquote, img, em, del with design tokens * fix(mothership): correctly scope blockquote paragraph margin reset to first/last child * improvement(mothership): restore italic on blockquotes * fix(mothership): widen img component prop type to satisfy Streamdown Components |
||
|
Siddharth Ganesan
|
9b333eaae9 | fix(mothership): fix tool hidden in ui (#4564) | ||
|
Waleed
|
22b5a1e577 |
fix(tables): cmd+a always selects all cells on the table page (#4562)
* fix(tables): cmd+a always selects all cells on the table page * fix(tables): move preventDefault inside non-empty guard |
||
|
Waleed
|
b7c34d4323 |
fix(file-viewer): prevent scroll jump to top during Mothership streaming (#4559)
* fix(file-viewer): prevent scroll jump to top during Mothership streaming - Fix root cause: MarkdownCheckboxCtx.Provider was conditionally rendered, causing the scroll container to unmount/remount when isStreaming flipped, resetting scrollTop to 0 on every stream start - Add useScrollAnchor hook with spacer element to preserve scroll position when streamed content temporarily shrinks the scroll container - Linger active session ID on complete to prevent streamingContent→undefined flicker between consecutive tool calls on the same file - Gate upsert activation on incoming session having renderable content - Fix shouldShowStreamingFilePanel to keep panel mounted during linger - Fix use-chat post-write navigation to work with lingered completed session - Fix useAutoScroll to check proximity before pinning to bottom on stream start * fix(file-viewer): preserve session linger in hydrate/upsert and clear spacer after stream * chore(file-viewer): trim verbose comments to match codebase style * fix(file-viewer): re-engage auto-scroll when user scrolls back to bottom * chore(file-viewer): update scroll-anchor tsdoc, remove test separators, add hydrate linger tests * fix(file-viewer): prevent false re-engage when spacer restoration triggers onScroll * refactor(file-viewer): extract shouldReengage as pure tested function Pulls the spacer-guard re-engage condition out of onScroll into an exported pure function so the false-re-engage invariant (spacer active → no re-engage) is covered by automated tests rather than relying on manual QA. Adds 8 unit tests for shouldReengage alongside the existing 15 for computeSpacerShortage. * chore(file-viewer): trim verbose inline comments in use-scroll-anchor * chore(file-viewer): final comment cleanup before merge |
||
|
Waleed
|
39f74aa353 |
feat(data-drains): add GCS, Azure Blob, BigQuery, Snowflake, and Datadog destinations (#4552)
* feat(data-drains): add GCS, Azure Blob, BigQuery, Snowflake, and Datadog destinations
* fix(data-drains): address PR review comments
* fix(data-drains): extract sleepUntilAborted, honor abort across all destinations
* fix(data-drains): widen BigQuery projectId max and dedupe parseServiceAccount
* fix(data-drains): tighten GCS bucket contract and expose Azure endpointSuffix
* improvement(data-drains): extract normalizePrefix and buildObjectKey to shared utils
* fix(data-drains): retry BigQuery network errors; tighten Azure accountKey contract
- BigQuery insertAll now wraps the fetch in try/catch inside the retry loop so DNS failures, socket resets, and timeouts are retried with backoff instead of propagating immediately.
- Align azureBlobCredentialsBodySchema with the runtime schema (min 64 / max 120 / base64 regex) so obviously invalid keys are rejected at the API boundary rather than at drain-run time.
* improvement(data-drains): consolidate parseRetryAfter; add Datadog NDJSON line context
- Extract a single parseRetryAfter helper (capped at 30s, returns number | null) into lib/data-drains/destinations/utils.ts and remove the five local copies in bigquery, datadog, gcs, snowflake, and webhook.
- Datadog parseNdjson now wraps JSON.parse in try/catch and surfaces the failing line index, matching BigQuery's parser.
* fix(data-drains): correct Datadog size guard and Snowflake VARIANT limit
- Datadog payload guard now checks the uncompressed size against the 5 MB limit and the wire size against the 6 MB compressed limit, so gzip cannot smuggle an oversized body past the client-side check.
- Snowflake VARIANT limit is 16 MiB (16,777,216 bytes), not 16,000,000 bytes — small payloads between 16 MB and 16 MiB were being rejected unnecessarily.
- Drop the unused apiKey field on Datadog PostInput; the key is already embedded in the prepared request headers.
* improvement(data-drains): consolidate backoffWithJitter into shared utils
Datadog, GCS, and webhook each had byte-identical backoff helpers (BASE 500ms, MAX 30s, jitter ±20%, Retry-After floor). Lift the helper into lib/data-drains/destinations/utils.ts alongside parseRetryAfter and sleepUntilAborted, and drop the per-file copies and their BASE_BACKOFF_MS/MAX_BACKOFF_MS constants.
* fix(data-drains): align destinations with live provider specs
Audited every destination against live AWS/GCS/Azure/BigQuery/Snowflake/
Datadog/webhook docs and applied spec-correctness fixes:
- S3: reserved bucket prefix amzn-s3-demo-, suffixes --x-s3/--table-s3;
metadata byte formula excludes x-amz-meta- prefix per AWS spec
- GCS: reject -./.- adjacency; UTF-8 prefix cap; forbid .well-known/
acme-challenge/ prefix; ASCII-only x-goog-meta-* enforcement
- BigQuery: insertId is 128 chars (not bytes); split DATASET_RE (ASCII)
and TABLE_RE (Unicode L/M/N + connectors); UTF-8 byte cap on tableId
- Snowflake: disambiguate org-account vs legacy locator account formats;
requestId+retry=true for idempotent retries; server-side timeout=600;
default column DATA uppercase to match unquoted canonical form
- Azure: endpoint suffix allowlist (4 sovereign clouds); accountKey
length(88) base64
- Webhook: url max(2048); CRLF/NUL rejection on bearer/secret/sig header
* fix(data-drains): address PR review on snowflake poll + shared NDJSON parsing
- snowflake pollStatement: per-attempt timeout via AbortSignal.any, retry on 429/5xx with Retry-After + jitter
- bigquery parseNdjson error messages now 1-indexed
- consolidate parseNdjson variants into shared parseNdjsonLines/parseNdjsonObjects in utils
* fix(data-drains): per-attempt fetch timeouts in gcs/bigquery, snowflake poll double-sleep
- gcs.fetchWithRetry + bigquery.postInsertAll now use AbortSignal.any with a per-attempt timeout so a hung TCP connection cannot stall the drain
- snowflake.pollStatement skips the next interval sleep when it just slept for retry backoff
* fix(data-drains): bigquery probe timeout + jittered retries, align Snowflake column default UI/docs
- bigquery test() probe now uses AbortSignal.any + per-attempt timeout
- bigquery insertAll retry switches to backoffWithJitter for thundering-herd avoidance
- Snowflake column placeholder + docs say DATA (uppercase) to match the code default
* fix(data-drains): mirror webhook signingSecret min length in form gate
isComplete now requires signingSecret >= 32 to match the contract/runtime
schema so the Save button can't enable on a value that will fail server-side.
* fix(data-drains): validate JSON client-side for Snowflake before binding
Switch Snowflake to parseNdjsonObjects so malformed rows are caught locally
with 1-indexed line numbers instead of failing the whole INSERT server-side.
Re-stringify each parsed object before binding to PARSE_JSON(?).
Drop the now-unused parseNdjsonLines helper.
* fix(data-drains): cross-cutting audit pass against live provider docs
- Azure: bound retryOptions on BlobServiceClient (SDK default tryTimeoutInMs is per-try unbounded; cap at 30s x 5 tries)
- Webhook contract: mirror runtime — signingSecret.max(512), bearerToken.max(4096) + CRLF/NUL refine, signatureHeader charset + CRLF/NUL refine
- S3 (lib + contract): reject bucket names with dash adjacent to dot; require https:// endpoint at the schema layer
- Snowflake: bind original NDJSON line bytes (re-stringifying a JSON.parse'd value loses bigint precision beyond 2^53-1); check pollStatement 200 body for the SQL error envelope (sqlState/code)
- Datadog: entry builder writes defaults first then user attrs then forced ddtags/message so user rows can't clobber routing fields; validate config.tags as comma-separated key:value pairs
- registry.tsx: tighten isComplete predicates to mirror contract minimums (GCS bucket >= 3, Azure containerName >= 3 / accountKey === 88, BigQuery projectId >= 6, Snowflake account >= 3)
* fix(data-drains): force ddsource/service overrides on Datadog entries
Previous fix placed ddsource/service before ...attrs, leaving them clobberable
by a user row field. Per Datadog docs, service + ddsource pick the processing
pipeline, so a drain's routing config must not be overridable per-row. Spread
attrs first, then force all four reserved fields (ddsource, service, ddtags,
message).
* fix(data-drains): preserve row-distinguishing index when BigQuery insertId overflows
Truncating from the left dropped the index suffix, so any overflow would
collapse all rows in a chunk to the same insertId and BigQuery would silently
dedupe them. Path is unreachable today (UUIDs keep raw ~85 chars), but the
overflow branch is now correct: hash the prefix, keep the index intact.
* fix(data-drains): refresh GCS token per retry, tighten Azure key regex
- gcs: rebuild Authorization header per attempt via buildHeaders so token
refresh from google-auth-library kicks in if a 5xx retry crosses the
hour-long token lifetime
- azure_blob: pin account-key regex to {0,2} trailing '=' (base64 of 64
bytes = exactly 88 chars with up to two '=' pad chars)
* fix(data-drains): address bugbot review of 6336948f6
- gcs: allow 1-char dot-separated bucket components (e.g. "a.bucket")
to match GCS naming rules — overall name is 3-63 (or up to 222 with
dots), but per-component minimum is 1 per Google's spec
- bigquery: drain the 401 response body before re-issuing the request
with a refreshed token so undici can return the socket to the
keep-alive pool
- snowflake: hoist getJwt() above the perAttempt timer in
executeStatement so JWT signing doesn't eat the network budget
(matches the order already used in pollStatement)
* fix(data-drains): allow org-account Snowflake identifier with region suffix
The account validation rejected `<orgname>-<acctname>.<region>.<cloud>`
because `ACCOUNT_LOCATOR_RE`'s first segment forbade hyphens, while
`ACCOUNT_ORG_RE` forbade dots. `normalizeAccountForJwt` already handles
this composite form. Widen the first segment of `ACCOUNT_LOCATOR_RE` to
allow hyphens so the boundary contract and the runtime schema accept
what the JWT layer was already designed to process.
* fix(data-drains): drain retryable response bodies in datadog/gcs loops
Mirrors the bigquery 401 fix. Without consuming the body before
sleeping, undici can't return the socket to the keep-alive pool, so
each retry leaks a TCP connection instead of reusing it.
* fix(data-drains): drain snowflake poll bodies on 202 and retryable status
Mirrors the bigquery/datadog/gcs drains. Long async statements can poll
many times against the same connection; without consuming the body
undici can't return the socket to the keep-alive pool, so each iteration
leaks a connection until GC.
* fix(data-drains): consume success bodies; check Snowflake sqlState on 200
- gcs: drain the body on success paths so undici can return the socket
to the keep-alive pool
- snowflake: drain the body on synchronous 200 OK and run the same
sqlState envelope check pollStatement already does — otherwise a
statement-level failure that completes synchronously would silently
return success
* fix(data-drains): drain datadog and bigquery probe success bodies
Same undici keep-alive issue as the prior fixes: postWithRetries
returned the Response on success without draining (callers only read
headers); the BigQuery `test()` probe returned without consuming the
body. Both now drain before returning.
* chore(data-drains): regenerate enum migration as 0206 after staging rebase
* fix(data-drains): cap snowflake poll retries; tighten datadog tags min length
|
||
|
Siddharth GanesanandTheodore Li
|
0b2cfaf7f7 |
feat(mothership): add superuser env selection (#4558)
* feat(table): live cell updates via SSE + per-table event buffer
Replaces the polling-based row refetch with a push-based SSE stream that
patches the React Query cache directly as cell-state events arrive.
Architecture:
- New per-table event buffer in apps/sim/lib/table/events.ts. Redis sorted-set
with monotonic eventId, 1h TTL, 5000-event cap, in-memory fallback. Modeled
after apps/sim/lib/execution/event-buffer.ts but stripped of complexity
tables don't need (no per-execution lifecycle, no id-batching, no write
queue serialization). ~150 lines instead of 700.
- writeWorkflowGroupState appends a fat event after each successful 'wrote'.
Status transitions carry executionId + jobId; terminal/partial transitions
also include the new output values inline so the client can patch row data
without a follow-up refetch.
- New SSE route at /api/table/[tableId]/events/stream?from=<lastEventId>.
Replays from buffer on connect, polls at 500ms (mirrors workflow execution
stream), heartbeat every 15s, signals 'pruned' if the caller fell off the
back of the buffer.
- Client hook useTableEventStream subscribes via EventSource. Reconnect-resume
with last-seen eventId. On 'pruned', invalidates the rows query and resumes
from the new earliest. Cache patches walk every cached query under
rowsRoot(tableId) so filter/sort variants all stay live.
- Removes refetchInterval from useTableRows and the per-page polling effect
from useInfiniteTableRows. React Query's refetchOnWindowFocus +
refetchOnReconnect cover the durability gap if any push is dropped.
Out of scope:
- Bulk-cancel events (cancellation path is being redesigned separately).
- Generalizing the workflow event-buffer module to a shared primitive (defer
until a third use case appears; for now the table buffer is the simpler
cousin of the workflow one).
* fix(table): drop run-mutation refetch so SSE patches aren't overwritten
useRunColumn.onSettled was canceling in-flight queries and invalidating the
rows query — leftover behavior from the polling era. With the SSE stream
now keeping the cache live via incremental patches, this refetch races the
stream and snaps the cache back to whatever DB shows at the refetch moment,
which can lag the just-arrived queued/running events. Cells appeared stuck
on the optimistic 'pending' even though the SSE was delivering the real
transitions.
* chore(table): simplify SSE plumbing — reuse helpers, drop dead polling code
- Reuse snapshotAndMutateRows for SSE cache patches instead of reimplementing
the page-walk + cache-shape detection. Adds a {cancelInFlight: false} opt
for the SSE caller (mutations still cancel as before).
- Drop client-side type duplication in use-table-event-stream — import
TableEvent and TableEventEntry from lib/table/events directly.
- Drop the now-dead mergePagePreservingIdentity + rowEqual from tables.ts;
their only caller was the polling effect that was removed earlier.
- Drop the defensive try/catch around appendTableEvent in cell-write — the
function is documented as never-throwing (returns null on failure).
- Combine INCR + ZADD into one Lua eval in events.ts. Halves Redis RTT per
cell-write. Lua returns the new eventId; the script splices it into the
pre-built entry JSON.
- Trim refs to plain let bindings inside the effect; trim stale
comments referencing the old polling implementation.
* fix(table): address PR review on SSE buffer
- TTL-expiry silent miss: when all keys expire, hgetall(meta) returns empty
so earliestEventId is undefined and the prune branch was skipped. Reconnect
with non-zero afterEventId now checks the seq counter — its absence (TTL
expired) signals pruned so the client refetches. Memory fallback mirrors.
- Unbounded ZRANGEBYSCORE: cap reads at TABLE_EVENT_READ_CHUNK = 500 events
per call. The route's 500ms poll loop drains chunks across ticks instead of
flushing 5000 entries (multi-MB) in one tick after a long disconnect.
- Pruned handler closes EventSource client-side: server-side close was firing
onerror and routing through the 500ms backoff path. Now we close
proactively, reset the reconnect attempt counter, and reconnect immediately
from the new earliest.
* Cross env copilot
* Force deploy
* Run migration
* Updates
* Fix migration
* Redeploy
* Make dev db push
* restore old migs
* Cross env copilot
* Add custom tools, skills, mcps to mothership
* Update migration
* Fix migs
* UPdate
* Fix types
* Fix
---------
Co-authored-by: Theodore Li <theo@sim.ai>
|
||
|
Waleed
|
8774f5c341 |
fix(deps): patch next-mdx-remote and opentelemetry CVEs (#4557)
- bump next-mdx-remote 5.0.0 → 6.0.0 (GHSA-g4xw-jxrg-5f6m / CVE-2026-0969, arbitrary code execution in MDX serialize) - bump @opentelemetry/sdk-node and exporter-trace-otlp-http 0.200.0 → 0.217.0 (GHSA-q7rr-3cgh-j5r3 / CVE-2026-44902, Prometheus exporter DoS) - align @opentelemetry/sdk-trace-base, sdk-trace-node, resources to ^2.7.0 to keep all @opentelemetry/* packages on a single core@2.7.1 instance |
||
|
Waleed
|
d895e0efee |
fix(security): authorize MCP subagent IDs, oauth workspace, credential admin demotion (#4551)
* fix(security): authorize MCP subagent IDs, oauth workspace, credential admin demotion
- handleSubagentToolCall and handleDirectToolCall now authorize user-supplied
workflowId/workspaceId via authorizeWorkflowByWorkspacePermission /
ensureWorkspaceAccess before forwarding downstream; resolvedWorkspaceId is
derived from the authorized workflow record instead of trusted from the body
- executeOAuthGetAuthLink verifies caller membership (write level) on the
target workspaceId before generating the OAuth link or writing
pendingCredentialDraft, closing the cross-workspace credential injection path
- POST /api/credentials/[id]/members wraps role updates in a transaction that
counts active admins and rejects demotion of the last admin (mirrors the
existing DELETE guard in the same file)
- GET /api/credentials/[id]/members returns uniform 404 for both missing and
inaccessible credentials to remove the existence oracle
* fix(security): address PR review — active-status guard, FOR UPDATE locks, workspaceId propagation
- credentials/members POST: add `current.status === 'active'` check to the
last-admin demotion guard so re-inviting a revoked admin as a non-admin role
no longer incorrectly hits the "Cannot demote the last admin" path
- credentials/members POST+DELETE: add `.for('update')` to the active-admin
count SELECT inside both transactions to serialize concurrent demotions and
eliminate the admin-count TOCTOU race under Postgres READ COMMITTED
- credentials/members POST: also lock the member row itself with `.for('update')`
so the role+status read and the subsequent UPDATE are atomic
- mcp/copilot handleDirectToolCall: thread the DB-verified workspaceId from the
authorization result into prepareExecutionContext instead of relying on
user-supplied args
- oauth handler: fix error message to mention both workspaceId and userId when
either is missing from the execution context
|
||
|
Waleed
|
3adbde404b |
fix(oauth): persist rotated Microsoft refresh tokens (#4554)
* fix(oauth): persist rotated Microsoft refresh tokens Microsoft Entra rotates refresh tokens on every refresh and expects clients to replace the stored token with the new one. The Microsoft provider config was missing supportsRefreshTokenRotation, so the rotated refresh_token returned by Azure AD was silently discarded and the original token from initial OAuth connect was reused indefinitely — causing periodic 'Failed to refresh access token' errors for Excel, Teams, Outlook, OneDrive, SharePoint, Planner, AD, and Dataverse integrations. * test(oauth): cover hyphenated Microsoft service IDs in rotation test |
||
|
Waleed
|
c47777fa33 |
fix(security): close cross-tenant IDOR gaps in OAuth credential and execution auth (#4549)
* fix(security): close IDOR gaps in OAuth credential and execution authorization Routes that called resolveOAuthAccountId followed by a conditional workspace permission check (only run when workspaceId was set) silently skipped all ownership validation on the legacy account-ID fallback path. Any authenticated user could supply a raw account.id to access another tenant's OAuth credentials. - Replace resolveOAuthAccountId + conditional perm check with authorizeCredentialUse in: auth/oauth/wealthbox/item, tools/gmail/label, tools/onedrive/files, tools/onedrive/folder, tools/outlook/folders, tools/wealthbox/item (routes 1, 3-7) - Add authorizeCredentialUse ownership gate before resolveVertexCredential in providers/route.ts (route 2) - Add verifyFileAccess check on the user-supplied file key before downloadFileFromStorage in tools/wordpress/upload (route 8) - Add workflowId param to PauseResumeManager methods (enqueueOrStartResume, beginPausedCancellation, completePausedCancellation, blockQueuedResumesForCancellation, clearPausedCancellationIntent, getPausedCancellationStatus, processQueuedResumes) and filter all pausedExecutions lookups by workflowId so callers cannot act on another tenant's paused execution by supplying a foreign executionId (route 9) - Update all call sites (cancel, resume, poll routes) to pass workflowId * fix(security): close verifyFileAccess bypass, thread workflowId to processQueuedResumes, fix log level - Fail closed in WordPress upload when userFile.key is present but authResult.userId is absent, preventing silent bypass of ownership check via JWT fallback path - Thread workflowId into processQueuedResumes in the async resume error-recovery path and in pause-persistence.ts to close residual cross-tenant gap - Change logger.error to logger.warn for credential access denial in OneDrive folder route to match all other routes in this PR * fix(security): thread workflowId through all processQueuedResumes call sites Closes residual cross-tenant IDOR gap where processQueuedResumes was called without a workflowId scope in persistPauseResult, startResumeExecution (success and error paths), and clearPausedCancellationIntent. workflowId was already in scope at each site — this wires it through to the existing optional parameter. * fix(security): remove any types, drop extraneous comments, normalize caught errors - catch (error: any) → catch (error) + toError(error).message in resume and cancel routes - Remove what-not-why inline comments from wordpress upload and onedrive/files routes - Remove redundant debug-only item breakdown log and the file-IDs log in onedrive/files - Trim extraneous DAG-edge comments from updateSnapshotAfterResume in HITL manager * fix: use logger.warn for credential access denial in outlook folders route * fix(security): make workflowId required in all HITL pause/resume methods All 7 method signatures (processQueuedResumes, enqueueOrStartResume, beginPausedCancellation, completePausedCancellation, blockQueuedResumesForCancellation, clearPausedCancellationIntent, getPausedCancellationStatus) previously accepted workflowId as optional. Every call site already supplies it — making it required closes the vulnerability at the type level so future callers cannot accidentally omit tenant scoping and silently fall back to an unscoped DB query. * fix(security): thread workflowId through internal HITL cancellation calls and remove dead branches in credential-access * fix(security): harden workflowId scoping and file key guard Replace falsy workflowId checks in PauseResumeManager (all methods now unconditionally apply the workflowId WHERE clause, preventing empty-string bypass). Flip WordPress upload file guard from truthy key check to explicit non-empty validation so key:"" fails closed with a 404 instead of silently skipping access control. |
||
|
Waleed
|
badfeae432 |
fix(security): SSRF fixes (#4548)
* fix: address SSRF and token-leakage security vulnerabilities
- Azure TTS SSRF: validate region against /^[a-z][a-z0-9-]{1,30}[a-z0-9]$/
in both the contract (tts.ts) and runtime guard in synthesizeWithAzure,
preventing user-supplied region from redirecting requests to arbitrary hosts
- HubSpot token in logs: remove fullResponse from logger.info call;
log only non-sensitive metadata (hub_id, hub_domain, user_id) instead
of the full introspection response which included the access token
- Wealthbox account takeover: replace hardcoded email with per-user identity
by fetching /v1/users/me; fall back to token-derived stable identifier
so distinct Wealthbox users no longer share the same email address
- Shopify SSRF: apply shopifyShopDomainSchema (.myshopify.com allowlist)
to shopDomain from cookie before using it to build the fetch URL
* fix(wealthbox): correct getUserInfo endpoint, auth header, and stable identity
- Bug 1: Change API endpoint from /v1/users/me to /v1/me (correct Wealthbox API path)
- Bug 2: Replace ACCESS_TOKEN header with Authorization: Bearer <token> (standard OAuth 2.0)
- Bug 3: Remove generateId() from returned id (was non-deterministic, caused duplicate accounts);
use refresh token (stable, long-lived) instead of access token (rotates every ~2 hours)
as the hash source for the fallback identity; return null if no token is available
* fix(security): hash wealthbox fallback token identity, guard undefined userId
- Replace base64 encoding with SHA-256 hash for fallback token-derived identity
so raw token bytes are never stored in the DB
- Return null early when Wealthbox API response lacks an id field to prevent
all such users colliding on the wealthbox-undefined account
* fix(auth): replace stale wealthbox userInfoUrl placeholder with actual endpoint
The dummy URL comment was rendered obsolete when getUserInfo was updated
to fetch from api.crmworkspace.com/v1/me. Align userInfoUrl with the real
endpoint used in the getUserInfo implementation.
* fix(auth): append generateId() suffix to Wealthbox account IDs to match codebase pattern
All other providers use `${stableId}-${generateId()}` so the account.create.after
hook can strip the UUID suffix, find stale sibling rows, and migrate credential FKs.
Without the suffix the migration logic is skipped and reconnections would hit
duplicate key conflicts instead of gracefully updating credentials.
|
||
|
Waleed
|
ae680afab2 |
improvement(data-drains): docs page, screenshots, and search/UI polish (#4547)
Adds the enterprise data-drains docs page with two screenshots, a search bar over the drains table, and a UI cleanup pass on the settings component (size-*, useMemo removal, text-sm fixes). The previously-proposed env-var reference feature has been dropped — drain credentials remain raw values, encrypted at rest by the existing pipeline. |
||
|
WaleedandClaude Opus 4.7
|
10f7d36f13 |
fix(settings): accurate View navigation after restore in recently deleted (#4546)
* fix(settings): accurate View navigation after restore in recently deleted
Files now deep-link to /files/{id} (was /files), and folders expand
the restored folder + its parent chain in the sidebar before navigating
to /w so the user actually lands on the item they restored.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix: fall back to archived folders for parent-chain lookup
The restored folder may not be in the active folders cache yet when
View is clicked (the invalidation+refetch fires from onSettled, after
onSuccess surfaces the View button). Merge archived folder data — where
the restored item still lives — into the lookup map so the expansion
loop can always resolve the folder and walk its parent chain.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* chore(styling): canonicalize size-* shorthand for equal height/width
Document the size-* shorthand as the canonical pattern across CLAUDE.md,
AGENTS.md, .claude rules, .cursor + .agents commands, and the emcn
design-review skill. Default icon size is size-[14px]. Treat
h-[Npx] w-[Npx] and h-N w-N pairs as refactor targets.
Also migrate the remaining occurrences in recently-deleted.tsx.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
WaleedandCursor
|
3ed8615b5d |
chore(cleanup): react-doctor dead code elimination, landing + docs overhaul, component modernization (#4544)
* fix(react-doctor): remove unused export types, useEffect clearTimeout missing, a11y fixes * fix(react-doctor): strip unused export types from contracts, copilot, stores, and components Remove export keyword from type/interface declarations confirmed to have zero importers across lib/api/contracts/tools/aws/, lib/api/contracts/*.ts, lib/copilot/generated/, stores/workflows/workflow/types.ts, ee/access-control, ee/data-retention, lib/logs/types.ts, and app/workspace component files. TypeScript and API validation both pass clean. Reduces unused-types count from 394 → 181 and fully eliminates the ✗ critical dead-code categories (exports, types, files now show as ⚠ warnings not ✗ errors). * docs improvements * fix(react-doctor): delete 50 unused files (dead barrels, unreachable components, stale utilities) Remove confirmed-unused barrel index.ts files across stores/, connectors/, executor/, lib/, and app/workspace/ that had zero importers. Also delete unreachable components (chat-history-skeleton, trace-spans, logs-list, template-profile, enterprise landing sections), stale utilities (buffered-stream, blob-to-data-url, queued-workflow-execution, compute-edit-sequence), and obsolete generated/contract files. TypeScript passes clean. * remove dead code * cleanup * more * fix(blog): restore DiffControlsDemo for v0-5 blog post * fix: restore ContactButton for enterprise blog post, export WIKIPEDIA_PAGE_CONTENT_OUTPUT_PROPERTIES * fix(react-doc): restore stripped exports and remove server-only dependency * chore(deps): update lockfile after removing server-only * added back some exports * docs * more * fix type issues * tc * fix docs search route Co-authored-by: Cursor <cursoragent@cursor.com> * fix(tag-dropdown): add missing isEqual import from es-toolkit --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
Waleed
|
aae93f8228 |
fix(execution): cap isolate memory at 128MB and recycle workers every 200 executions (#4543)
* fix(execution): cap isolate memory at 128MB and recycle workers every 100 executions * fix(execution): set IVM_MAX_EXECUTIONS_PER_WORKER env default to 100 * fix(execution): raise MAX_EXECUTIONS_PER_WORKER from 100 to 200 * fix(execution): update memory limit error messages from 256 MB to 128 MB |
||
|
Vikhyath Mondreti
|
94f60e79b2 | improvement(deps): remove unused remark deps (#4542) | ||
|
Vikhyath Mondreti
|
a170255633 | fix(script): biome format wrap (#4541) | ||
|
Vikhyath Mondreti
|
13666b162d |
fix(zustand): v5 selector stability issues (#4539)
* fix(zustand): v5 selector stability issues * address comments |
||
|
Waleed
|
6544e0a783 | fix(tables): inline editing center alignment in table cells (#4538) | ||
|
Waleed
|
f122b686a4 |
fix(uploads): write workspaceFiles row when issuing presigned URL (#4537)
* fix(uploads): write workspaceFiles row when issuing presigned URL * test(uploads): cover insertFileMetadata failure path in presigned route |
||
|
Vikhyath Mondreti
|
02f381d08c |
feat(pptx): replace viewer with custom preview (#4536)
* feat(pptx): replace viewer with custom preview * address comments * address comments * address bugbot * address comment * rm bun lock * resolve conflict * update credit comment * address comments * rm bun lock * resolve conflict * make previews consistent for zoom and scroll * zoom fix |
||
|
Waleed
|
c7130c6bf9 |
fix(tables): fix bulk ops truncation for tables larger than one page (#4532)
* fix(tables): fix bulk ops truncation for tables larger than one page Bulk operations (column-header delete, select-all copy/cut/delete/run) were silently truncated to the first 1000 rows because handlers only iterated the loaded pages from useInfiniteQuery. Fix: - Extract tableRowsInfiniteOptions factory (infiniteQueryOptions) so the hook and imperative drain share the same typed cache key - Add background drain via useEffect watching hasNextPage/isFetchingNextPage — chains fetchNextPage until getNextPageParam returns undefined - Add ensureAllRowsLoaded to use-table: reads cache via getQueryData + calls fetchNextPage in a while loop until the last page is partial - Await ensureAllRowsLoaded at every kind:'all' bulk-op entry point in table-grid (column delete, copy, cut, action-bar delete/run) - Add chunkBatchUpdates to send updates in MAX_BULK_OPERATION_SIZE=1000 chunks so server validation never rejects oversized batches - Fix undo-redo: make executeAction async and chunk clear-cells, update-cells, and delete-column cell-restore with mutateAsync loops Tests: 41 passing across use-table, tables queries, and use-table-undo * chore(tables): remove extraneous comments * fix(tables): add missing useEffect import; chunk range-selection delete and cut * fix(tables): add hasRunningGroupExecution and import it * fix(tables): define mergePagePreservingIdentity helper used in polling cache merge * fix(tables): define ROWS_POLL_INTERVAL_WHILE_RUNNING_MS constant used in polling loop * fix(tables): capture rowSel before await in delete handler; handle clipboard NotAllowedError * fix(tables): abort cut on clipboard NotAllowedError to prevent silent cell deletion * fix(tables): push undo before chunkBatchUpdates to survive partial chunk failures * fix(tables): use text input for number cells; idle poll backoff; csv error toast; column-cut drain * fix(tables): audit fixes — column-copy drain, polling scope, merge identity - Fix polling tick: move Promise.all inside else-branch so dirty[] stays in scope; keep hasDirty=true during active mutations so the short interval fires while chunked batch-updates are in flight - Add isColumnSelectionRef branch to handleCopy (mirrors handleCut fix): column-header Cmd+C now drains all pages before building clipboard content - Replace String(updatedAt) comparison in mergePagePreservingIdentity with Date.getTime() equality — handles ISO vs +00:00 timezone variants - Remove redundant batchUpdates.length > 0 guards at chunkBatchUpdates callsites (empty-array case is handled inside the function) - Export _mergePagePreservingIdentity for unit testing - Add 6 unit tests covering mergePagePreservingIdentity edge cases * improvement(tables): cleanup — extract components, stabilize callbacks, fix ref sync * improvement(tables): remove polling, eager drain, and parallelize batch updates - Drop the per-page polling loop — SSE stream already patches execution cell state in real time and invalidates on buffer prune; polling was redundant and burned CPU/network on every open table - Remove eager mount drain (fetchNextPage loop in use-table.ts); scroll handler and ensureAllRowsLoaded handle progressive/on-demand loading - Parallelize chunkBatchUpdates with a 3-worker pool instead of serial chunks, reducing bulk-op round-trips by ~3x - Delete mergePagePreservingIdentity and its tests (no longer called) * chore(tables): remove stale comments and dead defensive code - Fix chunkBatchUpdates JSDoc to reflect parallel dispatch (was "sequentially") - Inline CHUNK_CONCURRENCY=3, single-use constant needs no abstraction - Drop stale "Polls while any cell is in flight" from useTableRows JSDoc - Remove two generic "Validation errors surfaced by caller" comments - Remove ASCII separator line from workflow group mutations section - Remove dead `if (!variables) return` guard in useImportCsvIntoTable onSettled (TanStack v5 always provides variables to onSettled) * fix(tables): run-all selection sends all rows to server, not just loaded pages When rowSelection.kind === 'all', selectedRunScope now flags allRows: true. The action-bar run handlers pass rowIds: undefined to the server when allRows is set, matching the server contract (missing rowIds = run all eligible rows). Stop likewise routes through scope: 'all' instead of per-row cancels. Previously, selecting all rows and clicking Run would silently only run the rows loaded in the current infinite-query cache (potentially one page of 1000 on a 5000-row table). * test(tables): remove background-drain describe block (behavior intentionally removed) * fix(tables): surface CSV import error toast; remove dead hasRunningGroupExecution * fix(tables): make runWithoutRecording async-aware so undoRedoInProgress covers full undo execution * feat(tables): render URL cells with favicon and clickable link * feat(tables): clickable URL cells with favicons using tldts |
||
|
WaleedandClaude Opus 4.7
|
4cc00b2ef9 |
fix(security): enforce workspace scope on workflow middleware and validate shopify shop domain (#4535)
* fix(security): enforce workspace scope on workflow middleware and validate shopify shop domain
- validateWorkflowAccess now rejects workspace-scoped API keys whose
workspaceId doesn't match the workflow's workspace, closing a boundary
leak across /api/workflows/[id]/{log,paused,status} and
/api/resume/[workflowId]/[executionId]/[contextId]
- shopify authorize route now validates the resolved shop domain against
shopifyShopDomainSchema before proceeding
- adds middleware tests covering workspace/personal/session auth paths
* fix(shopify): disallow trailing hyphen in shop subdomain regex
* fix(shopify): align shop domain regex with shopify handle rules (3-60 lowercase, no edge hyphens)
* fix(security): widen shopify subdomain regex to allow up to 63 chars
Shopify and RFC 1123 allow labels up to 63 chars; the previous
{1,58} quantifier capped the subdomain at 60 chars and rejected
valid 61–63 char shops with a 400.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
WaleedandClaude Opus 4.7
|
773720ec7f |
fix(uploads): allow images/video/audio in mothership presigned route (#4534)
* fix(uploads): allow image/video/audio attachments in mothership presigned route The mothership branch of the presigned upload route called validateFileType, which only permits SUPPORTED_DOCUMENT_EXTENSIONS — rejecting PNG screenshots and other media users have always been able to attach via the legacy /api/files/upload mothership branch. Introduce validateAttachmentFileType, backed by the union of document, code, image, audio, and video extensions, and wire it into the mothership branch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(uploads): allow non-document execution outputs in presigned route The execution branch of the presigned upload route called validateFileType, which only permits documents — but workflow execution outputs are arbitrary by design (images, audio, video, code). The legacy /api/files/upload execution branch had no docs-only gate, so the staging refactor to presigned PUTs regressed parity. Switch execution to validateAttachmentFileType to match prior behavior. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(uploads): add coverage for attachment validator and dedupe extensions - Dedupe SUPPORTED_ATTACHMENT_EXTENSIONS (webm appears in both audio and video lists) - Reuse SUPPORTED_ATTACHMENT_EXTENSIONS in /api/files/upload to avoid drift with the presigned route - Add unit tests for validateAttachmentFileType - Add presigned route tests covering mothership/execution/knowledge-base validator selection and permission gating * fix(uploads): restore SUPPORTED_IMAGE_EXTENSIONS import in upload route Build broke because biome auto-fix collapsed my multi-import edit and dropped SUPPORTED_IMAGE_EXTENSIONS, which is still referenced for the generic-MIME image fallback at line 298. --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Waleed
|
1d3ca79779 |
chore(deps): audit and clean up dependencies (#4531)
* chore(deps): audit and clean up dependencies - Remove unused: chalk, chart.js, dotenv, encoding, entities, thread-stream, uuid, @opentelemetry/exporter-jaeger, critters, marked, redis, soap - Replace soap with hand-rolled Workday SOAP client - Migrate marked to unified pipeline for inbox responses - Bump zustand v5, @react-email/* - Align all @aws-sdk/* to 3.1032.0 - Move type-only deps to devDependencies - Remove duplicate drizzle-orm/postgres overrides * fix(workday): coerce SOAP scalar strings to typed booleans/numbers - XML parser returns leaf text as strings; `!"false"` evaluated to `false`, causing all organizations to report `isActive: false` - Add parseSoapBoolean and parseSoapNumber helpers and apply at consumer sites (Inactive, Total_Results) - Drop unused service/soapAction fields from WD_OPERATIONS map * fix(workday): coerce compensation amounts and guard Date marshaling - get-compensation returned Amount/Per_Unit_Amount/Individual_Target_Amount as strings (XML leaf text), violating the tool's number contract - Coerce via parseSoapNumber and widen plan type to number | string - Add defensive Date branch in marshal() so Date inputs serialize as ISO 8601 instead of String(date) |
||
|
Waleed
|
cb3a876d3e |
feat(knowledge): include sourceUrl in KB search results (#4533)
* feat(knowledge): include sourceUrl in KB search results * improvement(kb-search): mark sourceUrl nullable and cover non-null happy path in tests |
||
|
Vikhyath Mondreti
|
d0b0ede39e |
fix(mothership): misc ui bugs (#4528)
* fix(vfs); compiled check tool call natural language tool desc * fix few more ui/ux bugs |
||
|
Waleed
|
b74f8dac40 |
improvement(sandbox): expand document generation — style extraction, sandbox hardening, OOM errors, task guards (#4526)
* improvement(sandbox): expand document generation — style extraction, sandbox hardening, OOM errors, PPTX/DOCX/PDF task guards * fix(style): make pptx aspect-ratio regex attribute-order independent * fix(sandbox): clarify pptx null-guard message; fix bold=false inheritance sentinel in docx style extractor * chore(lint): suppress noTemplateCurlyInString in resolver tests — strings intentionally assert template literal preservation * fix(contracts): export ListWorkspaceFilesResponse type from workspace-files contract |
||
|
Waleed
|
50d4afd4a8 |
fix(logs): include subfolders when filtering logs by folder (#4525)
* fix(logs): include subfolders when filtering logs by folder * fix(logs): use pop() for O(1) dequeue in folder BFS * fix(logs): move folder expansion to server-only module to fix client bundle build |
||
|
Waleed
|
630db5cee0 |
improvement(mothership-chat): memoize message rows for long-transcript load speed (#4520)
* improvement(mothership-chat): memoize message rows for long-transcript load speed * improvement(mothership-chat): hoist EMPTY_BLOCKS constant to keep MessageContent memoized for block-free messages |
||
|
Waleed
|
3c33c20378 | improvement(peopledatalabs): add titlecase, placeholders, and 404 handling on search (#4519) | ||
|
Vikhyath Mondreti
|
29affdac58 |
improvement(search-replace): dedupe double indexed segments (#4517)
* improvement(search-replace): dedupe double indexed segments * cleanup error message * address comments |
||
|
WaleedandClaude Opus 4.7
|
9e9ddaa1aa |
feat(peopledatalabs): add People Data Labs integration (#4513)
* feat(peopledatalabs): add People Data Labs integration Add 11 PDL operations: person enrich/identify/search/bulk, company enrich/search/bulk/clean, location/school cleaners, and autocomplete. All endpoints, params, and response shapes verified against official PDL docs (scroll_token pagination, top-level likelihood on company enrich, per-item likelihood on bulk company, full autocomplete field enum). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(peopledatalabs): narrow conditions for fields not used by every operation - min_likelihood now only shows for pdl_person_enrich (Person Identify ignores it) - ticker, pdl_id, company_location now only show for pdl_company_enrich (Company Cleaner only accepts name/website/profile) Addresses Greptile P1 review on PR #4513. * fix(peopledatalabs): scope param renames to their operation Param renames (company_profile→profile, company_location→location, school_*→*, bulk_*_requests→requests, autocomplete_size→size, etc.) now run only when the matching operation is selected, and stale alternate-operation values are stripped from the request. This prevents values left over from a prior operation switch from leaking into the current API call (e.g. a company LinkedIn URL overwriting a person profile, or a stale search size overwriting autocomplete size). Addresses Cursor Bugbot review on PR #4513. * fix(peopledatalabs): use currentColor for icon fill The PeopleDataLabsIcon was hardcoded to white, leaving it invisible on light backgrounds when rendered outside its bgColor container (e.g., search results, menus, docs). Switch to currentColor so it inherits the surrounding text color. Addresses Cursor Bugbot review on PR #4513. * fix(peopledatalabs): scope shared fields (profile/location/name/website) per operation The block has subBlocks whose raw IDs collide with PDL API param names (profile, location for person; name, website for company). Their values persist across operation switches even though the UI hides them, so a person LinkedIn URL could leak into a Company Enrich request, etc. Reset these shared targets and repopulate them only from inputs that belong to the active operation. Addresses Greptile P1 review on PR #4513. * fix(peopledatalabs): scope `size` to search and autocomplete operations `size` is shared by the person/company search subBlock and the autocomplete_size alias. The previous logic still forwarded a stale search `size` to operations that don't accept it (e.g. enrich, clean, identify), and the autocomplete branch only cleared it when autocomplete_size was unset. Reset `size` up front and only repopulate it for the three operations that actually accept it. Found via final integration audit of PR #4513. * fix(peopledatalabs): restore `location` for Person Identify Person Identify's tool accepts `location`, and the subBlock is shown for both Enrich and Identify, but the prior reset only repopulated `result.location` for Enrich — so any value entered on Identify was silently dropped before reaching the API. Addresses Greptile P1 review on PR #4513. * fix(peopledatalabs): align endpoints + outputs with PDL API - person_identify: short-circuit on PDL 404 (no-match), matching the person_enrich pattern - company_search: drop unsupported `dataset` param (PDL company search docs do not list it) - block: expose `min_likelihood` for `pdl_company_enrich` (PDL Company Enrichment supports min_likelihood) - location_clean: surface `subregion`; drop phantom `latitude`/ `longitude` (PDL only returns `geo` as a "lat,lon" string) - school_clean: surface `domain` and `location_continent` from the nested `location` object - docs icon: switch fill to `currentColor` so the icon renders on light backgrounds * fix(peopledatalabs): restore `name` for Person Enrich / Identify The shared `name` reset at the top of `tools.config.params` was only repopulated for the company-side operations, so any programmatic `name` input to `pdl_person_enrich` or `pdl_person_identify` was silently dropped. Both PDL endpoints accept `name` as a full-name match parameter. * fix(peopledatalabs): restore `name` for Person Enrich Add the `name` parameter to `PdlPersonEnrichParams`, the tool's params definition, and the URL builder. PDL Person Enrichment accepts `name` as a full-name match alternative to first_name + last_name; without it, programmatic `name` input was silently dropped before reaching the API. * fix(peopledatalabs): isolate company `name` UI from person ops Rename Company Name subBlock id from `name` to `company_name` so a stale company value can't leak into Person Enrich/Identify when the user switches operations. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(peopledatalabs): honor programmatic inputs for clean_location/school `pdl_clean_location` and `pdl_clean_school` were only restoring values from UI subBlock IDs (`clean_location_input`, `school_*`). Programmatic callers using the declared `location`/`name`/`website`/`profile` inputs had their values dropped after the shared-field reset. Add fallbacks so both UI and programmatic inputs flow through. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(peopledatalabs): programmatic input fallbacks + required autocomplete text - Company Enrich and Clean Company now fall back to programmatic `params.profile` / `params.location` when the UI-scoped `company_profile` / `company_location` are absent. Mirrors the fallback pattern already used for `name`. - Autocomplete `text` subBlock is now required when operation is autocomplete — PDL requires it for nearly all field values. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
WaleedandVikhyath Mondreti
|
57f00e2a96 |
fix(table): fix table boolean, add dynamic row number col size, search & replace imporvements (#4515)
* fix(table): fix table boolean, add dynamic row number col size, fix style for search and replace * sidebar styling * refactor(search-replace): extract duplicate WORKFLOW_SEARCH_HIGHLIGHT_CLASS to shared constants * fix(tables): add missing onError toast to useAddTableColumn * fix(tables): add consistent onError toast handlers across all table mutations * subblock sweep * more resources supported * code restructuring * better organization * canonical modes search replace counterpart * more cases * tool inp edge case * more subblock cases * trigger canoncial index ops consolidation * minor codec fix * fix(tables): prevent double toast in workflow-sidebar and csv upload mutations * resolver fix for quoted js literals * code scan context * triple quotes case * escaped behaviour --------- Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai> |
||
|
Theodore LiandClaude Opus 4.7
|
eb871fc0de |
feat(table): live cell updates via SSE + per-table event buffer (#4508)
* feat(table): live cell updates via SSE + per-table event buffer
Replaces the polling-based row refetch with a push-based SSE stream that
patches the React Query cache directly as cell-state events arrive.
Architecture:
- New per-table event buffer in apps/sim/lib/table/events.ts. Redis sorted-set
with monotonic eventId, 1h TTL, 5000-event cap, in-memory fallback. Modeled
after apps/sim/lib/execution/event-buffer.ts but stripped of complexity
tables don't need (no per-execution lifecycle, no id-batching, no write
queue serialization). ~150 lines instead of 700.
- writeWorkflowGroupState appends a fat event after each successful 'wrote'.
Status transitions carry executionId + jobId; terminal/partial transitions
also include the new output values inline so the client can patch row data
without a follow-up refetch.
- New SSE route at /api/table/[tableId]/events/stream?from=<lastEventId>.
Replays from buffer on connect, polls at 500ms (mirrors workflow execution
stream), heartbeat every 15s, signals 'pruned' if the caller fell off the
back of the buffer.
- Client hook useTableEventStream subscribes via EventSource. Reconnect-resume
with last-seen eventId. On 'pruned', invalidates the rows query and resumes
from the new earliest. Cache patches walk every cached query under
rowsRoot(tableId) so filter/sort variants all stay live.
- Removes refetchInterval from useTableRows and the per-page polling effect
from useInfiniteTableRows. React Query's refetchOnWindowFocus +
refetchOnReconnect cover the durability gap if any push is dropped.
Out of scope:
- Bulk-cancel events (cancellation path is being redesigned separately).
- Generalizing the workflow event-buffer module to a shared primitive (defer
until a third use case appears; for now the table buffer is the simpler
cousin of the workflow one).
* fix(table): drop run-mutation refetch so SSE patches aren't overwritten
useRunColumn.onSettled was canceling in-flight queries and invalidating the
rows query — leftover behavior from the polling era. With the SSE stream
now keeping the cache live via incremental patches, this refetch races the
stream and snaps the cache back to whatever DB shows at the refetch moment,
which can lag the just-arrived queued/running events. Cells appeared stuck
on the optimistic 'pending' even though the SSE was delivering the real
transitions.
* chore(table): simplify SSE plumbing — reuse helpers, drop dead polling code
- Reuse snapshotAndMutateRows for SSE cache patches instead of reimplementing
the page-walk + cache-shape detection. Adds a {cancelInFlight: false} opt
for the SSE caller (mutations still cancel as before).
- Drop client-side type duplication in use-table-event-stream — import
TableEvent and TableEventEntry from lib/table/events directly.
- Drop the now-dead mergePagePreservingIdentity + rowEqual from tables.ts;
their only caller was the polling effect that was removed earlier.
- Drop the defensive try/catch around appendTableEvent in cell-write — the
function is documented as never-throwing (returns null on failure).
- Combine INCR + ZADD into one Lua eval in events.ts. Halves Redis RTT per
cell-write. Lua returns the new eventId; the script splices it into the
pre-built entry JSON.
- Trim refs to plain let bindings inside the effect; trim stale
comments referencing the old polling implementation.
* fix(table): address PR review on SSE buffer
- TTL-expiry silent miss: when all keys expire, hgetall(meta) returns empty
so earliestEventId is undefined and the prune branch was skipped. Reconnect
with non-zero afterEventId now checks the seq counter — its absence (TTL
expired) signals pruned so the client refetches. Memory fallback mirrors.
- Unbounded ZRANGEBYSCORE: cap reads at TABLE_EVENT_READ_CHUNK = 500 events
per call. The route's 500ms poll loop drains chunks across ticks instead of
flushing 5000 entries (multi-MB) in one tick after a long disconnect.
- Pruned handler closes EventSource client-side: server-side close was firing
onerror and routing through the 500ms backoff path. Now we close
proactively, reset the reconnect attempt counter, and reconnect immediately
from the new earliest.
* improvement(table): persist SSE lastEventId in sessionStorage
Tab refresh / navigate-away-and-back now resume the stream from where the
previous mount left off instead of replaying from from=0. Mirrors the
useExecutionStream pattern (saveExecutionPointer / loadExecutionPointer).
First-ever mounts and new tabs still start at 0 — sessionStorage is
per-tab, so the safe default applies. On a 'pruned' fallback the new
earliestEventId is also persisted so the next reconnect starts there.
* fix(table): include runningBlockIds + blockErrors in SSE event payload
The cell renderer's 'queued' vs 'running' vs 'pending-upstream' decision
reads exec.runningBlockIds + exec.blockErrors. Without those fields the
inFlight branch falls through to 'pending-upstream' (amber Pending pill)
even when the worker has already written status=running. The worker writes
both fields to DB; the SSE event was stripping them. Thread them through
events.ts → cell-write.ts → use-table-event-stream.ts.
* fix(table): show value once column output has landed mid-run
The cell renderer treated any `status: 'running'` event as in-flight,
even when the column's own output had already been written. During a
multi-block group run, partial-write events for a later block carry
the earlier block's outputs but tag only the later block as running
— that flipped the finished column back to the amber Pending pill
until the terminal `completed` event arrived.
Re-order the priority chain so the column's value wins over
`pending-upstream`. Active re-run of the column itself
(`blockRunning`) still wins over the stale value, so a re-run on a
previously-completed cell still surfaces the running pill before the
new value overwrites.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(table): address PR review nits on tables.ts
- Merge duplicate JSDoc on snapshotAndMutateRows into a single block
- Remove unused useQueryClient() calls from useTableRows and
useInfiniteTableRows (leftover from polling-era code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(table): apply biome formatting fixes
CI lint job flagged import order in two files and over-wrapped union
in lib/table/events.ts. No behavior change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Vikhyath Mondreti
|
6cb779601a |
feat(search-replace): search & replace, cut, deploy modal ui flicker (#4507)
* feat(search): workflow search and replace * fix alignment * fix hidden fields bug * fix loops/parallel badge case * resource resolver * add cut * update docs * address comments * make source code for func blocks dispay resolved code instead * fix match issue * fix padding |
||
|
Waleed
|
11563cfbc2 |
improvement(uploads): migrate remaining FormData uploads to presigned PUT (#4509)
* fix(uploads): switch mothership uploads to presigned PUT pattern * fix(uploads): drop unreachable size guard in mothership presigned branch * improvement(uploads): migrate profile-picture and workspace-logo uploads to presigned PUT * improvement(uploads): migrate workflow file-upload sub-block to presigned PUT * improvement(uploads): migrate execution-trigger file uploads to presigned PUT * fix(uploads): tighten permission checks and fix multipart customKey for mothership/execution * fix(uploads): require workspace write+ for execution presigned, admin-only for workspace-logos, suppress doubled error toast * fix(uploads): skip per-file invalidation in batch + extract shared API fallback - Add skipInvalidation flag to useUploadWorkspaceFile; file-upload sub-block now invalidates once after the batch instead of per file - Extract uploadViaApiFallback to lib/uploads/client/api-fallback.ts (DRY across 3 hooks) |
||
|
WaleedandClaude Opus 4.7
|
6a927c9cd3 |
fix(tables): optimistic updates for column delete/update (#4512)
* fix(tables): optimistic updates for column delete/update Add onMutate/onError to useDeleteColumn and useUpdateColumn so column deletes feel instant on large tables (no flash-back during the JSONB rewrite) and concurrent type changes don't race the in-flight delete's invalidation. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(tables): toast on delete-column failure, case-insensitive row cleanup, rename row-data keys Address greptile review: - useDeleteColumn now toasts on non-validation errors so users see when the column "snaps back" after a server/network failure - Row data cleanup matches keys case-insensitively in both useDeleteColumn and useUpdateColumn so a column stored as "Age" is cleaned even when the request uses "age" - useUpdateColumn now migrates row-data keys when updates.name is set, preventing blank cells during the server round-trip on a rename Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Waleed
|
235a62fd95 |
fix(hunter): align tools, block, and outputs with Hunter.io v2 API spec (#4511)
* fix(hunter): align tools, block, and outputs with Hunter.io v2 API spec * fix(hunter): match documented Discover response and coerce numeric employees |
||
|
Waleed
|
a2aa648f86 |
improvement(sandbox): upgrade pptx/docx/pdf bootstrap with image helpers, MIME guards, and 256 MB isolate limit (#4505)
* improvement(sandbox): upgrade pptx/docx/pdf bootstrap with image helpers, MIME guards, and 256 MB isolate limit * fix(sandbox): strict MIME allowlist and nullish coalescing in docx addImage * fix(sandbox): validate required opts in pdf drawImage to prevent silent origin placement * fix(sandbox): throw on malformed data URI in docx addImage * fix(sandbox): prevent opts from clobbering computed ImageRun data/type/transformation * fix(sandbox): prevent opts from clobbering fetched data in pptx addImage * fix(sandbox): validate required opts in pptx addImage * fix(sandbox): remove silent image/png fallback in docx addImage MIME parsing * fix(sandbox): consistency and cleanup pass on doc-gen tasks and worker - DOCX addImage: upfront width/height validation (matches PDF/PPTX pattern) - PDF embedImage: remove dead Buffer ternary; drop redundant size guard already enforced in getFileBase64 - isolated-vm-worker: add friendly MemoryLimitError branch in both execute paths so OOM produces a clear message instead of a raw V8 error |
||
|
WaleedandClaude Opus 4.7
|
6a006851fa |
improvement(apollo): align tools and block with Apollo API docs (#4487)
* improvement(apollo): align tools and block with Apollo API docs
* improvement(apollo): fix tool outputs to match Apollo API response shapes
* chore(apollo): regenerate docs for output changes
* fix(apollo): address PR review comments
* fix(apollo): allow skipped_contact_ids as hash per Apollo docs
* docs
* fix(apollo): add runtime guard for account_bulk_update empty body
* fix(apollo): require contact_attributes for bulk_update
* fix(apollo): add subblock id migrations for renamed opportunity fields
* fix(apollo): tighten account_bulk_update guard and accept object attrs
* fix(apollo): require contact_ids with object-form contact_attributes
* docs(apollo): clarify contact_bulk_update parameter requirements
* fix(apollo): handle flat and wrapped contact response shapes
* validate
* fix(apollo): mirror bulk_update guard, preserve update fields in migration, expose account_bulk_create options
* fix(apollo): don't clobber user contact_attributes in migration; simplify task_create created flag
* fix(apollo): drop undocumented task type, preserve mixed-array IDs, migrate note→task_notes
* fix(apollo): align tools and block with live API docs
Final pass over the Apollo integration after a per-tool forensic audit
against Apollo.io docs. Notable fixes:
- organization_enrich: GET+querystring -> POST+JSON body (canonical, non
master-key)
- organization_bulk_enrich: ?domains[]= -> JSON body { organizations }
- people_search: declare/forward organization_num_employees_ranges; fix
contact_email_status placeholder ("likely to engage", with spaces)
- account_bulk_create: surface failed_accounts and failed count
- contact_bulk_create: expand documented per-contact fields (CRM IDs,
phone_numbers, contact_emails, typed_custom_fields, etc.)
- sequence_add_contacts: surface remaining documented filter params
- task_create: confirm wire field name (note) and remap from task_notes
- types: tighten params/responses for the above
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs
* fix(apollo): add _removed_* migrations for retired opportunity subblocks
* fix(apollo): expose webhook_url subblock for people enrich phone reveal
* fix(apollo): drop colliding account_ids migration, enforce contact bulk limit, expose async toggle for accounts
* fix(apollo): cap account_attributes at 1000 in bulk update
* fix(apollo): drop bare-id merging in bulk update migration to avoid empty attribute objects
* fix(apollo): reject ambiguous account/contact_ids + array-form attributes
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
Waleed
|
408669dd85 | fix(md-render): inline code inherits heading size in mothership/templates/changelog (#4504) |