Commit Graph
7 Commits
Author SHA1 Message Date
Vikhyath Mondreti 37628f1566 fix(workflow-persistence): make persistMigratedBlocks' updated_at guard millisecond-tolerant (#5790)
* fix(workflow-persistence): make persistMigratedBlocks' updated_at guard millisecond-tolerant

* improvement(sockets): state improvements

* address comments
2026-07-21 14:36:11 -07:00
Waleed d078c84ee6 chore(typescript): upgrade to TypeScript 7 (native Go compiler) (#5521)
* chore(typescript): upgrade to TypeScript 7 (native Go compiler)

Bumps typescript to ^7.0.2 across every workspace package. Full
bun run type-check/lint/build/test all pass; apps/sim's type-check
(the one needing an 8GB heap bump) drops from ~55s to ~7s wall time.

Migration fixes required by TS7's stricter defaults:
- baseUrl removed: drop it from 5 tsconfigs (paths already resolved
  relative to tsconfig dir, so behavior is unchanged) and prefix the
  one bare (non-relative) paths entry each in apps/sim and
  apps/realtime with './'
- moduleResolution=node10 removed: switch packages/cli and
  packages/ts-sdk to "bundler", matching the rest of the monorepo
- types now defaults to [] instead of auto-including every @types/*
  package: add "types": ["node"] to the shared base tsconfig (this
  is fundamentally a Node monorepo, so this restores prior behavior
  in one place instead of duplicating it per-package), add explicit
  @types/node deps to packages that now rely on it transitively via
  @sim/db/@sim/logger, and add "declare module '*.css'" to the two
  packages with plain (non-module) CSS side-effect imports that
  TS7's stricter checker now flags
- packages/logger's isomorphic `typeof window` check no longer needs
  DOM lib in every consumer: replaced with `'window' in globalThis`
- packages/testing and apps/realtime's fetch/DOM mocks need DOM lib
  where they're compiled, since they model the browser Fetch API
- the `typescript` npm package no longer exports the classic
  Compiler API from its main entry (moved to unstable/ast subpaths);
  apps/sim's Function-block route used it at runtime to strip
  import statements from user code, so that one call site now uses
  Microsoft's official transition package, @typescript/typescript6
- Next.js 16.2.6's own TypeScript-detection heuristic hardcodes a
  path TS7 no longer ships, and its auto-install fallback assumes
  npm/pnpm; added @typescript/native-preview as a devDependency to
  apps/sim and apps/docs so Next detects a valid native compiler
  instead of trying (and failing) to auto-install one

Not merging yet: TS 7.0.2 published today and is still inside this
repo's bunfig.toml minimumReleaseAge (7-day) supply-chain gate, so
`bun install` will fail for everyone until 2026-07-15. Opening this
now to get it through review; hold the actual merge until then.

* fix(typescript): address Greptile review findings on TS7 upgrade

- packages/logger: 'window' in globalThis treats a shim that leaves
  globalThis.window explicitly undefined as browser-only, silently
  dropping production server logs. Restore the original
  typeof !== 'undefined' semantics via an inline cast instead, so it
  stays correct without requiring DOM lib in every consumer.
- packages/ts-sdk, packages/cli: both are tsc-built, published as
  Node ESM (package.json "type": "module" with an "exports" map).
  "moduleResolution": "bundler" is too permissive for that target -
  it accepts import patterns (e.g. extensionless relative imports)
  that Node's actual ESM resolver rejects at runtime. Switch both to
  "module"/"moduleResolution": "nodenext", the correct pairing for a
  published Node ESM package. Verified real tsc builds (not just
  --noEmit) still succeed for both.

* chore(bunfig): temporarily disable minimumReleaseAge gate for TS7 install

TS 7.0.2 published today, still inside the 7-day gate. Lowering to 0
to unblock this merge; will restore to 604800 in an immediate follow-up
commit right after merging.
2026-07-08 16:41:23 -07:00
Waleed 9d34fbea1b refactor(utils): consolidate duplicated helpers onto @sim/utils (#5509)
* refactor(utils): consolidate duplicated helpers onto @sim/utils

Replaces ~90 hand-rolled reimplementations of error-message extraction,
postgres error-code checks, sleep, Math.random, retry/backoff, object
filtering/omission, noop, string truncation, date/time formatting, email
normalization, and plain-object type guards with the shared @sim/utils
exports. Wires check:utils into CI (test-build.yml) so these patterns
don't regress.

* fix(test): mock @sim/utils/random instead of Math.random in schedule-execute tests

The schedules/execute route previously used Math.random() for jitter delay;
this consolidation PR switched it to randomInt() from @sim/utils/random,
which is backed by crypto.getRandomValues() rather than Math.random(). The
route.test.ts spies on Math.random() no longer had any effect, so jitter
became real random delay instead of the deterministic 0ms the tests expect,
causing intermittent 10s timeouts in CI.

* fix(retry): preserve uncapped Retry-After comparison in tools/index.ts

parseRetryAfter() caps its return value at 30s by default. tools/index.ts
compares the parsed Retry-After against a caller-configured maxDelayMs to
decide whether to skip a retry entirely -- capping before that comparison
silently defeats the skip check whenever maxDelayMs is configured above
30s, since a Retry-After between 30s and maxDelayMs would incorrectly look
"within limits" and get retried instead of skipped (caught by Cursor
Bugbot). Added an optional maxMs param (default unchanged) so tools/index.ts
can request the raw, uncapped value for its own comparison while
backoffWithJitter still clamps the actual sleep duration to maxDelayMs.
Added a regression test covering maxDelayMs > 30s.

* fix(utils): fall back to Intl-resolved abbreviation for unmapped timezones

getTimezoneAbbreviation only covered 9 hardcoded IANA zones and returned
the raw IANA string for everything else, degrading schedule descriptions
for zones like Europe/Berlin or America/Toronto (caught by Greptile). The
deleted local implementation in schedules/utils.ts resolved any valid IANA
timezone generically via Intl.DateTimeFormat's short timeZoneName. Restore
that as a fallback so only genuinely invalid timezone strings return
themselves unchanged.
2026-07-08 11:26:23 -07:00
Vikhyath Mondreti 43f53bb7d3 feat(execution): payload size bottlenecks with lazy execution value hydration, safer materialization, and batched parallel execution (#4560)
* improvement(resolver): lazy resolution for underlying fields greater than 10MB

* progress

* feat(parallel): batching

* codegen to allow inline substitution

* address comments

* ui inconsistencies

* cleanup redundant code

* address more comments

* address comments

* replace helper

* fix tests
2026-05-12 14:57:32 -07:00
Vikhyath Mondreti 1e5cc392d5 improvement(deployment): solve multiple client side races, and deployed state management issues (#4502)
* improvement(deploy): state transitions

* more fixes

* improvements and shift to outbox policy with eager call

* address comments

* ux improvement

* address comments

* address bugbot
2026-05-07 17:20:04 -07:00
Vikhyath Mondreti 09425553b3 fix(sanitize): repair malformed subblock states (#4490)
* fix(sanitize): repair malformed subblock states

* address comments

* address comment
2026-05-07 00:54:38 -07:00
5f0f0edd63 improvement(repo): separate realtime into separate app (#4262)
* improvement(repo): restructuring to make realtime image narrower scoped

* improvements

* chore(repo): rebase fixes and quality improvements for realtime split

Addresses merge-time issues and gaps from the realtime app split:
- Retarget stale vi.mock paths to @sim/workflow-persistence/subblocks
- Restore README branding, fix AGENTS.md script reference
- Restore TSDoc on workflow-persistence subblocks helpers
- Use toError() from @sim/utils/errors in save.ts
- Add vitest config + local mocks so @sim/audit tests run standalone
- Move socket.io-client to devDependencies in apps/realtime
- Add missing package COPY steps to docker/app.Dockerfile
- Add check:boundaries/check:realtime-prune scripts and wire into CI

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(security): consolidate crypto primitives into @sim/security

Move general-purpose crypto primitives out of apps/sim into the
@sim/security package so both apps/sim and apps/realtime can share them.

@sim/security exports (all pure, dependency-free):
  ./compare    safeCompare (constant-time HMAC-wrapped equality)
  ./encryption encrypt/decrypt (AES-256-GCM, iv:cipher:tag format)
  ./hash       sha256Hex
  ./tokens     generateSecureToken (base64url)

Migrate apps/sim call sites to use these + @sim/utils helpers:
  crypto.randomUUID()            -> generateId() from @sim/utils/id
  createHash('sha256').digest    -> sha256Hex
  timingSafeEqual on hashed hex  -> safeCompare
  new Promise(setTimeout)        -> sleep from @sim/utils/helpers

No behavior change: encryption format, digest output, and token
length are preserved exactly.

* refactor(copilot): use toError in remaining otel/finalize sites

Replace the last two `error instanceof Error ? error : new Error(String(error))`
patterns with toError from @sim/utils/errors. Completes the sweep of clean
candidates — no behavior change.

* refactor(security): consolidate HMAC-SHA256 primitives into @sim/security

Adds hmacSha256Hex and hmacSha256Base64 to @sim/security/hmac and migrates
15 webhook providers plus 5 other hot paths (deployment token signing,
outbound webhook requests, workspace notification delivery, notification
test route, Shopify OAuth callback) off bare `createHmac` calls. Secret
parameter accepts `string | Buffer` to cover base64-decoded Svix-style
secrets (Resend) and MS Teams' HMAC scheme. AWS SigV4 signing in S3 and
Textract tools intentionally retains direct `createHmac` usage — its
multi-step key derivation chain doesn't fit a generic helper.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore(packages): post-audit test + packaging polish

- Add safeCompare unit tests (identity, length mismatch, hex-nibble diff).
- Add Buffer-secret cases to hmac tests to lock in Svix/MS-Teams contract.
- Declare `reactflow` as a peerDependency on @sim/workflow-types — only used for type imports.
- Add a barrel export to @sim/workflow-persistence for consumers that prefer package-level imports; subpath exports retained.
- Document the data-field invariant in load.ts for loop/parallel subflow patching.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore(realtime): address PR review feedback

- Remove redundant SOCKET_PORT=3002 env from Dockerfile runner stage
  (env.PORT already defaults to 3002 via zod schema).
- Reorder PORT fallback so an explicitly-set SOCKET_PORT wins over
  the schema default for PORT; keeps SOCKET_PORT functional as an
  override instead of dead code.
- Add dedicated type-check CI step for @sim/realtime so TS errors
  surface pre-deploy (the Dockerfile runs source TS via Bun and has
  no implicit build-time type check).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore(realtime): remove unused SOCKET_PORT env var

SOCKET_PORT has lived in the socket server since the June 2025 refactor
but was never actually set in any deploy config — docker-compose.prod,
helm values/templates, .env.example, and docs all use PORT or the 3002
default exclusively. No self-hoster was ever pointed at SOCKET_PORT, so
removing it is safe.

Simplifies realtime port resolution to `env.PORT` (zod-validated with a
3002 default) and drops the orphaned sim-side schema entry.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Waleed Latif <walif6@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-22 23:06:16 -07:00