Commit Graph
23 Commits
Author SHA1 Message Date
Waleed 856fe0ffb6 fix(docker): upgrade bun to 1.3.14 (#6236)
* fix(docker): upgrade bun to 1.3.14 to unbreak the Next 16.3.0 server

Bun 1.3.13 cannot load Next 16.3.0's compiled server runtime. The app container
runs the Next server under Bun (`oven/bun:1.3.13-slim`, `bun apps/sim/bootstrap.js`),
so every app-page render threw and `/api/health` returned 500:

  ⨯ Error: Failed to load external module
    next/dist/compiled/next-server/app-page-turbo.runtime.prod.js:
    TypeError: Expected CommonJS module to have a function wrapper.
    If you weren't messing around with Bun's internals, this is a bug in Bun

Isolated to Bun, not Next, by loading that exact module in the real images:

  Next 16.2.12 + Bun 1.3.13 -> loads (why staging was fine before)
  Next 16.3.0  + Bun 1.3.13 -> CJS wrapper error
  Next 16.3.0  + Bun 1.3.14 -> loads

Bun 1.3.14 is the current stable and already fixes it, so this bumps every pin
rather than reverting the framework upgrade, which would only defer the same
latent Bun bug to the next attempt.

Why no gate caught it: local dev machines and this bump's own verification run
Bun 1.3.14, while the container and CI pinned 1.3.13 — and CI only *builds* the
image, it never boots one and probes `/api/health`. A container smoke test in CI
would have caught this before merge; that is worth adding separately.

* fix(docker): align the remaining bun pins with 1.3.14

Two pins were missed in the first pass because the search was scoped to
docker/, package.json and .github/workflows/:

- .devcontainer/Dockerfile still built on oven/bun:1.3.13-alpine
- PI_BUN_VERSION in apps/sim/scripts/pi-sandbox-packages.ts was still 1.3.13,
  despite being documented as mirroring the root packageManager field, so Pi
  sandbox images would have kept installing the Bun release that cannot load
  the Next 16.3.0 server runtime.

Fixed surgically rather than with a repo-wide replace: "1.3.13" also appears
inside SVG path data in apps/sim/components/icons.tsx and
apps/docs/components/icons.tsx, which a blind sed would have corrupted.
2026-08-03 19:23:52 -07:00
Theodore Li e1c3c7f6c9 feat(secrets): ingest env secrets at container runtime instead of fanning into ECS taskdef (#5189)
* feat(secrets): ingest env secrets at container runtime instead of fanning into ECS taskdef

The app/socket ECS taskdefs were ~42KB, ~93% of which was the secrets[] array:
268 pointer entries each restating the full ~78-char secret ARN, marching toward
the 64KB taskdef limit and growing ~150 bytes per hosted key added. The secret
blob itself is only ~18KB/268 keys.

Move secret delivery to container boot: new @sim/runtime-secrets loadRuntimeSecrets()
reads SIM_ENV_SECRET_ID, fetches the combined secret once, and hydrates process.env
(no-clobber, no-op when unset, fail-fast). Bootstrap entrypoints for app + realtime
await it before importing the real server (env-flags reads env at module load). The
app bootstrap is bun-bundled in the Dockerfile builder stage since it runs outside
the Next standalone bundle; realtime keeps full node_modules and runs the TS entry.

Backward-compatible: with the current fan-out taskdef the loader no-ops and the app
reads the injected env vars unchanged. The matching infra change (empty secrets[] +
SIM_ENV_SECRET_ID) ships separately, after this image is live.

* fix(runtime-secrets): address review feedback

- Move the binary-secret guard outside the retry loop (sendWithRetry) so a
  missing SecretString throws immediately instead of burning 3 attempts + backoff.
- Bound each Secrets Manager request with AbortSignal.timeout(5s) so a stalled
  response can't hang boot indefinitely.
- Drop the redundant @aws-sdk/client-secrets-manager pin from apps/realtime; it
  resolves transitively via @sim/runtime-secrets.
- Add a test for the non-retriable binary-secret path.
2026-06-24 16:23:37 -04:00
Waleed 5fba724818 chore(bun): bump bun to 1.3.13 (#4291) 2026-04-24 15:59:10 -07:00
5f0f0edd63 improvement(repo): separate realtime into separate app (#4262)
* improvement(repo): restructuring to make realtime image narrower scoped

* improvements

* chore(repo): rebase fixes and quality improvements for realtime split

Addresses merge-time issues and gaps from the realtime app split:
- Retarget stale vi.mock paths to @sim/workflow-persistence/subblocks
- Restore README branding, fix AGENTS.md script reference
- Restore TSDoc on workflow-persistence subblocks helpers
- Use toError() from @sim/utils/errors in save.ts
- Add vitest config + local mocks so @sim/audit tests run standalone
- Move socket.io-client to devDependencies in apps/realtime
- Add missing package COPY steps to docker/app.Dockerfile
- Add check:boundaries/check:realtime-prune scripts and wire into CI

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(security): consolidate crypto primitives into @sim/security

Move general-purpose crypto primitives out of apps/sim into the
@sim/security package so both apps/sim and apps/realtime can share them.

@sim/security exports (all pure, dependency-free):
  ./compare    safeCompare (constant-time HMAC-wrapped equality)
  ./encryption encrypt/decrypt (AES-256-GCM, iv:cipher:tag format)
  ./hash       sha256Hex
  ./tokens     generateSecureToken (base64url)

Migrate apps/sim call sites to use these + @sim/utils helpers:
  crypto.randomUUID()            -> generateId() from @sim/utils/id
  createHash('sha256').digest    -> sha256Hex
  timingSafeEqual on hashed hex  -> safeCompare
  new Promise(setTimeout)        -> sleep from @sim/utils/helpers

No behavior change: encryption format, digest output, and token
length are preserved exactly.

* refactor(copilot): use toError in remaining otel/finalize sites

Replace the last two `error instanceof Error ? error : new Error(String(error))`
patterns with toError from @sim/utils/errors. Completes the sweep of clean
candidates — no behavior change.

* refactor(security): consolidate HMAC-SHA256 primitives into @sim/security

Adds hmacSha256Hex and hmacSha256Base64 to @sim/security/hmac and migrates
15 webhook providers plus 5 other hot paths (deployment token signing,
outbound webhook requests, workspace notification delivery, notification
test route, Shopify OAuth callback) off bare `createHmac` calls. Secret
parameter accepts `string | Buffer` to cover base64-decoded Svix-style
secrets (Resend) and MS Teams' HMAC scheme. AWS SigV4 signing in S3 and
Textract tools intentionally retains direct `createHmac` usage — its
multi-step key derivation chain doesn't fit a generic helper.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore(packages): post-audit test + packaging polish

- Add safeCompare unit tests (identity, length mismatch, hex-nibble diff).
- Add Buffer-secret cases to hmac tests to lock in Svix/MS-Teams contract.
- Declare `reactflow` as a peerDependency on @sim/workflow-types — only used for type imports.
- Add a barrel export to @sim/workflow-persistence for consumers that prefer package-level imports; subpath exports retained.
- Document the data-field invariant in load.ts for loop/parallel subflow patching.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore(realtime): address PR review feedback

- Remove redundant SOCKET_PORT=3002 env from Dockerfile runner stage
  (env.PORT already defaults to 3002 via zod schema).
- Reorder PORT fallback so an explicitly-set SOCKET_PORT wins over
  the schema default for PORT; keeps SOCKET_PORT functional as an
  override instead of dead code.
- Add dedicated type-check CI step for @sim/realtime so TS errors
  surface pre-deploy (the Dockerfile runs source TS via Bun and has
  no implicit build-time type check).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore(realtime): remove unused SOCKET_PORT env var

SOCKET_PORT has lived in the socket server since the June 2025 refactor
but was never actually set in any deploy config — docker-compose.prod,
helm values/templates, .env.example, and docs all use PORT or the 3002
default exclusively. No self-hoster was ever pointed at SOCKET_PORT, so
removing it is safe.

Simplifies realtime port resolution to `env.PORT` (zod-validated with a
3002 default) and drops the orphaned sim-side schema entry.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Waleed Latif <walif6@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-22 23:06:16 -07:00
Waleed f91c1b614a chore(docker): add packages/utils to app and realtime Dockerfiles (#4229)
* chore(docker): add packages/utils to app and realtime Dockerfiles

* chore(docker): copy packages/utils in realtime runner stage
2026-04-18 18:00:44 -07:00
Vikhyath Mondreti d2c3c1c39e improvement(worker): configuration defaults (#3821)
* improvement(worker): configuration defaults

* update readmes

* realtime curl import
2026-03-27 19:54:56 -07:00
Waleed 8a481b612d chore(config): clean up bun, turbo, and next.js config (#3788)
* chore(config): clean up bun, turbo, and next.js config

* chore(ci): bump bun to 1.3.11 in dockerfiles and workflows
2026-03-26 13:04:23 -07:00
Waleed ae887185a1 fix(memory): upgrade bun from 1.3.9 to 1.3.10 (#3441) 2026-03-06 11:35:46 -08:00
Waleed f5dc180d9f fix(memory): upgrade bun from 1.3.3 to 1.3.9 (#3186) 2026-02-10 19:30:31 -08:00
ppippi-dev f8b1880575 fix(docker): resolve @sim/logger module not found in realtime container (#2637) 2025-12-29 23:06:28 -08:00
Waleed 71130c8b0a improvement(monorepo): added tsconfig package, resolved type errors in testing package (#2613) 2025-12-28 00:36:48 -08:00
Waleed cdc1a832d7 fix(docker): add logger package to realtime dockerfile (#2610) 2025-12-27 17:55:16 -08:00
Waleed d707d18ee6 fix(build): update dockerfile to contain testing package deps (#2591)
* fix(build): update dockerfile to contain testing package deps

* added logger package
2025-12-26 12:20:38 -08:00
Waleed b7f6bab282 feat(tests): added testing package, overhauled tests (#2586)
* feat(tests): added testing package, overhauled tests

* fix build
2025-12-25 16:06:47 -08:00
Waleed dda44f7382 fix(build): remove incompatible --frozen-lockfile and --omit dev from docker (#2341)
* fix(build): remove incompatible --frozen-lockfile and --omit dev from docker

* ack PR comments
2025-12-12 16:58:23 -08:00
Waleed 323e03520d feat(releases): tag releases to main with version numbers, speed up docker builds (#2337)
* feat(releases): tag releases to main with version numbers, speed up docker builds

* resize runners
2025-12-12 15:57:10 -08:00
Waleed c864d17991 chore(deps): upgrade to bun v1.3 (#2181)
* chore(deps): upgrade to bun v1.3

* ack PR comments
2025-12-04 00:43:01 -08:00
WaleedandConnor Mulholland 570b8d61f0 improvement(runners): added blacksmith optimizations to workflows and dockerfiles to enhance performance (#2055)
* added blacksmith optimizations to workflows and dockerfiles to enhance performance. please review before pushing to production

* remove cache from and cache to directives from docker based actions, per blacksmith docs

---------

Co-authored-by: Connor Mulholland <connormul@Connors-MacBook-Pro.local>
2025-11-19 13:07:03 -08:00
Waleed 1130befb17 fix(local): add missing deps, fix access patterns, update dockerfiles, updated turborepo (#1895)
* fix(local): add missing deps, fix access patterns, update dockerfiles

* upgrade turborepo
2025-11-11 12:01:59 -08:00
Waleed f345c4d1d8 fix(docker): updated docker to use non-root user for k8s/helm deployments (#1626)
* fix(docker): updated docker to use non-root user for k8s/helm deployments

* ack PR comments
2025-10-14 15:54:51 -07:00
Waleed 923595f57e fix(webhooks): use next public app url instead of request origin for webhook registration (#1596)
* fix(webhooks): use next public app url instead of request origin for webhook registration

* ack PR comments

* ci: pin Bun to v1.2.22 to avoid Bun 1.3 breaking changes
2025-10-10 17:19:51 -07:00
Vikhyath Mondreti 9de7a00373 improvement(code-structure): move db into separate package (#1364)
* improvement(code-structure): move db into separate package

* make db separate package

* remake bun lock

* update imports to not maintain two separate ones

* fix CI for tests by adding dummy url

* vercel build fix attempt

* update bun lock

* regenerate bun lock

* fix mocks

* remove db commands from apps/sim package json
2025-09-17 15:41:13 -07:00
Aditya Tripathi 00334e501f feat(ci): socket realtime image for hosting (#565)
* feat: socket server for self/local deployment

* ci: memory limit and redundant dependency install

* chore: update readme, devcontainer, cli package

* chore: add new dev scripts and update README for full development setup
2025-06-27 09:03:29 -07:00