mirror of
https://github.com/simstudioai/sim.git
synced 2026-10-24 01:23:02 +08:00
improvement(mothership): v0.2 (#4923)
* CONTRACTS * updates * Prompt caching * Fix regression * updates to prompt caching * Prompt caching trace * VFS updates * Changelog and plan * VFS update * Improvement/mothership (#4775) * ff * System role in cache * Fixes * Add dynamic user info * Improve tool search * improvement(platform): workspace UI/UX overhaul + integrations catalog Rework the workspace around the AI-workspace model: a Mothership home, a top-level Skills route, connected-credential and integration-detail pages, and a polished sidebar/settings surface. Replace the notifications store with a unified toast system (provider-level dismiss/pause, countdown ring). Integrations & catalog: - Add a BlockMeta layer (tags + catalog templates) scoped to catalog-visible integrations; every catalog integration carries >=7 grounded templates. - Rework the taxonomy: each block declares category tools|blocks|triggers. 3rd-party services are 'tools'; first-party primitives (postgres, mysql, knowledge, file, search, stt/tts, image/video generators, thinking, etc.) are 'blocks'. Versioned blocks follow the upgrade paradigm (old hidden, latest in toolbar/docs). - Generate integrations.json + tool docs canonically from block configs. Architecture & cleanup: - Consolidate block data extraction behind a single latest-version strategy (getCanonicalBlocksByCategory; version-consistent getBlockMeta). - Unify version-suffix handling in @sim/utils/string (stripVersionSuffix / isVersionedType, with tests); registry, generate-docs, tools/utils, and integrations all route through it. - Repair latent broken barrels, remove dead code, fix BlockMeta-related type errors and 5 broken docs links. Behavior-preserving for block execution and the toolbar's tool/block listing. * refactor(platform): remove forms, templates, and creators features Remove three standalone features and their supporting code: - Forms: form-deployment pages, API routes, execution path, and docs. - Templates: the template gallery (landing + workspace) and template APIs. - Creators: creator-profile routes and contracts. Add a super-user permissions module (lib/permissions/super-user) and an organizations API contract; update the audit/db/testing packages, billing, and the session/theme providers accordingly. * New doc setup * Fixes * Fixes * Logs tools * test(workflows): update archiveWorkflow update count after forms removal The forms feature was removed, dropping the form-table update from archiveWorkflow. Update the stale assertion from 8 to 7 tx.update calls. * Add finalization on error * Fixes * change dev CI to bun run db:push * Updates Please enter the commit message for your changes. Lines starting * Contrat update * improvement(nested): subagents * updates * upgrade * improvement(knowledge): polish tag filter dropdowns (#4816) * improvement(logs): object storage backed tracespans (#4787) * improvement(logs): obj storage backed tracespans * fix storage write context * fix tests * address comments * address comments * chore(db): remove migration 0219 to regenerate after staging merge Drops the 0219_robust_shard SQL, its snapshot, and the journal entry so the trace-spans/cost schema migration can be regenerated on top of the latest staging migration chain (avoids a number collision with staging's migrations). Co-authored-by: Cursor <cursoragent@cursor.com> * improvement(billing): accurate per-member usage via shared ledger helper Per-member/per-user usage in the org-member routes now adds the usage_log ledger to the currentPeriodCost baseline (which is no longer incremented), via a shared getOrgMemberLedgerByUser helper to avoid repeating the subscription→period→ledger lookup across the admin and member-facing routes. Co-authored-by: Cursor <cursoragent@cursor.com> * regen migrations * update migration * address comments * more code cleanup * incorrect type cast --------- Co-authored-by: Cursor <cursoragent@cursor.com> * improvement(providers): harden OpenAI-compatible providers + add tests (#4796) * improvement(providers): harden OpenAI-compatible providers + add tests * fix(vllm): let tool-loop errors propagate instead of returning silent partial success * fix(litellm): force tool_choice 'none' on final structured-output call The deferred final call used tool_choice 'auto', so the model could emit another tool_calls round instead of the structured answer, leaving content stale. Use 'none' (matching vLLM/Fireworks) on both the streaming and non-streaming final calls so the model must return the structured response. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(providers/ollama): drop tools from post-tool streaming call Ollama ignores tool_choice (not in its supported fields), so vLLM/Fireworks' tool_choice:'none' guard is a no-op here. Omit tools from the final streaming payload instead so the summarization turn can't emit dropped tool calls. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(litellm): spread payload into deferred final call so reasoning_effort carries over The non-streaming deferred finalPayload hand-picked fields and dropped reasoning_effort (and any future payload field), diverging from the streaming path which spreads ...payload. Spread payload here too for consistency. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(providers/ollama): restore enrichment TSDoc block Keeps parity with sibling Chat Completions providers (cerebras/mistral/xai). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(fireworks): restore TSDoc on utils helpers Restore the TSDoc blocks on supportsNativeStructuredOutputs, createReadableStreamFromOpenAIStream, and checkForForcedToolUsage — TSDoc is the codebase documentation standard and should not have been stripped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(litellm): remove inline rationale comments (codebase uses TSDoc) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore(providers/ollama): drop orphaned enrichment TSDoc The block documented a function that now lives in trace-enrichment.ts, so it documents nothing in this file. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * chore(copilot): deprecate mcp server (#4797) * chore(copilot): deprecate mcp * update error codes * deprecate copilot api v1 route * feat(integrations): hosted API keys for Findymail, Prospeo, and Wiza (#4777) * feat(integrations): hosted API keys for Findymail, Prospeo, and Wiza Add hosted-key support across all credit-consuming Findymail, Prospeo, and Wiza operations so Sim provides the key when a workspace has not brought its own. Register the three BYOK providers, consolidate Wiza's two-step reveal into a single polling wiza_individual_reveal op, and hide the API key field on hosted Sim for hosted operations. * fix(integrations): harden Wiza reveal polling, soften enrichment getCost guards Address Greptile + Cursor Bugbot review on #4777: return explicit failures from the Wiza individual_reveal poller instead of throwing (thrown errors were swallowed into a false queued success), short-circuit when the initial reveal is already terminal, tolerate transient 5xx/429 during polling, and return 0 (not throw) from Findymail getCost when the contacts/employees array is absent. * chore(integrations): biome formatting after wiza merge resolution * fix(wiza): type isTerminalReveal param structurally for next build typecheck * feat(enrichments): add Findymail, Prospeo, Wiza to work-email waterfall * feat(enrichments): add Wiza + Prospeo phone reveal to phone-number waterfall * feat(enrichments): opportunistic identifiers + LinkedIn URL input across work-email & phone cascades * fix(tables): reduce column header chevron size and fix sidebar shadow bleed (#4800) * feat(slack): add install + privacy section to integration landing page (#4799) * feat(slack): add install + privacy section to integration landing page Adds a hand-authored, slug-keyed landing-content module (separate from the generated integrations.json so it survives regeneration) and renders an install walkthrough + privacy-policy link on integration pages when present. Also refreshes generated docs (data-enrichment entry, icon mappings, tool mdx). * fix(landing): render privacy section independently, align CTA analytics label * docs(landing): clarify the Slack install button is behind sign-in * refactor(landing): bake integration landing content into generated json via docs-gen Moves landing content (install walkthrough + privacy) out of a render-time augment and into the generation pipeline: generate-docs reads the pure-data content map and writes landingContent into integrations.json, so the page reads a single source (integration.landingContent). Canonical types live in integrations/data/types.ts. * improvement(enrichments): align enrichments sidebar with design system (#4801) * improvement(enrichments): align enrichments sidebar with design system * fix(enrichments): consistent close button pattern and fix url link hover * fix(misc): upgrade path change for new better-auth version, billing issue for workflow block agent usage (#4803) * fix(misc): upgrade path change for new better-auth version, double-billing for workflow block agent usage * fail loudly if stripe sub id missing * fix(copilot): seq migration (#4804) * chore(db): drop redundant idx_webhook_on_workflow_id_block_id index (#4809) Removed because (workflow_id, block_id) is a left-prefix of idx_webhook_on_workflow_id_block_id_updated_at_desc, which fully covers it. The dropped index was non-unique and enforced no constraint. * perf(copilot): read chat transcripts from copilot_messages (R+1 cutover) (#4808) * perf(copilot): read chat transcripts from copilot_messages, not JSONB Flip user-facing chat reads from the legacy copilot_chats.messages JSONB array (5.7GB, 99% TOAST) to the normalized copilot_messages table via a new loadCopilotChatMessages helper ordered by seq NULLS LAST, created_at, id — the verified canonical order. Both chat-detail getters (getAccessibleCopilotChat, getAccessibleCopilotChatWithMessages) now drop the messages column from their metadata select (no more whole-array detoast on every load) and assemble the transcript from the table after authorization. This cascades to the copilot + mothership GET endpoints and to resolveOrCreateChat's conversationHistory (the LLM payload). The normalize/effective-transcript pipeline is source-agnostic (copilot_messages.content == a JSONB array element), so transcripts are byte-identical. Dual-write and the JSONB column stay in place as the internal-logic source and fallback; removing JSONB writes is a later step. Prod integrity verified before cutover: 0 messages missing, 0 NULL-seq, 0 dup keys/seq, 0 orphans, order-parity vs JSONB = 0 mismatches. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(copilot): cover auth-deny on a found row skips the messages query Address PR review: exercise the `if (!authorized) return null` contract — when the chat row exists but authorization fails, the getter returns null and never issues the copilot_messages read. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(tables): right-align run/stop in embedded toolbar; workflow cells format like normal cells (#4806) * fix(tables): right-align run/stop in the embedded table toolbar Add a right-aligned `trailing` slot to ResourceOptionsBar and move the embedded mothership table's run/stop control into it, so Filter + Sort stay left-aligned and run/stop sits opposite on the right. No-op for the search-bearing consumers (logs, resource list), which don't pass `trailing`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(tables): workflow-output cells format values like normal cells Workflow-output columns short-circuited in resolveCellRender and rendered their value as plain text, so a sim-resource URL / external URL / JSON / date produced by a workflow never got the chip, favicon link, or typed formatting a normal cell gets. Factor value formatting into a shared `resolveValueKind` helper used by both the workflow-value branch and the plain-cell branch; the workflow branch keeps the typewriter reveal for plain streaming text via a `typewriter` flag. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(tables): detect resource/URL links on workflow output regardless of column type Workflow output columns default to `json` (columnTypeForLeaf), so routing their values through the type-based formatter (a) gated chip/URL promotion behind `column.type === 'string'` — a URL produced by a json-typed output never became a chip — and (b) JSON.stringify'd plain string values, adding quotes and losing the typewriter reveal. Detect links (sim-resource chip / favicon URL) on the value string directly for workflow outputs, falling back to the plain `value` kind; plain cells keep the type-based formatting. Addresses Greptile P2 on #4806. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(icons): repair broken integration icon rendering (#4810) * fix(icons): repair broken integration icon rendering Two distinct bugs left integration icons broken on the /integrations page (visible at 32-40px, hidden at the toolbar's 16px): 1. Corrupted SVG paths (Notion, Greptile, Granola, Calendly, Grafana, Bedrock): over-minified data dropped elliptical-arc flag digits (e.g. `A1 1 0 5.9 7` instead of `A1 1 0 0 0 5.9 7`); Granola's cubic stream was truncated. Browsers abort path parsing at the first invalid arc flag, so each rendered as a fragment or blank. Replaced with correct path data from canonical sources, preserving each icon's existing fill/gradient and bgColor. 2. Invisible glyph (Bright Data): its icon uses fill='currentColor' but bgColor was '#FFFFFF', and every surface forces text-white on the glyph - white-on-white. Changed bgColor to Bright Data's brand blue (#3d7ffc) so the white glyph reads, matching the white-glyph-on-brand-chip convention. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(icons): restore Calendly dual-tone brand colors Addresses review feedback: the previous fix replaced the broken Calendly icon with a monochrome #006BFF path, dropping the cyan #0ae8f0 accent from the original dual-tone mark. Restored the two-tone logo (blue + cyan) using clean, valid path data, cropped to a tight square viewBox so it fills the chip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * improvement(icons): enlarge icons, fix Zoom contrast and Quiver chip - Zoom: glyph was blue-on-blue (#0B5CFF on #2D8CFF chip); switched to currentColor so it renders as a white glyph on the blue chip. - Quiver: chip bgColor #000000 -> #FFFFFF to match the icon's near-white box, and enlarged the mark slightly (viewBox crop). - Enlarged (tightened viewBox, verified no clipping): RevenueCat, Prospeo, Granola, Firecrawl, Enrich.so, and the AWS icons (RDS, DynamoDB, SQS, CloudFormation, Athena, CloudWatch, SES, Bedrock, S3). - ZoomInfo left unchanged: it is a full red rounded-square logo that already fills its frame, so a crop would clip it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(icons): use Bright Data wordmark on white chip; repair Circleback - Bright Data: replaced the flame glyph with the official two-tone 'bright data' wordmark (provided asset), centered in a symmetric viewBox. Reverted the chip bgColor from #3d7ffc to #FFFFFF since the blue wordmark is invisible on a blue chip (the wordmark is designed for a light background). - Circleback: a minifier had rounded the pattern's image scale to scale(0), collapsing the embedded logo to zero size (invisible). Restored the correct scale (1/280 = 0.00357142857) so the C. mark renders. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(docs): sync Quiver block color card to white chip Reflects the Quiver bgColor change (#000000 -> #FFFFFF) in the docs block info card. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * improvement(icons): enlarge AWS/Cloudflare/Dagster icons, fully white Zoom - Enlarged (tighter viewBox, render-verified, no clipping): Cloudflare, Dagster, and the red AWS icons AWS IAM, Identity Center, Secrets Manager, SES, STS. Identity Center was anomalously small (filled ~32% of its frame); the group is now sized consistently (~80% fill). - Zoom: the camera lens triangle was still #0B5CFF (blue-on-blue); switched it to currentColor so the whole camera renders white on the blue chip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(wiza): consolidate individual reveal into a single operation Merges the separate Start/Get Individual Reveal operations into one Individual Reveal operation in the Wiza docs and integrations data (operationCount 5 -> 4). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * improvement(icons): size remaining AWS icons to match the set (~80% fill) Bring RDS, DynamoDB, SQS, CloudFormation, Athena, CloudWatch and S3 up to the same ~80% fill as the AWS IAM/Identity Center/Secrets Manager/SES/STS group, so all AWS icons are visually consistent. Bedrock left as-is (already ~92% fill). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(icons): use Bright Data flame mark, enlarge ZoomInfo - Bright Data: the full 'bright data' wordmark was illegible at chip size. Replaced with just the flame-'i' brand mark (blue #4280f6 on the white chip), centered. - ZoomInfo: cropped the viewBox toward the white 'Zi' so it's larger; the red rounded-square background still fills the chip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * improvement(icons): enlarge CrowdStrike icon The falcon mark sat small in its chip because the icon used a wide 768x500 viewBox (letterboxed in the square chip). Switched to a square viewBox centered on the mark so it fills ~80%, consistent with the other icons. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(tables): serialize schema mutations to prevent parallel column clobber (#4812) * Make workflow description nullable * fix(tables): serialize schema mutations to prevent parallel column clobber * fix(tables): load workflow outside schema lock; use DbOrTx for getTableById * fix(tables): scale idle timeout in updateColumnType to avoid aborting large type changes * fix(tables): skip stale remap types when workflowId changes concurrently * fix(tables): scale idle timeout in updateColumnConstraints for large tables * fix(wait): resume live/draft async waits and preserve cell context on chained waits (#4814) * Make workflow description nullable * fix(wait): resume live/draft async waits and preserve cell context on chained waits * improvement(knowledge): polish tag filter dropdowns * improvement(knowledge): soften filter section labels * improvement(knowledge): soften list filter labels * fix(security): harden SSO domain registration, webhook path isolation, and CSV export (#4813) * fix(security): harden KB file access, SSO domain registration, webhook path isolation, env secrets, and CSV export * fix(sso): scope domain conflict query with indexed lower(domain) filter Address PR review: avoid a full-table scan on every SSO provider registration by filtering candidate rows in SQL with lower(domain) = <normalized>, keeping the in-memory ownership check. Also tighten the normalizeSSODomain TSDoc. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: condense env route security comments Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * icons update * chore(security): tighten inline comments in CSV export and KB file authorization Condense verbose comment blocks to concise TSDoc/single-line form; no behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): validate internal serve origin in KB file authorization Replace the bypassable isInternalFileUrl substring check in resolveInternalKbKey with an origin allow-list (base URL, internal API base URL, TRUSTED_ORIGINS). A crafted external host whose path is /api/files/serve/<victim-key> no longer resolves to the victim key. Relative same-origin URLs are unaffected. * style(sso): use idiomatic sql lower() comparison for domain conflict query Match the repo's prevailing `sql`lower(col) = value`` idiom for the case-insensitive SSO domain conflict lookup. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): align workspace env admin gate with hasWorkspaceAdminAccess Use the same admin check the secrets UI uses (owner, admin permission, or org-admin) so owners and org-admins are not wrongly denied their own decrypted workspace secrets, while read-only members remain restricted to names only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sso): rely on lower(domain) match for conflict detection, drop dead in-memory recheck Address PR review: the SQL `lower(domain) = <normalized>` predicate already excludes rows that the in-memory `normalizeSSODomain(...) === domain` recheck claimed to catch, making that recheck dead/misleading code. Match on the canonical lower-cased domain and filter purely by ownership. Malformed legacy values (wildcards, schemes, ports) never match an email domain at sign-in, so excluding them is not a gap. Test DB mock now applies the lower() predicate so the casing-variant case is genuinely exercised. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): scope webhook deploy path conflict to active webhooks findConflictingWebhookPathOwner omitted the isActive filter that the runtime dispatcher (findAllWebhooksForPath) applies, so an inactive but non-archived webhook from another workflow (e.g. after undeploy or failure auto-disable) would permanently block any new deployment on that path even though it never receives deliveries. Align the guard with the runtime isActive + archivedAt filter; the earliest-owner runtime check remains the authoritative cross-tenant protection. Also trims verbose TSDoc on the webhook path-isolation helpers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): exclude archived workflows from webhook deploy path conflict findConflictingWebhookPathOwner now joins workflow and filters isNull(workflow.archivedAt), matching the runtime dispatcher (findAllWebhooksForPath). A webhook on an archived workflow can never receive deliveries at runtime, so it must not block legitimate path reuse with a 409. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): anchor KB file ownership to earliest document in any state A KB file's owner is now the earliest document referencing its key regardless of state (active/archived/deleted/excluded); access is granted only when that owning document is still active. Closes the residual where an attacker could plant an active document to claim a file whose original document was archived or deleted. * updated greptile icon * revert(security): drop KB file authorization changes Reverts the knowledge-base file-access work (origin-pinning / owner-pinning / origin allow-list in verifyKBFileAccess) and its test. The other hardening fixes (SSO domain registration, webhook path isolation, workspace env secrets, CSV export) are unchanged. apps/sim/app/api/files/authorization.ts is restored to its origin/staging baseline. * fix(sso): treat caller's own user-scoped provider as owned during conflict check Self-hosters often register SSO user-scoped via the CLI script (no SSO_ORGANIZATION_ID). If they later enable organizations and reconfigure the same domain org-scoped through the UI, the conflict check previously treated their own user-scoped row as another tenant's and returned a misleading 409. Recognize the caller's own user-scoped provider as owned so that migration is allowed, while still blocking another user's or another org's domain. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * revert(security): remove workspace-env admin gate Defer to a credential-based access model (separate change). Restores GET /api/workspaces/[id]/environment to main behavior and removes the test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(security): consolidate webhook path-collision check into one helper Extract findConflictingWebhookPathOwner to lib/webhooks/utils.server.ts as the single source of truth for cross-tenant path-collision detection, used by both webhook creation paths (deploy sync and the manual /api/webhooks route). This also repairs two latent issues in the manual route's previous inline check, which queried with limit(1) and only webhook.archivedAt: - limit(1) inspected one arbitrary row, so a same-workflow row could mask a foreign collision (false negative). The shared helper scans all matching rows. - It omitted isActive/workflow.archivedAt, so inactive or archived-workflow webhooks (which never receive deliveries) permanently blocked path reuse. The helper mirrors the runtime dispatcher's filter. Same-workflow webhook reuse for upsert is now a separate, explicit lookup. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(security): block private/reserved IPs for hosted 1Password Connect SSRF (#4818) * fix(security): block private/reserved IPs for hosted 1Password Connect SSRF * test(security): use real isPrivateOrReservedIP and cover IPv6 edge cases * improvement(integrations): validate and expand devin, cursor, and greptile (#4820) * improvement(integrations): validate and expand devin, cursor, and greptile - devin: fix missing org_id path segment on all session endpoints, add 7 session sub-resource tools (list messages/attachments, get/append/replace tags, archive, terminate), pagination, and is_archived output - cursor: add get_api_key_info, list_models, list_repositories tools - greptile: align block and docs - normalize array outputs to default [] and tighten types * refactor(cursor): simplify list_repositories v2 array normalization Collapse the redundant `?? []` + `Array.isArray` double-guard into a single Array.isArray check, per PR review feedback. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(devin): scope session-tag mapping to tag ops and normalize array tag inputs - Only map sessionTags into the tools tags param for append/replace operations, preventing stale sessionTags state from clobbering create_session tags - Fall back to a wired tags value when sessionTags is empty for tag operations - Normalize tag inputs (string or wired string[]) via normalizeTags so array values from other blocks no longer throw on .split Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(cursor): restore base64 file data in legacy download_artifact metadata The legacy CursorBlock exposes only content + metadata (no v2 file output), so metadata.data was the only way legacy-block workflows could access downloaded artifact bytes. Restore the base64 data field and document it in the outputs/type instead of dropping it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(devin): coerce terminateArchive to archive flag for boolean-wired input * docs(integrations): regenerate tool docs for new devin and cursor operations --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(search-replace): don't auto-navigate when content edits invalidate the active match (#4819) * fix(search-replace): don't auto-navigate when content edits invalidate the active match * fix(search-replace): clear afterReplaceIndexRef on apply failure and zero matches * fix(search-replace): remove duplicate setActiveSearchTarget(null) on close * fix(search-replace): move afterReplaceIndexRef write inside handleApply past the guard * fix(search-replace): auto-navigate when hydration resolves with no prior active match * chore(search-replace): remove inline comments * fix(search-replace): revert !activeMatchId guard that caused immediate re-navigation after deselect * improvement(enrichments): limit company-info to fields both providers return (#4817) Hunter's company dataset returns null industry/foundedYear for many large companies (verified against the live API for Microsoft, Amazon, Google), so under the first-non-empty-wins cascade those columns appeared inconsistently across rows. Limit company-info outputs to employee count and description — the fields Hunter and PDL both reliably return — so every row is consistent. employeeCount is a string so Hunter's range bucket and PDL's exact count share the column. * fix(files): don't reject external URLs containing '..' in file parse validation (#4821) * fix(files): don't reject external URLs containing '..' in file parse validation The file block's file_fetch operation rejected any external URL whose path contained '..' (e.g. Slack files-pri slugs with a literal '...') with 'Access denied: path traversal detected'. Traversal checks only apply to local paths — external http(s) URLs are fetched with SSRF protection downstream and are never resolved against the filesystem, so they now short-circuit as valid. Internal /api/files/serve/ URLs keep full traversal protection. * test(files): fix external-URL assertion to handle undefined error * test(files): assert success explicitly in external-URL traversal test * fix(files): keep traversal protection for https URLs matching internal serve paths * feat(google-sheets): add row filtering to read with numeric operators (#4822) * feat(google-sheets): add row filtering to read with numeric operators Adds client-side row filtering to the Google Sheets read (v2) operation. Filter the returned rows by a header column using text operators (contains, not_contains, exact, not_equals, starts_with, ends_with) and numeric/ordering operators (gt, gte, lt, lte). Filtering lives in a pure, unit-tested helper (filterSheetRows) and runs over the fetched read range; an optional `filter` output reports whether the column was found and how many rows matched. Also hardens the surrounding tools: - trim spreadsheetId in write/update/append URL builders (matches read) - URL-encode the v1 read default range - expose valueInputOption for the update operation in the block Backwards compatible: with no filter requested, read output is byte- identical and the `filter` field is omitted. The filterMatchType union is widened additively (4 -> 10 values). * fix(google-sheets): correct filter metadata for missing column and header-only sheets - matchedRows is now 0 (not totalRows) when the filter column is not found, so it no longer contradicts applied=false / columnFound=false - columnFound now reflects an actual header lookup for empty/header-only sheets instead of being hardcoded true - add tests covering header-only and empty sheets with present/absent columns * fix(selectors): fetch all pages for paginated dropdown list routes (#4823) * fix(selectors): fetch all pages for paginated dropdown list routes Dropdown selectors fetched only the first page of paginated provider APIs, silently hiding results past page one. Add bounded server-side draining to the list routes across Microsoft Graph, Google, Notion, Atlassian, Linear, AWS CloudWatch, and offset/token REST APIs, plus a shared client-side drain cap in the selector hook. Response shapes, stored values, and tool execution are unchanged; CloudWatch list tools still honor a caller-supplied limit. Also fixes the Word file picker that was searching for .xlsx files. * fix(selectors): harden JSM and Monday pagination draining - JSM service-desk/request-type drains advance `start` by the actual row count returned (not the fixed page size) and stop on an empty page, so a short non-final page can't skip items. - Monday boards drain now checks `response.ok` per page, surfacing a mid-drain HTTP failure instead of treating it as an empty final page and returning a partial 200. * docs(selectors): clarify JSM drain advances start by actual row count The offset-advancement fix (advance `start` by the rows returned, not the fixed page size) landed in 7b19788a8; update the TSDoc to match so it no longer reads as advancing by `limit`. * fix(selectors): drain fetchPage in direct fetchList callers Making `fetchList` optional left three direct callers (outside the useSelectorOptions hook) calling it unguarded, which broke the build's type check. Route them through a shared `loadAllSelectorOptions` helper that uses `fetchList` when present and otherwise drains `fetchPage`. This also prevents a regression: `confluence.spaces` / `knowledge.documents` now paginate via `fetchPage` only, and these callers (search/replace, value resolution) would otherwise have silently returned no options. * chore(selectors): rename MAX_PAGE_PAGES to MAX_NOTION_PAGES for readability * fix(sso): re-check domain conflict before write and reject IP-address domains (#4825) * improvement(copilot): make copilot_messages the sole transcript store, remove JSONB dual-write (#4826) Stop writing/reading the legacy copilot_chats.messages JSONB column now that reads are cut over to copilot_messages. Make appendCopilotChatMessages the primary write (throws on failure instead of swallowing), repoint peripheral readers (workspace VFS, chat cleanup, data drains, fork, superuser import) to copilot_messages, and persist the assistant turn inside finalizeAssistantTurn's transaction so it commits atomically with the stream-marker clear. The column itself is dropped in a follow-up migration after this bakes. * feat(tables): expand filter operators (not-contains, starts/ends-with, not-in, empty) (#4827) Add does-not-contain ($ncontains), starts-with ($startsWith), ends-with ($endsWith), not-in-array ($nin, previously executed server-side but unexposed in the UI), and is-empty/is-not-empty ($empty) filter operators end-to-end — SQL builder, condition types, query-builder converters/constants, the filter UI, the Table tools/block descriptions, and docs. Also fix correctness bugs in the filter builder surfaced by the wider operator set: - Same-column AND rules (e.g. age > 18 AND age < 65, or name startsWith 'A' AND name endsWith 'Z') silently overwrote each other because the AND group was keyed by column name. They now merge into one operator object, which also makes Filter -> rules -> Filter round-trip losslessly for multi-operator columns. - $nin values were not split into an array like $in, and textual-match values like "123" were numeric-coerced (breaking the ILIKE path). - A non-boolean $empty operand from the raw API silently inverted the check; it now coerces 'true'/'false' strings and otherwise returns a 400. * improvement(copilot): stop persisting tool-call result outputs in transcripts (#4829) Opening a Mothership task could take many seconds because a single persisted assistant message in copilot_messages.content can reach hundreds of MB, almost entirely inside contentBlocks[].toolCall.result.output (e.g. a get_workflow_logs or run_workflow result). The DB query is ~2ms; the cost is detoasting that payload, shipping it to the browser, and parsing it. These outputs are dead weight on the Sim side: they are never rendered (the thread shows only tool name/title/status) and never replayed to the model (the upstream copilot service owns conversation memory). So drop result.output before it is persisted, keeping result.success/error plus the tool metadata. - add stripToolResultOutput() in persisted-message.ts - apply it in messages-store toRow (covers every write path) and in loadCopilotChatMessages (existing rows render fast on read) Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * feat(providers): add Together AI, Baseten, and Ollama Cloud model providers (#4830) * feat(providers): add Together AI, Baseten, and Ollama Cloud model providers * fix(providers): guard Ollama streaming fast-path with hasActiveTools Match Together/Baseten/Fireworks: when tools are supplied but all are filtered out (usageControl 'none'), take the single streaming call instead of an extra non-streaming round-trip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(providers): filter non-chat model types from Together model list * refactor(providers): dedupe Ollama Cloud upstream schema ollamaCloudUpstreamResponseSchema was byte-for-byte identical to ollamaUpstreamResponseSchema (both /api/tags endpoints return the same { models: [{ name }] } shape). Drop the duplicate and reuse the shared schema. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix(knowledge): calendar view sync, deduplicate popover animation classes, type-safe filter cast * cleanup(knowledge): remove TRIGGER_BORDER_CLASS duplication, inline displayLabel, drop enabledFilterParam alias --------- Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Waleed <walif6@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Theodore Li <theo@sim.ai> Co-authored-by: andresdjasso <andresdjasso@users.noreply.github.com> * feat(blocks): add BlockMeta to Quiver and Linq; fix invalid block config fields; update skills Block fixes: - Add QuiverBlockMeta (tags + 3 templates: icon generator, diagram creator, vectorizer) - Fix QuiverBlock: remove invalid tags field from BlockConfig, IntegrationType.Design → IntegrationType.AI (Design doesn't exist in the enum) - Fix GreptileBlock: remove invalid tags field from BlockConfig, IntegrationType.DeveloperTools → IntegrationType.DevOps - Fix LinqBlock: remove invalid tags field from BlockConfig (tags belong only in BlockMeta) Skills: - add-block: add dedicated BlockMeta section with structure, rules, and registration pattern; add BlockMeta checklist items - add-integration: add BlockMeta to block structure template, add rules clarifying that tags must NOT appear on BlockConfig and integrationType must be a valid enum value; update registry snippet to include blocksMeta; add checklist items * fix(integrations): fix category dropdown by defining missing LANDING_INTEGRATIONS_DATA_PATH and regenerating integrations.json The staging merge introduced landing-content.ts but forgot to define LANDING_INTEGRATIONS_DATA_PATH in generate-docs.ts, causing the script to crash before writing integrations.json. The stale JSON had integrationTypes (plural array) from an older script version, while the Integration type and workspace UI both read integrationType (singular string) — so ALL_CATEGORY_SECTIONS bucketed to undefined and the category filters never appeared in the dropdown. Fixed by adding the missing path constant and re-running the generator. integrations.json now has 192 entries with the correct integrationType field. * fix(sidebar): restore resize handle on all pages commit3109104582wrapped the resize handle in {(isCollapsed || isOnWorkflowPage) && ...} and added a useEffect that resets sidebar width to SIDEBAR_WIDTH.MIN whenever the user navigates away from a workflow page. Together these made the sidebar non-resizable on Tasks, Tables, Knowledge Base, and every other non-workflow page. Restore the staging behavior: always render the resize handle and remove the effect that forced the width reset on page transitions. * fix(sidebar): match staging onKeyDown and tabIndex on resize handle The resize handle was still conditionalizing onKeyDown and tabIndex on isCollapsed, blocking keyboard accessibility of the separator role when expanded. Staging always attaches both unconditionally. onKeyDown={isCollapsed ? handleEdgeKeyDown : undefined} → onKeyDown={handleEdgeKeyDown} tabIndex={isCollapsed ? 0 : undefined} → tabIndex={0} * feat(integrations): show connected credentials on integration detail page When navigating to /integrations/google-docs (or any integration), a Connected section now appears above the templates listing all workspace credentials tied to that provider. Each row links back to the credential detail page (/integrations/connected/${id}) for management actions. Pairs with the earlier change that routes connected items from the integrations list to the provider detail page instead of directly to the credential detail page. * fix(integrations): rename Add in chat to Add to Sim * fix(skills): rename Add button to Add to Sim * fix(platform): restore M1/M2/M3 regressions and LazyMotion on landing page M1 — Invitation guard: re-introduce usePermissionConfig().isInvitationsDisabled alongside the workspace inviteDisabledReason check. The flag now also respects NEXT_PUBLIC_DISABLE_INVITATIONS and EE permission-group disableInvitations, not just billing policy. M2 — Settings redirects: /settings/integrations and /settings/skills now server-redirect to /integrations and /skills respectively so old bookmarks and emails don't silently land on General. M3 — Starter block search exclusion: restore block.type !== 'starter' guard in the search store so users cannot add a duplicate Starter block via the command palette. LazyMotion: restore LazyMotion + domMax/domAnimation wrappers and m.* components in landing-preview-panel and landing-preview-home. The removal was accidental (the full motion bundle was left after an import cleanup), which caused the entire framer-motion feature set to load eagerly on the landing page. * fix(integrations): revert connected list to credential detail; remove settings redirects * feat(sidebar): restore workspace switcher search with updated styling Shows a search input in the workspace dropdown when the user has more than 3 workspaces (WORKSPACE_SEARCH_THRESHOLD). Keyboard navigation: ArrowDown/Up to move through results, Enter to switch, resets on close. Styled to match the current branch (border-1/surface-5 tokens, sm text, 11px Search icon) rather than the old staging styles. Highlight state is wired through chipVariants active prop so it follows the same active appearance as clicked/hovered items. * fix(sidebar): clean up workspace search — layout, memo, and effect guard * fix(sidebar): align workspace rename input selection style with workflow rename * perf(sidebar): eliminate React re-renders during sidebar drag resize Previously, every mousemove during resize called setSidebarWidth(), which both updated the --sidebar-width CSS variable (sync) and set Zustand state (async). This caused: 1. A 1-frame transition flash on mousedown — isResizing state had to round-trip through React before the is-resizing CSS class was applied, so the width transition fired for the first pixel of movement. 2. A React re-render per pixel dragged — components reading sidebarWidth from the store (avatars, usage-indicator) lagged one frame behind the container, making the + and ... buttons appear to jump ahead. New approach: - handleMouseDown adds is-resizing directly to the sidebar DOM node before any React involvement (synchronous, no frame lag). - mousemove writes only to the CSS custom property (zero React renders). - mouseup persists the final width to Zustand exactly once. - isResizing / setIsResizing state removed from the store and hook — they are no longer needed since the class is managed via direct DOM mutation. * perf(sidebar): add requestAnimationFrame throttle to resize mousemove handler * fix(sidebar): fix drag-right lag caused by WorkspaceChrome overflow-hidden transition The sidebar-container's is-resizing class correctly suppressed its own width transition, but the two wrapper divs in WorkspaceChrome both have transition-[width]/transition-transform with a 175ms ease. The outer wrapper also has overflow-hidden, so while the sidebar content was at the correct width instantly, it was visually clipped by the outer wrapper which was still animating — causing the + and ... buttons to appear to lag behind the resize line on drag-right (not on drag-left, since shrinking doesn't clip content). Fix: add sidebar-shell-outer/sidebar-shell-inner class names to both chrome wrappers, and suppress their transitions via html.sidebar-resizing rule when a drag is active. The html.sidebar-resizing class is toggled directly in the resize hook alongside is-resizing, so it takes effect synchronously on mousedown. * fix(icons): redesign Download icon to match Upload style; fix Upload/Download confusion Download icon was missing the tray/shelf line at the bottom that Upload has, making it look like a plain arrow rather than a matched pair. Updated Download to use the same viewBox, stroke weight, and three-path structure as Upload (tray + stem + arrowhead), just pointing down. Also fix 5 places where Upload (↑) was incorrectly used for download/export actions: - files.tsx: two Download action rows in toolbar and context menu - tables/table.tsx: Export CSV toolbar button - table-context-menu.tsx: Export CSV context menu item - logs.tsx: Export toolbar button - landing-preview-logs.tsx: decorative Export button Import CSV and actual upload actions correctly keep the Upload icon. * fix(icons): replace Upload with Download on all remaining export/download actions - panel.tsx: Export workflow dropdown item - context-menu.tsx: Export in sidebar workflow context menu - chat.tsx: Export chat button - output-panel.tsx: Export console CSV button - terminal.tsx: Export console CSV button - resource-content.tsx: Export table as CSV + Download file buttons * fix(icons): fix remaining Upload→Download on download actions in files and logs - action-bar.tsx: download button in files toolbar - file-row-context-menu.tsx: Download item in file context menu - file-download.tsx: both download buttons in log details file viewer * updated block skills, settings pages, modals, buttons -> chips, blocks missing metadata * updated skill modal * improvement(resource-header): refine breadcrumb truncation ux * Fixes * improvement(resource): add floating overflow text tooltips * wire up credits counter * improvement(resource-header): mute path dropdown title * refactor(resource-header): share floating-tooltip engine, prune dead overlay tooltips (#4844) Clean up the breadcrumb truncation feature for reuse and correctness: - Extract useFloatingTooltip / useIsOverflowing / FloatingTooltip into a shared floating-tooltip module. BreadcrumbSegment and FloatingOverflowText now consume one implementation instead of duplicating ~150 lines of positioning, velocity, overflow-detection, and portal logic. - Replace the hardcoded terminal-label regex in ResourceHeader with a typed `terminal` flag on BreadcrumbItem (set by the document chunk/loading crumbs), decoupling the generic header from knowledge-base copy. - Clear the path-popover close timeout on unmount and reuse the shared POPOVER_ANIMATION_CLASSES constant. - Drop the redundant manual overflow-state writes (fixes a sticky fade mask). - Revert FloatingOverflowText inside Combobox `overlayContent` back to plain truncating spans across files/logs/tables/scheduled-tasks/document: the combobox overlay is pointer-events-none, so the tooltip handlers never fired there. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * refactor(emcn,resource-header): address PR #4844 review feedback - useIsOverflowing now uses a callback ref so the ResizeObserver follows the element across mount/unmount/reassignment instead of capturing it once at mount. Safe for conditionally rendered consumers of the shared hook. (greptile P2) - Move POPOVER_ANIMATION_CLASSES out of chip-date-picker implementation internals into emcn/components/popover/popover-animation.ts, exported from the @/components/emcn barrel. Consumers now import from the module boundary. (greptile P2) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Prompt surface * Regenerated contracts * upgrade table and styling upgrade * Literalness gaps * Updates * Remove prefix for running code * Display names for reads * Tool catalog * fix schema to include integration * fix(files): align delete icon with tables view (Trash → Trash2) Co-Authored-By: waleed <waleed@simstudio.ai> * fix(mothership): preserve blockType for integration contexts in sent messages Integration mention chips were missing their provider icons in sent messages because blockType was dropped when mapping ChatContext to messageContexts. renderIntegrationTile returns null without blockType, silently hiding the icon. * fix(mothership): allow 'integration' resource type in chat resources API The VALID_RESOURCE_TYPES allowlist was missing 'integration', causing a 400 error when adding integrations to the Mothership resource tab — so they never persisted and disappeared on refresh. * fix(ui): Add "File" title next to file resource header * fix(ui): fix resource header columns being bolded * fix(resource): keep the floating tooltip from jumping on click Gate the focus-driven show behind :focus-visible so a mouse click (which focuses the trigger) no longer re-shows the tooltip anchored to the element's bottom edge. On click the tooltip now hides cleanly instead of jumping down; keyboard focus still shows it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * perf(sidebar): eliminate unnecessary re-renders in workspace switcher for non-search users - onMouseEnter: only set highlightedIndex when showSearch is true, preventing a state update + re-render on every workspace row hover for users with ≤ 3 workspaces where the search is never shown - onOpenChange: only reset workspaceSearch and highlightedIndex when showSearch is true, since both values are always already at their defaults for non-search users and setting them triggers a pointless re-render during dropdown close - data-workspace-row-idx: only set when showSearch is true since the scroll effect that reads this attribute is already gated on showSearch * feat(search): context-aware cmd-k results on the integrations page When cmd-k is opened on the integrations page, show two new result groups: connected accounts (visible even with empty input) and catalog integrations (appear once the user types). Selecting an OAuth integration deep-links to its detail page with ?connect=oauth so the connect modal auto-opens. Non-OAuth integrations navigate to the plain detail page. Both groups are gated to the integrations page only and respect the hideIntegrationsTab permission. The credentials fetch shares the same React Query cache key as the integrations page itself (no double fetch). * refactor(emcn): make the floating tooltip the one canonical Tooltip Replace the Radix-based emcn Tooltip with the cursor-following floating tooltip so every tooltip in the app uses one consistent style. Built on the shared floating-tooltip engine (relocated into emcn), not a parallel implementation. - Move the floating-tooltip engine into emcn/components/tooltip and export it from the barrel; re-point its consumers (FloatingOverflowText, resource-header) - Extend the FloatingTooltip bubble to render arbitrary children (+ role/id for a11y) so it can back general tooltips, not just overflow text - Rebuild emcn Tooltip (Root/Trigger/Content/Provider/Shortcut/Preview) on useFloatingTooltip — compound API preserved, ~350 call sites unchanged, legacy side/align props accepted and ignored (the tooltip follows the cursor). Removes @radix-ui/react-tooltip usage (package kept for a later cleanup; react-slot retained for asChild) Note: general tooltips now show instantly (no hover delay) and follow the cursor. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * style(emcn): put tooltip text on the design scale (text-caption) Replace the tooltip's ad-hoc `text-xs` + `leading-[18px]` with the semantic `text-caption` (12px) font-size token so the text styling is fully on the design scale and self-documenting, matching how the rest of the system is set up. The color already used the global `--text-body` token. No visual change (still 12px with a ~18px line height). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Mship byok * feat(sidebar): add empty task state and inline task creation - Show "No tasks yet" in the Tasks section (expanded and collapsed) when the list is empty - Clicking + now creates a task via the API and navigates directly to it, rather than navigating to home - Add isCreatingTaskRef guard to prevent double-click from spawning multiple tasks - Disable + button while creation is pending - Fall back to home navigation on creation error * invite, billing, home * Sandbox warning * improvement(seats): auto purchase seats on invitations into workspace (#4857) * improvement(seats): auto purchase seats on invitations into workspace * improve sampling for seat drift reconciler * address comments * feat(knowledge): align connector UI with integrations page styling - ConnectorTypeCard now matches integration rows: brand-colored rounded-xl tile, ArrowRight, title/subtitle hierarchy - ConnectorCard icon upgraded from flat surface-4 to branded tile (white icon on brand bg, graceful fallback) - Connector header badges use chipVariants instead of custom Button classes - Add-connector search input aligned to integrations style (h-[30px], rounded-lg, border-1) * fix(icons): trim Folder SVG viewBox to remove right-side whitespace The folder path only extends to x≈14.33 in a 15-unit viewBox, leaving ~0.5 units of empty space on the right. At 12px rendered size this produces ~0.4px extra gap (visible as ~1px on retina displays) compared to solid icons like the workflow color square. Trimming the viewBox to 14.5 units makes the folder fill its chip slot evenly. * fix(user-input): restore draft text synchronously to preserve contexts on nav The SSR-safe approach (empty useState + effect restore) created a timing window where the sync effect in useContextManagement fired with message='' before the value was set, clearing any restored contexts. Folder and workflow contexts (not re-added by applyAutoMentions) were lost on every nav-back. Revert to the staging approach: initialize value synchronously from the draft store so message is already populated when effects run, matching the behavior on staging. * fix(queue): render context chips in queued messages - Remove plainMentions from queued message rows so context chips render with icons, consistent with sent messages - Fix computeMentionRanges to use '/' prefix for skill contexts (content has the slash trigger restored at submit time, not '@') * fix(mothership): remove integrations from add-resource dropdown * fix(mothership): comment out integrations from add-resource dropdown * File serializer * Skills * chore(db): drop form, templates, template_creators, template_stars tables These tables backed the Forms and Templates platform features which were intentionally removed from this branch. Clean up the DB schema to match. * chore(db): add migration metadata for 0224 drop tables * block icons, sidebar, toolbar * chore: remove remaining dead code for template-profile feature - Remove 'template-profile' from SettingsSection union type - Remove 'template-profile' entry from SECTION_TITLES - Remove now-redundant template-profile guard in settings sidebar - Remove commented-out template-profile nav item * fix(multi-select): preserve anchor on range selection for tasks and folders After a shift+click range, the anchor (lastSelectedTaskId / lastSelectedFolderId) was being updated to the end of the range (toId). This caused subsequent shift+clicks to extend from the wrong point instead of the original click. Standard behavior: anchor stays at the initial click (fromId) so repeated shift+clicks always expand/contract relative to where you started. * Update skills * Load skill tool * Fix vfs dynamic context encoding * Media subagent * feat(emcn): add SearchInput component and unify search bars platform-wide - Add SearchInput to emcn: 30px chip-family filled search input matching the integrations page pattern (border-1, surface-5, leading Search icon) - Migrate all 22 search bars across settings, EE pages, and integrations to SearchInput (only layout classes allowed at callsites) - Rename Sim Keys -> Sim API Keys in nav/title; page copy now says API key - Remove components/ui input, label, and verified-badge; migrate consumers to emcn equivalents or raw inputs (table cell editor, wand prompt bar) - Delete dead EE skeleton files (data-drains, data-retention) - General settings: Home Page chip moves to header left as navigation * fix(files,tables): restore new-file editor autofocus and CSV import error toasts Both were dropped on the staging line and regressed vs production (main): - Files: the new-file editor autofocus chain (files.tsx -> file-viewer -> text-editor) was stripped by the react-doctor dead-code pass in #4544, which misread the prop-drilled `autoFocus` (consumed by an imperative `editor.focus()` effect) as unused. Restored the prop through all three layers and the one-shot focus effect so creating a new file focuses the editor immediately. - Tables: CSV import failures were silently logged with no user feedback. Restored the per-file and generic `toast.error` surfacing. * Move superagnet back into superagent * feat(home): score suggested actions by workspace signals - Derive the suggestion pool from the curated block template catalog (1,343 prompts across 172 blocks) instead of 15 hardcoded entries - Fix inverted relevance: prompts for connected providers are now boosted 4x (instantly runnable) instead of excluded; unconnected discounted 0.4x - Weight by featured (3x), popular category (1.5x), and resource gaps (no tables -> boost table starters; has KBs -> dampen KB-creation prompts) - Weighted sampling without replacement, max one suggestion per block - Connect rows weighted by catalog template count; 2 for fresh workspaces, 1 once something is connected - Key the catalog map by both versioned and base block types so gmail_v2 templates resolve (gmail, github, notion, linear were silently dropped) - Replace derive-in-effect state with a useMemo keyed by a shuffle nonce - Add suggested_action_clicked / suggested_actions_shuffled / suggested_actions_toggled PostHog events * billing, teammates * improvement(credentials): credentials invites, secrets tab wiring up (#4874) * improvement(credentials): move away from invite notion * wire up secrets ui/ux * address comments * get consistent styling by removing emcninput + text area * styling consistency * remove fallback * address comment: * refactor(ui): migrate settings & workspace UI to chip design system Migrate modals to ChipModal (showDivider, hint, resizable, size, leading, ChipModalTabs), standardize Chip variants, add ChipCombobox wrapper, and apply chip inputs/dropdowns across settings, knowledge, logs, tables, inbox, EE tabs. Render ChipModalTabs as a ChipSwitch segmented control. Align /settings/secrets detail with /integrations, refresh whitelabeling, and restore file-editor autofocus and CSV import error toasts. * Update media * fix(mothership): restore integrations to useAvailableResources for @ mention Integrations were fully removed from useAvailableResources which broke the @ mention menu since user-input shares the same hook. Now integrations are always included in the hook but excluded at the AddResourceDropdown component level, keeping them out of the sidebar + menu while remaining available for @ mention autocomplete. * refactor(settings): chip design-system consistency pass across all tabs Extract a shared chip-field shell (CHIP_FIELD_SHELL/CHIP_FIELD_INPUT) mirroring Input variant='chip' and route secrets, credential detail, and integrations credential detail through it (30px height, font-medium, focus ring). Add a Discard action to the secrets header when dirty. Group BYOK providers into Models/Search & web/Enrichment sections and align its tiles to the integrations tile. Normalize list-row typography to text-[14px]/text-[12px] and icon tiles to rounded-xl + border across api-keys, copilot, custom-tools, mcp, workflow-mcp-servers, credential-sets, and access-control. Tone the secrets Details chip and per-row affordances to ghost. Fix token correctness: raw tailwind colors to design tokens (data-drains), missing chip variant on the Snowflake role input, ColorInput chip-field reuse (whitelabeling), error token and icon sizes (workflow-mcp-servers), border token (mcp), no-results sizing (secrets), row chrome (recently-deleted), hover token and Button to Chip (access-control), and deduped textarea chrome (sso). * feat(settings): unify filter dropdowns on ChipSelect (integrations style) Add a ChipSelect emcn component — a filled chip trigger + chevron opening a DropdownMenu, matching the integrations category filter — supporting single select, multi-select (checkbox rows), grouped options, and optional in-menu search. Migrate every settings/EE filter dropdown off ChipCombobox to it: audit-logs (resource-type multi + time-range), data-retention, data-drains, general, admin (grouped tool picker), inbox status filter, and the workflow MCP-server pickers. The SSO provider-id field stays an editable combobox since it accepts free-text slugs. Also fix audit findings: chip the MCP client-secret input, normalize an MCP error-text size, and drop now-dead destination-icon code in data-drains. * fix(mentions): require explicit @ for integration mentions; decorate sent messages robustly - Bare integration names in prose (Monday, Notion, Clay) are no longer auto-converted to mentions or chipped — mention treatment is strictly opt-in via a token-starting @ (fixes the scunthorpe problem) - @-prefixed mentions still canonicalize casing (@slack -> @Slack) on both the keystroke fast-path and bulk paths (paste, template, draft, STT) - Sent/queued messages now self-sufficiently decorate @IntegrationName tokens via a text scan, covering messages sent before the input pass ran or authored outside the chat input - Integration contexts missing a resolvable blockType (messages persisted before blockType was saved) are backfilled by label lookup so their mention pills render the brand icon again * refactor(settings): section the API keys page like secrets Wrap Workspace, Personal, and the allow-personal-keys toggle in SettingsSection (muted label + divider) instead of bare bold headers, matching the secrets and BYOK pages. * fix(settings): ChipSelect renders above modals + full-width form mode Raise the ChipSelect menu to --z-popover so it layers above modal surfaces (--z-modal) instead of opening behind them. Add a fullWidth prop that stretches the trigger and right-aligns the chevron for form-field use, and apply it to the workflow MCP-server pickers. * fix(emcn): ChipSelect uses the emcn flat chevron, not lucide's square one The lucide ChevronDown is square; rendering it at the chip's 9x7 footprint stretched it. Switch to the custom emcn ChevronDown (built for that wide aspect), matching the integrations filter and ChipDropdown. * fix(emcn): ChipSelect trigger hugs its content (w-fit) In a stacked form layout the trigger was stretched by align-items: stretch, leaving an empty gap to the right of the value. Add w-fit so the chip sizes to its content (a compact pill) everywhere; fullWidth form selects are unaffected. * fix(emcn): ChipSelect uses a square lucide chevron Revert to lucide's ChevronDown sized square (size-[14px]) so it renders crisp, matching the standard select chevron used by Combobox. * renamed tasks to chats * rename and file change * Wspace resource refs * improvement(billing): wire up billing, org, teammates tabs + remove deprecated subscription tab (#4887) * improvement(billing): wire up billing, org, teammates tabs + remove depr subscription tab * pass exec timeout to tool routes * reuse helper * address comments * address disable comment * Vfs updates * chore(db): remove migration 0224 to regenerate on top of staging Co-authored-by: Cursor <cursoragent@cursor.com> * VFS updates and linter * fix type errors and regen migration? * File block v5 * Remove docs * Fix * Doccer image * Nuke the dag * chore(db): drop branch migration 0226 ahead of staging merge; will regenerate * chore(db): regenerate migration 0226 after staging merge * Fix lint * externalize before compaction in fallback' * Run workflow improvements * Loosen run options validation * Lint * fix save/discard chips to be consistent * fix(ui): remove smodal tabs in favor of chip modal tabs * update byok manager component * fix(ui): skip auto-scrolling on mouse highlight of workspace * Add creds to wspace context * Wscontext * fix(platform): restore settings redirects, forgot-password Enter submit, and tag tooltip visibility - Re-add SETTINGS_REDIRECTS so /settings/integrations and /settings/skills deep links redirect to their top-level routes instead of rendering an empty settings panel (accidentally removed in86da193cc3one minute aftercca5054cf6added it) - Add opt-in onSubmit to ChipModalField input/email variants and wire it in the forgot-password modal so Enter submits again (lost in the ChipModal conversion) - Knowledge tag tooltip: drop the max-h/overflow-y-auto clamp that the pointer-events-none floating tooltip made unreachable; truncate each tag row instead so all tags stay visible with bounded height * feat(usage-limits): org member specific limits (#4893) * feat(usage-limits): org member specific limits * revert feature flags * fix type issues * address comments * address comments and address other billing gapes' * fix * improvements * fix remaining credits display * fix * pass ws id to route validations * chore(db): remove migration 0226_third_spot before staging merge * chore(db): regenerate migration as 0227 after staging merge * Block * feat(telemetry): add posthog + audit coverage for new platform actions Audit log (compliance/permission-relevant only): - org_seat.provisioned — seat auto-purchased when an invite acceptance grows the org (actor = accepting user, includes seat delta) - org_plan.converted — Pro→Team conversion triggered by invite acceptance - org_seat.drift_reconciled — hourly cron healed a drifted seat count - credential_member.added/removed/role_changed — credential sharing surface was previously fully unaudited - table.created — parity with existing table.updated/deleted - skill updates now record skill.updated instead of mislabeled skill.created PostHog: - seats_provisioned, credential_shared/unshared, environment_updated/deleted (key counts only, never names/values) - table_import_started/completed — background CSV imports previously had zero failure observability - table_exported, file_downloaded, skill_updated - credential_connected now fires for OAuth completions (draft-hooks), credential_deleted for OAuth disconnects; previously only manual credentials were tracked - table_workflow_run gains deployment_mode (live/deployed/mixed) Also: logger.warn on all new credential-admin 403 denials (members + environment routes); invite-created audit enriched with enforcedFixedSeats/plan. Deliberately excluded as noise: per-hour dead-letter audit rows (would re-record the same stuck event every cron run) and a duplicate system-actor org_plan.converted in the Stripe outbox handler. * feat(home): fill textarea on suggested prompt click instead of sending Clicking a prompt action in the Suggested Actions panel now populates the Mothership user-input textarea (via applyAutoMentions) and focuses it with the caret at end, rather than immediately submitting. The user can review, edit, and send manually. * fix(templates): name owning integration in featured block template prompts Four featured prompts omitted their owning integration name, making them unbranded and disconnected from their title. Each prompt now explicitly names GitHub or Google Sheets so mentionifyIntegrations renders the chip and the copy reads as a self-contained agent-building instruction. * fix(templates): rewrite fragment prompts so each names its integration and reads as a complete instruction Featured and non-featured block template prompts that either omitted the owning integration's canonical name or were phrased as marketing fragments rather than natural user instructions have been rewritten. Each updated prompt now starts with an imperative verb ("Build a workflow that…"), names the owning integration explicitly so the @-mention chip renders correctly, and aligns with the entry's title. Files changed: salesforce, hubspot (×2), github, slack, airtable, firecrawl, iam. * fix(templates): name integration in remaining block template prompts - google_docs.ts: replace 'Google Doc' with 'Google Docs document' in 4 prompts; update title 'Meeting notes to Google Doc' to 'Meeting notes to Google Docs' - google_sheets.ts: replace 'Google Sheet' with 'Google Sheets spreadsheet/Google Sheets' in 3 non-featured prompts - slack.ts: replace 'Google Doc' with 'Google Docs document' in 'Daily standup summary' - stripe.ts: replace 'Google Sheet'/'Slacks' with 'Google Sheets'/'Slack' in 'Weekly metrics report' - reddit.ts: add 'Reddit' to 3 prompts that only referenced subreddits - notion.ts: rewrite featured prompt to start with a verb and name Notion - jira.ts: rewrite featured marketing-fragment prompt to start with a verb - linear.ts: rewrite featured marketing-fragment prompt to start with a verb - gmail.ts: rewrite featured marketing-fragment prompt to start with a verb and name Gmail * fix(icons): convert monochrome dark brand icons to currentColor for dark mode LinkupIcon, InfisicalIcon, IntercomIcon, LumaIcon, GranolaIcon, OnePasswordIcon, and RailwayIcon were hardcoded to black/near-black fills/strokes, making them invisible in bare DARK mode. Convert all to currentColor so they follow the theme-aware text color. Add iconColor: '#286efa' to IntercomBlock (Intercom brand blue is a confident mid-tone, safe on both themes). Two-tone icons (StagehandIcon, AgentPhoneIcon, QuiverIcon) are left unchanged because their white details are structural — converting to single-tone would destroy logo legibility. * removed color, user input, sidebar, suggested actions, chips/emcn * removed color migration * Table bump * improve(blocks): audit block catalog metadata for accuracy and fill gaps - Fix template prompts that claimed capabilities blocks don't have (fabricated triggers, non-existent tools) across ~98 integrations - Normalize integration tags to family conventions and valid union values - Align alsoIntegrations and modules with template prompt content - Add BlockMeta for circleback, imap, and rss trigger integrations - Add templates to clickhouse and greptile metas - Remove duplicate PageSpeed deploy-gate template * chore(telemetry): drop org_seat.drift_reconciled audit System self-heal bookkeeping doesn't belong in the user-facing audit trail — membership and seat-purchase changes are already audited, and the cron's logger output covers ops visibility. * chore(db): consolidate branch migrations into single 0227 * fix(emcn): make ChipModal scroll internally when content exceeds viewport The Modal→ChipModal migration dropped the old ModalBody scroll container: ChipModal renders ModalContent bare (no overflow-hidden) and its wrappers had no min-h-0 chain, so tall modals (e.g. New data drain with an S3 destination) overflowed max-h-[84vh] off-screen with no way to scroll. Complete the flex min-h-0 chain through ChipModal's frame and give ChipModalBody flex-1 min-h-0 overflow-y-auto — header/footer stay pinned, body scrolls only when constrained. Short modals are unaffected (max-h caps, it doesn't stretch), and dropdowns inside the body are Radix-portaled so the new scroll container cannot clip them. Five modals that had locally patched this with max-h-[Nvh] overrides keep working unchanged. * fix(emcn): don't close modal when dismissing a dropdown via outside click Radix dispatches pointer-down-outside to every open dismissable layer at once, so clicking outside an open dropdown/select inside a modal closed both the dropdown and the modal in one jarring step. ModalContent now prevents its own dismissal while a portaled popper layer is open — the first outside click closes just the popper, the next one closes the modal. * docs(skills): de-duplicate and correct agent docs; canonical styling tokens; mirror sim-sandbox rule * docs(skills): broaden boundary-raw-fetch scope note, sync cursor rule mirrors * fix(emcn): harden modal popper guard and exempt caret-anchored dropdowns from body scroll Adversarial review follow-ups to the ChipModal scroll + dismissal fixes: - Popper guard now requires data-state="open" inside the popper wrapper, so a dropdown that is merely animating closed no longer swallows the next outside click on the modal (DropdownMenu has an exit animation that keeps its wrapper mounted briefly) - Port the same guard to SModalContent for consistency - custom-tool-modal: opt its body out of the chrome scroll container (flex-none + overflow-visible); the caret-anchored EnvVar/Tag autocomplete dropdowns are absolute-positioned inside the body and must spill past its bounds rather than clip against a scroll boundary * refactor(billing): drop unnecessary useCallback wrappers from event handlers * fix(data-drains): complete chip migration of destination forms and fill missing placeholders Finishes the in-flight FormField → ChipModalField conversion for all destination form specs and adds the placeholders that several inputs never had (S3 bucket/region/access keys, Azure account key, Datadog API key, webhook signing secret/bearer token) — the cause of the New Data Drain modal showing placeholder-less inputs inconsistently. * fix(emcn): broaden modal popper guard to onInteractOutside Covers the focusOutside dismissal path too: when a popper's focus scope unwinds on close, the transient focus shift could still dismiss the modal (and simultaneous body pointer-events lock teardown could freeze the page). Same data-state="open" scoping as the pointer guard. * fix * fix(emcn): harden modal outside interactions * Remove migration 0227 to regenerate after merge Drop the 0227_org_member_usage_limit migration, its snapshot, and journal entry so it can be regenerated cleanly after merging improvement/platform. Co-authored-by: Cursor <cursoragent@cursor.com> * Regenerate migration as 0228 after merging improvement/platform drizzle-kit generate re-emits the org_member_usage_limit table, document uploaded_by, and table_run_dispatches triggered_by_user_id changes on top of improvement/platform's 0227 snapshot. Also collapse the admin.tsx emcn import left multi-line after merge conflict resolution. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix pre-existing type errors surfaced by post-merge type-check These were present on dev before the merge; the post-merge type-check made them visible. - workspace-vfs.ts: materializeEnvironment now emits full oauth integration objects (id, providerId, displayName, role) so the summary matches WorkspaceMdData['oauthIntegrations'] (consumer reads credentialId from id) - workflows/utils.ts: narrow resolved workflow rows to non-null workspaceId (the inArray filter already guarantees it) so the resolved result type holds Co-authored-by: Cursor <cursoragent@cursor.com> * Slack trigger * fix tool mark complete bugs * change thinking icon to just be blimp * fix tests * MCP fixes * Fix files * refactor use chat * multiple fixes * Fix deploys * fix billing, tracing, retry lifecycle issues * remove stale generated migration Drop migration 0228 and its metadata so it can be regenerated after merging staging. Co-authored-by: Cursor <cursoragent@cursor.com> * merge staging in * fix sockets err classification * sockets invite flow fix * remove stale generated migration 0229 before staging merge Co-authored-by: Cursor <cursoragent@cursor.com> * regen migration * Update bun lock * Fix schema issues * Fix lint * Fix build --------- Co-authored-by: Vikhyath Mondreti <vikhyathvikku@gmail.com> Co-authored-by: Theodore Li <theo@sim.ai> Co-authored-by: Waleed <walif6@gmail.com> Co-authored-by: Emir Karabeg <emirkarabeg@berkeley.edu> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai> Co-authored-by: andres <k62hc5kjst@privaterelay.appleid.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: andresdjasso <andresdjasso@users.noreply.github.com> Co-authored-by: waleed <waleed@simstudio.ai>
This commit is contained in:
Siddharth Ganesan
co-authored by
Cursor
Vikhyath Mondreti
Theodore Li
Waleed
Emir Karabeg
Vikhyath Mondreti
andres
Claude Opus 4.8
andresdjasso
waleed
Cursor
Vikhyath Mondreti
Theodore Li
Waleed
Emir Karabeg
Vikhyath Mondreti
andres
Claude Opus 4.8
andresdjasso
waleed
parent
540835a7a5
commit
39418f9615
+2
-1
@@ -24,5 +24,6 @@
|
||||
# Client-side stream consumption, hydration, and reconnect.
|
||||
/apps/sim/app/workspace/*/home/hooks/index.ts @simstudioai/mothership
|
||||
/apps/sim/app/workspace/*/home/hooks/use-chat.ts @simstudioai/mothership
|
||||
/apps/sim/app/workspace/*/home/hooks/use-file-preview-sessions.ts @simstudioai/mothership
|
||||
/apps/sim/app/workspace/*/home/hooks/preview/ @simstudioai/mothership
|
||||
/apps/sim/app/workspace/*/home/hooks/stream/ @simstudioai/mothership
|
||||
/apps/sim/hooks/queries/tasks.ts @simstudioai/mothership
|
||||
|
||||
@@ -35,8 +35,15 @@ jobs:
|
||||
- name: Install dependencies
|
||||
run: bun install --frozen-lockfile
|
||||
|
||||
- name: Apply migrations
|
||||
- name: Apply database schema changes
|
||||
working-directory: ./packages/db
|
||||
env:
|
||||
DATABASE_URL: ${{ github.ref == 'refs/heads/main' && secrets.DATABASE_URL || github.ref == 'refs/heads/dev' && secrets.DEV_DATABASE_URL || secrets.STAGING_DATABASE_URL }}
|
||||
run: bun run ./scripts/migrate.ts
|
||||
run: |
|
||||
if [ "${{ github.ref }}" = "refs/heads/dev" ]; then
|
||||
echo "Dev environment detected — pushing schema with drizzle-kit (db:push)"
|
||||
bun run db:push --force
|
||||
else
|
||||
echo "Applying versioned migrations (db:migrate)"
|
||||
bun run ./scripts/migrate.ts
|
||||
fi
|
||||
@@ -272,6 +272,7 @@ export const blockTypeToIconMap: Record<string, IconComponent> = {
|
||||
file: DocumentIcon,
|
||||
file_v2: DocumentIcon,
|
||||
file_v4: DocumentIcon,
|
||||
file_v5: DocumentIcon,
|
||||
findymail: FindymailIcon,
|
||||
firecrawl: FirecrawlIcon,
|
||||
fireflies: FirefliesIcon,
|
||||
|
||||
@@ -32,7 +32,7 @@ Trigger workflow from Slack events like mentions, messages, and reactions
|
||||
| Parameter | Type | Description |
|
||||
| --------- | ---- | ----------- |
|
||||
| `event` | object | Slack event data |
|
||||
| ↳ `event_type` | string | Type of Slack event \(e.g., app_mention, message\) |
|
||||
| ↳ `event_type` | string | Type of Slack payload: an Events API event \(e.g., app_mention, message\), an interactivity type \(e.g., block_actions\), or "slash_command" for slash commands |
|
||||
| ↳ `subtype` | string | Message subtype \(e.g., channel_join, channel_leave, bot_message, file_share\). Null for regular user messages |
|
||||
| ↳ `channel` | string | Slack channel ID where the event occurred |
|
||||
| ↳ `channel_name` | string | Human-readable channel name |
|
||||
@@ -40,13 +40,22 @@ Trigger workflow from Slack events like mentions, messages, and reactions
|
||||
| ↳ `user` | string | User ID who triggered the event |
|
||||
| ↳ `user_name` | string | Username who triggered the event |
|
||||
| ↳ `bot_id` | string | Bot ID if the message was sent by a bot. Null for human users |
|
||||
| ↳ `text` | string | Message text content |
|
||||
| ↳ `text` | string | Message text content. For slash commands, the text after the command. For interactivity, the source message text \(falls back to the triggering action value\) |
|
||||
| ↳ `timestamp` | string | Message timestamp from the triggering event |
|
||||
| ↳ `thread_ts` | string | Parent thread timestamp \(if message is in a thread\) |
|
||||
| ↳ `team_id` | string | Slack workspace/team ID |
|
||||
| ↳ `event_id` | string | Unique event identifier |
|
||||
| ↳ `reaction` | string | Emoji reaction name \(e.g., thumbsup\). Present for reaction_added/reaction_removed events |
|
||||
| ↳ `item_user` | string | User ID of the original message author. Present for reaction_added/reaction_removed events |
|
||||
| ↳ `command` | string | Slash command name including the leading slash \(e.g., /deploy\). Present for slash commands |
|
||||
| ↳ `action_id` | string | action_id of the first interactive element triggered. Present for block_actions \(button/select clicks\) |
|
||||
| ↳ `action_value` | string | Value carried by the first interactive element \(button value, selected option, date, etc.\). Present for block_actions |
|
||||
| ↳ `actions` | json | Full array of interactive actions from the payload, preserving every element and its value. Present for block_actions |
|
||||
| ↳ `response_url` | string | Temporary URL to post a response back to the originating message or command. Present for interactivity and slash commands |
|
||||
| ↳ `trigger_id` | string | Short-lived trigger ID used to open a modal in response. Present for interactivity and slash commands |
|
||||
| ↳ `callback_id` | string | Callback ID of the shortcut or view. Present for shortcuts and modal submissions |
|
||||
| ↳ `api_app_id` | string | Slack app ID. Present for interactivity and slash commands |
|
||||
| ↳ `message_ts` | string | Timestamp of the message the interaction originated from. Present for block_actions |
|
||||
| ↳ `hasFiles` | boolean | Whether the message has file attachments |
|
||||
| ↳ `files` | file[] | File attachments downloaded from the message \(if includeFiles is enabled and bot token is provided\) |
|
||||
|
||||
|
||||
@@ -39,6 +39,14 @@ async function main() {
|
||||
hasRedis: !!env.REDIS_URL,
|
||||
})
|
||||
|
||||
// Register the HTTP handler before Socket.IO attaches: engine.io captures
|
||||
// pre-existing `request` listeners and forwards only non-`/socket.io/`
|
||||
// requests to them, making it the single dispatcher for the shared port.
|
||||
// The handler itself is assigned after the room manager exists, before listen().
|
||||
// biome-ignore lint/style/useConst: must be declared before the request listener closure; assigned only after the room manager exists
|
||||
let httpHandler: ReturnType<typeof createHttpHandler> | undefined
|
||||
httpServer.on('request', (req, res) => httpHandler?.(req, res))
|
||||
|
||||
// Create Socket.IO server with Redis adapter if configured
|
||||
const io = await createSocketIOServer(httpServer)
|
||||
|
||||
@@ -49,8 +57,7 @@ async function main() {
|
||||
io.use(authenticateSocket)
|
||||
|
||||
// Set up HTTP handler for health checks and internal APIs
|
||||
const httpHandler = createHttpHandler(roomManager, logger)
|
||||
httpServer.on('request', httpHandler)
|
||||
httpHandler = createHttpHandler(roomManager, logger)
|
||||
|
||||
// Global error handlers
|
||||
process.on('uncaughtException', (error) => {
|
||||
|
||||
@@ -172,3 +172,59 @@ export const GET = withRouteHandler(async (req: NextRequest) => {
|
||||
)
|
||||
}
|
||||
})
|
||||
|
||||
export const DELETE = withRouteHandler(async (req: NextRequest) => {
|
||||
const userId = await getAuthorizedAdminUserId()
|
||||
if (!userId) {
|
||||
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
|
||||
}
|
||||
|
||||
const adminKey = env.MOTHERSHIP_API_ADMIN_KEY
|
||||
if (!adminKey) {
|
||||
return NextResponse.json({ error: 'MOTHERSHIP_API_ADMIN_KEY not configured' }, { status: 500 })
|
||||
}
|
||||
|
||||
const { searchParams } = new URL(req.url)
|
||||
const queryValidation = adminMothershipQuerySchema.safeParse(searchParamsToObject(searchParams))
|
||||
if (!queryValidation.success) return validationErrorResponse(queryValidation.error)
|
||||
const { env: environment, endpoint } = queryValidation.data
|
||||
|
||||
if (!isValidEndpoint(endpoint)) {
|
||||
return NextResponse.json({ error: 'invalid endpoint' }, { status: 400 })
|
||||
}
|
||||
|
||||
const baseUrl = await getMothershipUrl(environment, userId)
|
||||
if (!baseUrl) {
|
||||
return NextResponse.json(
|
||||
{ error: `No URL configured for environment: ${environment}` },
|
||||
{ status: 400 }
|
||||
)
|
||||
}
|
||||
|
||||
const forwardParams = new URLSearchParams()
|
||||
searchParams.forEach((value, key) => {
|
||||
if (key !== 'env' && key !== 'endpoint') {
|
||||
forwardParams.set(key, value)
|
||||
}
|
||||
})
|
||||
|
||||
const qs = forwardParams.toString()
|
||||
const targetUrl = `${baseUrl}/api/admin/${endpoint}${qs ? `?${qs}` : ''}`
|
||||
|
||||
try {
|
||||
const upstream = await fetch(targetUrl, {
|
||||
method: 'DELETE',
|
||||
headers: { 'x-api-key': adminKey },
|
||||
})
|
||||
|
||||
const data = await upstream.json()
|
||||
return NextResponse.json(data, { status: upstream.status })
|
||||
} catch (error) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
error: `Failed to reach mothership (${environment}): ${getErrorMessage(error, 'Unknown error')}`,
|
||||
},
|
||||
{ status: 502 }
|
||||
)
|
||||
}
|
||||
})
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
import { type NextRequest, NextResponse } from 'next/server'
|
||||
import { getMyMemberCreditsContract } from '@/lib/api/contracts/organization'
|
||||
import { parseRequest } from '@/lib/api/server'
|
||||
import { getSession } from '@/lib/auth'
|
||||
import { checkOrgMemberUsageLimit } from '@/lib/billing/calculations/usage-monitor'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
|
||||
/**
|
||||
* GET /api/billing/member-credits?workspaceId=...
|
||||
*
|
||||
* Returns the caller's OWN per-member usage and cap inside the workspace's
|
||||
* organization, in DOLLARS (the DB unit) so the client's `formatCredits` does the
|
||||
* single dollars→credits conversion. Own-data only, so no admin gate (unlike the
|
||||
* org/member admin route). Reuses {@link checkOrgMemberUsageLimit}, which yields a
|
||||
* null limit — and the chip falls back to the plan-level view — whenever no
|
||||
* per-member cap applies: non-hosted, the workspace isn't org-owned, or no cap is
|
||||
* set for this member.
|
||||
*/
|
||||
export const GET = withRouteHandler(async (request: NextRequest) => {
|
||||
const session = await getSession()
|
||||
if (!session?.user?.id) {
|
||||
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
|
||||
}
|
||||
|
||||
const parsed = await parseRequest(getMyMemberCreditsContract, request, {})
|
||||
if (!parsed.success) return parsed.response
|
||||
|
||||
const { workspaceId } = parsed.data.query
|
||||
const { currentUsage, limit } = await checkOrgMemberUsageLimit(session.user.id, workspaceId)
|
||||
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
data: {
|
||||
usedDollars: currentUsage,
|
||||
limitDollars: limit,
|
||||
},
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,134 @@
|
||||
/**
|
||||
* @vitest-environment node
|
||||
*/
|
||||
import { createMockRequest } from '@sim/testing'
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
|
||||
const {
|
||||
mockCheckInternalApiKey,
|
||||
mockRecordUsage,
|
||||
mockRecordCumulativeUsage,
|
||||
mockCheckAndBillOverageThreshold,
|
||||
} = vi.hoisted(() => ({
|
||||
mockCheckInternalApiKey: vi.fn(),
|
||||
mockRecordUsage: vi.fn(),
|
||||
mockRecordCumulativeUsage: vi.fn(),
|
||||
mockCheckAndBillOverageThreshold: vi.fn(),
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/copilot/request/http', () => ({
|
||||
checkInternalApiKey: mockCheckInternalApiKey,
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/copilot/request/otel', () => ({
|
||||
withIncomingGoSpan: (
|
||||
_headers: unknown,
|
||||
_span: unknown,
|
||||
_attrs: unknown,
|
||||
fn: (span: { setAttribute: () => void; setAttributes: () => void }) => unknown
|
||||
) => fn({ setAttribute: vi.fn(), setAttributes: vi.fn() }),
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/billing/core/usage-log', () => ({
|
||||
recordUsage: mockRecordUsage,
|
||||
recordCumulativeUsage: mockRecordCumulativeUsage,
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/billing/threshold-billing', () => ({
|
||||
checkAndBillOverageThreshold: mockCheckAndBillOverageThreshold,
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/core/config/feature-flags', () => ({
|
||||
isBillingEnabled: true,
|
||||
}))
|
||||
|
||||
import { POST } from '@/app/api/billing/update-cost/route'
|
||||
|
||||
describe('POST /api/billing/update-cost — workspaceId attribution', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
mockCheckInternalApiKey.mockReturnValue({ success: true })
|
||||
mockRecordUsage.mockResolvedValue(undefined)
|
||||
mockRecordCumulativeUsage.mockResolvedValue({ billed: true, delta: 0.5, total: 0.5 })
|
||||
mockCheckAndBillOverageThreshold.mockResolvedValue(undefined)
|
||||
})
|
||||
|
||||
it('stamps workspaceId onto recorded usage when provided (no idempotency key)', async () => {
|
||||
const res = await POST(
|
||||
createMockRequest(
|
||||
'POST',
|
||||
{ userId: 'user-1', cost: 0.5, model: 'gpt', source: 'mcp_copilot', workspaceId: 'ws-1' },
|
||||
{ 'x-api-key': 'internal' }
|
||||
)
|
||||
)
|
||||
expect(res.status).toBe(200)
|
||||
expect(mockRecordUsage).toHaveBeenCalledTimes(1)
|
||||
expect(mockRecordUsage.mock.calls[0][0]).toMatchObject({
|
||||
userId: 'user-1',
|
||||
workspaceId: 'ws-1',
|
||||
})
|
||||
})
|
||||
|
||||
it('records cumulative cost via monotonic top-up when an idempotency key is present', async () => {
|
||||
const res = await POST(
|
||||
createMockRequest(
|
||||
'POST',
|
||||
{
|
||||
userId: 'user-1',
|
||||
cost: 0.4662453,
|
||||
model: 'claude-opus-4.8',
|
||||
source: 'workspace-chat',
|
||||
workspaceId: 'ws-1',
|
||||
idempotencyKey: 'msg-1-billing',
|
||||
inputTokens: 461371,
|
||||
outputTokens: 1686,
|
||||
},
|
||||
{ 'x-api-key': 'internal' }
|
||||
)
|
||||
)
|
||||
expect(res.status).toBe(200)
|
||||
expect(mockRecordUsage).not.toHaveBeenCalled()
|
||||
expect(mockRecordCumulativeUsage).toHaveBeenCalledTimes(1)
|
||||
expect(mockRecordCumulativeUsage.mock.calls[0][0]).toMatchObject({
|
||||
userId: 'user-1',
|
||||
workspaceId: 'ws-1',
|
||||
source: 'workspace-chat',
|
||||
model: 'claude-opus-4.8',
|
||||
cost: 0.4662453,
|
||||
eventKey: 'update-cost:msg-1-billing',
|
||||
})
|
||||
expect(mockCheckAndBillOverageThreshold).toHaveBeenCalledWith('user-1')
|
||||
})
|
||||
|
||||
it('returns 409 and skips overage when the cumulative is not higher (duplicate flush)', async () => {
|
||||
mockRecordCumulativeUsage.mockResolvedValue({ billed: false, delta: 0, total: 0.4662453 })
|
||||
const res = await POST(
|
||||
createMockRequest(
|
||||
'POST',
|
||||
{
|
||||
userId: 'user-1',
|
||||
cost: 0.4662453,
|
||||
model: 'claude-opus-4.8',
|
||||
source: 'workspace-chat',
|
||||
workspaceId: 'ws-1',
|
||||
idempotencyKey: 'msg-1-billing',
|
||||
},
|
||||
{ 'x-api-key': 'internal' }
|
||||
)
|
||||
)
|
||||
expect(res.status).toBe(409)
|
||||
expect(mockCheckAndBillOverageThreshold).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('rejects with 400 when workspaceId is omitted (contract-required, fail loud)', async () => {
|
||||
const res = await POST(
|
||||
createMockRequest(
|
||||
'POST',
|
||||
{ userId: 'user-1', cost: 0.5, model: 'gpt', source: 'copilot' },
|
||||
{ 'x-api-key': 'internal' }
|
||||
)
|
||||
)
|
||||
expect(res.status).toBe(400)
|
||||
expect(mockRecordUsage).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
@@ -4,7 +4,7 @@ import { toError } from '@sim/utils/errors'
|
||||
import { type NextRequest, NextResponse } from 'next/server'
|
||||
import { billingUpdateCostContract } from '@/lib/api/contracts/subscription'
|
||||
import { parseRequest } from '@/lib/api/server'
|
||||
import { recordUsage } from '@/lib/billing/core/usage-log'
|
||||
import { recordCumulativeUsage, recordUsage } from '@/lib/billing/core/usage-log'
|
||||
import { checkAndBillOverageThreshold } from '@/lib/billing/threshold-billing'
|
||||
import { BillingRouteOutcome } from '@/lib/copilot/generated/trace-attribute-values-v1'
|
||||
import { TraceAttr } from '@/lib/copilot/generated/trace-attributes-v1'
|
||||
@@ -12,7 +12,6 @@ import { TraceSpan } from '@/lib/copilot/generated/trace-spans-v1'
|
||||
import { checkInternalApiKey } from '@/lib/copilot/request/http'
|
||||
import { withIncomingGoSpan } from '@/lib/copilot/request/otel'
|
||||
import { isBillingEnabled } from '@/lib/core/config/feature-flags'
|
||||
import { type AtomicClaimResult, billingIdempotency } from '@/lib/core/idempotency/service'
|
||||
import { generateRequestId } from '@/lib/core/utils/request'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
|
||||
@@ -42,8 +41,6 @@ export const POST = withRouteHandler((req: NextRequest) =>
|
||||
async function updateCostInner(req: NextRequest, span: Span): Promise<NextResponse> {
|
||||
const requestId = generateRequestId()
|
||||
const startTime = Date.now()
|
||||
let claim: AtomicClaimResult | null = null
|
||||
let usageCommitted = false
|
||||
|
||||
try {
|
||||
logger.info(`[${requestId}] Update cost request started`)
|
||||
@@ -110,7 +107,7 @@ async function updateCostInner(req: NextRequest, span: Span): Promise<NextRespon
|
||||
|
||||
if (!parsed.success) return parsed.response
|
||||
|
||||
const { userId, cost, model, inputTokens, outputTokens, source, idempotencyKey } =
|
||||
const { userId, cost, model, inputTokens, outputTokens, source, idempotencyKey, workspaceId } =
|
||||
parsed.data.body
|
||||
const isMcp = source === 'mcp_copilot'
|
||||
|
||||
@@ -125,28 +122,6 @@ async function updateCostInner(req: NextRequest, span: Span): Promise<NextRespon
|
||||
...(idempotencyKey ? { [TraceAttr.BillingIdempotencyKey]: idempotencyKey } : {}),
|
||||
})
|
||||
|
||||
claim = idempotencyKey
|
||||
? await billingIdempotency.atomicallyClaim('update-cost', idempotencyKey)
|
||||
: null
|
||||
|
||||
if (claim && !claim.claimed) {
|
||||
logger.warn(`[${requestId}] Duplicate billing update rejected`, {
|
||||
idempotencyKey,
|
||||
userId,
|
||||
source,
|
||||
})
|
||||
span.setAttribute(TraceAttr.BillingOutcome, BillingRouteOutcome.DuplicateIdempotencyKey)
|
||||
span.setAttribute(TraceAttr.HttpStatusCode, 409)
|
||||
return NextResponse.json(
|
||||
{
|
||||
success: false,
|
||||
error: 'Duplicate request: idempotency key already processed',
|
||||
requestId,
|
||||
},
|
||||
{ status: 409 }
|
||||
)
|
||||
}
|
||||
|
||||
logger.info(`[${requestId}] Processing cost update`, {
|
||||
userId,
|
||||
cost,
|
||||
@@ -154,33 +129,85 @@ async function updateCostInner(req: NextRequest, span: Span): Promise<NextRespon
|
||||
source,
|
||||
})
|
||||
|
||||
await recordUsage({
|
||||
userId,
|
||||
entries: [
|
||||
{
|
||||
category: 'model',
|
||||
source,
|
||||
description: model,
|
||||
cost,
|
||||
eventKey: idempotencyKey ? `update-cost:${idempotencyKey}` : undefined,
|
||||
sourceReference: idempotencyKey ? `update-cost:${idempotencyKey}` : requestId,
|
||||
metadata: { inputTokens, outputTokens },
|
||||
},
|
||||
],
|
||||
})
|
||||
usageCommitted = true
|
||||
|
||||
logger.info(`[${requestId}] Recorded usage`, {
|
||||
userId,
|
||||
addedCost: cost,
|
||||
source,
|
||||
})
|
||||
|
||||
// Check if user has hit overage threshold and bill incrementally
|
||||
await checkAndBillOverageThreshold(userId)
|
||||
// Go sends the request's CUMULATIVE cost, possibly more than once (a
|
||||
// mid-loop provider-error flush, then the recovered terminal flush, plus
|
||||
// abort-race duplicates). Record it as a monotonic top-up: one ledger row
|
||||
// per request holds the MAX cumulative and we bill only the delta, so
|
||||
// partial + complete flushes converge to the true total exactly once — no
|
||||
// under-billing on recovery, no over-billing on duplicates. When there is
|
||||
// no idempotency key (shouldn't happen for real requests) we fall back to a
|
||||
// plain append so cost is never silently dropped.
|
||||
let billed = true
|
||||
if (idempotencyKey) {
|
||||
const result = await recordCumulativeUsage({
|
||||
userId,
|
||||
workspaceId,
|
||||
source,
|
||||
model,
|
||||
cost,
|
||||
eventKey: `update-cost:${idempotencyKey}`,
|
||||
metadata: { inputTokens, outputTokens },
|
||||
})
|
||||
billed = result.billed
|
||||
logger.info(`[${requestId}] Cumulative cost top-up`, {
|
||||
userId,
|
||||
source,
|
||||
cumulativeCost: cost,
|
||||
billedDelta: result.delta,
|
||||
newTotal: result.total,
|
||||
billed: result.billed,
|
||||
})
|
||||
} else {
|
||||
await recordUsage({
|
||||
userId,
|
||||
workspaceId,
|
||||
entries: [
|
||||
{
|
||||
category: 'model',
|
||||
source,
|
||||
description: model,
|
||||
cost,
|
||||
sourceReference: requestId,
|
||||
metadata: { inputTokens, outputTokens },
|
||||
},
|
||||
],
|
||||
})
|
||||
logger.info(`[${requestId}] Recorded usage (no idempotency key)`, {
|
||||
userId,
|
||||
addedCost: cost,
|
||||
source,
|
||||
})
|
||||
}
|
||||
|
||||
const duration = Date.now() - startTime
|
||||
|
||||
// Same-or-lower cumulative than already recorded: nothing new to bill. Tell
|
||||
// the caller via 409 (its existing "duplicate" outcome) without re-running
|
||||
// overage billing.
|
||||
if (!billed) {
|
||||
logger.info(`[${requestId}] Duplicate/non-increasing cumulative cost; no new charge`, {
|
||||
idempotencyKey,
|
||||
userId,
|
||||
cost,
|
||||
})
|
||||
span.setAttribute(TraceAttr.BillingOutcome, BillingRouteOutcome.DuplicateIdempotencyKey)
|
||||
span.setAttribute(TraceAttr.HttpStatusCode, 409)
|
||||
span.setAttribute(TraceAttr.BillingDurationMs, duration)
|
||||
return NextResponse.json(
|
||||
{
|
||||
success: false,
|
||||
error: 'Duplicate request: cumulative cost already recorded',
|
||||
requestId,
|
||||
},
|
||||
{ status: 409 }
|
||||
)
|
||||
}
|
||||
|
||||
// Check if user has hit overage threshold and bill incrementally. Reads the
|
||||
// (now topped-up) ledger total and is idempotent against billedOverage, so
|
||||
// it is safe to run on every flush that records new cost.
|
||||
await checkAndBillOverageThreshold(userId)
|
||||
|
||||
logger.info(`[${requestId}] Cost update completed successfully`, {
|
||||
userId,
|
||||
duration,
|
||||
@@ -208,22 +235,9 @@ async function updateCostInner(req: NextRequest, span: Span): Promise<NextRespon
|
||||
duration,
|
||||
})
|
||||
|
||||
if (claim?.claimed && !usageCommitted) {
|
||||
await billingIdempotency
|
||||
.release(claim.normalizedKey, claim.storageMethod)
|
||||
.catch((releaseErr) => {
|
||||
logger.warn(`[${requestId}] Failed to release idempotency claim`, {
|
||||
error: toError(releaseErr).message,
|
||||
normalizedKey: claim?.normalizedKey,
|
||||
})
|
||||
})
|
||||
} else if (claim?.claimed && usageCommitted) {
|
||||
logger.warn(
|
||||
`[${requestId}] Error occurred after usage committed; retaining idempotency claim to prevent double-billing`,
|
||||
{ normalizedKey: claim.normalizedKey }
|
||||
)
|
||||
}
|
||||
|
||||
// The cumulative top-up runs in a single transaction (and a plain append is
|
||||
// a single insert), so a failure here leaves nothing partially committed —
|
||||
// a retry re-evaluates the max idempotently. No claim to release.
|
||||
span.setAttribute(TraceAttr.BillingOutcome, BillingRouteOutcome.InternalError)
|
||||
span.setAttribute(TraceAttr.HttpStatusCode, 500)
|
||||
span.setAttribute(TraceAttr.BillingDurationMs, duration)
|
||||
|
||||
@@ -0,0 +1,114 @@
|
||||
/**
|
||||
* @vitest-environment node
|
||||
*/
|
||||
import { createMockRequest } from '@sim/testing'
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
|
||||
const {
|
||||
mockFlags,
|
||||
mockDbLimit,
|
||||
mockCheckInternalApiKey,
|
||||
mockCheckServerSideUsageLimits,
|
||||
mockCheckOrgMemberUsageLimit,
|
||||
} = vi.hoisted(() => ({
|
||||
mockFlags: { isHosted: true },
|
||||
mockDbLimit: vi.fn(),
|
||||
mockCheckInternalApiKey: vi.fn(),
|
||||
mockCheckServerSideUsageLimits: vi.fn(),
|
||||
mockCheckOrgMemberUsageLimit: vi.fn(),
|
||||
}))
|
||||
|
||||
vi.mock('@sim/db', () => ({
|
||||
db: {
|
||||
select: () => ({ from: () => ({ where: () => ({ limit: mockDbLimit }) }) }),
|
||||
},
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/billing/calculations/usage-monitor', () => ({
|
||||
checkServerSideUsageLimits: mockCheckServerSideUsageLimits,
|
||||
checkOrgMemberUsageLimit: mockCheckOrgMemberUsageLimit,
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/copilot/request/http', () => ({
|
||||
checkInternalApiKey: mockCheckInternalApiKey,
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/copilot/request/otel', () => ({
|
||||
withIncomingGoSpan: (
|
||||
_headers: unknown,
|
||||
_span: unknown,
|
||||
_attrs: unknown,
|
||||
fn: (span: { setAttribute: () => void; setAttributes: () => void }) => unknown
|
||||
) => fn({ setAttribute: vi.fn(), setAttributes: vi.fn() }),
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/core/config/feature-flags', () => ({
|
||||
get isHosted() {
|
||||
return mockFlags.isHosted
|
||||
},
|
||||
}))
|
||||
|
||||
import { POST } from '@/app/api/copilot/api-keys/validate/route'
|
||||
|
||||
function request(body: Record<string, unknown>) {
|
||||
return createMockRequest('POST', body, { 'x-api-key': 'internal' })
|
||||
}
|
||||
|
||||
describe('POST /api/copilot/api-keys/validate — per-member enforcement', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
mockFlags.isHosted = true
|
||||
mockCheckInternalApiKey.mockReturnValue({ success: true })
|
||||
mockDbLimit.mockResolvedValue([{ id: 'user-1' }])
|
||||
mockCheckServerSideUsageLimits.mockResolvedValue({
|
||||
isExceeded: false,
|
||||
currentUsage: 0,
|
||||
limit: 100,
|
||||
})
|
||||
mockCheckOrgMemberUsageLimit.mockResolvedValue({
|
||||
isExceeded: false,
|
||||
currentUsage: 0,
|
||||
limit: null,
|
||||
})
|
||||
})
|
||||
|
||||
it('returns 402 when the pooled/personal limit is exceeded (existing behavior)', async () => {
|
||||
mockCheckServerSideUsageLimits.mockResolvedValue({
|
||||
isExceeded: true,
|
||||
currentUsage: 200,
|
||||
limit: 100,
|
||||
})
|
||||
const res = await POST(request({ userId: 'user-1', workspaceId: 'ws-1' }))
|
||||
expect(res.status).toBe(402)
|
||||
expect(mockCheckOrgMemberUsageLimit).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('returns 402 when the per-member org-workspace cap is exceeded', async () => {
|
||||
mockCheckOrgMemberUsageLimit.mockResolvedValue({
|
||||
isExceeded: true,
|
||||
currentUsage: 5,
|
||||
limit: 4,
|
||||
})
|
||||
const res = await POST(request({ userId: 'user-1', workspaceId: 'ws-1' }))
|
||||
expect(res.status).toBe(402)
|
||||
expect(mockCheckOrgMemberUsageLimit).toHaveBeenCalledWith('user-1', 'ws-1')
|
||||
})
|
||||
|
||||
it('returns 200 when under both limits', async () => {
|
||||
const res = await POST(request({ userId: 'user-1', workspaceId: 'ws-1' }))
|
||||
expect(res.status).toBe(200)
|
||||
})
|
||||
|
||||
it('rejects with 400 when workspaceId is omitted (contract-required, fail closed)', async () => {
|
||||
const res = await POST(request({ userId: 'user-1' }))
|
||||
expect(res.status).toBe(400)
|
||||
expect(mockCheckOrgMemberUsageLimit).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('skips the per-member check when not hosted', async () => {
|
||||
mockFlags.isHosted = false
|
||||
const res = await POST(request({ userId: 'user-1', workspaceId: 'ws-1' }))
|
||||
expect(res.status).toBe(200)
|
||||
expect(mockCheckOrgMemberUsageLimit).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
@@ -5,12 +5,16 @@ import { eq } from 'drizzle-orm'
|
||||
import { type NextRequest, NextResponse } from 'next/server'
|
||||
import { validateCopilotApiKeyContract } from '@/lib/api/contracts/copilot'
|
||||
import { parseRequest, validationErrorResponse } from '@/lib/api/server'
|
||||
import { checkServerSideUsageLimits } from '@/lib/billing/calculations/usage-monitor'
|
||||
import {
|
||||
checkOrgMemberUsageLimit,
|
||||
checkServerSideUsageLimits,
|
||||
} from '@/lib/billing/calculations/usage-monitor'
|
||||
import { CopilotValidateOutcome } from '@/lib/copilot/generated/trace-attribute-values-v1'
|
||||
import { TraceAttr } from '@/lib/copilot/generated/trace-attributes-v1'
|
||||
import { TraceSpan } from '@/lib/copilot/generated/trace-spans-v1'
|
||||
import { checkInternalApiKey } from '@/lib/copilot/request/http'
|
||||
import { withIncomingGoSpan } from '@/lib/copilot/request/otel'
|
||||
import { isHosted } from '@/lib/core/config/feature-flags'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
|
||||
const logger = createLogger('CopilotApiKeysValidate')
|
||||
@@ -71,7 +75,7 @@ export const POST = withRouteHandler((req: NextRequest) =>
|
||||
)
|
||||
if (!parsed.success) return parsed.response
|
||||
|
||||
const { userId } = parsed.data.body
|
||||
const { userId, workspaceId } = parsed.data.body
|
||||
span.setAttribute(TraceAttr.UserId, userId)
|
||||
|
||||
const [existingUser] = await db.select().from(user).where(eq(user.id, userId)).limit(1)
|
||||
@@ -104,6 +108,28 @@ export const POST = withRouteHandler((req: NextRequest) =>
|
||||
return new NextResponse(null, { status: 402 })
|
||||
}
|
||||
|
||||
// Per-member org-workspace cap (hosted-only). Blocks the mothership/copilot
|
||||
// chat request itself when the user is over their personal credit limit for
|
||||
// the org that owns this workspace, independent of the pooled org limit.
|
||||
// workspaceId is contract-required, so the gate can't be silently skipped.
|
||||
if (isHosted) {
|
||||
const memberCheck = await checkOrgMemberUsageLimit(userId, workspaceId)
|
||||
if (memberCheck.isExceeded) {
|
||||
logger.info('[API VALIDATION] Per-member org usage limit exceeded', {
|
||||
userId,
|
||||
workspaceId,
|
||||
currentUsage: memberCheck.currentUsage,
|
||||
limit: memberCheck.limit,
|
||||
})
|
||||
span.setAttribute(
|
||||
TraceAttr.CopilotValidateOutcome,
|
||||
CopilotValidateOutcome.UsageExceeded
|
||||
)
|
||||
span.setAttribute(TraceAttr.HttpStatusCode, 402)
|
||||
return new NextResponse(null, { status: 402 })
|
||||
}
|
||||
}
|
||||
|
||||
span.setAttribute(TraceAttr.CopilotValidateOutcome, CopilotValidateOutcome.Ok)
|
||||
span.setAttribute(TraceAttr.HttpStatusCode, 200)
|
||||
return new NextResponse(null, { status: 200 })
|
||||
|
||||
@@ -1,168 +0,0 @@
|
||||
import { createLogger } from '@sim/logger'
|
||||
import { type NextRequest, NextResponse } from 'next/server'
|
||||
import {
|
||||
addCopilotAutoAllowedToolContract,
|
||||
removeCopilotAutoAllowedToolContract,
|
||||
} from '@/lib/api/contracts/copilot'
|
||||
import { parseRequest } from '@/lib/api/server'
|
||||
import { getSession } from '@/lib/auth'
|
||||
import { TraceAttr } from '@/lib/copilot/generated/trace-attributes-v1'
|
||||
import { fetchGo } from '@/lib/copilot/request/go/fetch'
|
||||
import { getMothershipBaseURL } from '@/lib/copilot/server/agent-url'
|
||||
import { env } from '@/lib/core/config/env'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
|
||||
const logger = createLogger('CopilotAutoAllowedToolsAPI')
|
||||
|
||||
/** Headers for server-to-server calls to the copilot backend. */
|
||||
function copilotHeaders(): Record<string, string> {
|
||||
const headers: Record<string, string> = {
|
||||
'Content-Type': 'application/json',
|
||||
}
|
||||
if (env.COPILOT_API_KEY) {
|
||||
headers['x-api-key'] = env.COPILOT_API_KEY
|
||||
}
|
||||
return headers
|
||||
}
|
||||
|
||||
/**
|
||||
* GET - Fetch user's auto-allowed integration tools
|
||||
*/
|
||||
export const GET = withRouteHandler(async () => {
|
||||
try {
|
||||
const session = await getSession()
|
||||
|
||||
if (!session?.user?.id) {
|
||||
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
|
||||
}
|
||||
|
||||
const userId = session.user.id
|
||||
const mothershipBaseURL = await getMothershipBaseURL({ userId })
|
||||
|
||||
const res = await fetchGo(
|
||||
`${mothershipBaseURL}/api/tool-preferences/auto-allowed?userId=${encodeURIComponent(userId)}`,
|
||||
{
|
||||
method: 'GET',
|
||||
headers: copilotHeaders(),
|
||||
spanName: 'sim → go /api/tool-preferences/auto-allowed',
|
||||
operation: 'list_auto_allowed_tools',
|
||||
attributes: { [TraceAttr.UserId]: userId },
|
||||
}
|
||||
)
|
||||
|
||||
if (!res.ok) {
|
||||
logger.warn('Copilot returned error for list auto-allowed', { status: res.status })
|
||||
return NextResponse.json({ autoAllowedTools: [] })
|
||||
}
|
||||
|
||||
const payload = await res.json()
|
||||
return NextResponse.json({ autoAllowedTools: payload?.autoAllowedTools || [] })
|
||||
} catch (error) {
|
||||
logger.error('Failed to fetch auto-allowed tools', { error })
|
||||
return NextResponse.json({ error: 'Internal server error' }, { status: 500 })
|
||||
}
|
||||
})
|
||||
|
||||
/**
|
||||
* POST - Add a tool to the auto-allowed list
|
||||
*/
|
||||
export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
try {
|
||||
const session = await getSession()
|
||||
|
||||
if (!session?.user?.id) {
|
||||
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
|
||||
}
|
||||
|
||||
const userId = session.user.id
|
||||
const mothershipBaseURL = await getMothershipBaseURL({ userId })
|
||||
const parsed = await parseRequest(
|
||||
addCopilotAutoAllowedToolContract,
|
||||
request,
|
||||
{},
|
||||
{
|
||||
validationErrorResponse: () =>
|
||||
NextResponse.json({ error: 'toolId must be a string' }, { status: 400 }),
|
||||
invalidJsonResponse: () =>
|
||||
NextResponse.json({ error: 'toolId must be a string' }, { status: 400 }),
|
||||
}
|
||||
)
|
||||
if (!parsed.success) return parsed.response
|
||||
const { toolId } = parsed.data.body
|
||||
|
||||
const res = await fetchGo(`${mothershipBaseURL}/api/tool-preferences/auto-allowed`, {
|
||||
method: 'POST',
|
||||
headers: copilotHeaders(),
|
||||
body: JSON.stringify({ userId, toolId }),
|
||||
spanName: 'sim → go /api/tool-preferences/auto-allowed',
|
||||
operation: 'add_auto_allowed_tool',
|
||||
attributes: { [TraceAttr.UserId]: userId, [TraceAttr.ToolId]: toolId },
|
||||
})
|
||||
|
||||
if (!res.ok) {
|
||||
logger.warn('Copilot returned error for add auto-allowed', { status: res.status })
|
||||
return NextResponse.json({ error: 'Failed to add tool' }, { status: 500 })
|
||||
}
|
||||
|
||||
const payload = await res.json()
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
autoAllowedTools: payload?.autoAllowedTools || [],
|
||||
})
|
||||
} catch (error) {
|
||||
logger.error('Failed to add auto-allowed tool', { error })
|
||||
return NextResponse.json({ error: 'Internal server error' }, { status: 500 })
|
||||
}
|
||||
})
|
||||
|
||||
/**
|
||||
* DELETE - Remove a tool from the auto-allowed list
|
||||
*/
|
||||
export const DELETE = withRouteHandler(async (request: NextRequest) => {
|
||||
try {
|
||||
const session = await getSession()
|
||||
|
||||
if (!session?.user?.id) {
|
||||
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
|
||||
}
|
||||
|
||||
const userId = session.user.id
|
||||
const mothershipBaseURL = await getMothershipBaseURL({ userId })
|
||||
const parsed = await parseRequest(
|
||||
removeCopilotAutoAllowedToolContract,
|
||||
request,
|
||||
{},
|
||||
{
|
||||
validationErrorResponse: () =>
|
||||
NextResponse.json({ error: 'toolId query parameter is required' }, { status: 400 }),
|
||||
}
|
||||
)
|
||||
if (!parsed.success) return parsed.response
|
||||
const { toolId } = parsed.data.query
|
||||
|
||||
const res = await fetchGo(
|
||||
`${mothershipBaseURL}/api/tool-preferences/auto-allowed?userId=${encodeURIComponent(userId)}&toolId=${encodeURIComponent(toolId)}`,
|
||||
{
|
||||
method: 'DELETE',
|
||||
headers: copilotHeaders(),
|
||||
spanName: 'sim → go /api/tool-preferences/auto-allowed',
|
||||
operation: 'remove_auto_allowed_tool',
|
||||
attributes: { [TraceAttr.UserId]: userId, [TraceAttr.ToolId]: toolId },
|
||||
}
|
||||
)
|
||||
|
||||
if (!res.ok) {
|
||||
logger.warn('Copilot returned error for remove auto-allowed', { status: res.status })
|
||||
return NextResponse.json({ error: 'Failed to remove tool' }, { status: 500 })
|
||||
}
|
||||
|
||||
const payload = await res.json()
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
autoAllowedTools: payload?.autoAllowedTools || [],
|
||||
})
|
||||
} catch (error) {
|
||||
logger.error('Failed to remove auto-allowed tool', { error })
|
||||
return NextResponse.json({ error: 'Internal server error' }, { status: 500 })
|
||||
}
|
||||
})
|
||||
@@ -0,0 +1,121 @@
|
||||
import { db } from '@sim/db'
|
||||
import { settings, user } from '@sim/db/schema'
|
||||
import { getErrorMessage } from '@sim/utils/errors'
|
||||
import { eq } from 'drizzle-orm'
|
||||
import { type NextRequest, NextResponse } from 'next/server'
|
||||
import {
|
||||
deleteCopilotByokKeyContract,
|
||||
listCopilotByokKeysContract,
|
||||
upsertCopilotByokKeyContract,
|
||||
} from '@/lib/api/contracts/copilot'
|
||||
import { parseRequest } from '@/lib/api/server'
|
||||
import { getSession } from '@/lib/auth'
|
||||
import { SIM_AGENT_API_URL } from '@/lib/copilot/constants'
|
||||
import { getMothershipSourceEnvHeaders } from '@/lib/copilot/server/agent-url'
|
||||
import { env } from '@/lib/core/config/env'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
|
||||
/**
|
||||
* Enterprise BYOK key management for the current workspace's mothership.
|
||||
*
|
||||
* Unlike the cross-environment admin inspector (`/api/admin/mothership`), this
|
||||
* talks to the SAME copilot the workspace's mothership actually runs on —
|
||||
* `SIM_AGENT_API_URL` (local in dev, prod copilot in prod) — and authenticates
|
||||
* with the hosted internal key (`COPILOT_API_KEY`), the exact credential
|
||||
* mothership chat uses. Copilot requires that key (`SIM_AGENT_API_KEY`) and
|
||||
* rejects self-hosted callers, so BYOK can only ever be written through our
|
||||
* hosted Sim. The route is superuser-gated; the workspace id rides in the
|
||||
* request and is resolved by the caller from the route.
|
||||
*/
|
||||
async function getAuthorizedSuperUserId(): Promise<string | null> {
|
||||
const session = await getSession()
|
||||
if (!session?.user?.id) return null
|
||||
|
||||
const [currentUser] = await db
|
||||
.select({ role: user.role, superUserModeEnabled: settings.superUserModeEnabled })
|
||||
.from(user)
|
||||
.leftJoin(settings, eq(settings.userId, user.id))
|
||||
.where(eq(user.id, session.user.id))
|
||||
.limit(1)
|
||||
|
||||
const authorized = currentUser?.role === 'admin' && (currentUser.superUserModeEnabled ?? false)
|
||||
return authorized ? session.user.id : null
|
||||
}
|
||||
|
||||
async function forwardToCopilot(
|
||||
method: 'GET' | 'POST' | 'DELETE',
|
||||
query: URLSearchParams,
|
||||
body?: string
|
||||
) {
|
||||
const headers: Record<string, string> = { ...getMothershipSourceEnvHeaders() }
|
||||
if (env.COPILOT_API_KEY) headers['x-api-key'] = env.COPILOT_API_KEY
|
||||
if (body !== undefined) headers['Content-Type'] = 'application/json'
|
||||
|
||||
const qs = query.toString()
|
||||
const targetUrl = `${SIM_AGENT_API_URL}/api/admin/byok${qs ? `?${qs}` : ''}`
|
||||
|
||||
try {
|
||||
const upstream = await fetch(targetUrl, {
|
||||
method,
|
||||
headers,
|
||||
...(body !== undefined ? { body } : {}),
|
||||
})
|
||||
const text = await upstream.text()
|
||||
// boundary-raw-fetch: copilot returns JSON; tolerate an empty body.
|
||||
const data = text ? JSON.parse(text) : {}
|
||||
return NextResponse.json(data, { status: upstream.status })
|
||||
} catch (error) {
|
||||
return NextResponse.json(
|
||||
{ error: `Failed to reach copilot: ${getErrorMessage(error, 'Unknown error')}` },
|
||||
{ status: 502 }
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
export const GET = withRouteHandler(async (req: NextRequest) => {
|
||||
const userId = await getAuthorizedSuperUserId()
|
||||
if (!userId) {
|
||||
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
|
||||
}
|
||||
|
||||
const parsed = await parseRequest(listCopilotByokKeysContract, req, {})
|
||||
if (!parsed.success) return parsed.response
|
||||
|
||||
return forwardToCopilot(
|
||||
'GET',
|
||||
new URLSearchParams({ workspaceId: parsed.data.query.workspaceId })
|
||||
)
|
||||
})
|
||||
|
||||
export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
const userId = await getAuthorizedSuperUserId()
|
||||
if (!userId) {
|
||||
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
|
||||
}
|
||||
|
||||
const parsed = await parseRequest(upsertCopilotByokKeyContract, req, {})
|
||||
if (!parsed.success) return parsed.response
|
||||
|
||||
// Bind the audit field to the authenticated superuser, ignoring any
|
||||
// client-supplied createdBy so provisioning is always attributable.
|
||||
const body = JSON.stringify({ ...parsed.data.body, createdBy: userId })
|
||||
return forwardToCopilot('POST', new URLSearchParams(), body)
|
||||
})
|
||||
|
||||
export const DELETE = withRouteHandler(async (req: NextRequest) => {
|
||||
const userId = await getAuthorizedSuperUserId()
|
||||
if (!userId) {
|
||||
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
|
||||
}
|
||||
|
||||
const parsed = await parseRequest(deleteCopilotByokKeyContract, req, {})
|
||||
if (!parsed.success) return parsed.response
|
||||
|
||||
return forwardToCopilot(
|
||||
'DELETE',
|
||||
new URLSearchParams({
|
||||
workspaceId: parsed.data.query.workspaceId,
|
||||
provider: parsed.data.query.provider,
|
||||
})
|
||||
)
|
||||
})
|
||||
@@ -0,0 +1,68 @@
|
||||
import { createLogger } from '@sim/logger'
|
||||
import { type NextRequest, NextResponse } from 'next/server'
|
||||
import { validateCopilotByokContract } from '@/lib/api/contracts/copilot'
|
||||
import { parseRequest } from '@/lib/api/server'
|
||||
import { isWorkspaceOnEnterprisePlan } from '@/lib/billing/core/subscription'
|
||||
import { checkInternalApiKey } from '@/lib/copilot/request/http'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import { verifyEffectiveSuperUser } from '@/lib/permissions/super-user'
|
||||
import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils'
|
||||
|
||||
const logger = createLogger('CopilotByokValidate')
|
||||
|
||||
/**
|
||||
* Authoritative entitlement gate for enterprise BYOK, called server-to-server by
|
||||
* the mothership (Go) before it uses a workspace's own provider key. Gated by
|
||||
* INTERNAL_API_SECRET — never exposed to the browser.
|
||||
*
|
||||
* Returns 200 when EITHER:
|
||||
* - the requesting user is a superuser admin (platform admin with superuser
|
||||
* mode on), who may use BYOK on any workspace for management/testing; OR
|
||||
* - the user is a member of the workspace (prevents one org from causing
|
||||
* another org's stored key to be used) AND the workspace is on an
|
||||
* enterprise plan.
|
||||
*
|
||||
* Any other case returns 403 (not entitled) or 401 (bad internal auth). The Go
|
||||
* caller fails closed to hosted keys on anything but a 200.
|
||||
*/
|
||||
export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
const auth = checkInternalApiKey(req)
|
||||
if (!auth.success) {
|
||||
return new NextResponse(null, { status: 401 })
|
||||
}
|
||||
|
||||
const parsed = await parseRequest(validateCopilotByokContract, req, {})
|
||||
if (!parsed.success) return parsed.response
|
||||
|
||||
const { workspaceId, userId } = parsed.data.body
|
||||
|
||||
try {
|
||||
// Superuser admins may use BYOK on any workspace (management/testing).
|
||||
const { effectiveSuperUser } = await verifyEffectiveSuperUser(userId)
|
||||
if (effectiveSuperUser) {
|
||||
return new NextResponse(null, { status: 200 })
|
||||
}
|
||||
|
||||
// Everyone else must be a workspace member on an enterprise plan. The
|
||||
// membership check prevents one org from using another org's stored key.
|
||||
const permission = await getUserEntityPermissions(userId, 'workspace', workspaceId)
|
||||
if (!permission) {
|
||||
logger.warn('BYOK validate denied: user is not a member of the workspace', {
|
||||
workspaceId,
|
||||
userId,
|
||||
})
|
||||
return new NextResponse(null, { status: 403 })
|
||||
}
|
||||
|
||||
const eligible = await isWorkspaceOnEnterprisePlan(workspaceId)
|
||||
if (!eligible) {
|
||||
logger.warn('BYOK validate denied: workspace is not on an enterprise plan', { workspaceId })
|
||||
return new NextResponse(null, { status: 403 })
|
||||
}
|
||||
|
||||
return new NextResponse(null, { status: 200 })
|
||||
} catch (error) {
|
||||
logger.error('BYOK validation failed', { error, workspaceId })
|
||||
return NextResponse.json({ error: 'Failed to validate BYOK entitlement' }, { status: 500 })
|
||||
}
|
||||
})
|
||||
@@ -58,25 +58,19 @@ export const POST = withRouteHandler((request: NextRequest) =>
|
||||
[TraceAttr.UserId]: authenticatedUserId,
|
||||
})
|
||||
|
||||
if (!chatId) {
|
||||
const run = await getLatestRunForStream(streamId, authenticatedUserId).catch((err) => {
|
||||
logger.warn('getLatestRunForStream failed while resolving chatId for abort', {
|
||||
streamId,
|
||||
error: getErrorMessage(err),
|
||||
})
|
||||
return null
|
||||
const run = await getLatestRunForStream(streamId, authenticatedUserId).catch((err) => {
|
||||
logger.warn('getLatestRunForStream failed while resolving abort context', {
|
||||
streamId,
|
||||
error: getErrorMessage(err),
|
||||
})
|
||||
if (run?.chatId) {
|
||||
chatId = run.chatId
|
||||
}
|
||||
return null
|
||||
})
|
||||
if (!chatId && run?.chatId) {
|
||||
chatId = run.chatId
|
||||
}
|
||||
const workspaceId = run?.workspaceId ?? undefined
|
||||
if (chatId) rootSpan.setAttribute(TraceAttr.ChatId, chatId)
|
||||
|
||||
// Local abort before Go — lets the lifecycle classifier see
|
||||
// `signal.aborted` with an explicit-stop reason before Go's
|
||||
// context-canceled error propagates back. Go's endpoint runs
|
||||
// second for billing-ledger flush; Go's context is already
|
||||
// cancelled by then.
|
||||
const aborted = await abortActiveStream(streamId)
|
||||
rootSpan.setAttribute(TraceAttr.CopilotAbortLocalAborted, aborted)
|
||||
|
||||
@@ -101,6 +95,7 @@ export const POST = withRouteHandler((request: NextRequest) =>
|
||||
messageId: streamId,
|
||||
userId: authenticatedUserId,
|
||||
...(chatId ? { chatId } : {}),
|
||||
...(workspaceId ? { workspaceId } : {}),
|
||||
}),
|
||||
spanName: 'sim → go /api/streams/explicit-abort',
|
||||
operation: 'explicit_abort',
|
||||
|
||||
@@ -1,330 +0,0 @@
|
||||
/**
|
||||
* Tests for copilot stats API route
|
||||
*
|
||||
* @vitest-environment node
|
||||
*/
|
||||
import { copilotHttpMock, copilotHttpMockFns, createEnvMock, createMockRequest } from '@sim/testing'
|
||||
import { NextRequest } from 'next/server'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
|
||||
const { mockFetch, mockGetMothershipBaseURL } = vi.hoisted(() => ({
|
||||
mockFetch: vi.fn(),
|
||||
mockGetMothershipBaseURL: vi.fn(),
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/copilot/request/http', () => copilotHttpMock)
|
||||
|
||||
vi.mock('@/lib/copilot/constants', () => ({
|
||||
SIM_AGENT_API_URL_DEFAULT: 'https://agent.sim.example.com',
|
||||
SIM_AGENT_API_URL: 'https://agent.sim.example.com',
|
||||
COPILOT_MODES: ['ask', 'build', 'plan'] as const,
|
||||
COPILOT_REQUEST_MODES: ['ask', 'build', 'plan', 'agent'] as const,
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/copilot/server/agent-url', () => ({
|
||||
getMothershipBaseURL: mockGetMothershipBaseURL,
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/core/config/env', () => createEnvMock({ COPILOT_API_KEY: 'test-api-key' }))
|
||||
|
||||
import { POST } from '@/app/api/copilot/stats/route'
|
||||
|
||||
// `fetchGo` reads `response.status` and `response.headers.get('content-length')`
|
||||
// to stamp span attributes, so mock responses need both fields or the call
|
||||
// path throws before the route handler sees the body.
|
||||
function buildMockResponse(init: {
|
||||
ok: boolean
|
||||
status?: number
|
||||
json: () => Promise<unknown>
|
||||
}): Record<string, unknown> {
|
||||
return {
|
||||
ok: init.ok,
|
||||
status: init.status ?? (init.ok ? 200 : 500),
|
||||
headers: new Headers(),
|
||||
json: init.json,
|
||||
}
|
||||
}
|
||||
|
||||
describe('Copilot Stats API Route', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
mockGetMothershipBaseURL.mockResolvedValue('https://agent.sim.example.com')
|
||||
global.fetch = mockFetch
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks()
|
||||
})
|
||||
|
||||
describe('POST', () => {
|
||||
it('should return 401 when user is not authenticated', async () => {
|
||||
copilotHttpMockFns.mockAuthenticateCopilotRequestSessionOnly.mockResolvedValueOnce({
|
||||
userId: null,
|
||||
isAuthenticated: false,
|
||||
})
|
||||
|
||||
const req = createMockRequest('POST', {
|
||||
messageId: 'message-123',
|
||||
diffCreated: true,
|
||||
diffAccepted: false,
|
||||
})
|
||||
|
||||
const response = await POST(req)
|
||||
|
||||
expect(response.status).toBe(401)
|
||||
const responseData = await response.json()
|
||||
expect(responseData).toEqual({ error: 'Unauthorized' })
|
||||
})
|
||||
|
||||
it('should successfully forward stats to Sim Agent', async () => {
|
||||
copilotHttpMockFns.mockAuthenticateCopilotRequestSessionOnly.mockResolvedValueOnce({
|
||||
userId: 'user-123',
|
||||
isAuthenticated: true,
|
||||
})
|
||||
|
||||
mockFetch.mockResolvedValueOnce(
|
||||
buildMockResponse({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ success: true }),
|
||||
})
|
||||
)
|
||||
|
||||
const req = createMockRequest('POST', {
|
||||
messageId: 'message-123',
|
||||
diffCreated: true,
|
||||
diffAccepted: true,
|
||||
})
|
||||
|
||||
const response = await POST(req)
|
||||
|
||||
expect(response.status).toBe(200)
|
||||
const responseData = await response.json()
|
||||
expect(responseData).toEqual({ success: true })
|
||||
|
||||
expect(mockFetch).toHaveBeenCalledWith(
|
||||
'https://agent.sim.example.com/api/stats',
|
||||
expect.objectContaining({
|
||||
method: 'POST',
|
||||
headers: expect.objectContaining({
|
||||
'Content-Type': 'application/json',
|
||||
'x-api-key': 'test-api-key',
|
||||
}),
|
||||
body: JSON.stringify({
|
||||
messageId: 'message-123',
|
||||
diffCreated: true,
|
||||
diffAccepted: true,
|
||||
}),
|
||||
})
|
||||
)
|
||||
})
|
||||
|
||||
it('should return 400 for invalid request body - missing messageId', async () => {
|
||||
copilotHttpMockFns.mockAuthenticateCopilotRequestSessionOnly.mockResolvedValueOnce({
|
||||
userId: 'user-123',
|
||||
isAuthenticated: true,
|
||||
})
|
||||
|
||||
const req = createMockRequest('POST', {
|
||||
diffCreated: true,
|
||||
diffAccepted: false,
|
||||
})
|
||||
|
||||
const response = await POST(req)
|
||||
|
||||
expect(response.status).toBe(400)
|
||||
const responseData = await response.json()
|
||||
expect(responseData.error).toBe('Invalid request body for copilot stats')
|
||||
})
|
||||
|
||||
it('should return 400 for invalid request body - missing diffCreated', async () => {
|
||||
copilotHttpMockFns.mockAuthenticateCopilotRequestSessionOnly.mockResolvedValueOnce({
|
||||
userId: 'user-123',
|
||||
isAuthenticated: true,
|
||||
})
|
||||
|
||||
const req = createMockRequest('POST', {
|
||||
messageId: 'message-123',
|
||||
diffAccepted: false,
|
||||
})
|
||||
|
||||
const response = await POST(req)
|
||||
|
||||
expect(response.status).toBe(400)
|
||||
const responseData = await response.json()
|
||||
expect(responseData.error).toBe('Invalid request body for copilot stats')
|
||||
})
|
||||
|
||||
it('should return 400 for invalid request body - missing diffAccepted', async () => {
|
||||
copilotHttpMockFns.mockAuthenticateCopilotRequestSessionOnly.mockResolvedValueOnce({
|
||||
userId: 'user-123',
|
||||
isAuthenticated: true,
|
||||
})
|
||||
|
||||
const req = createMockRequest('POST', {
|
||||
messageId: 'message-123',
|
||||
diffCreated: true,
|
||||
})
|
||||
|
||||
const response = await POST(req)
|
||||
|
||||
expect(response.status).toBe(400)
|
||||
const responseData = await response.json()
|
||||
expect(responseData.error).toBe('Invalid request body for copilot stats')
|
||||
})
|
||||
|
||||
it('should return 400 when upstream Sim Agent returns error', async () => {
|
||||
copilotHttpMockFns.mockAuthenticateCopilotRequestSessionOnly.mockResolvedValueOnce({
|
||||
userId: 'user-123',
|
||||
isAuthenticated: true,
|
||||
})
|
||||
|
||||
mockFetch.mockResolvedValueOnce(
|
||||
buildMockResponse({
|
||||
ok: false,
|
||||
json: () => Promise.resolve({ error: 'Invalid message ID' }),
|
||||
})
|
||||
)
|
||||
|
||||
const req = createMockRequest('POST', {
|
||||
messageId: 'invalid-message',
|
||||
diffCreated: true,
|
||||
diffAccepted: false,
|
||||
})
|
||||
|
||||
const response = await POST(req)
|
||||
|
||||
expect(response.status).toBe(400)
|
||||
const responseData = await response.json()
|
||||
expect(responseData).toEqual({ success: false, error: 'Invalid message ID' })
|
||||
})
|
||||
|
||||
it('should handle upstream error with message field', async () => {
|
||||
copilotHttpMockFns.mockAuthenticateCopilotRequestSessionOnly.mockResolvedValueOnce({
|
||||
userId: 'user-123',
|
||||
isAuthenticated: true,
|
||||
})
|
||||
|
||||
mockFetch.mockResolvedValueOnce(
|
||||
buildMockResponse({
|
||||
ok: false,
|
||||
json: () => Promise.resolve({ message: 'Rate limit exceeded' }),
|
||||
})
|
||||
)
|
||||
|
||||
const req = createMockRequest('POST', {
|
||||
messageId: 'message-123',
|
||||
diffCreated: true,
|
||||
diffAccepted: false,
|
||||
})
|
||||
|
||||
const response = await POST(req)
|
||||
|
||||
expect(response.status).toBe(400)
|
||||
const responseData = await response.json()
|
||||
expect(responseData).toEqual({ success: false, error: 'Rate limit exceeded' })
|
||||
})
|
||||
|
||||
it('should handle upstream error with no JSON response', async () => {
|
||||
copilotHttpMockFns.mockAuthenticateCopilotRequestSessionOnly.mockResolvedValueOnce({
|
||||
userId: 'user-123',
|
||||
isAuthenticated: true,
|
||||
})
|
||||
|
||||
mockFetch.mockResolvedValueOnce(
|
||||
buildMockResponse({
|
||||
ok: false,
|
||||
json: () => Promise.reject(new Error('Not JSON')),
|
||||
})
|
||||
)
|
||||
|
||||
const req = createMockRequest('POST', {
|
||||
messageId: 'message-123',
|
||||
diffCreated: true,
|
||||
diffAccepted: false,
|
||||
})
|
||||
|
||||
const response = await POST(req)
|
||||
|
||||
expect(response.status).toBe(400)
|
||||
const responseData = await response.json()
|
||||
expect(responseData).toEqual({ success: false, error: 'Upstream error' })
|
||||
})
|
||||
|
||||
it('should handle network errors gracefully', async () => {
|
||||
copilotHttpMockFns.mockAuthenticateCopilotRequestSessionOnly.mockResolvedValueOnce({
|
||||
userId: 'user-123',
|
||||
isAuthenticated: true,
|
||||
})
|
||||
|
||||
mockFetch.mockRejectedValueOnce(new Error('Network error'))
|
||||
|
||||
const req = createMockRequest('POST', {
|
||||
messageId: 'message-123',
|
||||
diffCreated: true,
|
||||
diffAccepted: false,
|
||||
})
|
||||
|
||||
const response = await POST(req)
|
||||
|
||||
expect(response.status).toBe(500)
|
||||
const responseData = await response.json()
|
||||
expect(responseData.error).toBe('Failed to forward copilot stats')
|
||||
})
|
||||
|
||||
it('should handle JSON parsing errors in request body', async () => {
|
||||
copilotHttpMockFns.mockAuthenticateCopilotRequestSessionOnly.mockResolvedValueOnce({
|
||||
userId: 'user-123',
|
||||
isAuthenticated: true,
|
||||
})
|
||||
|
||||
const req = new NextRequest('http://localhost:3000/api/copilot/stats', {
|
||||
method: 'POST',
|
||||
body: '{invalid-json',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
})
|
||||
|
||||
const response = await POST(req)
|
||||
|
||||
expect(response.status).toBe(400)
|
||||
const responseData = await response.json()
|
||||
expect(responseData.error).toBe('Invalid request body for copilot stats')
|
||||
})
|
||||
|
||||
it('should forward stats with diffCreated=false and diffAccepted=false', async () => {
|
||||
copilotHttpMockFns.mockAuthenticateCopilotRequestSessionOnly.mockResolvedValueOnce({
|
||||
userId: 'user-123',
|
||||
isAuthenticated: true,
|
||||
})
|
||||
|
||||
mockFetch.mockResolvedValueOnce(
|
||||
buildMockResponse({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ success: true }),
|
||||
})
|
||||
)
|
||||
|
||||
const req = createMockRequest('POST', {
|
||||
messageId: 'message-456',
|
||||
diffCreated: false,
|
||||
diffAccepted: false,
|
||||
})
|
||||
|
||||
const response = await POST(req)
|
||||
|
||||
expect(response.status).toBe(200)
|
||||
|
||||
expect(mockFetch).toHaveBeenCalledWith(
|
||||
expect.any(String),
|
||||
expect.objectContaining({
|
||||
body: JSON.stringify({
|
||||
messageId: 'message-456',
|
||||
diffCreated: false,
|
||||
diffAccepted: false,
|
||||
}),
|
||||
})
|
||||
)
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -1,73 +0,0 @@
|
||||
import { type NextRequest, NextResponse } from 'next/server'
|
||||
import { copilotStatsContract } from '@/lib/api/contracts/copilot'
|
||||
import { parseRequest, validationErrorResponse } from '@/lib/api/server'
|
||||
import { fetchGo } from '@/lib/copilot/request/go/fetch'
|
||||
import {
|
||||
authenticateCopilotRequestSessionOnly,
|
||||
createInternalServerErrorResponse,
|
||||
createRequestTracker,
|
||||
createUnauthorizedResponse,
|
||||
} from '@/lib/copilot/request/http'
|
||||
import { getMothershipBaseURL } from '@/lib/copilot/server/agent-url'
|
||||
import { env } from '@/lib/core/config/env'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
|
||||
export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
const tracker = createRequestTracker()
|
||||
try {
|
||||
const { userId, isAuthenticated } = await authenticateCopilotRequestSessionOnly()
|
||||
if (!isAuthenticated || !userId) {
|
||||
return createUnauthorizedResponse()
|
||||
}
|
||||
|
||||
const parsed = await parseRequest(
|
||||
copilotStatsContract,
|
||||
req,
|
||||
{},
|
||||
{
|
||||
validationErrorResponse: (error) =>
|
||||
validationErrorResponse(error, 'Invalid request body for copilot stats'),
|
||||
invalidJsonResponse: () =>
|
||||
NextResponse.json(
|
||||
{ error: 'Invalid request body for copilot stats', details: [] },
|
||||
{ status: 400 }
|
||||
),
|
||||
}
|
||||
)
|
||||
if (!parsed.success) return parsed.response
|
||||
|
||||
const { messageId, diffCreated, diffAccepted } = parsed.data.body
|
||||
|
||||
const payload: Record<string, any> = {
|
||||
messageId,
|
||||
diffCreated,
|
||||
diffAccepted,
|
||||
}
|
||||
|
||||
const mothershipBaseURL = await getMothershipBaseURL({ userId })
|
||||
const agentRes = await fetchGo(`${mothershipBaseURL}/api/stats`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...(env.COPILOT_API_KEY ? { 'x-api-key': env.COPILOT_API_KEY } : {}),
|
||||
},
|
||||
body: JSON.stringify(payload),
|
||||
spanName: 'sim → go /api/stats',
|
||||
operation: 'stats_ingest',
|
||||
})
|
||||
|
||||
let agentJson: any = null
|
||||
try {
|
||||
agentJson = await agentRes.json()
|
||||
} catch {}
|
||||
|
||||
if (!agentRes.ok) {
|
||||
const message = (agentJson && (agentJson.error || agentJson.message)) || 'Upstream error'
|
||||
return NextResponse.json({ success: false, error: message }, { status: 400 })
|
||||
}
|
||||
|
||||
return NextResponse.json({ success: true })
|
||||
} catch (error) {
|
||||
return createInternalServerErrorResponse('Failed to forward copilot stats')
|
||||
}
|
||||
})
|
||||
@@ -5,6 +5,12 @@ import type { NextRequest } from 'next/server'
|
||||
import { NextResponse } from 'next/server'
|
||||
import { fileServeParamsSchema, fileServeQuerySchema } from '@/lib/api/contracts/storage-transfer'
|
||||
import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
|
||||
import {
|
||||
DocCompileUserError,
|
||||
getE2BDocFormat,
|
||||
loadCompiledDocByExt,
|
||||
} from '@/lib/copilot/tools/server/files/doc-compile'
|
||||
import { isE2BDocEnabled } from '@/lib/core/config/feature-flags'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import { runSandboxTask } from '@/lib/execution/sandbox/run-task'
|
||||
import { CopilotFiles, isUsingCloudStorage } from '@/lib/uploads'
|
||||
@@ -72,14 +78,55 @@ async function compileDocumentIfNeeded(
|
||||
if (raw) return { buffer, contentType: getContentType(filename) }
|
||||
|
||||
const ext = filename.slice(filename.lastIndexOf('.')).toLowerCase()
|
||||
const extNoDot = ext.replace(/^\./, '')
|
||||
const format = COMPILABLE_FORMATS[ext]
|
||||
if (!format) return { buffer, contentType: getContentType(filename) }
|
||||
|
||||
const magicLen = format.magic.length
|
||||
if (buffer.length >= magicLen && buffer.subarray(0, magicLen).equals(format.magic)) {
|
||||
// Already a binary file (uploaded or pre-compiled)? Serve as-is.
|
||||
if (format) {
|
||||
const magicLen = format.magic.length
|
||||
if (buffer.length >= magicLen && buffer.subarray(0, magicLen).equals(format.magic)) {
|
||||
return { buffer, contentType: getContentType(filename) }
|
||||
}
|
||||
}
|
||||
|
||||
// .xlsx is a ZIP container with no JS compile path. An uploaded/binary xlsx
|
||||
// must short-circuit here (it isn't in COMPILABLE_FORMATS) — otherwise every
|
||||
// xlsx open would utf-8-decode the whole binary and do an always-miss S3 GET.
|
||||
// Only a Python-source xlsx (UTF-8 text, no ZIP magic) falls through.
|
||||
if (
|
||||
extNoDot === 'xlsx' &&
|
||||
buffer.length >= ZIP_MAGIC.length &&
|
||||
buffer.subarray(0, ZIP_MAGIC.length).equals(ZIP_MAGIC)
|
||||
) {
|
||||
return { buffer, contentType: getContentType(filename) }
|
||||
}
|
||||
|
||||
// Generated docs render from a content-addressed compiled binary that is built
|
||||
// exactly ONCE per edit_content/create (at write time) and stored in S3. Serve
|
||||
// only LOADS it — it must never compile, or it would re-run E2B on every preview
|
||||
// fetch, including against the incomplete source mid-generation. A hit returns
|
||||
// the (possibly partial) committed doc; a miss in the E2B regime means the doc
|
||||
// is still being generated → 409, and the client polls until the artifact lands.
|
||||
if (workspaceId && (format || extNoDot === 'xlsx')) {
|
||||
const source = buffer.toString('utf-8')
|
||||
// Load the prebuilt artifact directly from S3 (content-addressed). No extra
|
||||
// in-memory layer here: the store is the source of truth, the client (react
|
||||
// query) already caches the bytes, and this branch never recomputes.
|
||||
const stored = await loadCompiledDocByExt(workspaceId, source, extNoDot)
|
||||
if (stored) {
|
||||
return { buffer: stored.buffer, contentType: stored.contentType }
|
||||
}
|
||||
|
||||
if (isE2BDocEnabled && getE2BDocFormat(filename)) {
|
||||
// Artifact not built yet (still generating, or the source didn't compile at
|
||||
// write time). Signal "not ready" without compiling — handled as 409.
|
||||
throw new DocCompileUserError('Document is still being generated')
|
||||
}
|
||||
}
|
||||
|
||||
if (!format) return { buffer, contentType: getContentType(filename) }
|
||||
|
||||
// E2B disabled and no stored artifact → compile JS source via isolated-vm.
|
||||
const code = buffer.toString('utf-8')
|
||||
const cacheKey = sha256Hex(`${ext}${code}${workspaceId ?? ''}`)
|
||||
const cached = compiledDocCache.get(cacheKey)
|
||||
@@ -106,6 +153,24 @@ function getWorkspaceIdForCompile(key: string): string | undefined {
|
||||
return parseWorkspaceFileKey(key) ?? undefined
|
||||
}
|
||||
|
||||
const IMMUTABLE_CACHE_CONTROL = 'private, max-age=31536000, immutable'
|
||||
const WORKSPACE_REVALIDATE_CACHE_CONTROL = 'private, no-cache, must-revalidate'
|
||||
|
||||
/**
|
||||
* Cache-Control for a served file. A versioned request (`?v=<updatedAt>`) addresses
|
||||
* content-immutable bytes — generated docs are content-addressed and the version
|
||||
* bumps on every edit — so the browser may cache it indefinitely; re-opens and
|
||||
* focus refetches then resolve from cache with no round trip. Unversioned workspace
|
||||
* reads stay revalidated because the same storage key is edited in place.
|
||||
*/
|
||||
function resolveServeCacheControl(
|
||||
versioned: boolean,
|
||||
context: string | undefined
|
||||
): string | undefined {
|
||||
if (versioned) return IMMUTABLE_CACHE_CONTROL
|
||||
return context === 'workspace' ? WORKSPACE_REVALIDATE_CACHE_CONTROL : undefined
|
||||
}
|
||||
|
||||
export const GET = withRouteHandler(
|
||||
async (request: NextRequest, { params }: { params: Promise<{ path: string[] }> }) => {
|
||||
try {
|
||||
@@ -143,8 +208,10 @@ export const GET = withRouteHandler(
|
||||
|
||||
const query = fileServeQuerySchema.parse({
|
||||
raw: request.nextUrl.searchParams.get('raw'),
|
||||
v: request.nextUrl.searchParams.get('v'),
|
||||
})
|
||||
const raw = query.raw === '1'
|
||||
const versioned = query.v != null
|
||||
|
||||
const authResult = await checkSessionOrInternalAuth(request, { requireWorkflowId: false })
|
||||
|
||||
@@ -159,11 +226,22 @@ export const GET = withRouteHandler(
|
||||
const userId = authResult.userId
|
||||
|
||||
if (isUsingCloudStorage()) {
|
||||
return await handleCloudProxy(cloudKey, userId, raw, request.signal)
|
||||
return await handleCloudProxy(cloudKey, userId, raw, versioned, request.signal)
|
||||
}
|
||||
|
||||
return await handleLocalFile(cloudKey, userId, raw, request.signal)
|
||||
return await handleLocalFile(cloudKey, userId, raw, versioned, request.signal)
|
||||
} catch (error) {
|
||||
// An in-progress/incomplete doc source fails to compile — this is expected
|
||||
// mid-generation, not a server fault. Return 409 (not 500) so it isn't an
|
||||
// alarming error; the client re-fetches once the doc finishes (the serve
|
||||
// URL is busted on the file's updatedAt).
|
||||
if (error instanceof DocCompileUserError) {
|
||||
logger.info('Serve: document still compiling, returning 409', {
|
||||
message: error.message,
|
||||
})
|
||||
return NextResponse.json({ error: 'Document is still being generated' }, { status: 409 })
|
||||
}
|
||||
|
||||
logger.error('Error serving file:', error)
|
||||
|
||||
if (error instanceof FileNotFoundError) {
|
||||
@@ -179,6 +257,7 @@ async function handleLocalFile(
|
||||
filename: string,
|
||||
userId: string,
|
||||
raw: boolean,
|
||||
versioned: boolean,
|
||||
signal: AbortSignal | undefined
|
||||
): Promise<NextResponse> {
|
||||
const ownerKey = `user:${userId}`
|
||||
@@ -225,7 +304,7 @@ async function handleLocalFile(
|
||||
buffer: fileBuffer,
|
||||
contentType,
|
||||
filename: displayName,
|
||||
cacheControl: contextParam === 'workspace' ? 'private, no-cache, must-revalidate' : undefined,
|
||||
cacheControl: resolveServeCacheControl(versioned, contextParam),
|
||||
})
|
||||
} catch (error) {
|
||||
logger.error('Error reading local file:', error)
|
||||
@@ -237,6 +316,7 @@ async function handleCloudProxy(
|
||||
cloudKey: string,
|
||||
userId: string,
|
||||
raw = false,
|
||||
versioned = false,
|
||||
signal: AbortSignal | undefined = undefined
|
||||
): Promise<NextResponse> {
|
||||
const ownerKey = `user:${userId}`
|
||||
@@ -291,7 +371,7 @@ async function handleCloudProxy(
|
||||
buffer: fileBuffer,
|
||||
contentType,
|
||||
filename: displayName,
|
||||
cacheControl: context === 'workspace' ? 'private, no-cache, must-revalidate' : undefined,
|
||||
cacheControl: resolveServeCacheControl(versioned, context),
|
||||
})
|
||||
} catch (error) {
|
||||
logger.error('Error downloading from cloud storage:', error)
|
||||
|
||||
@@ -12,10 +12,22 @@ import {
|
||||
import { NextRequest } from 'next/server'
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
|
||||
const { mockExecuteInE2B, mockExecuteInIsolatedVM, mockUploadFile } = vi.hoisted(() => ({
|
||||
const {
|
||||
mockExecuteInE2B,
|
||||
mockExecuteInIsolatedVM,
|
||||
mockGetWorkspaceFile,
|
||||
mockUpdateWorkspaceFileContent,
|
||||
mockUploadFile,
|
||||
mockValidateWorkspaceFileWriteTarget,
|
||||
mockWriteWorkspaceFileByPath,
|
||||
} = vi.hoisted(() => ({
|
||||
mockExecuteInE2B: vi.fn(),
|
||||
mockExecuteInIsolatedVM: vi.fn(),
|
||||
mockGetWorkspaceFile: vi.fn(),
|
||||
mockUpdateWorkspaceFileContent: vi.fn(),
|
||||
mockUploadFile: vi.fn(),
|
||||
mockValidateWorkspaceFileWriteTarget: vi.fn(),
|
||||
mockWriteWorkspaceFileByPath: vi.fn(),
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/execution/isolated-vm', () => ({
|
||||
@@ -37,9 +49,40 @@ vi.mock('@/lib/copilot/request/tools/files', () => ({
|
||||
},
|
||||
normalizeOutputWorkspaceFileName: vi.fn((p: string) => p.replace(/^files\//, '')),
|
||||
resolveOutputFormat: vi.fn(() => 'json'),
|
||||
getOutputFileDeclarations: vi.fn((params: Record<string, any>) => {
|
||||
if (Array.isArray(params.outputs?.files)) {
|
||||
return params.outputs.files.map((file: Record<string, any>) => ({
|
||||
path: file.path,
|
||||
mode: file.mode === 'overwrite' ? 'overwrite' : 'create',
|
||||
sandboxPath: file.sandboxPath,
|
||||
mimeType: file.mimeType,
|
||||
format: file.format,
|
||||
}))
|
||||
}
|
||||
return params.outputPath
|
||||
? [
|
||||
{
|
||||
path: params.overwriteFileId || params.outputPath,
|
||||
mode: params.overwriteFileId ? 'overwrite' : 'create',
|
||||
sandboxPath: params.outputSandboxPath,
|
||||
mimeType: params.outputMimeType,
|
||||
format: params.outputFormat,
|
||||
formatPath: params.outputPath,
|
||||
overwriteFileId: params.overwriteFileId,
|
||||
},
|
||||
]
|
||||
: []
|
||||
}),
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/copilot/vfs/resource-writer', () => ({
|
||||
validateWorkspaceFileWriteTarget: mockValidateWorkspaceFileWriteTarget,
|
||||
writeWorkspaceFileByPath: mockWriteWorkspaceFileByPath,
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/uploads/contexts/workspace/workspace-file-manager', () => ({
|
||||
getWorkspaceFile: mockGetWorkspaceFile,
|
||||
updateWorkspaceFileContent: mockUpdateWorkspaceFileContent,
|
||||
uploadWorkspaceFile: vi.fn(),
|
||||
}))
|
||||
|
||||
@@ -79,6 +122,35 @@ describe('Function Execute API Route', () => {
|
||||
stdout: 'e2b output',
|
||||
sandboxId: 'test-sandbox-id',
|
||||
})
|
||||
mockGetWorkspaceFile.mockResolvedValue({
|
||||
id: 'wf_existing',
|
||||
name: 'existing.png',
|
||||
size: 10,
|
||||
type: 'image/png',
|
||||
url: '/api/files/view/existing',
|
||||
key: 'workspace/existing.png',
|
||||
})
|
||||
mockUpdateWorkspaceFileContent.mockResolvedValue({
|
||||
id: 'wf_existing',
|
||||
name: 'existing.png',
|
||||
size: 20,
|
||||
type: 'image/png',
|
||||
url: '/api/files/view/existing',
|
||||
key: 'workspace/existing.png',
|
||||
})
|
||||
mockValidateWorkspaceFileWriteTarget.mockImplementation(async ({ target }) => ({
|
||||
mode: target.mode,
|
||||
vfsPath: target.path,
|
||||
}))
|
||||
mockWriteWorkspaceFileByPath.mockImplementation(async ({ target, buffer }) => ({
|
||||
id: `wf_${String(target.path).split('/').pop()?.replace(/\W+/g, '_') || 'file'}`,
|
||||
name: String(target.path).split('/').pop() || 'file',
|
||||
vfsPath: target.path,
|
||||
downloadUrl: `/api/files/view/${encodeURIComponent(target.path)}`,
|
||||
mode: target.mode,
|
||||
size: buffer.length,
|
||||
contentType: target.mimeType || 'application/octet-stream',
|
||||
}))
|
||||
})
|
||||
|
||||
describe('Security Tests', () => {
|
||||
@@ -268,6 +340,196 @@ describe('Function Execute API Route', () => {
|
||||
expect(isLargeValueRef(data.output.result.text)).toBe(true)
|
||||
})
|
||||
|
||||
it('exports multiple declared sandbox output files', async () => {
|
||||
featureFlagsMock.isE2bEnabled = true
|
||||
mockExecuteInE2B.mockResolvedValueOnce({
|
||||
result: 'done',
|
||||
stdout: 'ok',
|
||||
sandboxId: 'sandbox-123',
|
||||
exportedFiles: {
|
||||
'/home/user/chart.png': 'iVBORw0KGgo=',
|
||||
'/home/user/summary.json': '{"ok":true}',
|
||||
},
|
||||
})
|
||||
|
||||
const req = createMockRequest('POST', {
|
||||
code: 'print("done")',
|
||||
language: 'python',
|
||||
workspaceId: 'workspace-1',
|
||||
outputs: {
|
||||
files: [
|
||||
{
|
||||
path: 'files/reports/chart.png',
|
||||
mode: 'create',
|
||||
sandboxPath: '/home/user/chart.png',
|
||||
mimeType: 'image/png',
|
||||
},
|
||||
{
|
||||
path: 'files/reports/summary.json',
|
||||
mode: 'overwrite',
|
||||
sandboxPath: '/home/user/summary.json',
|
||||
mimeType: 'application/json',
|
||||
},
|
||||
],
|
||||
},
|
||||
})
|
||||
|
||||
const response = await POST(req)
|
||||
const data = await response.json()
|
||||
|
||||
expect(response.status).toBe(200)
|
||||
expect(data.success).toBe(true)
|
||||
expect(mockExecuteInE2B).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
outputSandboxPaths: ['/home/user/chart.png', '/home/user/summary.json'],
|
||||
})
|
||||
)
|
||||
expect(mockValidateWorkspaceFileWriteTarget).toHaveBeenCalledTimes(2)
|
||||
expect(mockWriteWorkspaceFileByPath).toHaveBeenCalledTimes(2)
|
||||
expect(mockWriteWorkspaceFileByPath).toHaveBeenNthCalledWith(
|
||||
1,
|
||||
expect.objectContaining({
|
||||
target: expect.objectContaining({ path: 'files/reports/chart.png', mode: 'create' }),
|
||||
})
|
||||
)
|
||||
expect(mockWriteWorkspaceFileByPath).toHaveBeenNthCalledWith(
|
||||
2,
|
||||
expect.objectContaining({
|
||||
target: expect.objectContaining({
|
||||
path: 'files/reports/summary.json',
|
||||
mode: 'overwrite',
|
||||
}),
|
||||
})
|
||||
)
|
||||
expect(data.output.result.files).toHaveLength(2)
|
||||
expect(data.resources).toEqual([
|
||||
expect.objectContaining({ path: 'files/reports/chart.png' }),
|
||||
expect.objectContaining({ path: 'files/reports/summary.json' }),
|
||||
])
|
||||
})
|
||||
|
||||
it('prevalidates all sandbox output destinations before writing any files', async () => {
|
||||
featureFlagsMock.isE2bEnabled = true
|
||||
mockExecuteInE2B.mockResolvedValueOnce({
|
||||
result: 'done',
|
||||
stdout: 'ok',
|
||||
sandboxId: 'sandbox-123',
|
||||
exportedFiles: {
|
||||
'/home/user/first.json': '{"first":true}',
|
||||
'/home/user/second.json': '{"second":true}',
|
||||
},
|
||||
})
|
||||
mockValidateWorkspaceFileWriteTarget
|
||||
.mockResolvedValueOnce({ mode: 'create', vfsPath: 'files/first.json' })
|
||||
.mockRejectedValueOnce(new Error('Directory not yet created: files/missing'))
|
||||
|
||||
const req = createMockRequest('POST', {
|
||||
code: 'print("done")',
|
||||
language: 'python',
|
||||
workspaceId: 'workspace-1',
|
||||
outputs: {
|
||||
files: [
|
||||
{
|
||||
path: 'files/first.json',
|
||||
mode: 'create',
|
||||
sandboxPath: '/home/user/first.json',
|
||||
},
|
||||
{
|
||||
path: 'files/missing/second.json',
|
||||
mode: 'create',
|
||||
sandboxPath: '/home/user/second.json',
|
||||
},
|
||||
],
|
||||
},
|
||||
})
|
||||
|
||||
const response = await POST(req)
|
||||
const data = await response.json()
|
||||
|
||||
expect(response.status).toBe(400)
|
||||
expect(data.success).toBe(false)
|
||||
expect(data.error).toContain('Directory not yet created')
|
||||
expect(mockWriteWorkspaceFileByPath).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('rejects duplicate sandbox output destinations before writing files', async () => {
|
||||
featureFlagsMock.isE2bEnabled = true
|
||||
mockExecuteInE2B.mockResolvedValueOnce({
|
||||
result: 'done',
|
||||
stdout: 'ok',
|
||||
sandboxId: 'sandbox-123',
|
||||
exportedFiles: {
|
||||
'/home/user/first.json': '{"first":true}',
|
||||
'/home/user/second.json': '{"second":true}',
|
||||
},
|
||||
})
|
||||
mockValidateWorkspaceFileWriteTarget.mockResolvedValue({
|
||||
mode: 'create',
|
||||
vfsPath: 'files/dupe.json',
|
||||
})
|
||||
|
||||
const req = createMockRequest('POST', {
|
||||
code: 'print("done")',
|
||||
language: 'python',
|
||||
workspaceId: 'workspace-1',
|
||||
outputs: {
|
||||
files: [
|
||||
{
|
||||
path: 'files/dupe.json',
|
||||
mode: 'create',
|
||||
sandboxPath: '/home/user/first.json',
|
||||
},
|
||||
{
|
||||
path: 'files/dupe.json',
|
||||
mode: 'create',
|
||||
sandboxPath: '/home/user/second.json',
|
||||
},
|
||||
],
|
||||
},
|
||||
})
|
||||
|
||||
const response = await POST(req)
|
||||
const data = await response.json()
|
||||
|
||||
expect(response.status).toBe(400)
|
||||
expect(data.success).toBe(false)
|
||||
expect(data.error).toContain('Duplicate sandbox output destination')
|
||||
expect(mockWriteWorkspaceFileByPath).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('returns a targeted error when a declared sandbox output is missing', async () => {
|
||||
featureFlagsMock.isE2bEnabled = true
|
||||
mockExecuteInE2B.mockResolvedValueOnce({
|
||||
result: 'done',
|
||||
stdout: 'ok',
|
||||
sandboxId: 'sandbox-123',
|
||||
exportedFiles: {},
|
||||
})
|
||||
|
||||
const req = createMockRequest('POST', {
|
||||
code: 'print("done")',
|
||||
language: 'python',
|
||||
workspaceId: 'workspace-1',
|
||||
outputs: {
|
||||
files: [
|
||||
{
|
||||
path: 'files/missing.json',
|
||||
mode: 'create',
|
||||
sandboxPath: '/home/user/missing.json',
|
||||
},
|
||||
],
|
||||
},
|
||||
})
|
||||
|
||||
const response = await POST(req)
|
||||
const data = await response.json()
|
||||
|
||||
expect(response.status).toBe(500)
|
||||
expect(data.success).toBe(false)
|
||||
expect(data.error).toContain('Sandbox file "/home/user/missing.json" was not found')
|
||||
expect(mockWriteWorkspaceFileByPath).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('should return computed result for multi-line code', async () => {
|
||||
mockExecuteInIsolatedVM.mockResolvedValueOnce({ result: 10, stdout: '' })
|
||||
|
||||
|
||||
@@ -5,9 +5,15 @@ import { parseRequest } from '@/lib/api/server'
|
||||
import { checkInternalAuth } from '@/lib/auth/hybrid'
|
||||
import {
|
||||
FORMAT_TO_CONTENT_TYPE,
|
||||
getOutputFileDeclarations,
|
||||
normalizeOutputWorkspaceFileName,
|
||||
type OutputFileDeclaration,
|
||||
resolveOutputFormat,
|
||||
} from '@/lib/copilot/request/tools/files'
|
||||
import {
|
||||
validateWorkspaceFileWriteTarget,
|
||||
writeWorkspaceFileByPath,
|
||||
} from '@/lib/copilot/vfs/resource-writer'
|
||||
import { isE2bEnabled } from '@/lib/core/config/feature-flags'
|
||||
import { generateRequestId } from '@/lib/core/utils/request'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
@@ -29,7 +35,6 @@ import {
|
||||
import { compactExecutionPayload } from '@/lib/execution/payloads/serializer'
|
||||
import { materializeLargeValueRef } from '@/lib/execution/payloads/store'
|
||||
import { isExecutionResourceLimitError } from '@/lib/execution/resource-errors'
|
||||
import { uploadWorkspaceFile } from '@/lib/uploads/contexts/workspace/workspace-file-manager'
|
||||
import { getWorkflowById } from '@/lib/workflows/utils'
|
||||
import { escapeRegExp, normalizeName, REFERENCE } from '@/executor/constants'
|
||||
import { type OutputSchema, resolveBlockReference } from '@/executor/utils/block-reference'
|
||||
@@ -48,6 +53,8 @@ const TAG_PATTERN = createReferencePattern()
|
||||
|
||||
const E2B_JS_WRAPPER_LINES = 3
|
||||
const E2B_PYTHON_WRAPPER_LINES = 1
|
||||
const MAX_SANDBOX_OUTPUT_FILES = 20
|
||||
const MAX_SANDBOX_OUTPUT_BYTES = 50 * 1024 * 1024
|
||||
|
||||
/** Matches valid JS identifier names (letters, digits, underscore; no leading digit). */
|
||||
const SAFE_IDENTIFIER = /^[a-zA-Z_][a-zA-Z0-9_]*$/
|
||||
@@ -672,6 +679,37 @@ function resolveCodeVariables(
|
||||
* This handles the common case where print() or console.log() adds a trailing \n
|
||||
* that users don't expect to see in the output
|
||||
*/
|
||||
/**
|
||||
* Heuristic: did the sandbox die from an infrastructure failure (OOM kill,
|
||||
* timeout, lost connection) rather than a normal code error? Python/JS code
|
||||
* exceptions surface via execution.error; an OOM kill instead makes runCode
|
||||
* throw, often with an empty or cryptic message.
|
||||
*/
|
||||
function isLikelySandboxKill(error: any): boolean {
|
||||
const msg = `${error?.name ?? ''} ${error?.message ?? ''} ${error?.code ?? ''}`
|
||||
.toLowerCase()
|
||||
.trim()
|
||||
if (!msg) return true
|
||||
return [
|
||||
'out of memory',
|
||||
'oom',
|
||||
'killed',
|
||||
'sigkill',
|
||||
'code 137',
|
||||
'signal 9',
|
||||
'terminated',
|
||||
'econnreset',
|
||||
'epipe',
|
||||
'socket hang up',
|
||||
'connection closed',
|
||||
'connection reset',
|
||||
'websocket',
|
||||
'timed out',
|
||||
'timeout',
|
||||
'deadline',
|
||||
].some((s) => msg.includes(s))
|
||||
}
|
||||
|
||||
function cleanStdout(stdout: string): string {
|
||||
if (stdout.endsWith('\n')) {
|
||||
return stdout.slice(0, -1)
|
||||
@@ -865,6 +903,8 @@ async function maybeExportSandboxFileToWorkspace(args: {
|
||||
outputFormat?: string
|
||||
outputMimeType?: string
|
||||
outputSandboxPath?: string
|
||||
overwriteFileId?: string
|
||||
outputMode?: 'create' | 'overwrite'
|
||||
exportedFileContent?: string
|
||||
stdout: string
|
||||
executionTime: number
|
||||
@@ -877,6 +917,8 @@ async function maybeExportSandboxFileToWorkspace(args: {
|
||||
outputFormat,
|
||||
outputMimeType,
|
||||
outputSandboxPath,
|
||||
overwriteFileId,
|
||||
outputMode,
|
||||
exportedFileContent,
|
||||
stdout,
|
||||
executionTime,
|
||||
@@ -933,28 +975,282 @@ async function maybeExportSandboxFileToWorkspace(args: {
|
||||
? Buffer.from(exportedFileContent, 'base64')
|
||||
: Buffer.from(exportedFileContent, 'utf-8')
|
||||
|
||||
const uploaded = await uploadWorkspaceFile(
|
||||
resolvedWorkspaceId,
|
||||
authUserId,
|
||||
fileBuffer,
|
||||
fileName,
|
||||
resolvedMimeType
|
||||
const targetPath = overwriteFileId || outputPath
|
||||
const mode = outputMode ?? (overwriteFileId ? 'overwrite' : 'create')
|
||||
|
||||
try {
|
||||
const written = await writeWorkspaceFileByPath({
|
||||
workspaceId: resolvedWorkspaceId,
|
||||
userId: authUserId,
|
||||
target: {
|
||||
path: targetPath,
|
||||
mode,
|
||||
mimeType: outputMimeType,
|
||||
},
|
||||
buffer: fileBuffer,
|
||||
inferredMimeType: resolvedMimeType,
|
||||
})
|
||||
logger.info('Sandbox file exported to workspace', {
|
||||
fileId: written.id,
|
||||
vfsPath: written.vfsPath,
|
||||
sandboxPath: outputSandboxPath,
|
||||
mode,
|
||||
mimeType: resolvedMimeType,
|
||||
size: fileBuffer.length,
|
||||
})
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
output: {
|
||||
result: {
|
||||
message: `Sandbox file exported to ${written.vfsPath}`,
|
||||
fileId: written.id,
|
||||
fileName: written.name,
|
||||
vfsPath: written.vfsPath,
|
||||
downloadUrl: written.downloadUrl,
|
||||
sandboxPath: outputSandboxPath,
|
||||
},
|
||||
stdout: cleanStdout(stdout),
|
||||
executionTime,
|
||||
},
|
||||
resources: [{ type: 'file', id: written.id, title: written.name, path: written.vfsPath }],
|
||||
})
|
||||
} catch (error) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
success: false,
|
||||
error: error instanceof Error ? error.message : 'Failed to export sandbox file',
|
||||
output: { result: null, stdout: cleanStdout(stdout), executionTime },
|
||||
},
|
||||
{ status: 400 }
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
async function maybeExportSandboxFilesToWorkspace(args: {
|
||||
authUserId: string
|
||||
workflowId?: string
|
||||
workspaceId?: string
|
||||
outputFiles: OutputFileDeclaration[]
|
||||
exportedFiles?: Record<string, string>
|
||||
exportedFileContent?: string
|
||||
stdout: string
|
||||
executionTime: number
|
||||
}) {
|
||||
const sandboxFiles = args.outputFiles.filter((file) => file.sandboxPath)
|
||||
if (sandboxFiles.length === 0) return null
|
||||
if (sandboxFiles.length > MAX_SANDBOX_OUTPUT_FILES) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
success: false,
|
||||
error: `Too many sandbox output files requested (${sandboxFiles.length}). Maximum is ${MAX_SANDBOX_OUTPUT_FILES}.`,
|
||||
output: {
|
||||
result: null,
|
||||
stdout: cleanStdout(args.stdout),
|
||||
executionTime: args.executionTime,
|
||||
},
|
||||
},
|
||||
{ status: 400 }
|
||||
)
|
||||
}
|
||||
|
||||
if (sandboxFiles.length === 1) {
|
||||
const file = sandboxFiles[0]
|
||||
return maybeExportSandboxFileToWorkspace({
|
||||
authUserId: args.authUserId,
|
||||
workflowId: args.workflowId,
|
||||
workspaceId: args.workspaceId,
|
||||
outputPath: file.formatPath ?? file.path,
|
||||
outputFormat: file.format,
|
||||
outputMimeType: file.mimeType,
|
||||
outputSandboxPath: file.sandboxPath,
|
||||
outputMode: file.mode,
|
||||
exportedFileContent:
|
||||
(file.sandboxPath ? args.exportedFiles?.[file.sandboxPath] : undefined) ??
|
||||
args.exportedFileContent,
|
||||
stdout: args.stdout,
|
||||
executionTime: args.executionTime,
|
||||
})
|
||||
}
|
||||
|
||||
const resolvedWorkspaceId =
|
||||
args.workspaceId ||
|
||||
(args.workflowId ? (await getWorkflowById(args.workflowId))?.workspaceId : undefined)
|
||||
if (!resolvedWorkspaceId) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
success: false,
|
||||
error: 'Workspace context required to save sandbox files to workspace',
|
||||
output: {
|
||||
result: null,
|
||||
stdout: cleanStdout(args.stdout),
|
||||
executionTime: args.executionTime,
|
||||
},
|
||||
},
|
||||
{ status: 400 }
|
||||
)
|
||||
}
|
||||
|
||||
const preparedFiles = []
|
||||
let totalOutputBytes = 0
|
||||
for (const file of sandboxFiles) {
|
||||
const sandboxPath = file.sandboxPath!
|
||||
const content = args.exportedFiles?.[sandboxPath]
|
||||
if (content === undefined) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
success: false,
|
||||
error: `Sandbox file "${sandboxPath}" was not found or could not be read`,
|
||||
output: {
|
||||
result: null,
|
||||
stdout: cleanStdout(args.stdout),
|
||||
executionTime: args.executionTime,
|
||||
},
|
||||
},
|
||||
{ status: 500 }
|
||||
)
|
||||
}
|
||||
const fileName = normalizeOutputWorkspaceFileName(file.formatPath ?? file.path)
|
||||
const resolvedMimeType =
|
||||
file.mimeType ||
|
||||
FORMAT_TO_CONTENT_TYPE[resolveOutputFormat(fileName, file.format)] ||
|
||||
'application/octet-stream'
|
||||
const isBinary = !new Set(Object.values(FORMAT_TO_CONTENT_TYPE)).has(resolvedMimeType)
|
||||
const size = Buffer.byteLength(content, isBinary ? 'base64' : 'utf-8')
|
||||
totalOutputBytes += size
|
||||
if (totalOutputBytes > MAX_SANDBOX_OUTPUT_BYTES) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
success: false,
|
||||
error: `Sandbox output files exceed ${MAX_SANDBOX_OUTPUT_BYTES} bytes total`,
|
||||
output: {
|
||||
result: null,
|
||||
stdout: cleanStdout(args.stdout),
|
||||
executionTime: args.executionTime,
|
||||
},
|
||||
},
|
||||
{ status: 400 }
|
||||
)
|
||||
}
|
||||
preparedFiles.push({
|
||||
file,
|
||||
sandboxPath,
|
||||
content,
|
||||
resolvedMimeType,
|
||||
isBinary,
|
||||
size,
|
||||
target: {
|
||||
path: file.path,
|
||||
mode: file.mode ?? 'create',
|
||||
mimeType: file.mimeType,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
let validationPaths: string[]
|
||||
try {
|
||||
const validations = await Promise.all(
|
||||
preparedFiles.map((prepared) =>
|
||||
validateWorkspaceFileWriteTarget({
|
||||
workspaceId: resolvedWorkspaceId,
|
||||
userId: args.authUserId,
|
||||
target: prepared.target,
|
||||
})
|
||||
)
|
||||
)
|
||||
validationPaths = validations.map((validation) => validation.vfsPath)
|
||||
} catch (error) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
success: false,
|
||||
error: error instanceof Error ? error.message : 'Invalid sandbox output destination',
|
||||
output: {
|
||||
result: null,
|
||||
stdout: cleanStdout(args.stdout),
|
||||
executionTime: args.executionTime,
|
||||
},
|
||||
},
|
||||
{ status: 400 }
|
||||
)
|
||||
}
|
||||
const duplicateDestination = validationPaths.find(
|
||||
(vfsPath, index) => validationPaths.indexOf(vfsPath) !== index
|
||||
)
|
||||
if (duplicateDestination) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
success: false,
|
||||
error: `Duplicate sandbox output destination: ${duplicateDestination}`,
|
||||
output: {
|
||||
result: null,
|
||||
stdout: cleanStdout(args.stdout),
|
||||
executionTime: args.executionTime,
|
||||
},
|
||||
},
|
||||
{ status: 400 }
|
||||
)
|
||||
}
|
||||
|
||||
const writtenFiles = []
|
||||
try {
|
||||
for (const prepared of preparedFiles) {
|
||||
const buffer = prepared.isBinary
|
||||
? Buffer.from(prepared.content, 'base64')
|
||||
: Buffer.from(prepared.content, 'utf-8')
|
||||
const written = await writeWorkspaceFileByPath({
|
||||
workspaceId: resolvedWorkspaceId,
|
||||
userId: args.authUserId,
|
||||
target: prepared.target,
|
||||
buffer,
|
||||
inferredMimeType: prepared.resolvedMimeType,
|
||||
})
|
||||
logger.info('Sandbox file exported to workspace', {
|
||||
fileId: written.id,
|
||||
vfsPath: written.vfsPath,
|
||||
sandboxPath: prepared.sandboxPath,
|
||||
mode: prepared.file.mode ?? 'create',
|
||||
mimeType: prepared.resolvedMimeType,
|
||||
size: prepared.size,
|
||||
})
|
||||
writtenFiles.push({ ...written, sandboxPath: prepared.sandboxPath })
|
||||
}
|
||||
} catch (error) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
success: false,
|
||||
error: error instanceof Error ? error.message : 'Failed to export sandbox files',
|
||||
output: {
|
||||
result: null,
|
||||
stdout: cleanStdout(args.stdout),
|
||||
executionTime: args.executionTime,
|
||||
},
|
||||
},
|
||||
{ status: 400 }
|
||||
)
|
||||
}
|
||||
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
output: {
|
||||
result: {
|
||||
message: `Sandbox file exported to files/${fileName}`,
|
||||
fileId: uploaded.id,
|
||||
fileName,
|
||||
downloadUrl: uploaded.url,
|
||||
sandboxPath: outputSandboxPath,
|
||||
message: `Exported ${writtenFiles.length} sandbox files`,
|
||||
files: writtenFiles.map((file) => ({
|
||||
fileId: file.id,
|
||||
fileName: file.name,
|
||||
vfsPath: file.vfsPath,
|
||||
backingVfsPath: file.backingVfsPath,
|
||||
downloadUrl: file.downloadUrl,
|
||||
sandboxPath: file.sandboxPath,
|
||||
})),
|
||||
},
|
||||
stdout: cleanStdout(stdout),
|
||||
executionTime,
|
||||
stdout: cleanStdout(args.stdout),
|
||||
executionTime: args.executionTime,
|
||||
},
|
||||
resources: [{ type: 'file', id: uploaded.id, title: fileName }],
|
||||
resources: writtenFiles.map((file) => ({
|
||||
type: 'file',
|
||||
id: file.id,
|
||||
title: file.name,
|
||||
path: file.vfsPath,
|
||||
})),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -990,6 +1286,8 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
outputFormat,
|
||||
outputMimeType,
|
||||
outputSandboxPath,
|
||||
overwriteFileId,
|
||||
outputs,
|
||||
envVars = {},
|
||||
blockData = {},
|
||||
blockNameMapping = {},
|
||||
@@ -1007,6 +1305,26 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
_sandboxFiles,
|
||||
} = body
|
||||
sourceCodeForErrors = sourceCode
|
||||
const outputFiles = getOutputFileDeclarations({
|
||||
outputs,
|
||||
outputPath,
|
||||
outputFormat,
|
||||
outputMimeType,
|
||||
outputSandboxPath,
|
||||
overwriteFileId,
|
||||
})
|
||||
const outputSandboxPaths = outputFiles
|
||||
.map((file) => file.sandboxPath)
|
||||
.filter((path): path is string => Boolean(path))
|
||||
if (outputSandboxPaths.length > MAX_SANDBOX_OUTPUT_FILES) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
success: false,
|
||||
error: `Too many sandbox output files requested (${outputSandboxPaths.length}). Maximum is ${MAX_SANDBOX_OUTPUT_FILES}.`,
|
||||
},
|
||||
{ status: 400 }
|
||||
)
|
||||
}
|
||||
|
||||
const executionParams = { ...params }
|
||||
executionParams._context = undefined
|
||||
@@ -1108,12 +1426,14 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
sandboxId,
|
||||
error: shellError,
|
||||
exportedFileContent,
|
||||
exportedFiles,
|
||||
} = await executeShellInE2B({
|
||||
code: resolvedCode,
|
||||
envs: shellEnvs,
|
||||
timeoutMs: timeout,
|
||||
sandboxFiles: _sandboxFiles,
|
||||
outputSandboxPath,
|
||||
outputSandboxPaths,
|
||||
})
|
||||
const executionTime = Date.now() - execStart
|
||||
|
||||
@@ -1136,15 +1456,13 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
)
|
||||
}
|
||||
|
||||
if (outputSandboxPath) {
|
||||
const fileExportResponse = await maybeExportSandboxFileToWorkspace({
|
||||
if (outputSandboxPaths.length > 0 || outputSandboxPath) {
|
||||
const fileExportResponse = await maybeExportSandboxFilesToWorkspace({
|
||||
authUserId: auth.userId,
|
||||
workflowId,
|
||||
workspaceId,
|
||||
outputPath,
|
||||
outputFormat,
|
||||
outputMimeType,
|
||||
outputSandboxPath,
|
||||
outputFiles,
|
||||
exportedFiles,
|
||||
exportedFileContent,
|
||||
stdout: shellStdout,
|
||||
executionTime,
|
||||
@@ -1237,12 +1555,14 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
sandboxId,
|
||||
error: e2bError,
|
||||
exportedFileContent,
|
||||
exportedFiles,
|
||||
} = await executeInE2B({
|
||||
code: codeForE2B,
|
||||
language: CodeLanguage.JavaScript,
|
||||
timeoutMs: timeout,
|
||||
sandboxFiles: _sandboxFiles,
|
||||
outputSandboxPath,
|
||||
outputSandboxPaths,
|
||||
})
|
||||
const executionTime = Date.now() - execStart
|
||||
stdout += e2bStdout
|
||||
@@ -1273,15 +1593,13 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
)
|
||||
}
|
||||
|
||||
if (outputSandboxPath) {
|
||||
const fileExportResponse = await maybeExportSandboxFileToWorkspace({
|
||||
if (outputSandboxPaths.length > 0 || outputSandboxPath) {
|
||||
const fileExportResponse = await maybeExportSandboxFilesToWorkspace({
|
||||
authUserId: auth.userId,
|
||||
workflowId,
|
||||
workspaceId,
|
||||
outputPath,
|
||||
outputFormat,
|
||||
outputMimeType,
|
||||
outputSandboxPath,
|
||||
outputFiles,
|
||||
exportedFiles,
|
||||
exportedFileContent,
|
||||
stdout,
|
||||
executionTime,
|
||||
@@ -1324,12 +1642,14 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
sandboxId,
|
||||
error: e2bError,
|
||||
exportedFileContent,
|
||||
exportedFiles,
|
||||
} = await executeInE2B({
|
||||
code: codeForE2B,
|
||||
language: CodeLanguage.Python,
|
||||
timeoutMs: timeout,
|
||||
sandboxFiles: _sandboxFiles,
|
||||
outputSandboxPath,
|
||||
outputSandboxPaths,
|
||||
})
|
||||
const executionTime = Date.now() - execStart
|
||||
stdout += e2bStdout
|
||||
@@ -1360,15 +1680,13 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
)
|
||||
}
|
||||
|
||||
if (outputSandboxPath) {
|
||||
const fileExportResponse = await maybeExportSandboxFileToWorkspace({
|
||||
if (outputSandboxPaths.length > 0 || outputSandboxPath) {
|
||||
const fileExportResponse = await maybeExportSandboxFilesToWorkspace({
|
||||
authUserId: auth.userId,
|
||||
workflowId,
|
||||
workspaceId,
|
||||
outputPath,
|
||||
outputFormat,
|
||||
outputMimeType,
|
||||
outputSandboxPath,
|
||||
outputFiles,
|
||||
exportedFiles,
|
||||
exportedFileContent,
|
||||
stdout,
|
||||
executionTime,
|
||||
@@ -1546,6 +1864,24 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
)
|
||||
}
|
||||
|
||||
if (isLikelySandboxKill(error)) {
|
||||
const underlying = (error?.message || String(error)).slice(0, 300)
|
||||
logger.warn(`[${requestId}] Sandbox terminated before completion (likely OOM or timeout)`, {
|
||||
executionTime,
|
||||
underlying,
|
||||
})
|
||||
const killResponse = {
|
||||
success: false,
|
||||
error:
|
||||
'The sandbox was terminated before finishing — most likely it ran out of memory or hit the time limit while processing large or combined inputs. Mount and process fewer/smaller files at once (e.g. one file at a time), or stream and aggregate incrementally instead of loading everything into memory. ' +
|
||||
`(underlying: ${underlying || 'no detail; sandbox died'})`,
|
||||
output: { result: null, stdout: cleanStdout(stdout), executionTime },
|
||||
}
|
||||
return routeContext
|
||||
? functionJsonResponse(killResponse, routeContext, { status: 500 })
|
||||
: NextResponse.json(killResponse, { status: 500 })
|
||||
}
|
||||
|
||||
logger.error(`[${requestId}] Function execution failed`, {
|
||||
error: error.message || 'Unknown error',
|
||||
stack: error.stack,
|
||||
|
||||
@@ -4,6 +4,7 @@ import { type NextRequest, NextResponse } from 'next/server'
|
||||
import { guardrailsValidateContract } from '@/lib/api/contracts'
|
||||
import { parseRequest } from '@/lib/api/server'
|
||||
import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
|
||||
import { checkActorUsageLimits } from '@/lib/billing/calculations/usage-monitor'
|
||||
import { generateRequestId } from '@/lib/core/utils/request'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import { validateHallucination } from '@/lib/guardrails/validate_hallucination'
|
||||
@@ -175,6 +176,17 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
}
|
||||
throw err
|
||||
}
|
||||
|
||||
// Gate the actor's usage before incurring hosted LLM + RAG cost. In a normal
|
||||
// workflow run this already passed at preprocessing; this also blocks direct
|
||||
// calls to this route by an over-limit or frozen actor.
|
||||
const usage = await checkActorUsageLimits(auth.userId, resolvedWorkspaceId)
|
||||
if (usage.isExceeded) {
|
||||
return NextResponse.json(
|
||||
{ error: usage.message || 'Usage limit exceeded. Please upgrade your plan to continue.' },
|
||||
{ status: 402 }
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
const inputStr = convertInputToString(input)
|
||||
@@ -216,6 +228,33 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
requestId
|
||||
)
|
||||
|
||||
// Bill the guardrail's LLM scoring cost (hallucination only; BYOK/non-hosted
|
||||
// already resolve to 0). Attributed to the caller + the workflow's workspace
|
||||
// so it lands in the per-member meter. Best-effort — never fail validation on
|
||||
// a billing error.
|
||||
if (
|
||||
resolvedWorkspaceId &&
|
||||
typeof validationResult.cost === 'number' &&
|
||||
validationResult.cost > 0
|
||||
) {
|
||||
const { recordUsage } = await import('@/lib/billing/core/usage-log')
|
||||
await recordUsage({
|
||||
userId: auth.userId,
|
||||
workspaceId: resolvedWorkspaceId,
|
||||
entries: [
|
||||
{
|
||||
category: 'model',
|
||||
source: 'workflow',
|
||||
description: `guardrail-hallucination:${model ?? 'unknown'}`,
|
||||
cost: validationResult.cost,
|
||||
sourceReference: `guardrail:${workflowId ?? 'unknown'}:${requestId}`,
|
||||
},
|
||||
],
|
||||
}).catch((billingError) => {
|
||||
logger.error(`[${requestId}] Failed to record guardrail usage`, { error: billingError })
|
||||
})
|
||||
}
|
||||
|
||||
logger.info(`[${requestId}] Validation completed`, {
|
||||
passed: validationResult.passed,
|
||||
hasError: !!validationResult.error,
|
||||
@@ -301,6 +340,7 @@ async function executeValidation(
|
||||
reasoning?: string
|
||||
detectedEntities?: any[]
|
||||
maskedText?: string
|
||||
cost?: number
|
||||
}> {
|
||||
// Use TypeScript validators for all validation types
|
||||
if (validationType === 'json') {
|
||||
|
||||
@@ -43,6 +43,10 @@ vi.mock('@/lib/knowledge/documents/service', () => ({
|
||||
KnowledgeBaseFileOwnershipError: class KnowledgeBaseFileOwnershipError extends Error {},
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/billing/calculations/usage-monitor', () => ({
|
||||
checkActorUsageLimits: vi.fn().mockResolvedValue({ isExceeded: false }),
|
||||
}))
|
||||
|
||||
vi.mock('@sim/audit', () => auditMock)
|
||||
|
||||
import {
|
||||
@@ -343,7 +347,8 @@ describe('Knowledge Base Documents API Route', () => {
|
||||
expect(vi.mocked(createSingleDocument)).toHaveBeenCalledWith(
|
||||
validDocumentData,
|
||||
'kb-123',
|
||||
expect.any(String)
|
||||
expect.any(String),
|
||||
'user-123'
|
||||
)
|
||||
})
|
||||
|
||||
@@ -444,7 +449,8 @@ describe('Knowledge Base Documents API Route', () => {
|
||||
expect(vi.mocked(createDocumentRecords)).toHaveBeenCalledWith(
|
||||
validBulkData.documents,
|
||||
'kb-123',
|
||||
expect.any(String)
|
||||
expect.any(String),
|
||||
'user-123'
|
||||
)
|
||||
expect(vi.mocked(processDocumentsWithQueue)).toHaveBeenCalled()
|
||||
})
|
||||
|
||||
@@ -12,6 +12,7 @@ import {
|
||||
import { parseRequest } from '@/lib/api/server'
|
||||
import { getSession } from '@/lib/auth'
|
||||
import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
|
||||
import { checkActorUsageLimits } from '@/lib/billing/calculations/usage-monitor'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import {
|
||||
bulkDocumentOperation,
|
||||
@@ -163,11 +164,27 @@ export const POST = withRouteHandler(
|
||||
|
||||
const kbWorkspaceId = accessCheck.knowledgeBase?.workspaceId
|
||||
|
||||
// Gate KB indexing (pooled + per-member) before accepting work. Runs even for
|
||||
// legacy KBs with no workspace — the uploader's pooled/frozen status is still
|
||||
// enforced (per-member is simply skipped when there's no org workspace). The
|
||||
// authoritative backstop also runs in processDocumentAsync for non-HTTP paths
|
||||
// (connector/cron/retry).
|
||||
const usage = await checkActorUsageLimits(userId, kbWorkspaceId)
|
||||
if (usage.isExceeded) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
error: usage.message || 'Usage limit exceeded. Please upgrade your plan to continue.',
|
||||
},
|
||||
{ status: 402 }
|
||||
)
|
||||
}
|
||||
|
||||
if (body.bulk === true) {
|
||||
const createdDocuments = await createDocumentRecords(
|
||||
body.documents,
|
||||
knowledgeBaseId,
|
||||
requestId
|
||||
requestId,
|
||||
userId
|
||||
)
|
||||
|
||||
logger.info(
|
||||
@@ -247,7 +264,12 @@ export const POST = withRouteHandler(
|
||||
}
|
||||
|
||||
const { bulk: _bulk, workflowId: _workflowId, ...singleDocumentData } = body
|
||||
const newDocument = await createSingleDocument(singleDocumentData, knowledgeBaseId, requestId)
|
||||
const newDocument = await createSingleDocument(
|
||||
singleDocumentData,
|
||||
knowledgeBaseId,
|
||||
requestId,
|
||||
userId
|
||||
)
|
||||
|
||||
try {
|
||||
const { PlatformEvents } = await import('@/lib/core/telemetry')
|
||||
|
||||
@@ -27,6 +27,10 @@ vi.mock('@/lib/auth/hybrid', () => hybridAuthMock)
|
||||
vi.mock('@/app/api/knowledge/utils', () => knowledgeApiUtilsMock)
|
||||
vi.mock('@sim/audit', () => auditMock)
|
||||
|
||||
vi.mock('@/lib/billing/calculations/usage-monitor', () => ({
|
||||
checkActorUsageLimits: vi.fn().mockResolvedValue({ isExceeded: false }),
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/knowledge/documents/service', () => ({
|
||||
createDocumentRecords: vi.fn(),
|
||||
deleteDocument: vi.fn(),
|
||||
|
||||
@@ -10,6 +10,7 @@ import { type NextRequest, NextResponse } from 'next/server'
|
||||
import { upsertKnowledgeDocumentContract } from '@/lib/api/contracts/knowledge'
|
||||
import { parseRequest } from '@/lib/api/server'
|
||||
import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
|
||||
import { checkActorUsageLimits } from '@/lib/billing/calculations/usage-monitor'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import {
|
||||
createDocumentRecords,
|
||||
@@ -72,6 +73,21 @@ export const POST = withRouteHandler(
|
||||
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
|
||||
}
|
||||
|
||||
// Gate usage before any create/delete so an over-limit upsert is rejected up
|
||||
// front and never deletes the existing (already-indexed) document. Runs even
|
||||
// for legacy KBs with no workspace — the uploader's pooled/frozen status is
|
||||
// still enforced (per-member is skipped when there's no org workspace).
|
||||
const kbWorkspaceId = accessCheck.knowledgeBase?.workspaceId
|
||||
const usage = await checkActorUsageLimits(userId, kbWorkspaceId)
|
||||
if (usage.isExceeded) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
error: usage.message || 'Usage limit exceeded. Please upgrade your plan to continue.',
|
||||
},
|
||||
{ status: 402 }
|
||||
)
|
||||
}
|
||||
|
||||
let existingDocumentId: string | null = null
|
||||
let isUpdate = false
|
||||
|
||||
@@ -129,7 +145,8 @@ export const POST = withRouteHandler(
|
||||
},
|
||||
],
|
||||
knowledgeBaseId,
|
||||
requestId
|
||||
requestId,
|
||||
userId
|
||||
)
|
||||
|
||||
const firstDocument = createdDocuments[0]
|
||||
|
||||
@@ -158,7 +158,9 @@ describe('Knowledge Search API Route', () => {
|
||||
parallelLimit: 15,
|
||||
singleQueryOptimized: true,
|
||||
})
|
||||
mockGenerateSearchEmbedding.mockClear().mockResolvedValue([0.1, 0.2, 0.3, 0.4, 0.5])
|
||||
mockGenerateSearchEmbedding
|
||||
.mockClear()
|
||||
.mockResolvedValue({ embedding: [0.1, 0.2, 0.3, 0.4, 0.5], isBYOK: false })
|
||||
mockGetDocumentMetadataByIds.mockClear().mockResolvedValue({
|
||||
doc1: { filename: 'Document 1', sourceUrl: null },
|
||||
doc2: { filename: 'Document 2', sourceUrl: null },
|
||||
@@ -997,7 +999,7 @@ describe('Knowledge Search API Route', () => {
|
||||
singleQueryOptimized: true,
|
||||
})
|
||||
|
||||
mockGenerateSearchEmbedding.mockResolvedValue([0.1, 0.2, 0.3])
|
||||
mockGenerateSearchEmbedding.mockResolvedValue({ embedding: [0.1, 0.2, 0.3], isBYOK: false })
|
||||
mockGetDocumentMetadataByIds.mockResolvedValue({
|
||||
'doc-active': {
|
||||
filename: 'Active Document.pdf',
|
||||
@@ -1145,7 +1147,7 @@ describe('Knowledge Search API Route', () => {
|
||||
singleQueryOptimized: true,
|
||||
})
|
||||
|
||||
mockGenerateSearchEmbedding.mockResolvedValue([0.1, 0.2, 0.3])
|
||||
mockGenerateSearchEmbedding.mockResolvedValue({ embedding: [0.1, 0.2, 0.3], isBYOK: false })
|
||||
mockGetDocumentMetadataByIds.mockResolvedValue({
|
||||
'doc-active-combined': { filename: 'Active Combined Search.pdf', sourceUrl: null },
|
||||
})
|
||||
|
||||
@@ -4,7 +4,8 @@ import { authorizeWorkflowByWorkspacePermission } from '@sim/workflow-authz'
|
||||
import { type NextRequest, NextResponse } from 'next/server'
|
||||
import { knowledgeSearchBodySchema } from '@/lib/api/contracts/knowledge'
|
||||
import { parseJsonBody, validationErrorResponse } from '@/lib/api/server'
|
||||
import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
|
||||
import { AuthType, checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
|
||||
import { checkActorUsageLimits } from '@/lib/billing/calculations/usage-monitor'
|
||||
import { PlatformEvents } from '@/lib/core/telemetry'
|
||||
import { generateRequestId } from '@/lib/core/utils/request'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
@@ -37,7 +38,7 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
const parsedBody = await parseJsonBody(request)
|
||||
if (!parsedBody.success) return parsedBody.response
|
||||
const body = parsedBody.data as Record<string, unknown>
|
||||
const { workflowId, ...searchParams } = body
|
||||
const { workflowId, skipUsageBilling, ...searchParams } = body
|
||||
|
||||
const auth = await checkSessionOrInternalAuth(request, { requireWorkflowId: false })
|
||||
if (!auth.success || !auth.userId) {
|
||||
@@ -45,6 +46,11 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
}
|
||||
const userId = auth.userId
|
||||
|
||||
// Only the internal workflow tool may suppress route metering (it rolls the
|
||||
// cost into the executor's usage instead). Session/API-key callers cannot set
|
||||
// skipUsageBilling to dodge their own embedding/reranker charge.
|
||||
const shouldMeter = !(skipUsageBilling === true && auth.authType === AuthType.INTERNAL_JWT)
|
||||
|
||||
if (workflowId) {
|
||||
const authorization = await authorizeWorkflowByWorkspacePermission({
|
||||
workflowId: workflowId as string,
|
||||
@@ -219,6 +225,19 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
)
|
||||
}
|
||||
|
||||
// Gate the actor before incurring hosted embedding cost, unless this is the
|
||||
// internal workflow tool (already gated at preprocessing, rolls cost up). Tag-only
|
||||
// search is free, so only the query path is gated.
|
||||
if (shouldMeter && hasQuery) {
|
||||
const usage = await checkActorUsageLimits(userId, workspaceId)
|
||||
if (usage.isExceeded) {
|
||||
return NextResponse.json(
|
||||
{ error: usage.message || 'Usage limit exceeded. Please upgrade your plan to continue.' },
|
||||
{ status: 402 }
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
const queryEmbeddingPromise = hasQuery
|
||||
? generateSearchEmbedding(validatedData.query!, queryEmbeddingModel, workspaceId)
|
||||
: Promise.resolve(null)
|
||||
@@ -273,7 +292,7 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
} else if (hasQuery && hasFilters) {
|
||||
logger.debug(`[${requestId}] Executing tag + vector search with filters:`, structuredFilters)
|
||||
const strategy = getQueryStrategy(accessibleKbIds.length, candidateTopK)
|
||||
const queryVector = JSON.stringify(await queryEmbeddingPromise)
|
||||
const queryVector = JSON.stringify((await queryEmbeddingPromise)?.embedding ?? null)
|
||||
|
||||
results = await handleTagAndVectorSearch({
|
||||
knowledgeBaseIds: accessibleKbIds,
|
||||
@@ -284,7 +303,7 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
})
|
||||
} else if (hasQuery && !hasFilters) {
|
||||
const strategy = getQueryStrategy(accessibleKbIds.length, candidateTopK)
|
||||
const queryVector = JSON.stringify(await queryEmbeddingPromise)
|
||||
const queryVector = JSON.stringify((await queryEmbeddingPromise)?.embedding ?? null)
|
||||
|
||||
results = await handleVectorOnlySearch({
|
||||
knowledgeBaseIds: accessibleKbIds,
|
||||
@@ -359,7 +378,11 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
validatedData.query!,
|
||||
getEmbeddingModelInfo(queryEmbeddingModel).tokenizerProvider
|
||||
)
|
||||
cost = calculateCost(queryEmbeddingModel, tokenCount.count, 0, false)
|
||||
// BYOK query embeddings incur no Sim cost, so don't bill (or roll up) them.
|
||||
const queryEmbeddingResult = await queryEmbeddingPromise
|
||||
if (!queryEmbeddingResult?.isBYOK) {
|
||||
cost = calculateCost(queryEmbeddingModel, tokenCount.count, 0, false)
|
||||
}
|
||||
} catch (error) {
|
||||
logger.warn(`[${requestId}] Failed to calculate cost for search query`, {
|
||||
error: getErrorMessage(error, 'Unknown error'),
|
||||
@@ -393,6 +416,29 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
}
|
||||
}
|
||||
|
||||
// Record query-embedding + reranker cost for standalone callers (UI, copilot,
|
||||
// guardrail RAG). The workflow tool sets skipUsageBilling and rolls the cost
|
||||
// up via the executor instead, so this never double-bills; BYOK already
|
||||
// resolved to 0 above.
|
||||
if (shouldMeter && workspaceId && cost && cost.total > 0) {
|
||||
const { recordUsage } = await import('@/lib/billing/core/usage-log')
|
||||
await recordUsage({
|
||||
userId,
|
||||
workspaceId,
|
||||
entries: [
|
||||
{
|
||||
category: 'model',
|
||||
source: 'knowledge-base',
|
||||
description: queryEmbeddingModel,
|
||||
cost: cost.total,
|
||||
sourceReference: `kb-search:${requestId}`,
|
||||
},
|
||||
],
|
||||
}).catch((billingError) => {
|
||||
logger.error(`[${requestId}] Failed to record KB search usage`, { error: billingError })
|
||||
})
|
||||
}
|
||||
|
||||
const tagDefsResults = await Promise.all(
|
||||
accessibleKbIds.map(async (kbId) => {
|
||||
try {
|
||||
|
||||
@@ -184,7 +184,7 @@ describe('Knowledge Search Utils', () => {
|
||||
}),
|
||||
})
|
||||
)
|
||||
expect(result).toEqual([0.1, 0.2, 0.3])
|
||||
expect(result.embedding).toEqual([0.1, 0.2, 0.3])
|
||||
|
||||
// Clean up
|
||||
Object.keys(env).forEach((key) => delete (env as any)[key])
|
||||
@@ -214,7 +214,7 @@ describe('Knowledge Search Utils', () => {
|
||||
}),
|
||||
})
|
||||
)
|
||||
expect(result).toEqual([0.1, 0.2, 0.3])
|
||||
expect(result.embedding).toEqual([0.1, 0.2, 0.3])
|
||||
|
||||
// Clean up
|
||||
Object.keys(env).forEach((key) => delete (env as any)[key])
|
||||
|
||||
@@ -1,64 +1,14 @@
|
||||
import { db } from '@sim/db'
|
||||
import {
|
||||
jobExecutionLogs,
|
||||
pausedExecutions,
|
||||
permissions,
|
||||
workflow,
|
||||
workflowDeploymentVersion,
|
||||
workflowExecutionLogs,
|
||||
} from '@sim/db/schema'
|
||||
import { createLogger } from '@sim/logger'
|
||||
import {
|
||||
and,
|
||||
asc,
|
||||
desc,
|
||||
eq,
|
||||
gt,
|
||||
gte,
|
||||
inArray,
|
||||
isNotNull,
|
||||
isNull,
|
||||
lt,
|
||||
lte,
|
||||
ne,
|
||||
or,
|
||||
type SQL,
|
||||
sql,
|
||||
} from 'drizzle-orm'
|
||||
import type { NextRequest } from 'next/server'
|
||||
import { NextResponse } from 'next/server'
|
||||
import { listLogsContract, type WorkflowLogSummary } from '@/lib/api/contracts/logs'
|
||||
import { listLogsContract } from '@/lib/api/contracts/logs'
|
||||
import { parseRequest } from '@/lib/api/server'
|
||||
import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import { jobCostTotal } from '@/lib/logs/fetch-log-detail'
|
||||
import { buildFilterConditions } from '@/lib/logs/filters'
|
||||
import { expandFolderIdsWithDescendants } from '@/lib/logs/folder-expansion'
|
||||
import { listLogs } from '@/lib/logs/list-logs'
|
||||
|
||||
const logger = createLogger('LogsAPI')
|
||||
|
||||
type SortBy = 'date' | 'duration' | 'cost' | 'status'
|
||||
type SortOrder = 'asc' | 'desc'
|
||||
|
||||
interface CursorData {
|
||||
v: string | number | null
|
||||
id: string
|
||||
}
|
||||
|
||||
function encodeCursor(data: CursorData): string {
|
||||
return Buffer.from(JSON.stringify(data)).toString('base64')
|
||||
}
|
||||
|
||||
function decodeCursor(cursor: string): CursorData | null {
|
||||
try {
|
||||
const parsed = JSON.parse(Buffer.from(cursor, 'base64').toString())
|
||||
if (typeof parsed?.id !== 'string') return null
|
||||
return parsed as CursorData
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
export const GET = withRouteHandler(async (request: NextRequest) => {
|
||||
const authResult = await checkSessionOrInternalAuth(request, { requireWorkflowId: false })
|
||||
if (!authResult.success || !authResult.userId) {
|
||||
@@ -73,402 +23,15 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
|
||||
if (!parsed.success) return parsed.response
|
||||
|
||||
const params = parsed.data.query
|
||||
const sortBy = params.sortBy as SortBy
|
||||
const sortOrder = params.sortOrder as SortOrder
|
||||
const cursor = params.cursor ? decodeCursor(params.cursor) : null
|
||||
|
||||
const workflowSortExpr: SQL<unknown> = (() => {
|
||||
switch (sortBy) {
|
||||
case 'duration':
|
||||
return sql`${workflowExecutionLogs.totalDurationMs}`
|
||||
case 'cost':
|
||||
// Indexed projection of the usage_log ledger (dollars); no live aggregation.
|
||||
return sql`${workflowExecutionLogs.costTotal}`
|
||||
case 'status':
|
||||
return sql`${workflowExecutionLogs.status}`
|
||||
default:
|
||||
return sql`${workflowExecutionLogs.startedAt}`
|
||||
}
|
||||
})()
|
||||
|
||||
const jobSortExpr: SQL<unknown> = (() => {
|
||||
switch (sortBy) {
|
||||
case 'duration':
|
||||
return sql`${jobExecutionLogs.totalDurationMs}`
|
||||
case 'cost':
|
||||
return sql`(${jobExecutionLogs.cost}->>'total')::numeric`
|
||||
case 'status':
|
||||
return sql`${jobExecutionLogs.status}`
|
||||
default:
|
||||
return sql`${jobExecutionLogs.startedAt}`
|
||||
}
|
||||
})()
|
||||
|
||||
const dir = sortOrder === 'asc' ? asc : desc
|
||||
const nullsLast = sql`NULLS LAST`
|
||||
const orderByClause = (expr: SQL): SQL => sql`${dir(expr)} ${nullsLast}`
|
||||
|
||||
const buildCursorCondition = (sortExpr: unknown, idCol: unknown): SQL | undefined => {
|
||||
if (!cursor) return undefined
|
||||
const v = cursor.v
|
||||
const id = cursor.id
|
||||
const cmp = sortOrder === 'asc' ? sql`>` : sql`<`
|
||||
if (v === null) {
|
||||
return sql`(${sortExpr} IS NULL AND ${idCol} ${cmp} ${id})`
|
||||
}
|
||||
return sql`((${sortExpr} IS NOT NULL AND ${sortExpr} ${cmp} ${v}) OR (${sortExpr} = ${v} AND ${idCol} ${cmp} ${id}) OR ${sortExpr} IS NULL)`
|
||||
}
|
||||
|
||||
const fetchSize = params.limit + 1
|
||||
|
||||
// Build workflow log conditions
|
||||
const workflowConditions: SQL[] = [eq(workflowExecutionLogs.workspaceId, params.workspaceId)]
|
||||
|
||||
if (params.level && params.level !== 'all') {
|
||||
const levels = params.level.split(',').filter(Boolean)
|
||||
const levelConditions: SQL[] = []
|
||||
|
||||
for (const level of levels) {
|
||||
if (level === 'error') {
|
||||
levelConditions.push(eq(workflowExecutionLogs.level, 'error'))
|
||||
} else if (level === 'info') {
|
||||
const c = and(
|
||||
eq(workflowExecutionLogs.level, 'info'),
|
||||
isNotNull(workflowExecutionLogs.endedAt)
|
||||
)
|
||||
if (c) levelConditions.push(c)
|
||||
} else if (level === 'running') {
|
||||
const c = and(
|
||||
eq(workflowExecutionLogs.level, 'info'),
|
||||
isNull(workflowExecutionLogs.endedAt)
|
||||
)
|
||||
if (c) levelConditions.push(c)
|
||||
} else if (level === 'pending') {
|
||||
const c = and(
|
||||
eq(workflowExecutionLogs.level, 'info'),
|
||||
or(
|
||||
sql`(${pausedExecutions.totalPauseCount} > 0 AND ${pausedExecutions.resumedCount} < ${pausedExecutions.totalPauseCount})`,
|
||||
and(
|
||||
isNotNull(pausedExecutions.status),
|
||||
sql`${pausedExecutions.status} != 'fully_resumed'`
|
||||
)
|
||||
)
|
||||
)
|
||||
if (c) levelConditions.push(c)
|
||||
}
|
||||
}
|
||||
|
||||
if (levelConditions.length > 0) {
|
||||
workflowConditions.push(
|
||||
levelConditions.length === 1 ? levelConditions[0] : or(...levelConditions)!
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
if (params.folderIds) {
|
||||
params.folderIds = await expandFolderIdsWithDescendants(params.workspaceId, params.folderIds)
|
||||
}
|
||||
|
||||
const commonFilters = buildFilterConditions(params, { useSimpleLevelFilter: false })
|
||||
if (commonFilters) workflowConditions.push(commonFilters)
|
||||
|
||||
const workflowCursorCond = buildCursorCondition(workflowSortExpr, workflowExecutionLogs.id)
|
||||
if (workflowCursorCond) workflowConditions.push(workflowCursorCond)
|
||||
|
||||
// Decide whether to include job logs
|
||||
const hasWorkflowSpecificFilters = !!(
|
||||
params.workflowIds ||
|
||||
params.folderIds ||
|
||||
params.workflowName ||
|
||||
params.folderName
|
||||
)
|
||||
const triggersList = params.triggers?.split(',').filter(Boolean) || []
|
||||
const triggersExcludeJobs =
|
||||
triggersList.length > 0 && !triggersList.includes('all') && !triggersList.includes('mothership')
|
||||
const levelList =
|
||||
params.level && params.level !== 'all' ? params.level.split(',').filter(Boolean) : []
|
||||
const levelExcludesJobs =
|
||||
levelList.length > 0 && !levelList.some((l) => l === 'error' || l === 'info')
|
||||
const includeJobLogs = !hasWorkflowSpecificFilters && !triggersExcludeJobs && !levelExcludesJobs
|
||||
|
||||
const workflowQuery = db
|
||||
.select({
|
||||
id: workflowExecutionLogs.id,
|
||||
workflowId: workflowExecutionLogs.workflowId,
|
||||
executionId: workflowExecutionLogs.executionId,
|
||||
deploymentVersionId: workflowExecutionLogs.deploymentVersionId,
|
||||
level: workflowExecutionLogs.level,
|
||||
status: workflowExecutionLogs.status,
|
||||
trigger: workflowExecutionLogs.trigger,
|
||||
startedAt: workflowExecutionLogs.startedAt,
|
||||
endedAt: workflowExecutionLogs.endedAt,
|
||||
totalDurationMs: workflowExecutionLogs.totalDurationMs,
|
||||
costTotal: workflowExecutionLogs.costTotal,
|
||||
createdAt: workflowExecutionLogs.createdAt,
|
||||
workflowName: workflow.name,
|
||||
workflowDescription: workflow.description,
|
||||
workflowFolderId: workflow.folderId,
|
||||
workflowUserId: workflow.userId,
|
||||
workflowWorkspaceId: workflow.workspaceId,
|
||||
workflowCreatedAt: workflow.createdAt,
|
||||
workflowUpdatedAt: workflow.updatedAt,
|
||||
pausedStatus: pausedExecutions.status,
|
||||
pausedTotalPauseCount: pausedExecutions.totalPauseCount,
|
||||
pausedResumedCount: pausedExecutions.resumedCount,
|
||||
deploymentVersion: workflowDeploymentVersion.version,
|
||||
deploymentVersionName: workflowDeploymentVersion.name,
|
||||
sortValue: sql<unknown>`${workflowSortExpr}`.as('sort_value'),
|
||||
})
|
||||
.from(workflowExecutionLogs)
|
||||
.leftJoin(pausedExecutions, eq(pausedExecutions.executionId, workflowExecutionLogs.executionId))
|
||||
.leftJoin(
|
||||
workflowDeploymentVersion,
|
||||
eq(workflowDeploymentVersion.id, workflowExecutionLogs.deploymentVersionId)
|
||||
)
|
||||
.leftJoin(workflow, eq(workflowExecutionLogs.workflowId, workflow.id))
|
||||
.innerJoin(
|
||||
permissions,
|
||||
and(
|
||||
eq(permissions.entityType, 'workspace'),
|
||||
eq(permissions.entityId, workflowExecutionLogs.workspaceId),
|
||||
eq(permissions.userId, userId)
|
||||
)
|
||||
)
|
||||
.where(and(...workflowConditions))
|
||||
.orderBy(orderByClause(workflowSortExpr), dir(workflowExecutionLogs.id))
|
||||
.limit(fetchSize)
|
||||
|
||||
const jobConditions: SQL[] = [eq(jobExecutionLogs.workspaceId, params.workspaceId)]
|
||||
|
||||
if (includeJobLogs) {
|
||||
jobConditions.push(
|
||||
sql`EXISTS (SELECT 1 FROM ${permissions} WHERE ${permissions.entityType} = 'workspace' AND ${permissions.entityId} = ${jobExecutionLogs.workspaceId} AND ${permissions.userId} = ${userId})`
|
||||
)
|
||||
|
||||
if (params.level && params.level !== 'all') {
|
||||
const levels = params.level.split(',').filter(Boolean)
|
||||
const jobLevelConditions: SQL[] = []
|
||||
for (const level of levels) {
|
||||
if (level === 'error') {
|
||||
jobLevelConditions.push(eq(jobExecutionLogs.level, 'error'))
|
||||
} else if (level === 'info') {
|
||||
const c = and(eq(jobExecutionLogs.level, 'info'), isNotNull(jobExecutionLogs.endedAt))
|
||||
if (c) jobLevelConditions.push(c)
|
||||
}
|
||||
}
|
||||
if (jobLevelConditions.length > 0) {
|
||||
jobConditions.push(
|
||||
jobLevelConditions.length === 1 ? jobLevelConditions[0] : or(...jobLevelConditions)!
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
if (triggersList.length > 0 && !triggersList.includes('all')) {
|
||||
jobConditions.push(inArray(jobExecutionLogs.trigger, triggersList))
|
||||
}
|
||||
|
||||
if (params.startDate) {
|
||||
jobConditions.push(gte(jobExecutionLogs.startedAt, new Date(params.startDate)))
|
||||
}
|
||||
if (params.endDate) {
|
||||
jobConditions.push(lte(jobExecutionLogs.startedAt, new Date(params.endDate)))
|
||||
}
|
||||
|
||||
if (params.search) {
|
||||
jobConditions.push(sql`${jobExecutionLogs.executionId} ILIKE ${`%${params.search}%`}`)
|
||||
}
|
||||
if (params.executionId) {
|
||||
jobConditions.push(eq(jobExecutionLogs.executionId, params.executionId))
|
||||
}
|
||||
|
||||
if (params.costOperator && params.costValue !== undefined) {
|
||||
const costField = sql`(${jobExecutionLogs.cost}->>'total')::numeric`
|
||||
const ops = {
|
||||
'=': sql`=`,
|
||||
'>': sql`>`,
|
||||
'<': sql`<`,
|
||||
'>=': sql`>=`,
|
||||
'<=': sql`<=`,
|
||||
'!=': sql`!=`,
|
||||
} as const
|
||||
jobConditions.push(sql`${costField} ${ops[params.costOperator]} ${params.costValue}`)
|
||||
}
|
||||
|
||||
if (params.durationOperator && params.durationValue !== undefined) {
|
||||
const durationOps: Record<
|
||||
string,
|
||||
(field: typeof jobExecutionLogs.totalDurationMs, val: number) => SQL | undefined
|
||||
> = {
|
||||
'=': (f, v) => eq(f, v),
|
||||
'>': (f, v) => gt(f, v),
|
||||
'<': (f, v) => lt(f, v),
|
||||
'>=': (f, v) => gte(f, v),
|
||||
'<=': (f, v) => lte(f, v),
|
||||
'!=': (f, v) => ne(f, v),
|
||||
}
|
||||
const durationCond = durationOps[params.durationOperator]?.(
|
||||
jobExecutionLogs.totalDurationMs,
|
||||
params.durationValue
|
||||
)
|
||||
if (durationCond) jobConditions.push(durationCond)
|
||||
}
|
||||
|
||||
const jobCursorCond = buildCursorCondition(jobSortExpr, jobExecutionLogs.id)
|
||||
if (jobCursorCond) jobConditions.push(jobCursorCond)
|
||||
}
|
||||
|
||||
const jobQuery = includeJobLogs
|
||||
? db
|
||||
.select({
|
||||
id: jobExecutionLogs.id,
|
||||
executionId: jobExecutionLogs.executionId,
|
||||
level: jobExecutionLogs.level,
|
||||
status: jobExecutionLogs.status,
|
||||
trigger: jobExecutionLogs.trigger,
|
||||
startedAt: jobExecutionLogs.startedAt,
|
||||
endedAt: jobExecutionLogs.endedAt,
|
||||
totalDurationMs: jobExecutionLogs.totalDurationMs,
|
||||
cost: jobExecutionLogs.cost,
|
||||
createdAt: jobExecutionLogs.createdAt,
|
||||
jobTitle: sql<string | null>`${jobExecutionLogs.executionData}->'trigger'->>'source'`,
|
||||
sortValue: sql<unknown>`${jobSortExpr}`.as('sort_value'),
|
||||
})
|
||||
.from(jobExecutionLogs)
|
||||
.where(and(...jobConditions))
|
||||
.orderBy(orderByClause(jobSortExpr), dir(jobExecutionLogs.id))
|
||||
.limit(fetchSize)
|
||||
: Promise.resolve([])
|
||||
|
||||
const [workflowRows, jobRows] = await Promise.all([workflowQuery, jobQuery])
|
||||
|
||||
type RowWithSort = {
|
||||
id: string
|
||||
sortValue: unknown
|
||||
summary: WorkflowLogSummary
|
||||
}
|
||||
|
||||
const workflowMapped: RowWithSort[] = workflowRows.map((log) => {
|
||||
const totalPauseCount = Number(log.pausedTotalPauseCount ?? 0)
|
||||
const resumedCount = Number(log.pausedResumedCount ?? 0)
|
||||
const hasPendingPause =
|
||||
(totalPauseCount > 0 && resumedCount < totalPauseCount) ||
|
||||
(log.pausedStatus !== null && log.pausedStatus !== 'fully_resumed')
|
||||
|
||||
const summary: WorkflowLogSummary = {
|
||||
id: log.id,
|
||||
workflowId: log.workflowId,
|
||||
executionId: log.executionId,
|
||||
deploymentVersionId: log.deploymentVersionId,
|
||||
deploymentVersion: log.deploymentVersion ?? null,
|
||||
deploymentVersionName: log.deploymentVersionName ?? null,
|
||||
level: log.level,
|
||||
status: log.status,
|
||||
duration: log.totalDurationMs ? `${log.totalDurationMs}ms` : null,
|
||||
trigger: log.trigger,
|
||||
createdAt: log.startedAt.toISOString(),
|
||||
workflow: log.workflowId
|
||||
? {
|
||||
id: log.workflowId,
|
||||
name: log.workflowName,
|
||||
description: log.workflowDescription,
|
||||
folderId: log.workflowFolderId,
|
||||
userId: log.workflowUserId,
|
||||
workspaceId: log.workflowWorkspaceId,
|
||||
createdAt: log.workflowCreatedAt?.toISOString() ?? null,
|
||||
updatedAt: log.workflowUpdatedAt?.toISOString() ?? null,
|
||||
}
|
||||
: null,
|
||||
jobTitle: null,
|
||||
// List cost is the cost_total projection (faithful ledger sum). Null until
|
||||
// completion (running) or until the one-time legacy backfill populates it.
|
||||
cost: log.costTotal != null ? { total: Number(log.costTotal) } : null,
|
||||
pauseSummary: {
|
||||
status: log.pausedStatus ?? null,
|
||||
total: totalPauseCount,
|
||||
resumed: resumedCount,
|
||||
},
|
||||
hasPendingPause,
|
||||
}
|
||||
return { id: log.id, sortValue: log.sortValue, summary }
|
||||
})
|
||||
|
||||
const jobMapped: RowWithSort[] = (jobRows as Awaited<typeof jobQuery>).map((log) => {
|
||||
const summary: WorkflowLogSummary = {
|
||||
id: log.id,
|
||||
workflowId: null,
|
||||
executionId: log.executionId,
|
||||
deploymentVersionId: null,
|
||||
deploymentVersion: null,
|
||||
deploymentVersionName: null,
|
||||
level: log.level,
|
||||
status: log.status,
|
||||
duration: log.totalDurationMs ? `${log.totalDurationMs}ms` : null,
|
||||
trigger: log.trigger,
|
||||
createdAt: log.startedAt.toISOString(),
|
||||
workflow: null,
|
||||
jobTitle: log.jobTitle ?? null,
|
||||
cost: jobCostTotal(log.cost),
|
||||
pauseSummary: { status: null, total: 0, resumed: 0 },
|
||||
hasPendingPause: false,
|
||||
}
|
||||
return { id: log.id, sortValue: log.sortValue, summary }
|
||||
})
|
||||
|
||||
const compareSortValues = (a: unknown, b: unknown): number => {
|
||||
if (a instanceof Date && b instanceof Date) return a.getTime() - b.getTime()
|
||||
if (typeof a === 'number' && typeof b === 'number') return a - b
|
||||
const aStr = String(a)
|
||||
const bStr = String(b)
|
||||
if (sortBy === 'date') {
|
||||
return new Date(aStr).getTime() - new Date(bStr).getTime()
|
||||
}
|
||||
const aNum = Number(aStr)
|
||||
const bNum = Number(bStr)
|
||||
if (!Number.isNaN(aNum) && !Number.isNaN(bNum)) return aNum - bNum
|
||||
return aStr.localeCompare(bStr)
|
||||
}
|
||||
|
||||
const merged = [...workflowMapped, ...jobMapped].sort((a, b) => {
|
||||
const aNull = a.sortValue === null || a.sortValue === undefined
|
||||
const bNull = b.sortValue === null || b.sortValue === undefined
|
||||
// Mirror SQL's NULLS LAST for both ASC and DESC so the cursor stays consistent.
|
||||
if (aNull && !bNull) return 1
|
||||
if (!aNull && bNull) return -1
|
||||
if (!aNull && !bNull) {
|
||||
const cmp = compareSortValues(a.sortValue, b.sortValue)
|
||||
if (cmp !== 0) return sortOrder === 'asc' ? cmp : -cmp
|
||||
}
|
||||
const idCmp = a.id.localeCompare(b.id)
|
||||
return sortOrder === 'asc' ? idCmp : -idCmp
|
||||
})
|
||||
|
||||
const page = merged.slice(0, params.limit)
|
||||
const hasMore = merged.length > params.limit
|
||||
let nextCursor: string | null = null
|
||||
if (hasMore && page.length > 0) {
|
||||
const last = page[page.length - 1]
|
||||
const v = last.sortValue
|
||||
const cursorV =
|
||||
v instanceof Date
|
||||
? v.toISOString()
|
||||
: typeof v === 'number' || typeof v === 'string'
|
||||
? v
|
||||
: v == null
|
||||
? null
|
||||
: String(v)
|
||||
nextCursor = encodeCursor({ v: cursorV, id: last.id })
|
||||
}
|
||||
const result = await listLogs(params, userId)
|
||||
|
||||
logger.debug('Listed logs', {
|
||||
workspaceId: params.workspaceId,
|
||||
count: page.length,
|
||||
hasMore,
|
||||
sortBy,
|
||||
sortOrder,
|
||||
count: result.data.length,
|
||||
hasMore: result.nextCursor !== null,
|
||||
sortBy: params.sortBy,
|
||||
sortOrder: params.sortOrder,
|
||||
})
|
||||
|
||||
return NextResponse.json({
|
||||
data: page.map((row) => row.summary),
|
||||
nextCursor,
|
||||
})
|
||||
return NextResponse.json(result)
|
||||
})
|
||||
|
||||
@@ -105,7 +105,7 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
workspaceId,
|
||||
type: 'mothership',
|
||||
title: null,
|
||||
model: 'claude-opus-4-6',
|
||||
model: 'claude-opus-4-8',
|
||||
updatedAt: now,
|
||||
lastSeenAt: now,
|
||||
})
|
||||
|
||||
@@ -14,8 +14,9 @@ import {
|
||||
import { runHeadlessCopilotLifecycle } from '@/lib/copilot/request/lifecycle/headless'
|
||||
import { requestExplicitStreamAbort } from '@/lib/copilot/request/session/explicit-abort'
|
||||
import type { StreamEvent } from '@/lib/copilot/request/types'
|
||||
import { isE2BDocEnabled } from '@/lib/core/config/feature-flags'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import { buildMothershipToolsForRequest } from '@/lib/mothership/settings/runtime'
|
||||
import { buildUserSkillTool } from '@/lib/mothership/skills'
|
||||
import {
|
||||
assertActiveWorkspaceAccess,
|
||||
getUserEntityPermissions,
|
||||
@@ -96,15 +97,33 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
messages,
|
||||
responseFormat,
|
||||
workspaceId,
|
||||
userId,
|
||||
userId: bodyUserId,
|
||||
chatId,
|
||||
messageId: providedMessageId,
|
||||
requestId: providedRequestId,
|
||||
fileAttachments,
|
||||
workflowId,
|
||||
executionId,
|
||||
userMetadata,
|
||||
} = validation.data.body
|
||||
|
||||
// Bind the billing actor to the authenticated identity. The executor mints
|
||||
// the internal JWT with the workflow owner's userId, so a token issued for
|
||||
// one user must never be used to attribute mothership-block cost to another
|
||||
// user via a forged body.userId. When the token carries a userId we require
|
||||
// the body to match it; the JWT userId is authoritative.
|
||||
if (auth.userId && auth.userId !== bodyUserId) {
|
||||
logger.warn('Mothership execute userId does not match authenticated identity', {
|
||||
tokenUserId: auth.userId,
|
||||
bodyUserId,
|
||||
})
|
||||
return NextResponse.json(
|
||||
{ error: 'userId does not match authenticated identity' },
|
||||
{ status: 403 }
|
||||
)
|
||||
}
|
||||
const userId = auth.userId ?? bodyUserId
|
||||
|
||||
await assertActiveWorkspaceAccess(workspaceId, userId)
|
||||
|
||||
const effectiveChatId = chatId || generateId()
|
||||
@@ -116,34 +135,32 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
workflowId,
|
||||
executionId,
|
||||
})
|
||||
const [workspaceContext, integrationTools, mothershipToolRuntime, userPermission] =
|
||||
await Promise.all([
|
||||
generateWorkspaceContext(workspaceId, userId),
|
||||
buildIntegrationToolSchemas(userId, messageId, undefined, workspaceId),
|
||||
buildMothershipToolsForRequest({ workspaceId, userId }),
|
||||
getUserEntityPermissions(userId, 'workspace', workspaceId).catch(() => null),
|
||||
])
|
||||
const workspaceContextWithMothershipTools = [
|
||||
workspaceContext,
|
||||
mothershipToolRuntime.catalogContext,
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join('\n\n')
|
||||
|
||||
const [workspaceContext, integrationTools, userSkillTool, userPermission] = await Promise.all([
|
||||
generateWorkspaceContext(workspaceId, userId),
|
||||
buildIntegrationToolSchemas(userId, messageId, undefined, workspaceId),
|
||||
buildUserSkillTool(workspaceId),
|
||||
getUserEntityPermissions(userId, 'workspace', workspaceId).catch(() => null),
|
||||
])
|
||||
const requestPayload: Record<string, unknown> = {
|
||||
messages,
|
||||
responseFormat,
|
||||
userId,
|
||||
// Go's auth middleware reads workspaceId off the request body to forward
|
||||
// to /api/copilot/api-keys/validate (per-member org usage gate). Omitting
|
||||
// it makes that validation 400 ("API key validation failed"), which kills
|
||||
// the block. The chat path sends it via buildCopilotRequestPayload; the
|
||||
// block path must too.
|
||||
workspaceId,
|
||||
chatId: effectiveChatId,
|
||||
mode: 'agent',
|
||||
messageId,
|
||||
isHosted: true,
|
||||
workspaceContext: workspaceContextWithMothershipTools,
|
||||
workspaceContext,
|
||||
...(isE2BDocEnabled ? { docCompiler: 'python' } : {}),
|
||||
...(userMetadata ? { userMetadata } : {}),
|
||||
...(fileAttachments && fileAttachments.length > 0 ? { fileAttachments } : {}),
|
||||
...(integrationTools.length > 0 ? { integrationTools } : {}),
|
||||
...(mothershipToolRuntime.tools.length > 0
|
||||
? { mothershipTools: mothershipToolRuntime.tools }
|
||||
: {}),
|
||||
...(userSkillTool ? { mothershipTools: [userSkillTool] } : {}),
|
||||
...(userPermission ? { userPermission } : {}),
|
||||
}
|
||||
|
||||
@@ -159,6 +176,7 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
streamId: messageId,
|
||||
userId,
|
||||
chatId: effectiveChatId,
|
||||
workspaceId,
|
||||
}).catch((error) => {
|
||||
reqLogger.warn('Failed to send explicit abort for mothership execution', {
|
||||
error: toError(error).message,
|
||||
|
||||
@@ -1,91 +0,0 @@
|
||||
import { db, settings, user } from '@sim/db'
|
||||
import { createLogger } from '@sim/logger'
|
||||
import { eq } from 'drizzle-orm'
|
||||
import { type NextRequest, NextResponse } from 'next/server'
|
||||
import {
|
||||
getMothershipSettingsContract,
|
||||
updateMothershipSettingsContract,
|
||||
} from '@/lib/api/contracts/mothership-settings'
|
||||
import { parseRequest } from '@/lib/api/server'
|
||||
import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
|
||||
import { generateRequestId } from '@/lib/core/utils/request'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import {
|
||||
getMothershipSettings,
|
||||
updateMothershipSettings,
|
||||
} from '@/lib/mothership/settings/operations'
|
||||
import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils'
|
||||
|
||||
const logger = createLogger('MothershipSettingsAPI')
|
||||
|
||||
async function isEffectiveSuperUser(userId: string): Promise<boolean> {
|
||||
const [row] = await db
|
||||
.select({
|
||||
role: user.role,
|
||||
superUserModeEnabled: settings.superUserModeEnabled,
|
||||
})
|
||||
.from(user)
|
||||
.leftJoin(settings, eq(settings.userId, user.id))
|
||||
.where(eq(user.id, userId))
|
||||
.limit(1)
|
||||
|
||||
return row?.role === 'admin' && (row.superUserModeEnabled ?? false)
|
||||
}
|
||||
|
||||
export const GET = withRouteHandler(async (request: NextRequest) => {
|
||||
const requestId = generateRequestId()
|
||||
|
||||
try {
|
||||
const auth = await checkSessionOrInternalAuth(request, { requireWorkflowId: false })
|
||||
if (!auth.success || !auth.userId) {
|
||||
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
|
||||
}
|
||||
if (!(await isEffectiveSuperUser(auth.userId))) {
|
||||
return NextResponse.json({ error: 'Super admin mode required' }, { status: 403 })
|
||||
}
|
||||
|
||||
const parsed = await parseRequest(getMothershipSettingsContract, request, {})
|
||||
if (!parsed.success) return parsed.response
|
||||
|
||||
const { workspaceId } = parsed.data.query
|
||||
const userPermission = await getUserEntityPermissions(auth.userId, 'workspace', workspaceId)
|
||||
if (!userPermission) {
|
||||
return NextResponse.json({ error: 'Access denied' }, { status: 403 })
|
||||
}
|
||||
|
||||
const settings = await getMothershipSettings(workspaceId)
|
||||
return NextResponse.json({ data: settings })
|
||||
} catch (error) {
|
||||
logger.error(`[${requestId}] Error fetching Mothership settings`, error)
|
||||
return NextResponse.json({ error: 'Failed to fetch Mothership settings' }, { status: 500 })
|
||||
}
|
||||
})
|
||||
|
||||
export const PUT = withRouteHandler(async (request: NextRequest) => {
|
||||
const requestId = generateRequestId()
|
||||
|
||||
try {
|
||||
const auth = await checkSessionOrInternalAuth(request, { requireWorkflowId: false })
|
||||
if (!auth.success || !auth.userId) {
|
||||
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
|
||||
}
|
||||
if (!(await isEffectiveSuperUser(auth.userId))) {
|
||||
return NextResponse.json({ error: 'Super admin mode required' }, { status: 403 })
|
||||
}
|
||||
|
||||
const parsed = await parseRequest(updateMothershipSettingsContract, request, {})
|
||||
if (!parsed.success) return parsed.response
|
||||
|
||||
const { workspaceId } = parsed.data.body
|
||||
const userPermission = await getUserEntityPermissions(auth.userId, 'workspace', workspaceId)
|
||||
if (!userPermission || (userPermission !== 'admin' && userPermission !== 'write')) {
|
||||
return NextResponse.json({ error: 'Write permission required' }, { status: 403 })
|
||||
}
|
||||
|
||||
const settings = await updateMothershipSettings(parsed.data.body)
|
||||
return NextResponse.json({ success: true, data: settings })
|
||||
} catch (error) {
|
||||
logger.error(`[${requestId}] Error updating Mothership settings`, error)
|
||||
return NextResponse.json({ error: 'Failed to update Mothership settings' }, { status: 500 })
|
||||
}
|
||||
})
|
||||
@@ -0,0 +1,154 @@
|
||||
/**
|
||||
* @vitest-environment node
|
||||
*/
|
||||
import { auditMock, createMockRequest, createSession } from '@sim/testing'
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
|
||||
const {
|
||||
mockGetSession,
|
||||
mockIsOrganizationOwnerOrAdmin,
|
||||
mockGetOrgMemberUsageLimit,
|
||||
mockGetOrgMemberWorkspaceUsage,
|
||||
mockSetOrgMemberUsageLimit,
|
||||
mockFlags,
|
||||
} = vi.hoisted(() => ({
|
||||
mockGetSession: vi.fn(),
|
||||
mockIsOrganizationOwnerOrAdmin: vi.fn(),
|
||||
mockGetOrgMemberUsageLimit: vi.fn(),
|
||||
mockGetOrgMemberWorkspaceUsage: vi.fn(),
|
||||
mockSetOrgMemberUsageLimit: vi.fn(),
|
||||
mockFlags: { isHosted: true },
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/auth', () => ({
|
||||
auth: { api: { getSession: vi.fn() } },
|
||||
getSession: mockGetSession,
|
||||
}))
|
||||
|
||||
vi.mock('@sim/audit', () => auditMock)
|
||||
|
||||
vi.mock('@/lib/billing/core/organization', () => ({
|
||||
isOrganizationOwnerOrAdmin: mockIsOrganizationOwnerOrAdmin,
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/billing/organizations/member-limits', () => ({
|
||||
getOrgMemberUsageLimit: mockGetOrgMemberUsageLimit,
|
||||
getOrgMemberWorkspaceUsage: mockGetOrgMemberWorkspaceUsage,
|
||||
setOrgMemberUsageLimit: mockSetOrgMemberUsageLimit,
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/core/config/feature-flags', () => ({
|
||||
get isHosted() {
|
||||
return mockFlags.isHosted
|
||||
},
|
||||
}))
|
||||
|
||||
import { GET, PUT } from '@/app/api/organizations/[id]/members/[memberId]/usage-limit/route'
|
||||
|
||||
function context() {
|
||||
return { params: Promise.resolve({ id: 'org-1', memberId: 'user-2' }) }
|
||||
}
|
||||
|
||||
function putRequest(body: unknown) {
|
||||
return createMockRequest('PUT', body)
|
||||
}
|
||||
|
||||
function getRequest() {
|
||||
return createMockRequest('GET')
|
||||
}
|
||||
|
||||
describe('GET /api/organizations/[id]/members/[memberId]/usage-limit', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
mockFlags.isHosted = true
|
||||
mockGetSession.mockResolvedValue(createSession({ userId: 'admin-1' }))
|
||||
mockIsOrganizationOwnerOrAdmin.mockResolvedValue(true)
|
||||
mockGetOrgMemberWorkspaceUsage.mockResolvedValue(1) // $1 -> 200 credits
|
||||
mockGetOrgMemberUsageLimit.mockResolvedValue(2) // $2 -> 400 credits
|
||||
})
|
||||
|
||||
it('returns 401 without a session', async () => {
|
||||
mockGetSession.mockResolvedValue(null)
|
||||
const res = await GET(getRequest(), context())
|
||||
expect(res.status).toBe(401)
|
||||
})
|
||||
|
||||
it('returns 404 when not hosted', async () => {
|
||||
mockFlags.isHosted = false
|
||||
const res = await GET(getRequest(), context())
|
||||
expect(res.status).toBe(404)
|
||||
})
|
||||
|
||||
it('returns 403 for non-admin callers', async () => {
|
||||
mockIsOrganizationOwnerOrAdmin.mockResolvedValue(false)
|
||||
const res = await GET(getRequest(), context())
|
||||
expect(res.status).toBe(403)
|
||||
})
|
||||
|
||||
it('returns credits used and limit converted to credits', async () => {
|
||||
const res = await GET(getRequest(), context())
|
||||
expect(res.status).toBe(200)
|
||||
await expect(res.json()).resolves.toEqual({
|
||||
success: true,
|
||||
data: {
|
||||
creditsUsed: 200,
|
||||
creditLimit: 400,
|
||||
},
|
||||
})
|
||||
})
|
||||
|
||||
it('returns null creditLimit when no cap is set', async () => {
|
||||
mockGetOrgMemberUsageLimit.mockResolvedValue(null)
|
||||
const res = await GET(getRequest(), context())
|
||||
const body = await res.json()
|
||||
expect(body.data.creditLimit).toBeNull()
|
||||
})
|
||||
})
|
||||
|
||||
describe('PUT /api/organizations/[id]/members/[memberId]/usage-limit', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
mockFlags.isHosted = true
|
||||
mockGetSession.mockResolvedValue(createSession({ userId: 'admin-1' }))
|
||||
mockIsOrganizationOwnerOrAdmin.mockResolvedValue(true)
|
||||
mockSetOrgMemberUsageLimit.mockResolvedValue(undefined)
|
||||
})
|
||||
|
||||
it('returns 404 when not hosted', async () => {
|
||||
mockFlags.isHosted = false
|
||||
const res = await PUT(putRequest({ creditLimit: 400 }), context())
|
||||
expect(res.status).toBe(404)
|
||||
expect(mockSetOrgMemberUsageLimit).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('returns 403 for non-admin callers', async () => {
|
||||
mockIsOrganizationOwnerOrAdmin.mockResolvedValue(false)
|
||||
const res = await PUT(putRequest({ creditLimit: 400 }), context())
|
||||
expect(res.status).toBe(403)
|
||||
expect(mockSetOrgMemberUsageLimit).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('persists the limit as dollars (credits / 200) and audits', async () => {
|
||||
const res = await PUT(putRequest({ creditLimit: 400 }), context())
|
||||
expect(res.status).toBe(200)
|
||||
expect(mockSetOrgMemberUsageLimit).toHaveBeenCalledWith('org-1', 'user-2', 2, 'admin-1')
|
||||
expect(auditMock.recordAudit).toHaveBeenCalledTimes(1)
|
||||
await expect(res.json()).resolves.toEqual({
|
||||
success: true,
|
||||
message: 'Member credit limit updated successfully',
|
||||
data: { creditLimit: 400 },
|
||||
})
|
||||
})
|
||||
|
||||
it('clears the cap when creditLimit is null', async () => {
|
||||
const res = await PUT(putRequest({ creditLimit: null }), context())
|
||||
expect(res.status).toBe(200)
|
||||
expect(mockSetOrgMemberUsageLimit).toHaveBeenCalledWith('org-1', 'user-2', null, 'admin-1')
|
||||
})
|
||||
|
||||
it('rejects a negative credit limit with 400', async () => {
|
||||
const res = await PUT(putRequest({ creditLimit: -5 }), context())
|
||||
expect(res.status).toBe(400)
|
||||
expect(mockSetOrgMemberUsageLimit).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,130 @@
|
||||
import { AuditAction, AuditResourceType, recordAudit } from '@sim/audit'
|
||||
import { createLogger } from '@sim/logger'
|
||||
import { type NextRequest, NextResponse } from 'next/server'
|
||||
import {
|
||||
getOrganizationMemberUsageLimitContract,
|
||||
updateOrganizationMemberUsageLimitContract,
|
||||
} from '@/lib/api/contracts/organization'
|
||||
import { parseRequest } from '@/lib/api/server'
|
||||
import { getSession } from '@/lib/auth'
|
||||
import { isOrganizationOwnerOrAdmin } from '@/lib/billing/core/organization'
|
||||
import { creditsToDollars, dollarsToCredits } from '@/lib/billing/credits/conversion'
|
||||
import {
|
||||
getOrgMemberUsageLimit,
|
||||
getOrgMemberWorkspaceUsage,
|
||||
setOrgMemberUsageLimit,
|
||||
} from '@/lib/billing/organizations/member-limits'
|
||||
import { isHosted } from '@/lib/core/config/feature-flags'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
|
||||
const logger = createLogger('OrgMemberUsageLimitAPI')
|
||||
|
||||
/**
|
||||
* GET /api/organizations/[id]/members/[memberId]/usage-limit
|
||||
*
|
||||
* Returns the member's current-period credits used inside the org's workspaces
|
||||
* and their per-member credit cap (both in credits). Owner/admin only and
|
||||
* hosted-only (the feature is meaningless where Sim does not own the DB/billing).
|
||||
* `memberId` is the target user id, so external members are supported.
|
||||
*/
|
||||
export const GET = withRouteHandler(
|
||||
async (request: NextRequest, context: { params: Promise<{ id: string; memberId: string }> }) => {
|
||||
const session = await getSession()
|
||||
if (!session?.user?.id) {
|
||||
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
|
||||
}
|
||||
|
||||
if (!isHosted) {
|
||||
return NextResponse.json({ error: 'Not found' }, { status: 404 })
|
||||
}
|
||||
|
||||
const parsed = await parseRequest(getOrganizationMemberUsageLimitContract, request, context)
|
||||
if (!parsed.success) return parsed.response
|
||||
|
||||
const { id: organizationId, memberId } = parsed.data.params
|
||||
|
||||
const hasAccess = await isOrganizationOwnerOrAdmin(session.user.id, organizationId)
|
||||
if (!hasAccess) {
|
||||
return NextResponse.json({ error: 'Forbidden - Admin access required' }, { status: 403 })
|
||||
}
|
||||
|
||||
const [usage, limitDollars] = await Promise.all([
|
||||
getOrgMemberWorkspaceUsage(organizationId, memberId),
|
||||
getOrgMemberUsageLimit(organizationId, memberId),
|
||||
])
|
||||
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
data: {
|
||||
creditsUsed: dollarsToCredits(usage),
|
||||
creditLimit: limitDollars === null ? null : dollarsToCredits(limitDollars),
|
||||
},
|
||||
})
|
||||
}
|
||||
)
|
||||
|
||||
/**
|
||||
* PUT /api/organizations/[id]/members/[memberId]/usage-limit
|
||||
*
|
||||
* Sets (or clears, when `creditLimit` is null) the member's per-org credit cap.
|
||||
* Owner/admin only and hosted-only. The target need not be an org `member` row,
|
||||
* so external members are supported.
|
||||
*/
|
||||
export const PUT = withRouteHandler(
|
||||
async (request: NextRequest, context: { params: Promise<{ id: string; memberId: string }> }) => {
|
||||
const session = await getSession()
|
||||
if (!session?.user?.id) {
|
||||
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
|
||||
}
|
||||
|
||||
if (!isHosted) {
|
||||
return NextResponse.json({ error: 'Not found' }, { status: 404 })
|
||||
}
|
||||
|
||||
const parsed = await parseRequest(updateOrganizationMemberUsageLimitContract, request, context)
|
||||
if (!parsed.success) return parsed.response
|
||||
|
||||
const { id: organizationId, memberId } = parsed.data.params
|
||||
const { creditLimit } = parsed.data.body
|
||||
|
||||
const hasAccess = await isOrganizationOwnerOrAdmin(session.user.id, organizationId)
|
||||
if (!hasAccess) {
|
||||
return NextResponse.json({ error: 'Forbidden - Admin access required' }, { status: 403 })
|
||||
}
|
||||
|
||||
const limitDollars = creditLimit === null ? null : creditsToDollars(creditLimit)
|
||||
await setOrgMemberUsageLimit(organizationId, memberId, limitDollars, session.user.id)
|
||||
|
||||
logger.info('Updated per-member usage limit', {
|
||||
organizationId,
|
||||
memberId,
|
||||
creditLimit,
|
||||
updatedBy: session.user.id,
|
||||
})
|
||||
|
||||
recordAudit({
|
||||
workspaceId: null,
|
||||
actorId: session.user.id,
|
||||
action: AuditAction.ORG_MEMBER_USAGE_LIMIT_CHANGED,
|
||||
resourceType: AuditResourceType.ORGANIZATION,
|
||||
resourceId: organizationId,
|
||||
actorName: session.user.name ?? undefined,
|
||||
actorEmail: session.user.email ?? undefined,
|
||||
description:
|
||||
creditLimit === null
|
||||
? `Cleared credit limit for member ${memberId}`
|
||||
: `Set credit limit for member ${memberId} to ${creditLimit} credits`,
|
||||
metadata: {
|
||||
targetUserId: memberId,
|
||||
creditLimit,
|
||||
},
|
||||
request,
|
||||
})
|
||||
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
message: 'Member credit limit updated successfully',
|
||||
data: { creditLimit },
|
||||
})
|
||||
}
|
||||
)
|
||||
@@ -11,6 +11,7 @@ import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
|
||||
import { generateRequestId } from '@/lib/core/utils/request'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import { captureServerEvent } from '@/lib/posthog/server'
|
||||
import { isBuiltinSkillId } from '@/lib/workflows/skills/builtin-skills'
|
||||
import { deleteSkill, listSkills, upsertSkills } from '@/lib/workflows/skills/operations'
|
||||
import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils'
|
||||
|
||||
@@ -47,8 +48,9 @@ export const GET = withRouteHandler(async (request: NextRequest) => {
|
||||
}
|
||||
|
||||
const result = await listSkills({ workspaceId })
|
||||
const data = result.map((s) => ({ ...s, readOnly: isBuiltinSkillId(s.id) }))
|
||||
|
||||
return NextResponse.json({ data: result }, { status: 200 })
|
||||
return NextResponse.json({ data }, { status: 200 })
|
||||
} catch (error) {
|
||||
logger.error(`[${requestId}] Error fetching skills:`, error)
|
||||
return NextResponse.json({ error: 'Failed to fetch skills' }, { status: 500 })
|
||||
|
||||
@@ -0,0 +1,141 @@
|
||||
/**
|
||||
* @vitest-environment node
|
||||
*/
|
||||
import { createMockRequest } from '@sim/testing'
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
|
||||
const {
|
||||
mockGetSession,
|
||||
mockRecordUsage,
|
||||
mockCheckActorUsageLimits,
|
||||
mockGetWorkspaceBilledAccountUserId,
|
||||
mockVerifyWorkspaceMembership,
|
||||
mockChatRows,
|
||||
} = vi.hoisted(() => ({
|
||||
mockGetSession: vi.fn(),
|
||||
mockRecordUsage: vi.fn(),
|
||||
mockCheckActorUsageLimits: vi.fn(),
|
||||
mockGetWorkspaceBilledAccountUserId: vi.fn(),
|
||||
mockVerifyWorkspaceMembership: vi.fn(),
|
||||
mockChatRows: { value: [] as Array<Record<string, unknown>> },
|
||||
}))
|
||||
|
||||
vi.mock('@sim/db', () => ({
|
||||
db: {
|
||||
select: () => {
|
||||
const chain: Record<string, unknown> = {}
|
||||
chain.from = () => chain
|
||||
chain.leftJoin = () => chain
|
||||
chain.where = () => chain
|
||||
chain.limit = () => Promise.resolve(mockChatRows.value)
|
||||
return chain
|
||||
},
|
||||
},
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/auth', () => ({ getSession: mockGetSession }))
|
||||
|
||||
vi.mock('@/lib/billing/core/usage-log', () => ({ recordUsage: mockRecordUsage }))
|
||||
|
||||
vi.mock('@/lib/billing/calculations/usage-monitor', () => ({
|
||||
checkActorUsageLimits: mockCheckActorUsageLimits,
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/workspaces/utils', () => ({
|
||||
getWorkspaceBilledAccountUserId: mockGetWorkspaceBilledAccountUserId,
|
||||
}))
|
||||
|
||||
vi.mock('@/app/api/workflows/utils', () => ({
|
||||
verifyWorkspaceMembership: mockVerifyWorkspaceMembership,
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/core/config/env', () => ({ env: { ELEVENLABS_API_KEY: 'test-key' } }))
|
||||
|
||||
vi.mock('@/lib/core/config/feature-flags', () => ({
|
||||
isBillingEnabled: false,
|
||||
getCostMultiplier: () => 1,
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/core/rate-limiter', () => ({
|
||||
RateLimiter: class {
|
||||
checkRateLimitDirect = vi.fn().mockResolvedValue({ allowed: true })
|
||||
},
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/core/security/deployment', () => ({ validateAuthToken: vi.fn(() => false) }))
|
||||
|
||||
import { POST } from '@/app/api/speech/token/route'
|
||||
|
||||
const publicChatRow = {
|
||||
id: 'chat-1',
|
||||
userId: 'owner-1',
|
||||
isActive: true,
|
||||
authType: 'public',
|
||||
password: null,
|
||||
workspaceId: 'ws-1',
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
mockChatRows.value = []
|
||||
mockGetSession.mockResolvedValue({ user: { id: 'member-1' } })
|
||||
mockRecordUsage.mockResolvedValue(undefined)
|
||||
mockCheckActorUsageLimits.mockResolvedValue({ isExceeded: false })
|
||||
mockGetWorkspaceBilledAccountUserId.mockResolvedValue('billed-acct')
|
||||
mockVerifyWorkspaceMembership.mockResolvedValue('admin')
|
||||
global.fetch = vi.fn().mockResolvedValue({
|
||||
ok: true,
|
||||
json: async () => ({ token: 'tok-123' }),
|
||||
// double-cast-allowed: minimal fetch stub for the ElevenLabs token call
|
||||
}) as unknown as typeof fetch
|
||||
})
|
||||
|
||||
describe('POST /api/speech/token — usage attribution', () => {
|
||||
it('editor voice: bills the session user and stamps the verified workspace', async () => {
|
||||
const res = await POST(createMockRequest('POST', { workspaceId: 'ws-1' }))
|
||||
|
||||
expect(res.status).toBe(200)
|
||||
expect(mockVerifyWorkspaceMembership).toHaveBeenCalledWith('member-1', 'ws-1')
|
||||
expect(mockRecordUsage).toHaveBeenCalledTimes(1)
|
||||
expect(mockRecordUsage.mock.calls[0][0]).toMatchObject({
|
||||
userId: 'member-1',
|
||||
workspaceId: 'ws-1',
|
||||
})
|
||||
})
|
||||
|
||||
it('editor voice: rejects an unverified workspace id (requires an attributable workspace)', async () => {
|
||||
mockVerifyWorkspaceMembership.mockResolvedValue(null)
|
||||
|
||||
const res = await POST(createMockRequest('POST', { workspaceId: 'ws-not-mine' }))
|
||||
|
||||
expect(res.status).toBe(400)
|
||||
expect(mockRecordUsage).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('deployed chat: bills the workspace billed account and stamps the chat workspace', async () => {
|
||||
mockChatRows.value = [publicChatRow]
|
||||
|
||||
const res = await POST(createMockRequest('POST', { chatId: 'chat-1' }))
|
||||
|
||||
expect(res.status).toBe(200)
|
||||
expect(mockGetSession).not.toHaveBeenCalled()
|
||||
expect(mockGetWorkspaceBilledAccountUserId).toHaveBeenCalledWith('ws-1')
|
||||
expect(mockRecordUsage.mock.calls[0][0]).toMatchObject({
|
||||
userId: 'billed-acct',
|
||||
workspaceId: 'ws-1',
|
||||
})
|
||||
})
|
||||
|
||||
it('deployed chat: falls back to the chat owner when no billed account resolves', async () => {
|
||||
mockChatRows.value = [publicChatRow]
|
||||
mockGetWorkspaceBilledAccountUserId.mockResolvedValue(null)
|
||||
|
||||
const res = await POST(createMockRequest('POST', { chatId: 'chat-1' }))
|
||||
|
||||
expect(res.status).toBe(200)
|
||||
expect(mockRecordUsage.mock.calls[0][0]).toMatchObject({
|
||||
userId: 'owner-1',
|
||||
workspaceId: 'ws-1',
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -1,13 +1,13 @@
|
||||
import { createHash } from 'node:crypto'
|
||||
import { db } from '@sim/db'
|
||||
import { chat } from '@sim/db/schema'
|
||||
import { chat, workflow } from '@sim/db/schema'
|
||||
import { createLogger } from '@sim/logger'
|
||||
import { getErrorMessage } from '@sim/utils/errors'
|
||||
import { eq } from 'drizzle-orm'
|
||||
import { type NextRequest, NextResponse } from 'next/server'
|
||||
import { speechTokenBodySchema } from '@/lib/api/contracts/media/speech'
|
||||
import { getSession } from '@/lib/auth'
|
||||
import { checkServerSideUsageLimits } from '@/lib/billing/calculations/usage-monitor'
|
||||
import { checkActorUsageLimits } from '@/lib/billing/calculations/usage-monitor'
|
||||
import { recordUsage } from '@/lib/billing/core/usage-log'
|
||||
import { env } from '@/lib/core/config/env'
|
||||
import { getCostMultiplier, isBillingEnabled } from '@/lib/core/config/feature-flags'
|
||||
@@ -15,6 +15,8 @@ import { RateLimiter } from '@/lib/core/rate-limiter'
|
||||
import { validateAuthToken } from '@/lib/core/security/deployment'
|
||||
import { getClientIp } from '@/lib/core/utils/request'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import { getWorkspaceBilledAccountUserId } from '@/lib/workspaces/utils'
|
||||
import { verifyWorkspaceMembership } from '@/app/api/workflows/utils'
|
||||
|
||||
const logger = createLogger('SpeechTokenAPI')
|
||||
|
||||
@@ -41,7 +43,7 @@ const rateLimiter = new RateLimiter()
|
||||
async function validateChatAuth(
|
||||
request: NextRequest,
|
||||
chatId: string
|
||||
): Promise<{ valid: boolean; ownerId?: string }> {
|
||||
): Promise<{ valid: boolean; ownerId?: string; workspaceId?: string | null }> {
|
||||
try {
|
||||
const chatResult = await db
|
||||
.select({
|
||||
@@ -50,8 +52,10 @@ async function validateChatAuth(
|
||||
isActive: chat.isActive,
|
||||
authType: chat.authType,
|
||||
password: chat.password,
|
||||
workspaceId: workflow.workspaceId,
|
||||
})
|
||||
.from(chat)
|
||||
.leftJoin(workflow, eq(workflow.id, chat.workflowId))
|
||||
.where(eq(chat.id, chatId))
|
||||
.limit(1)
|
||||
|
||||
@@ -62,13 +66,13 @@ async function validateChatAuth(
|
||||
const chatData = chatResult[0]
|
||||
|
||||
if (chatData.authType === 'public') {
|
||||
return { valid: true, ownerId: chatData.userId }
|
||||
return { valid: true, ownerId: chatData.userId, workspaceId: chatData.workspaceId }
|
||||
}
|
||||
|
||||
const cookieName = `chat_auth_${chatId}`
|
||||
const authCookie = request.cookies.get(cookieName)
|
||||
if (authCookie && validateAuthToken(authCookie.value, chatId, chatData.password)) {
|
||||
return { valid: true, ownerId: chatData.userId }
|
||||
return { valid: true, ownerId: chatData.userId, workspaceId: chatData.workspaceId }
|
||||
}
|
||||
|
||||
return { valid: false }
|
||||
@@ -86,19 +90,43 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
body.success && typeof body.data.chatId === 'string' ? body.data.chatId : undefined
|
||||
|
||||
let billingUserId: string | undefined
|
||||
let workspaceId: string | undefined
|
||||
|
||||
if (chatId) {
|
||||
const chatAuth = await validateChatAuth(request, chatId)
|
||||
if (!chatAuth.valid) {
|
||||
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
|
||||
}
|
||||
billingUserId = chatAuth.ownerId
|
||||
// A deployed chat is used by anonymous end-users, so the cost belongs to the
|
||||
// workspace's billed account (the deployment's payer) — matching how the
|
||||
// chat's workflow execution bills. Fall back to the chat owner only when no
|
||||
// billed account resolves.
|
||||
workspaceId = chatAuth.workspaceId ?? undefined
|
||||
const billedAccountUserId = workspaceId
|
||||
? await getWorkspaceBilledAccountUserId(workspaceId)
|
||||
: null
|
||||
billingUserId = billedAccountUserId ?? chatAuth.ownerId
|
||||
} else {
|
||||
const session = await getSession()
|
||||
if (!session?.user?.id) {
|
||||
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
|
||||
}
|
||||
billingUserId = session.user.id
|
||||
// Editor voice: only attribute to a workspace the caller actually belongs to,
|
||||
// so a client-supplied id can't misattribute (or dodge) per-member usage.
|
||||
const requestedWorkspaceId =
|
||||
body.success && typeof body.data.workspaceId === 'string'
|
||||
? body.data.workspaceId
|
||||
: undefined
|
||||
if (requestedWorkspaceId) {
|
||||
const permission = await verifyWorkspaceMembership(session.user.id, requestedWorkspaceId)
|
||||
if (permission) workspaceId = requestedWorkspaceId
|
||||
}
|
||||
// Editor voice is always workspace-scoped; require an attributable workspace
|
||||
// so per-member usage can't be skipped and the cost stamped workspace-less.
|
||||
if (!workspaceId) {
|
||||
return NextResponse.json({ error: 'Workspace context is required.' }, { status: 400 })
|
||||
}
|
||||
}
|
||||
|
||||
if (isBillingEnabled) {
|
||||
@@ -121,12 +149,13 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
}
|
||||
|
||||
if (billingUserId) {
|
||||
const usageCheck = await checkServerSideUsageLimits(billingUserId)
|
||||
const usageCheck = await checkActorUsageLimits(billingUserId, workspaceId)
|
||||
if (usageCheck.isExceeded) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
error:
|
||||
usageCheck.message || 'Usage limit exceeded. Please upgrade your plan to continue.',
|
||||
scope: usageCheck.scope,
|
||||
},
|
||||
{ status: 402 }
|
||||
)
|
||||
@@ -162,6 +191,7 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
|
||||
await recordUsage({
|
||||
userId: billingUserId,
|
||||
workspaceId,
|
||||
entries: [
|
||||
{
|
||||
category: 'fixed',
|
||||
|
||||
@@ -37,6 +37,7 @@ export const POST = withRouteHandler(async (request: NextRequest, { params }: Ro
|
||||
rowIds,
|
||||
limit,
|
||||
requestId,
|
||||
triggeredByUserId: auth.userId,
|
||||
})
|
||||
|
||||
return NextResponse.json({ success: true, data: { dispatchId } })
|
||||
|
||||
@@ -67,6 +67,7 @@ export const POST = withRouteHandler(async (request: NextRequest, { params }: Ro
|
||||
group: validated.group,
|
||||
outputColumns: validated.outputColumns,
|
||||
autoRun: validated.autoRun,
|
||||
actorUserId: authResult.userId,
|
||||
},
|
||||
requestId
|
||||
)
|
||||
@@ -103,6 +104,7 @@ export const PATCH = withRouteHandler(async (request: NextRequest, { params }: R
|
||||
{
|
||||
tableId,
|
||||
groupId: validated.groupId,
|
||||
actorUserId: authResult.userId,
|
||||
...(validated.workflowId !== undefined ? { workflowId: validated.workflowId } : {}),
|
||||
...(validated.name !== undefined ? { name: validated.name } : {}),
|
||||
...(validated.dependencies !== undefined ? { dependencies: validated.dependencies } : {}),
|
||||
|
||||
@@ -285,7 +285,7 @@ export const POST = withRouteHandler(async (request: NextRequest, { params }: Ro
|
||||
const inserted = insertedRows.length
|
||||
// Fire trigger + scheduler AFTER the tx commits — both read through the
|
||||
// global db connection and would otherwise see no rows.
|
||||
dispatchAfterBatchInsert(finalTable, insertedRows, requestId)
|
||||
dispatchAfterBatchInsert(finalTable, insertedRows, requestId, authResult.userId)
|
||||
|
||||
logger.info(`[${requestId}] Append CSV imported`, {
|
||||
tableId: table.id,
|
||||
|
||||
@@ -129,6 +129,7 @@ export const PATCH = withRouteHandler(async (request: NextRequest, context: RowR
|
||||
rowId,
|
||||
data: validated.data as RowData,
|
||||
workspaceId: validated.workspaceId,
|
||||
actorUserId: authResult.userId,
|
||||
},
|
||||
table,
|
||||
requestId
|
||||
|
||||
@@ -358,6 +358,7 @@ export const PUT = withRouteHandler(
|
||||
filter: validated.filter as Filter,
|
||||
data: validated.data as RowData,
|
||||
limit: validated.limit,
|
||||
actorUserId: authResult.userId,
|
||||
},
|
||||
requestId
|
||||
)
|
||||
@@ -544,6 +545,7 @@ export const PATCH = withRouteHandler(
|
||||
tableId,
|
||||
updates: validated.updates as Array<{ rowId: string; data: RowData }>,
|
||||
workspaceId: validated.workspaceId,
|
||||
actorUserId: authResult.userId,
|
||||
},
|
||||
table,
|
||||
requestId
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { Buffer, isUtf8 } from 'buffer'
|
||||
import { createLogger } from '@sim/logger'
|
||||
import { getErrorMessage } from '@sim/utils/errors'
|
||||
import { generateShortId } from '@sim/utils/id'
|
||||
@@ -7,8 +8,10 @@ import { parseRequest } from '@/lib/api/server'
|
||||
import { checkInternalAuth } from '@/lib/auth/hybrid'
|
||||
import { splitWorkspaceFilePath } from '@/lib/copilot/tools/server/files/workspace-file'
|
||||
import { acquireLock, releaseLock } from '@/lib/core/config/redis'
|
||||
import { generateRequestId } from '@/lib/core/utils/request'
|
||||
import { ensureAbsoluteUrl } from '@/lib/core/utils/urls'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import { isSupportedFileType, parseBuffer } from '@/lib/file-parsers'
|
||||
import { ensureWorkspaceFileFolderPath } from '@/lib/uploads/contexts/workspace/workspace-file-folder-manager'
|
||||
import {
|
||||
fetchWorkspaceFileBuffer,
|
||||
@@ -18,11 +21,14 @@ import {
|
||||
uploadWorkspaceFile,
|
||||
} from '@/lib/uploads/contexts/workspace/workspace-file-manager'
|
||||
import { getFileExtension, getMimeTypeFromExtension } from '@/lib/uploads/utils/file-utils'
|
||||
import { downloadFileFromStorage } from '@/lib/uploads/utils/file-utils.server'
|
||||
import { performMoveWorkspaceFileItems } from '@/lib/workspace-files/orchestration'
|
||||
import {
|
||||
assertActiveWorkspaceAccess,
|
||||
isWorkspaceAccessDeniedError,
|
||||
} from '@/lib/workspaces/permissions/utils'
|
||||
import { assertToolFileAccess } from '@/app/api/files/authorization'
|
||||
import type { UserFile } from '@/executor/types'
|
||||
|
||||
export const dynamic = 'force-dynamic'
|
||||
|
||||
@@ -122,6 +128,46 @@ const extractFileIdsFromInput = (fileInput: unknown): string[] => {
|
||||
.filter((id) => id.length > 0)
|
||||
}
|
||||
|
||||
/** Per-file download cap for the content operation. Aligned with the durable large-value ceiling. */
|
||||
const MAX_GET_CONTENT_FILE_BYTES = 64 * 1024 * 1024
|
||||
/** Combined extracted-text cap so the content array stays within the large-value-ref ceiling. */
|
||||
const MAX_GET_CONTENT_TOTAL_BYTES = 64 * 1024 * 1024
|
||||
|
||||
const isLikelyTextBuffer = (buffer: Buffer): boolean => isUtf8(buffer) && !buffer.includes(0)
|
||||
|
||||
/**
|
||||
* Download a stored file and extract its text content. Parseable types (PDF, DOCX,
|
||||
* CSV, etc.) go through the shared file-parsers; other UTF-8 files are returned as
|
||||
* raw text; binary files yield a short placeholder rather than corrupt bytes.
|
||||
*/
|
||||
const extractUserFileTextContent = async (
|
||||
userFile: UserFile,
|
||||
requestId: string
|
||||
): Promise<string> => {
|
||||
const buffer = await downloadFileFromStorage(userFile, requestId, logger, {
|
||||
maxBytes: MAX_GET_CONTENT_FILE_BYTES,
|
||||
})
|
||||
|
||||
const extension = getFileExtension(userFile.name)
|
||||
if (extension && isSupportedFileType(extension)) {
|
||||
try {
|
||||
const result = await parseBuffer(buffer, extension)
|
||||
return result.content ?? ''
|
||||
} catch (error) {
|
||||
logger.warn('Falling back to raw text after parser failure', {
|
||||
name: userFile.name,
|
||||
error: getErrorMessage(error, 'Unknown error'),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
if (isLikelyTextBuffer(buffer)) {
|
||||
return buffer.toString('utf-8')
|
||||
}
|
||||
|
||||
return `[Binary file: ${userFile.name} (${userFile.type || 'application/octet-stream'}, ${buffer.length} bytes). Cannot extract text content.]`
|
||||
}
|
||||
|
||||
export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
const auth = await checkInternalAuth(request, { requireWorkflowId: false })
|
||||
if (!auth.success) {
|
||||
@@ -231,6 +277,69 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
})
|
||||
}
|
||||
|
||||
case 'content': {
|
||||
const { fileId, fileInput } = body
|
||||
const requestId = generateRequestId()
|
||||
|
||||
const selectedFileIds = Array.isArray(fileId)
|
||||
? fileId.map((id) => id.trim()).filter(Boolean)
|
||||
: fileId
|
||||
? normalizeFileIdList(fileId)
|
||||
: extractFileIdsFromInput(fileInput)
|
||||
const selectedInputFiles = fileId ? [] : extractUserFilesFromInput(fileInput)
|
||||
|
||||
if (selectedFileIds.length === 0 && selectedInputFiles.length === 0) {
|
||||
return NextResponse.json({ success: false, error: 'File is required' }, { status: 400 })
|
||||
}
|
||||
|
||||
const workspaceFiles = await Promise.all(
|
||||
selectedFileIds.map((id) => getWorkspaceFile(workspaceId, id))
|
||||
)
|
||||
const missingFileId = selectedFileIds.find((_, index) => !workspaceFiles[index])
|
||||
if (missingFileId) {
|
||||
return NextResponse.json(
|
||||
{ success: false, error: `File not found: "${missingFileId}"` },
|
||||
{ status: 404 }
|
||||
)
|
||||
}
|
||||
|
||||
const userFiles: UserFile[] = workspaceFiles
|
||||
.map((file) => workspaceFileToUserFile(file))
|
||||
.filter((file): file is NonNullable<ReturnType<typeof workspaceFileToUserFile>> =>
|
||||
Boolean(file)
|
||||
)
|
||||
.concat(selectedInputFiles)
|
||||
|
||||
const contents: string[] = []
|
||||
let totalBytes = 0
|
||||
for (const userFile of userFiles) {
|
||||
const denied = await assertToolFileAccess(userFile.key, userId, requestId, logger)
|
||||
if (denied) return denied
|
||||
|
||||
const content = await extractUserFileTextContent(userFile, requestId)
|
||||
totalBytes += Buffer.byteLength(content, 'utf8')
|
||||
if (totalBytes > MAX_GET_CONTENT_TOTAL_BYTES) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
success: false,
|
||||
error: `Combined file content is too large to return safely. Maximum is ${
|
||||
MAX_GET_CONTENT_TOTAL_BYTES / (1024 * 1024)
|
||||
} MB.`,
|
||||
},
|
||||
{ status: 413 }
|
||||
)
|
||||
}
|
||||
contents.push(content)
|
||||
}
|
||||
|
||||
logger.info('File content extracted', { count: contents.length })
|
||||
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
data: { contents },
|
||||
})
|
||||
}
|
||||
|
||||
case 'write': {
|
||||
const { fileName, content, contentType } = body
|
||||
const { folderSegments, leafName } = splitWorkspaceFilePath(fileName)
|
||||
|
||||
@@ -5,6 +5,7 @@ import {
|
||||
v1UploadKnowledgeDocumentContract,
|
||||
} from '@/lib/api/contracts/v1/knowledge'
|
||||
import { parseRequest } from '@/lib/api/server'
|
||||
import { checkActorUsageLimits } from '@/lib/billing/calculations/usage-monitor'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import {
|
||||
createSingleDocument,
|
||||
@@ -139,6 +140,18 @@ export const POST = withRouteHandler(
|
||||
)
|
||||
if (result instanceof NextResponse) return result
|
||||
|
||||
// Fast usage gate before the storage write + indexing (the async backstop
|
||||
// in processDocumentAsync still covers non-HTTP paths).
|
||||
const usage = await checkActorUsageLimits(userId, workspaceId)
|
||||
if (usage.isExceeded) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
error: usage.message || 'Usage limit exceeded. Please upgrade your plan to continue.',
|
||||
},
|
||||
{ status: 402 }
|
||||
)
|
||||
}
|
||||
|
||||
const buffer = Buffer.from(await file.arrayBuffer())
|
||||
const contentType = file.type || 'application/octet-stream'
|
||||
|
||||
@@ -158,7 +171,8 @@ export const POST = withRouteHandler(
|
||||
mimeType: contentType,
|
||||
},
|
||||
knowledgeBaseId,
|
||||
requestId
|
||||
requestId,
|
||||
userId
|
||||
)
|
||||
|
||||
const documentData: DocumentData = {
|
||||
|
||||
@@ -42,6 +42,10 @@ vi.mock('@/app/api/knowledge/search/utils', () => ({
|
||||
|
||||
vi.mock('@/app/api/knowledge/utils', () => knowledgeApiUtilsMock)
|
||||
|
||||
vi.mock('@/lib/billing/calculations/usage-monitor', () => ({
|
||||
checkActorUsageLimits: vi.fn().mockResolvedValue({ isExceeded: false }),
|
||||
}))
|
||||
|
||||
vi.mock('@/app/api/v1/middleware', () => ({
|
||||
authenticateRequest: mockAuthenticateRequest,
|
||||
validateWorkspaceAccess: mockValidateWorkspaceAccess,
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
import { type NextRequest, NextResponse } from 'next/server'
|
||||
import { v1KnowledgeSearchContract } from '@/lib/api/contracts/v1/knowledge'
|
||||
import { parseRequest } from '@/lib/api/server'
|
||||
import { checkActorUsageLimits } from '@/lib/billing/calculations/usage-monitor'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import { ALL_TAG_SLOTS } from '@/lib/knowledge/constants'
|
||||
import { recordSearchEmbeddingUsage } from '@/lib/knowledge/embeddings'
|
||||
import { getDocumentTagDefinitions } from '@/lib/knowledge/tags/service'
|
||||
import { buildUndefinedTagsError, validateTagValue } from '@/lib/knowledge/tags/utils'
|
||||
import type { StructuredFilter } from '@/lib/knowledge/types'
|
||||
@@ -37,6 +39,18 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
const accessError = await validateWorkspaceAccess(rateLimit, userId, workspaceId)
|
||||
if (accessError) return accessError
|
||||
|
||||
// A query incurs hosted embedding (+ optional rerank) cost — gate the actor's
|
||||
// usage and frozen status before spending. Tag-only search is free, so skip it.
|
||||
if (query && query.trim().length > 0) {
|
||||
const usage = await checkActorUsageLimits(userId, workspaceId)
|
||||
if (usage.isExceeded) {
|
||||
return NextResponse.json(
|
||||
{ error: usage.message || 'Usage limit exceeded. Please upgrade your plan to continue.' },
|
||||
{ status: 402 }
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
const knowledgeBaseIds = Array.isArray(parsed.data.body.knowledgeBaseIds)
|
||||
? parsed.data.body.knowledgeBaseIds
|
||||
: [parsed.data.body.knowledgeBaseIds]
|
||||
@@ -148,6 +162,7 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
const queryEmbeddingModel = embeddingModels[0]
|
||||
|
||||
let results: SearchResult[]
|
||||
let queryEmbeddingIsBYOK: boolean | null = null
|
||||
|
||||
if (!hasQuery && hasFilters) {
|
||||
results = await handleTagOnlySearch({
|
||||
@@ -157,9 +172,13 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
})
|
||||
} else if (hasQuery && hasFilters) {
|
||||
const strategy = getQueryStrategy(accessibleKbIds.length, topK)
|
||||
const queryVector = JSON.stringify(
|
||||
await generateSearchEmbedding(query!, queryEmbeddingModel, workspaceId)
|
||||
const queryEmbeddingResult = await generateSearchEmbedding(
|
||||
query!,
|
||||
queryEmbeddingModel,
|
||||
workspaceId
|
||||
)
|
||||
queryEmbeddingIsBYOK = queryEmbeddingResult.isBYOK
|
||||
const queryVector = JSON.stringify(queryEmbeddingResult.embedding)
|
||||
results = await handleTagAndVectorSearch({
|
||||
knowledgeBaseIds: accessibleKbIds,
|
||||
topK,
|
||||
@@ -169,9 +188,13 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
})
|
||||
} else if (hasQuery) {
|
||||
const strategy = getQueryStrategy(accessibleKbIds.length, topK)
|
||||
const queryVector = JSON.stringify(
|
||||
await generateSearchEmbedding(query!, queryEmbeddingModel, workspaceId)
|
||||
const queryEmbeddingResult = await generateSearchEmbedding(
|
||||
query!,
|
||||
queryEmbeddingModel,
|
||||
workspaceId
|
||||
)
|
||||
queryEmbeddingIsBYOK = queryEmbeddingResult.isBYOK
|
||||
const queryVector = JSON.stringify(queryEmbeddingResult.embedding)
|
||||
results = await handleVectorOnlySearch({
|
||||
knowledgeBaseIds: accessibleKbIds,
|
||||
topK,
|
||||
@@ -185,6 +208,17 @@ export const POST = withRouteHandler(async (request: NextRequest) => {
|
||||
)
|
||||
}
|
||||
|
||||
if (queryEmbeddingIsBYOK !== null) {
|
||||
await recordSearchEmbeddingUsage({
|
||||
userId,
|
||||
workspaceId,
|
||||
embeddingModel: queryEmbeddingModel,
|
||||
query: query!,
|
||||
isBYOK: queryEmbeddingIsBYOK,
|
||||
sourceReference: `v1-kb-search:${requestId}`,
|
||||
})
|
||||
}
|
||||
|
||||
const tagDefsResults = await Promise.all(
|
||||
accessibleKbIds.map(async (kbId) => {
|
||||
try {
|
||||
|
||||
@@ -144,6 +144,7 @@ export const PATCH = withRouteHandler(async (request: NextRequest, context: RowR
|
||||
rowId,
|
||||
data: rowDataNameToId(validated.data as RowData, idByName),
|
||||
workspaceId: validated.workspaceId,
|
||||
actorUserId: userId,
|
||||
},
|
||||
table,
|
||||
requestId
|
||||
|
||||
@@ -350,6 +350,7 @@ export const PUT = withRouteHandler(async (request: NextRequest, context: TableR
|
||||
filter: filterNamesToIds(validated.filter as Filter, idByName),
|
||||
data: patchData,
|
||||
limit: validated.limit,
|
||||
actorUserId: userId,
|
||||
},
|
||||
requestId
|
||||
)
|
||||
|
||||
@@ -7,6 +7,10 @@ import { wandGenerateContract } from '@/lib/api/contracts'
|
||||
import { parseRequest } from '@/lib/api/server'
|
||||
import { getBYOKKey } from '@/lib/api-key/byok'
|
||||
import { getSession } from '@/lib/auth'
|
||||
import {
|
||||
checkActorUsageLimits,
|
||||
checkBillingBlocked,
|
||||
} from '@/lib/billing/calculations/usage-monitor'
|
||||
import { recordUsage } from '@/lib/billing/core/usage-log'
|
||||
import { checkAndBillOverageThreshold } from '@/lib/billing/threshold-billing'
|
||||
import { env } from '@/lib/core/config/env'
|
||||
@@ -14,7 +18,6 @@ import { getCostMultiplier, isBillingEnabled } from '@/lib/core/config/feature-f
|
||||
import { generateRequestId } from '@/lib/core/utils/request'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import { enrichTableSchema } from '@/lib/table/llm/wand'
|
||||
import { getWorkspaceBilledAccountUserId } from '@/lib/workspaces/utils'
|
||||
import { verifyWorkspaceMembership } from '@/app/api/workflows/utils'
|
||||
import { extractResponseText, parseResponsesUsage } from '@/providers/openai/utils'
|
||||
import { getModelPricing } from '@/providers/utils'
|
||||
@@ -170,6 +173,7 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
stream = false,
|
||||
history = [],
|
||||
workflowId,
|
||||
workspaceId: requestedWorkspaceId,
|
||||
generationType,
|
||||
wandContext = {},
|
||||
} = body
|
||||
@@ -221,23 +225,33 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
{ status: 403 }
|
||||
)
|
||||
}
|
||||
} else if (requestedWorkspaceId) {
|
||||
// No workflow entity to resolve from (e.g. table-schema wand or an
|
||||
// unhydrated editor); attribute to the workspace the wand is running in,
|
||||
// but only when the caller is a member so usage can't be misattributed.
|
||||
const permission = await verifyWorkspaceMembership(session.user.id, requestedWorkspaceId)
|
||||
if (permission) {
|
||||
workspaceId = requestedWorkspaceId
|
||||
}
|
||||
}
|
||||
|
||||
let billingUserId = session.user.id
|
||||
if (workspaceId) {
|
||||
const workspaceBilledAccountUserId = await getWorkspaceBilledAccountUserId(workspaceId)
|
||||
if (!workspaceBilledAccountUserId) {
|
||||
logger.error(`[${requestId}] Unable to resolve billed account for workspace`, {
|
||||
workspaceId,
|
||||
})
|
||||
return NextResponse.json(
|
||||
{ success: false, error: 'Unable to resolve billing account for this workspace' },
|
||||
{ status: 500 }
|
||||
)
|
||||
}
|
||||
billingUserId = workspaceBilledAccountUserId
|
||||
// Per-member usage must be attributable to an org workspace. The editor always
|
||||
// supplies a workflow or a workspace the caller belongs to; refuse to run rather
|
||||
// than stamp usage workspace-less (which would silently skip the per-member cap).
|
||||
if (!workspaceId) {
|
||||
return NextResponse.json(
|
||||
{ success: false, error: 'Workspace context is required.' },
|
||||
{ status: 400 }
|
||||
)
|
||||
}
|
||||
|
||||
// Wand is always an interactive, session-authenticated editor action, so the
|
||||
// person using it is the billing actor — matching client-side executions and
|
||||
// editor voice rather than the workspace billed account. deriveBillingContext
|
||||
// still routes payment to the org for org-scoped members; per-member usage is
|
||||
// attributed to the member who actually used the wand.
|
||||
const billingUserId = session.user.id
|
||||
|
||||
let isBYOK = false
|
||||
let activeOpenAIKey = openaiApiKey
|
||||
|
||||
@@ -258,6 +272,35 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
)
|
||||
}
|
||||
|
||||
// BYOK incurs no Sim-metered cost, so it skips usage gating — but a frozen /
|
||||
// billing-blocked account is locked out of everything, so still check that.
|
||||
// Non-BYOK runs the full actor gate, which already includes the billing-blocked
|
||||
// check, so it isn't repeated here.
|
||||
if (isBYOK) {
|
||||
const blocked = await checkBillingBlocked(billingUserId)
|
||||
if (blocked.blocked) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
success: false,
|
||||
error: blocked.message || 'Account is not in good standing. Please contact support.',
|
||||
},
|
||||
{ status: 402 }
|
||||
)
|
||||
}
|
||||
} else {
|
||||
const usage = await checkActorUsageLimits(billingUserId, workspaceId)
|
||||
if (usage.isExceeded) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
success: false,
|
||||
error: usage.message || 'Usage limit exceeded. Please upgrade your plan to continue.',
|
||||
scope: usage.scope,
|
||||
},
|
||||
{ status: 402 }
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
let finalSystemPrompt =
|
||||
systemPrompt ||
|
||||
'You are a helpful AI assistant. Generate content exactly as requested by the user.'
|
||||
|
||||
@@ -8,6 +8,7 @@ import { generateRequestId } from '@/lib/core/utils/request'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import { captureServerEvent } from '@/lib/posthog/server'
|
||||
import { performActivateVersion } from '@/lib/workflows/orchestration'
|
||||
import { updateDeploymentVersionMetadata } from '@/lib/workflows/persistence/utils'
|
||||
import { validateWorkflowPermissions } from '@/lib/workflows/utils'
|
||||
import { createErrorResponse, createSuccessResponse } from '@/app/api/workflows/utils'
|
||||
|
||||
@@ -173,37 +174,21 @@ export const PATCH = withRouteHandler(
|
||||
})
|
||||
}
|
||||
|
||||
// Handle name/description updates
|
||||
const updateData: { name?: string; description?: string | null } = {}
|
||||
if (name !== undefined) {
|
||||
updateData.name = name
|
||||
}
|
||||
if (description !== undefined) {
|
||||
updateData.description = description
|
||||
}
|
||||
|
||||
const [updated] = await db
|
||||
.update(workflowDeploymentVersion)
|
||||
.set(updateData)
|
||||
.where(
|
||||
and(
|
||||
eq(workflowDeploymentVersion.workflowId, id),
|
||||
eq(workflowDeploymentVersion.version, versionNum)
|
||||
)
|
||||
)
|
||||
.returning({
|
||||
id: workflowDeploymentVersion.id,
|
||||
name: workflowDeploymentVersion.name,
|
||||
description: workflowDeploymentVersion.description,
|
||||
})
|
||||
// Handle name/description updates (shared with the update_deployment_version copilot tool)
|
||||
const updated = await updateDeploymentVersionMetadata({
|
||||
workflowId: id,
|
||||
version: versionNum,
|
||||
name,
|
||||
description,
|
||||
})
|
||||
|
||||
if (!updated) {
|
||||
return createErrorResponse('Deployment version not found', 404)
|
||||
}
|
||||
|
||||
logger.info(`[${requestId}] Updated deployment version ${version} for workflow ${id}`, {
|
||||
name: updateData.name,
|
||||
description: updateData.description,
|
||||
name,
|
||||
description,
|
||||
})
|
||||
|
||||
return createSuccessResponse({ name: updated.name, description: updated.description })
|
||||
|
||||
@@ -827,7 +827,10 @@ async function handleExecutePost(
|
||||
}
|
||||
|
||||
const effectiveWorkflowStateOverride =
|
||||
sanitizedWorkflowStateOverride || cachedWorkflowData || undefined
|
||||
// double-cast-allowed: workflowStateSchema is structurally a supertype of the executor's reactflow-typed override (edges[].style is Record<string, unknown> vs CSSProperties); validated bodies carry store-shaped values so the runtime shape matches
|
||||
(sanitizedWorkflowStateOverride as unknown as ExecutionMetadata['workflowStateOverride']) ||
|
||||
cachedWorkflowData ||
|
||||
undefined
|
||||
const largeValueExecutionIds = [executionId]
|
||||
const largeValueKeys: string[] = []
|
||||
const fileKeys: string[] = []
|
||||
|
||||
@@ -20,6 +20,7 @@ import {
|
||||
} from '@sim/testing'
|
||||
import { NextRequest } from 'next/server'
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { getWorkflowResponseDataSchema } from '@/lib/api/contracts/workflows'
|
||||
|
||||
const mockLoadWorkflowFromNormalizedTables =
|
||||
workflowsPersistenceUtilsMockFns.mockLoadWorkflowFromNormalizedTables
|
||||
@@ -182,6 +183,55 @@ describe('Workflow By ID API Route', () => {
|
||||
expect(data.data.id).toBe('workflow-123')
|
||||
})
|
||||
|
||||
it('omits null workflow description from state metadata so response validates', async () => {
|
||||
const mockWorkflow = {
|
||||
id: 'workflow-null-description',
|
||||
userId: 'user-123',
|
||||
name: 'No Description Workflow',
|
||||
description: null,
|
||||
workspaceId: 'workspace-456',
|
||||
folderId: null,
|
||||
sortOrder: 0,
|
||||
color: '#3972F6',
|
||||
lastSynced: new Date(),
|
||||
createdAt: new Date(),
|
||||
updatedAt: new Date(),
|
||||
isDeployed: false,
|
||||
deployedAt: null,
|
||||
isPublicApi: false,
|
||||
locked: false,
|
||||
runCount: 0,
|
||||
lastRunAt: null,
|
||||
archivedAt: null,
|
||||
variables: {},
|
||||
}
|
||||
|
||||
mockGetSession({ user: { id: 'user-123' } })
|
||||
mockGetWorkflowById.mockResolvedValue(mockWorkflow)
|
||||
mockAuthorizeWorkflowByWorkspacePermission.mockResolvedValue({
|
||||
allowed: true,
|
||||
status: 200,
|
||||
workflow: mockWorkflow,
|
||||
workspacePermission: 'admin',
|
||||
})
|
||||
mockLoadWorkflowFromNormalizedTables.mockResolvedValue({
|
||||
blocks: {},
|
||||
edges: [],
|
||||
loops: {},
|
||||
parallels: {},
|
||||
})
|
||||
|
||||
const req = new NextRequest('http://localhost:3000/api/workflows/workflow-null-description')
|
||||
const params = Promise.resolve({ id: 'workflow-null-description' })
|
||||
|
||||
const response = await GET(req, { params })
|
||||
const data = await response.json()
|
||||
|
||||
expect(response.status).toBe(200)
|
||||
expect(data.data.state.metadata).toEqual({ name: 'No Description Workflow' })
|
||||
expect(getWorkflowResponseDataSchema.safeParse(data.data).success).toBe(true)
|
||||
})
|
||||
|
||||
it.concurrent('should allow access when user has workspace permissions', async () => {
|
||||
const mockWorkflow = {
|
||||
id: 'workflow-123',
|
||||
|
||||
@@ -108,6 +108,12 @@ export const GET = withRouteHandler(
|
||||
stampedVariables[variableId] = { ...variable, workflowId }
|
||||
}
|
||||
}
|
||||
const workflowStateMetadata = {
|
||||
name: responseWorkflowData.name,
|
||||
...(typeof responseWorkflowData.description === 'string'
|
||||
? { description: responseWorkflowData.description }
|
||||
: {}),
|
||||
}
|
||||
|
||||
if (snapshot.normalizedData) {
|
||||
const finalWorkflowData = {
|
||||
@@ -120,10 +126,7 @@ export const GET = withRouteHandler(
|
||||
lastSaved: Date.now(),
|
||||
isDeployed: responseWorkflowData.isDeployed || false,
|
||||
deployedAt: responseWorkflowData.deployedAt,
|
||||
metadata: {
|
||||
name: responseWorkflowData.name,
|
||||
description: responseWorkflowData.description,
|
||||
},
|
||||
metadata: workflowStateMetadata,
|
||||
},
|
||||
variables: stampedVariables,
|
||||
}
|
||||
@@ -145,10 +148,7 @@ export const GET = withRouteHandler(
|
||||
lastSaved: Date.now(),
|
||||
isDeployed: responseWorkflowData.isDeployed || false,
|
||||
deployedAt: responseWorkflowData.deployedAt,
|
||||
metadata: {
|
||||
name: responseWorkflowData.name,
|
||||
description: responseWorkflowData.description,
|
||||
},
|
||||
metadata: workflowStateMetadata,
|
||||
},
|
||||
variables: stampedVariables,
|
||||
}
|
||||
|
||||
@@ -31,6 +31,19 @@ export function createSuccessResponse(data: any) {
|
||||
* This is the single source of truth for redeployment detection — used by
|
||||
* both the /deploy and /status endpoints to ensure consistent results.
|
||||
*/
|
||||
/**
|
||||
* Pure redeployment-change comparison shared by checkNeedsRedeployment and the
|
||||
* VFS deployment serializer so both surfaces agree. Returns false when either
|
||||
* side is missing.
|
||||
*/
|
||||
export function computeNeedsRedeployment(
|
||||
currentSnapshot: WorkflowState | null | undefined,
|
||||
activeState: WorkflowState | null | undefined
|
||||
): boolean {
|
||||
if (!activeState || !currentSnapshot) return false
|
||||
return hasWorkflowChanged(currentSnapshot, activeState)
|
||||
}
|
||||
|
||||
export async function checkNeedsRedeployment(workflowId: string): Promise<boolean> {
|
||||
const [active] = await db
|
||||
.select({ state: workflowDeploymentVersion.state })
|
||||
@@ -44,12 +57,8 @@ export async function checkNeedsRedeployment(workflowId: string): Promise<boolea
|
||||
.orderBy(desc(workflowDeploymentVersion.createdAt))
|
||||
.limit(1)
|
||||
|
||||
if (!active?.state) return false
|
||||
|
||||
const currentState = await loadWorkflowDeploymentSnapshot(workflowId)
|
||||
if (!currentState) return false
|
||||
|
||||
return hasWorkflowChanged(currentState, active.state as WorkflowState)
|
||||
return computeNeedsRedeployment(currentState, (active?.state as WorkflowState) ?? null)
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -1,120 +0,0 @@
|
||||
import { createLogger } from '@sim/logger'
|
||||
import { toError } from '@sim/utils/errors'
|
||||
import { type NextRequest, NextResponse } from 'next/server'
|
||||
import type { z } from 'zod'
|
||||
import { defineRouteContract } from '@/lib/api/contracts'
|
||||
import { getValidationErrorMessage, parseRequest } from '@/lib/api/server'
|
||||
import { getSession } from '@/lib/auth'
|
||||
import { MAX_DOCUMENT_PREVIEW_CODE_BYTES } from '@/lib/execution/constants'
|
||||
import { runSandboxTask, SandboxUserCodeError } from '@/lib/execution/sandbox/run-task'
|
||||
import { verifyWorkspaceMembership } from '@/app/api/workflows/utils'
|
||||
import type { SandboxTaskId } from '@/sandbox-tasks/registry'
|
||||
|
||||
/**
|
||||
* Config for a document preview route handler.
|
||||
*
|
||||
* All three document preview endpoints (PDF / PPTX / DOCX) share the same
|
||||
* shape: auth → workspace membership → JSON body parse → `code` validation →
|
||||
* size guard → `runSandboxTask(taskId, ...)` → binary response. The only
|
||||
* differences between them are the sandbox task, the response MIME type, and
|
||||
* the logger/label used for the 500 path.
|
||||
*/
|
||||
export interface DocumentPreviewRouteConfig {
|
||||
/** Sandbox task registered in `apps/sim/sandbox-tasks/registry.ts`. */
|
||||
taskId: SandboxTaskId
|
||||
/** Content-Type of the binary returned on success. */
|
||||
contentType: string
|
||||
/** Short label used for the logger name + 500 log message. */
|
||||
label: 'PDF' | 'PPTX' | 'DOCX'
|
||||
/** Route params schema owned by the concrete route.ts boundary. */
|
||||
routeParamsSchema: z.ZodType<{ id: string }>
|
||||
/** JSON body schema owned by the concrete route.ts boundary. */
|
||||
previewBodySchema: z.ZodType<{ code: string }>
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a Next.js POST handler for one of the document preview endpoints.
|
||||
*
|
||||
* Everything security-relevant (session, workspace membership, JSON shape,
|
||||
* empty/oversized code) is enforced before we ever reach the isolated-vm
|
||||
* sandbox, and `runSandboxTask` is always invoked with the session owner key
|
||||
* + `req.signal` so pool fairness and client-disconnect cancellation behave
|
||||
* identically across all three formats.
|
||||
*/
|
||||
export function createDocumentPreviewRoute(config: DocumentPreviewRouteConfig) {
|
||||
const logger = createLogger(`${config.label}PreviewAPI`)
|
||||
|
||||
const previewContract = defineRouteContract({
|
||||
method: 'POST',
|
||||
path: '/api/workspaces/[id]/_preview',
|
||||
params: config.routeParamsSchema,
|
||||
body: config.previewBodySchema,
|
||||
response: { mode: 'json', schema: config.previewBodySchema },
|
||||
})
|
||||
|
||||
return async function POST(req: NextRequest, context: { params: Promise<{ id: string }> }) {
|
||||
const paramsResult = config.routeParamsSchema.safeParse(await context.params)
|
||||
if (!paramsResult.success) {
|
||||
return NextResponse.json(
|
||||
{ error: getValidationErrorMessage(paramsResult.error, 'Invalid route parameters') },
|
||||
{ status: 400 }
|
||||
)
|
||||
}
|
||||
const { id: workspaceId } = paramsResult.data
|
||||
|
||||
try {
|
||||
const session = await getSession()
|
||||
if (!session?.user?.id) {
|
||||
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
|
||||
}
|
||||
|
||||
const membership = await verifyWorkspaceMembership(session.user.id, workspaceId)
|
||||
if (!membership) {
|
||||
return NextResponse.json({ error: 'Insufficient permissions' }, { status: 403 })
|
||||
}
|
||||
|
||||
const parsed = await parseRequest(previewContract, req, context, {
|
||||
validationErrorResponse: (error) =>
|
||||
NextResponse.json(
|
||||
{ error: getValidationErrorMessage(error, 'code is required') },
|
||||
{ status: 400 }
|
||||
),
|
||||
invalidJsonResponse: () =>
|
||||
NextResponse.json({ error: 'Invalid or missing JSON body' }, { status: 400 }),
|
||||
})
|
||||
if (!parsed.success) return parsed.response
|
||||
const { code } = parsed.data.body
|
||||
|
||||
if (Buffer.byteLength(code, 'utf-8') > MAX_DOCUMENT_PREVIEW_CODE_BYTES) {
|
||||
return NextResponse.json({ error: 'code exceeds maximum size' }, { status: 413 })
|
||||
}
|
||||
|
||||
const buffer = await runSandboxTask(
|
||||
config.taskId,
|
||||
{ code, workspaceId },
|
||||
{ ownerKey: `user:${session.user.id}`, signal: req.signal }
|
||||
)
|
||||
|
||||
return new NextResponse(new Uint8Array(buffer), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': config.contentType,
|
||||
'Content-Length': String(buffer.length),
|
||||
'Cache-Control': 'private, no-store',
|
||||
},
|
||||
})
|
||||
} catch (err) {
|
||||
const message = toError(err).message
|
||||
if (err instanceof SandboxUserCodeError) {
|
||||
logger.warn(`${config.label} preview user code failed`, {
|
||||
error: message,
|
||||
errorName: err.name,
|
||||
workspaceId,
|
||||
})
|
||||
return NextResponse.json({ error: message, errorName: err.name }, { status: 422 })
|
||||
}
|
||||
logger.error(`${config.label} preview generation failed`, { error: message, workspaceId })
|
||||
return NextResponse.json({ error: message }, { status: 500 })
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,226 +0,0 @@
|
||||
/**
|
||||
* @vitest-environment node
|
||||
*/
|
||||
import { authMockFns, workflowsApiUtilsMock, workflowsApiUtilsMockFns } from '@sim/testing'
|
||||
import { NextRequest } from 'next/server'
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { MAX_DOCUMENT_PREVIEW_CODE_BYTES } from '@/lib/execution/constants'
|
||||
|
||||
const { mockRunSandboxTask, SandboxUserCodeError } = vi.hoisted(() => {
|
||||
class SandboxUserCodeError extends Error {
|
||||
constructor(message: string, name: string) {
|
||||
super(message)
|
||||
this.name = name
|
||||
}
|
||||
}
|
||||
return { mockRunSandboxTask: vi.fn(), SandboxUserCodeError }
|
||||
})
|
||||
|
||||
const mockVerifyWorkspaceMembership = workflowsApiUtilsMockFns.mockVerifyWorkspaceMembership
|
||||
|
||||
vi.mock('@/app/api/workflows/utils', () => workflowsApiUtilsMock)
|
||||
|
||||
vi.mock('@/lib/execution/sandbox/run-task', () => ({
|
||||
runSandboxTask: mockRunSandboxTask,
|
||||
SandboxUserCodeError,
|
||||
}))
|
||||
|
||||
import { POST } from '@/app/api/workspaces/[id]/docx/preview/route'
|
||||
|
||||
const DOCX_MIME = 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'
|
||||
|
||||
describe('DOCX preview API route', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
authMockFns.mockGetSession.mockResolvedValue({ user: { id: 'user-1' } })
|
||||
mockVerifyWorkspaceMembership.mockResolvedValue(true)
|
||||
mockRunSandboxTask.mockResolvedValue(Buffer.from('PK\x03\x04docx'))
|
||||
})
|
||||
|
||||
it('returns a generated DOCX for authorized workspace members', async () => {
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/docx/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ code: 'return 1' }),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(200)
|
||||
expect(response.headers.get('Content-Type')).toBe(DOCX_MIME)
|
||||
expect(response.headers.get('Cache-Control')).toBe('private, no-store')
|
||||
expect(mockVerifyWorkspaceMembership).toHaveBeenCalledWith('user-1', 'workspace-1')
|
||||
expect(mockRunSandboxTask).toHaveBeenCalledWith(
|
||||
'docx-generate',
|
||||
{ code: 'return 1', workspaceId: 'workspace-1' },
|
||||
{ ownerKey: 'user:user-1', signal: request.signal }
|
||||
)
|
||||
expect(Buffer.from(await response.arrayBuffer()).toString()).toBe('PK\x03\x04docx')
|
||||
})
|
||||
|
||||
it('rejects requests without code', async () => {
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/docx/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({}),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(400)
|
||||
await expect(response.json()).resolves.toEqual({ error: 'code is required' })
|
||||
expect(mockRunSandboxTask).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('rejects oversized preview source payloads', async () => {
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/docx/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ code: 'x'.repeat(MAX_DOCUMENT_PREVIEW_CODE_BYTES + 1) }),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(413)
|
||||
await expect(response.json()).resolves.toEqual({ error: 'code exceeds maximum size' })
|
||||
expect(mockRunSandboxTask).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('returns 401 for unauthenticated requests', async () => {
|
||||
authMockFns.mockGetSession.mockResolvedValue(null)
|
||||
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/docx/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ code: 'return 1' }),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(401)
|
||||
await expect(response.json()).resolves.toEqual({ error: 'Unauthorized' })
|
||||
expect(mockVerifyWorkspaceMembership).not.toHaveBeenCalled()
|
||||
expect(mockRunSandboxTask).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('returns 403 when the user is not a workspace member', async () => {
|
||||
mockVerifyWorkspaceMembership.mockResolvedValue(false)
|
||||
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/docx/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ code: 'return 1' }),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(403)
|
||||
await expect(response.json()).resolves.toEqual({ error: 'Insufficient permissions' })
|
||||
expect(mockRunSandboxTask).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('returns 400 for requests with invalid JSON bodies', async () => {
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/docx/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: '{ not valid json',
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(400)
|
||||
await expect(response.json()).resolves.toEqual({ error: 'Invalid or missing JSON body' })
|
||||
expect(mockRunSandboxTask).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('returns 500 when DOCX generation throws', async () => {
|
||||
mockRunSandboxTask.mockRejectedValue(new Error('boom: sandbox failed'))
|
||||
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/docx/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ code: 'return 1' }),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(500)
|
||||
await expect(response.json()).resolves.toEqual({ error: 'boom: sandbox failed' })
|
||||
})
|
||||
|
||||
it('returns 422 when user code throws inside the sandbox', async () => {
|
||||
mockRunSandboxTask.mockRejectedValue(
|
||||
new SandboxUserCodeError('Invalid or unexpected token', 'SyntaxError')
|
||||
)
|
||||
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/docx/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ code: 'const x = ' }),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(422)
|
||||
await expect(response.json()).resolves.toEqual({
|
||||
error: 'Invalid or unexpected token',
|
||||
errorName: 'SyntaxError',
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -1,20 +0,0 @@
|
||||
import { workspaceParamsSchema, workspacePreviewBodySchema } from '@/lib/api/contracts/workspaces'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import { createDocumentPreviewRoute } from '@/app/api/workspaces/[id]/_preview/create-preview-route'
|
||||
|
||||
export const dynamic = 'force-dynamic'
|
||||
export const runtime = 'nodejs'
|
||||
|
||||
/**
|
||||
* POST /api/workspaces/[id]/docx/preview
|
||||
* Compile docx source code and return the binary DOCX for streaming preview.
|
||||
*/
|
||||
export const POST = withRouteHandler(
|
||||
createDocumentPreviewRoute({
|
||||
taskId: 'docx-generate',
|
||||
contentType: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
label: 'DOCX',
|
||||
routeParamsSchema: workspaceParamsSchema,
|
||||
previewBodySchema: workspacePreviewBodySchema,
|
||||
})
|
||||
)
|
||||
@@ -4,6 +4,9 @@ import { type NextRequest, NextResponse } from 'next/server'
|
||||
import { workspaceFileCompiledCheckContract } from '@/lib/api/contracts/workspace-files'
|
||||
import { parseRequest } from '@/lib/api/server'
|
||||
import { getSession } from '@/lib/auth'
|
||||
import { getE2BDocFormat } from '@/lib/copilot/tools/server/files/doc-compile'
|
||||
import { runE2BCompiledCheck } from '@/lib/copilot/tools/server/files/doc-recalc'
|
||||
import { isE2BDocEnabled } from '@/lib/core/config/feature-flags'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import { BINARY_DOC_TASKS, MAX_DOCUMENT_PREVIEW_CODE_BYTES } from '@/lib/execution/constants'
|
||||
import { runSandboxTask, SandboxUserCodeError } from '@/lib/execution/sandbox/run-task'
|
||||
@@ -51,11 +54,15 @@ export const GET = withRouteHandler(
|
||||
}
|
||||
|
||||
const ext = fileRecord.name.split('.').pop()?.toLowerCase() ?? ''
|
||||
// In the E2B regime ALL four formats compile in the doc sandbox (Node for
|
||||
// pptx/docx, Python for pdf/xlsx). Gate on the flag (not the stored MIME) so
|
||||
// a stale file can't trigger an E2B compile when the sandbox is disabled.
|
||||
const e2bFmt = isE2BDocEnabled ? getE2BDocFormat(fileRecord.name) : null
|
||||
const taskId = BINARY_DOC_TASKS[ext]
|
||||
const isMermaidFile = ext === 'mmd' || ext === 'mermaid'
|
||||
if (!taskId && !isMermaidFile) {
|
||||
if (!e2bFmt && !taskId && !isMermaidFile) {
|
||||
return NextResponse.json(
|
||||
{ error: `Compiled check only supports .docx, .pptx, .pdf, and .mmd files` },
|
||||
{ error: `Compiled check only supports .docx, .pptx, .pdf, .xlsx, and .mmd files` },
|
||||
{ status: 422 }
|
||||
)
|
||||
}
|
||||
@@ -81,6 +88,20 @@ export const GET = withRouteHandler(
|
||||
return NextResponse.json(await validateMermaidSource(code))
|
||||
}
|
||||
|
||||
if (e2bFmt) {
|
||||
// Loads the compile-once artifact if present, else compiles via E2B once
|
||||
// (and recalc-scans xlsx formulas). Only a script error is { ok: false };
|
||||
// infra failures rethrow → 500, so the agent isn't told to "fix its script"
|
||||
// during an E2B/S3 outage.
|
||||
const result = await runE2BCompiledCheck({
|
||||
source: code,
|
||||
fileName: fileRecord.name,
|
||||
workspaceId,
|
||||
ext,
|
||||
})
|
||||
return NextResponse.json(result)
|
||||
}
|
||||
|
||||
try {
|
||||
if (!taskId) {
|
||||
return NextResponse.json({ error: 'Unsupported compiled check target' }, { status: 422 })
|
||||
|
||||
@@ -1,224 +0,0 @@
|
||||
/**
|
||||
* @vitest-environment node
|
||||
*/
|
||||
import { authMockFns, workflowsApiUtilsMock, workflowsApiUtilsMockFns } from '@sim/testing'
|
||||
import { NextRequest } from 'next/server'
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { MAX_DOCUMENT_PREVIEW_CODE_BYTES } from '@/lib/execution/constants'
|
||||
|
||||
const { mockRunSandboxTask, SandboxUserCodeError } = vi.hoisted(() => {
|
||||
class SandboxUserCodeError extends Error {
|
||||
constructor(message: string, name: string) {
|
||||
super(message)
|
||||
this.name = name
|
||||
}
|
||||
}
|
||||
return { mockRunSandboxTask: vi.fn(), SandboxUserCodeError }
|
||||
})
|
||||
|
||||
const mockVerifyWorkspaceMembership = workflowsApiUtilsMockFns.mockVerifyWorkspaceMembership
|
||||
|
||||
vi.mock('@/app/api/workflows/utils', () => workflowsApiUtilsMock)
|
||||
|
||||
vi.mock('@/lib/execution/sandbox/run-task', () => ({
|
||||
runSandboxTask: mockRunSandboxTask,
|
||||
SandboxUserCodeError,
|
||||
}))
|
||||
|
||||
import { POST } from '@/app/api/workspaces/[id]/pdf/preview/route'
|
||||
|
||||
describe('PDF preview API route', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
authMockFns.mockGetSession.mockResolvedValue({ user: { id: 'user-1' } })
|
||||
mockVerifyWorkspaceMembership.mockResolvedValue(true)
|
||||
mockRunSandboxTask.mockResolvedValue(Buffer.from('%PDF-test'))
|
||||
})
|
||||
|
||||
it('returns a generated PDF for authorized workspace members', async () => {
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/pdf/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ code: 'return 1' }),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(200)
|
||||
expect(response.headers.get('Content-Type')).toBe('application/pdf')
|
||||
expect(response.headers.get('Cache-Control')).toBe('private, no-store')
|
||||
expect(mockVerifyWorkspaceMembership).toHaveBeenCalledWith('user-1', 'workspace-1')
|
||||
expect(mockRunSandboxTask).toHaveBeenCalledWith(
|
||||
'pdf-generate',
|
||||
{ code: 'return 1', workspaceId: 'workspace-1' },
|
||||
{ ownerKey: 'user:user-1', signal: request.signal }
|
||||
)
|
||||
expect(Buffer.from(await response.arrayBuffer()).toString()).toBe('%PDF-test')
|
||||
})
|
||||
|
||||
it('rejects requests without code', async () => {
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/pdf/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({}),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(400)
|
||||
await expect(response.json()).resolves.toEqual({ error: 'code is required' })
|
||||
expect(mockRunSandboxTask).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('rejects oversized preview source payloads', async () => {
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/pdf/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ code: 'x'.repeat(MAX_DOCUMENT_PREVIEW_CODE_BYTES + 1) }),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(413)
|
||||
await expect(response.json()).resolves.toEqual({ error: 'code exceeds maximum size' })
|
||||
expect(mockRunSandboxTask).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('returns 401 for unauthenticated requests', async () => {
|
||||
authMockFns.mockGetSession.mockResolvedValue(null)
|
||||
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/pdf/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ code: 'return 1' }),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(401)
|
||||
await expect(response.json()).resolves.toEqual({ error: 'Unauthorized' })
|
||||
expect(mockVerifyWorkspaceMembership).not.toHaveBeenCalled()
|
||||
expect(mockRunSandboxTask).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('returns 403 when the user is not a workspace member', async () => {
|
||||
mockVerifyWorkspaceMembership.mockResolvedValue(false)
|
||||
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/pdf/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ code: 'return 1' }),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(403)
|
||||
await expect(response.json()).resolves.toEqual({ error: 'Insufficient permissions' })
|
||||
expect(mockRunSandboxTask).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('returns 400 for requests with invalid JSON bodies', async () => {
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/pdf/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: '{ not valid json',
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(400)
|
||||
await expect(response.json()).resolves.toEqual({ error: 'Invalid or missing JSON body' })
|
||||
expect(mockRunSandboxTask).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('returns 500 when PDF generation throws', async () => {
|
||||
mockRunSandboxTask.mockRejectedValue(new Error('boom: sandbox failed'))
|
||||
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/pdf/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ code: 'return 1' }),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(500)
|
||||
await expect(response.json()).resolves.toEqual({ error: 'boom: sandbox failed' })
|
||||
})
|
||||
|
||||
it('returns 422 when user code throws inside the sandbox', async () => {
|
||||
mockRunSandboxTask.mockRejectedValue(
|
||||
new SandboxUserCodeError('Invalid or unexpected token', 'SyntaxError')
|
||||
)
|
||||
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/pdf/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ code: 'const x = ' }),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(422)
|
||||
await expect(response.json()).resolves.toEqual({
|
||||
error: 'Invalid or unexpected token',
|
||||
errorName: 'SyntaxError',
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -1,20 +0,0 @@
|
||||
import { workspaceParamsSchema, workspacePreviewBodySchema } from '@/lib/api/contracts/workspaces'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import { createDocumentPreviewRoute } from '@/app/api/workspaces/[id]/_preview/create-preview-route'
|
||||
|
||||
export const dynamic = 'force-dynamic'
|
||||
export const runtime = 'nodejs'
|
||||
|
||||
/**
|
||||
* POST /api/workspaces/[id]/pdf/preview
|
||||
* Compile PDF-Lib source code and return the binary PDF for streaming preview.
|
||||
*/
|
||||
export const POST = withRouteHandler(
|
||||
createDocumentPreviewRoute({
|
||||
taskId: 'pdf-generate',
|
||||
contentType: 'application/pdf',
|
||||
label: 'PDF',
|
||||
routeParamsSchema: workspaceParamsSchema,
|
||||
previewBodySchema: workspacePreviewBodySchema,
|
||||
})
|
||||
)
|
||||
@@ -1,226 +0,0 @@
|
||||
/**
|
||||
* @vitest-environment node
|
||||
*/
|
||||
import { authMockFns, workflowsApiUtilsMock, workflowsApiUtilsMockFns } from '@sim/testing'
|
||||
import { NextRequest } from 'next/server'
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { MAX_DOCUMENT_PREVIEW_CODE_BYTES } from '@/lib/execution/constants'
|
||||
|
||||
const { mockRunSandboxTask, SandboxUserCodeError } = vi.hoisted(() => {
|
||||
class SandboxUserCodeError extends Error {
|
||||
constructor(message: string, name: string) {
|
||||
super(message)
|
||||
this.name = name
|
||||
}
|
||||
}
|
||||
return { mockRunSandboxTask: vi.fn(), SandboxUserCodeError }
|
||||
})
|
||||
|
||||
const mockVerifyWorkspaceMembership = workflowsApiUtilsMockFns.mockVerifyWorkspaceMembership
|
||||
|
||||
vi.mock('@/app/api/workflows/utils', () => workflowsApiUtilsMock)
|
||||
|
||||
vi.mock('@/lib/execution/sandbox/run-task', () => ({
|
||||
runSandboxTask: mockRunSandboxTask,
|
||||
SandboxUserCodeError,
|
||||
}))
|
||||
|
||||
import { POST } from '@/app/api/workspaces/[id]/pptx/preview/route'
|
||||
|
||||
const PPTX_MIME = 'application/vnd.openxmlformats-officedocument.presentationml.presentation'
|
||||
|
||||
describe('PPTX preview API route', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
authMockFns.mockGetSession.mockResolvedValue({ user: { id: 'user-1' } })
|
||||
mockVerifyWorkspaceMembership.mockResolvedValue(true)
|
||||
mockRunSandboxTask.mockResolvedValue(Buffer.from('PK\x03\x04pptx'))
|
||||
})
|
||||
|
||||
it('returns a generated PPTX for authorized workspace members', async () => {
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/pptx/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ code: 'return 1' }),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(200)
|
||||
expect(response.headers.get('Content-Type')).toBe(PPTX_MIME)
|
||||
expect(response.headers.get('Cache-Control')).toBe('private, no-store')
|
||||
expect(mockVerifyWorkspaceMembership).toHaveBeenCalledWith('user-1', 'workspace-1')
|
||||
expect(mockRunSandboxTask).toHaveBeenCalledWith(
|
||||
'pptx-generate',
|
||||
{ code: 'return 1', workspaceId: 'workspace-1' },
|
||||
{ ownerKey: 'user:user-1', signal: request.signal }
|
||||
)
|
||||
expect(Buffer.from(await response.arrayBuffer()).toString()).toBe('PK\x03\x04pptx')
|
||||
})
|
||||
|
||||
it('rejects requests without code', async () => {
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/pptx/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({}),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(400)
|
||||
await expect(response.json()).resolves.toEqual({ error: 'code is required' })
|
||||
expect(mockRunSandboxTask).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('rejects oversized preview source payloads', async () => {
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/pptx/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ code: 'x'.repeat(MAX_DOCUMENT_PREVIEW_CODE_BYTES + 1) }),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(413)
|
||||
await expect(response.json()).resolves.toEqual({ error: 'code exceeds maximum size' })
|
||||
expect(mockRunSandboxTask).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('returns 401 for unauthenticated requests', async () => {
|
||||
authMockFns.mockGetSession.mockResolvedValue(null)
|
||||
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/pptx/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ code: 'return 1' }),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(401)
|
||||
await expect(response.json()).resolves.toEqual({ error: 'Unauthorized' })
|
||||
expect(mockVerifyWorkspaceMembership).not.toHaveBeenCalled()
|
||||
expect(mockRunSandboxTask).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('returns 403 when the user is not a workspace member', async () => {
|
||||
mockVerifyWorkspaceMembership.mockResolvedValue(false)
|
||||
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/pptx/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ code: 'return 1' }),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(403)
|
||||
await expect(response.json()).resolves.toEqual({ error: 'Insufficient permissions' })
|
||||
expect(mockRunSandboxTask).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('returns 400 for requests with invalid JSON bodies', async () => {
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/pptx/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: '{ not valid json',
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(400)
|
||||
await expect(response.json()).resolves.toEqual({ error: 'Invalid or missing JSON body' })
|
||||
expect(mockRunSandboxTask).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('returns 500 when PPTX generation throws', async () => {
|
||||
mockRunSandboxTask.mockRejectedValue(new Error('boom: sandbox failed'))
|
||||
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/pptx/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ code: 'return 1' }),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(500)
|
||||
await expect(response.json()).resolves.toEqual({ error: 'boom: sandbox failed' })
|
||||
})
|
||||
|
||||
it('returns 422 when user code throws inside the sandbox', async () => {
|
||||
mockRunSandboxTask.mockRejectedValue(
|
||||
new SandboxUserCodeError('Invalid or unexpected token', 'SyntaxError')
|
||||
)
|
||||
|
||||
const request = new NextRequest(
|
||||
'http://localhost:3000/api/workspaces/workspace-1/pptx/preview',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({ code: 'const x = ' }),
|
||||
}
|
||||
)
|
||||
|
||||
const response = await POST(request, {
|
||||
params: Promise.resolve({ id: 'workspace-1' }),
|
||||
})
|
||||
|
||||
expect(response.status).toBe(422)
|
||||
await expect(response.json()).resolves.toEqual({
|
||||
error: 'Invalid or unexpected token',
|
||||
errorName: 'SyntaxError',
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -1,20 +0,0 @@
|
||||
import { workspaceParamsSchema, workspacePreviewBodySchema } from '@/lib/api/contracts/workspaces'
|
||||
import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
|
||||
import { createDocumentPreviewRoute } from '@/app/api/workspaces/[id]/_preview/create-preview-route'
|
||||
|
||||
export const dynamic = 'force-dynamic'
|
||||
export const runtime = 'nodejs'
|
||||
|
||||
/**
|
||||
* POST /api/workspaces/[id]/pptx/preview
|
||||
* Compile PptxGenJS source code and return the binary PPTX for streaming preview.
|
||||
*/
|
||||
export const POST = withRouteHandler(
|
||||
createDocumentPreviewRoute({
|
||||
taskId: 'pptx-generate',
|
||||
contentType: 'application/vnd.openxmlformats-officedocument.presentationml.presentation',
|
||||
label: 'PPTX',
|
||||
routeParamsSchema: workspaceParamsSchema,
|
||||
previewBodySchema: workspacePreviewBodySchema,
|
||||
})
|
||||
)
|
||||
@@ -5,10 +5,10 @@ import { createLogger } from '@sim/logger'
|
||||
import { toError } from '@sim/utils/errors'
|
||||
import { cn } from '@/lib/core/utils/cn'
|
||||
import type { WorkspaceFileRecord } from '@/lib/uploads/contexts/workspace'
|
||||
import { useWorkspaceFileBinary } from '@/hooks/queries/workspace-files'
|
||||
import { PDF_PAGE_SKELETON, PreviewError, resolvePreviewError } from './preview-shared'
|
||||
import { PreviewToolbar } from './preview-toolbar'
|
||||
import { bindPreviewWheelZoom } from './preview-wheel-zoom'
|
||||
import { useDocPreviewBinary } from './use-doc-preview-binary'
|
||||
|
||||
const logger = createLogger('DocxPreview')
|
||||
|
||||
@@ -62,21 +62,15 @@ function fitDocxToContainer(host: HTMLElement, viewport: HTMLElement, zoomPercen
|
||||
export const DocxPreview = memo(function DocxPreview({
|
||||
file,
|
||||
workspaceId,
|
||||
streamingContent,
|
||||
}: {
|
||||
file: WorkspaceFileRecord
|
||||
workspaceId: string
|
||||
streamingContent?: string
|
||||
}) {
|
||||
const containerRef = useRef<HTMLDivElement>(null)
|
||||
const scrollContainerRef = useRef<HTMLDivElement>(null)
|
||||
const lastSuccessfulHtmlRef = useRef('')
|
||||
const zoomPercentRef = useRef(100)
|
||||
const {
|
||||
data: fileData,
|
||||
isLoading,
|
||||
error: fetchError,
|
||||
} = useWorkspaceFileBinary(workspaceId, file.id, file.key)
|
||||
const preview = useDocPreviewBinary(workspaceId, file)
|
||||
const fileData = preview.data
|
||||
const [renderError, setRenderError] = useState<string | null>(null)
|
||||
const [rendering, setRendering] = useState(false)
|
||||
const [hasRenderedPreview, setHasRenderedPreview] = useState(false)
|
||||
@@ -182,7 +176,7 @@ export const DocxPreview = memo(function DocxPreview({
|
||||
}, [pageCount, documentRenderVersion])
|
||||
|
||||
useEffect(() => {
|
||||
if (!containerRef.current || !fileData || streamingContent !== undefined) return
|
||||
if (!containerRef.current || !fileData) return
|
||||
|
||||
let cancelled = false
|
||||
|
||||
@@ -200,7 +194,6 @@ export const DocxPreview = memo(function DocxPreview({
|
||||
})
|
||||
if (!cancelled && containerRef.current) {
|
||||
applyPostRenderStyling()
|
||||
lastSuccessfulHtmlRef.current = containerRef.current.innerHTML
|
||||
setHasRenderedPreview(true)
|
||||
setDocumentRenderVersion((version) => version + 1)
|
||||
}
|
||||
@@ -221,86 +214,12 @@ export const DocxPreview = memo(function DocxPreview({
|
||||
return () => {
|
||||
cancelled = true
|
||||
}
|
||||
}, [fileData, streamingContent, applyPostRenderStyling])
|
||||
}, [fileData, applyPostRenderStyling])
|
||||
|
||||
useEffect(() => {
|
||||
if (streamingContent === undefined || !containerRef.current) return
|
||||
if (streamingContent.trim().length === 0) return
|
||||
|
||||
let cancelled = false
|
||||
const controller = new AbortController()
|
||||
|
||||
const debounceTimer = setTimeout(async () => {
|
||||
const container = containerRef.current
|
||||
if (!container || cancelled) return
|
||||
|
||||
const previousHtml = lastSuccessfulHtmlRef.current
|
||||
|
||||
try {
|
||||
setRendering(true)
|
||||
|
||||
// boundary-raw-fetch: route returns binary DOCX (read via response.arrayBuffer()), not JSON
|
||||
const response = await fetch(`/api/workspaces/${workspaceId}/docx/preview`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ code: streamingContent }),
|
||||
signal: controller.signal,
|
||||
})
|
||||
if (!response.ok) {
|
||||
const err = await response.json().catch(() => ({ error: 'Preview failed' }))
|
||||
throw new Error(err.error || 'Preview failed')
|
||||
}
|
||||
|
||||
const arrayBuffer = await response.arrayBuffer()
|
||||
if (cancelled || !containerRef.current) return
|
||||
if (arrayBuffer.byteLength === 0) return
|
||||
|
||||
const { renderAsync } = await import('docx-preview')
|
||||
if (cancelled || !containerRef.current) return
|
||||
|
||||
containerRef.current.innerHTML = ''
|
||||
await renderAsync(new Uint8Array(arrayBuffer), containerRef.current, undefined, {
|
||||
inWrapper: true,
|
||||
ignoreWidth: false,
|
||||
ignoreHeight: false,
|
||||
})
|
||||
|
||||
if (!cancelled && containerRef.current) {
|
||||
applyPostRenderStyling()
|
||||
lastSuccessfulHtmlRef.current = containerRef.current.innerHTML
|
||||
setHasRenderedPreview(true)
|
||||
setDocumentRenderVersion((version) => version + 1)
|
||||
}
|
||||
} catch (err) {
|
||||
if (!cancelled && !(err instanceof DOMException && err.name === 'AbortError')) {
|
||||
if (containerRef.current && previousHtml) {
|
||||
containerRef.current.innerHTML = previousHtml
|
||||
applyPostRenderStyling()
|
||||
setHasRenderedPreview(true)
|
||||
setDocumentRenderVersion((version) => version + 1)
|
||||
}
|
||||
const msg = toError(err).message || 'Failed to render document'
|
||||
logger.info('Transient DOCX streaming preview error (suppressed)', { error: msg })
|
||||
}
|
||||
} finally {
|
||||
if (!cancelled) {
|
||||
setRendering(false)
|
||||
}
|
||||
}
|
||||
}, 500)
|
||||
|
||||
return () => {
|
||||
cancelled = true
|
||||
clearTimeout(debounceTimer)
|
||||
controller.abort()
|
||||
}
|
||||
}, [streamingContent, workspaceId, applyPostRenderStyling])
|
||||
|
||||
const error = streamingContent !== undefined ? null : resolvePreviewError(fetchError, renderError)
|
||||
const error = resolvePreviewError(preview.error, renderError)
|
||||
if (error) return <PreviewError label='document' error={error} />
|
||||
|
||||
const showSkeleton =
|
||||
!hasRenderedPreview && (streamingContent !== undefined || isLoading || rendering)
|
||||
const showSkeleton = !hasRenderedPreview && (!fileData || rendering)
|
||||
|
||||
const scrollToPage = (page: number) => {
|
||||
const scrollContainer = scrollContainerRef.current
|
||||
|
||||
@@ -36,7 +36,11 @@ const TEXT_EDITABLE_EXTENSIONS = new Set([
|
||||
...SUPPORTED_CODE_EXTENSIONS,
|
||||
])
|
||||
|
||||
const IFRAME_PREVIEWABLE_MIME_TYPES = new Set(['application/pdf', 'text/x-pdflibjs'])
|
||||
const IFRAME_PREVIEWABLE_MIME_TYPES = new Set([
|
||||
'application/pdf',
|
||||
'text/x-pdflibjs',
|
||||
'text/x-python-pdf',
|
||||
])
|
||||
const IFRAME_PREVIEWABLE_EXTENSIONS = new Set(['pdf'])
|
||||
|
||||
const IMAGE_PREVIEWABLE_MIME_TYPES = new Set(['image/png', 'image/jpeg', 'image/gif', 'image/webp'])
|
||||
@@ -78,6 +82,7 @@ const DOCX_PREVIEWABLE_EXTENSIONS = new Set(['docx'])
|
||||
|
||||
const XLSX_PREVIEWABLE_MIME_TYPES = new Set([
|
||||
'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
|
||||
'text/x-python-xlsx',
|
||||
])
|
||||
const XLSX_PREVIEWABLE_EXTENSIONS = new Set(['xlsx'])
|
||||
|
||||
|
||||
+20
-109
@@ -2,7 +2,6 @@
|
||||
|
||||
import { memo, useCallback, useEffect, useMemo, useRef, useState } from 'react'
|
||||
import { createLogger } from '@sim/logger'
|
||||
import { toError } from '@sim/utils/errors'
|
||||
import dynamic from 'next/dynamic'
|
||||
import { Skeleton } from '@/components/emcn'
|
||||
import type { WorkspaceFileRecord } from '@/lib/uploads/contexts/workspace'
|
||||
@@ -10,6 +9,7 @@ import { getFileExtension } from '@/lib/uploads/utils/file-utils'
|
||||
import { useWorkspaceFileBinary } from '@/hooks/queries/workspace-files'
|
||||
import { resolveFileCategory } from './file-category'
|
||||
import type { StreamingMode } from './text-editor-state'
|
||||
import { useDocPreviewBinary } from './use-doc-preview-binary'
|
||||
|
||||
export type { StreamingMode } from './text-editor-state'
|
||||
|
||||
@@ -89,14 +89,7 @@ export function FileViewer({
|
||||
}
|
||||
|
||||
if (category === 'iframe-previewable') {
|
||||
return (
|
||||
<IframePreview
|
||||
key={file.id}
|
||||
file={file}
|
||||
workspaceId={workspaceId}
|
||||
streamingContent={streamingContent}
|
||||
/>
|
||||
)
|
||||
return <IframePreview key={file.id} file={file} workspaceId={workspaceId} />
|
||||
}
|
||||
|
||||
if (category === 'image-previewable') {
|
||||
@@ -112,37 +105,15 @@ export function FileViewer({
|
||||
}
|
||||
|
||||
if (category === 'docx-previewable') {
|
||||
return (
|
||||
<DocxPreview
|
||||
key={file.id}
|
||||
file={file}
|
||||
workspaceId={workspaceId}
|
||||
streamingContent={streamingContent}
|
||||
/>
|
||||
)
|
||||
return <DocxPreview key={file.id} file={file} workspaceId={workspaceId} />
|
||||
}
|
||||
|
||||
if (category === 'pptx-previewable') {
|
||||
return (
|
||||
<PptxPreview
|
||||
key={file.id}
|
||||
file={file}
|
||||
workspaceId={workspaceId}
|
||||
streamingContent={streamingContent}
|
||||
/>
|
||||
)
|
||||
return <PptxPreview key={file.id} file={file} workspaceId={workspaceId} />
|
||||
}
|
||||
|
||||
if (category === 'xlsx-previewable') {
|
||||
return (
|
||||
<XlsxPreview
|
||||
file={file}
|
||||
workspaceId={workspaceId}
|
||||
canEdit={canEdit}
|
||||
onSaveStatusChange={onSaveStatusChange}
|
||||
saveRef={saveRef}
|
||||
/>
|
||||
)
|
||||
return <XlsxPreview key={file.id} file={file} workspaceId={workspaceId} />
|
||||
}
|
||||
|
||||
return <UnsupportedPreview file={file} />
|
||||
@@ -151,91 +122,31 @@ export function FileViewer({
|
||||
const IframePreview = memo(function IframePreview({
|
||||
file,
|
||||
workspaceId,
|
||||
streamingContent,
|
||||
}: {
|
||||
file: WorkspaceFileRecord
|
||||
workspaceId: string
|
||||
streamingContent?: string
|
||||
}) {
|
||||
const [streamingBuffer, setStreamingBuffer] = useState<ArrayBuffer | null>(null)
|
||||
const streamingBufferRef = useRef<ArrayBuffer | null>(null)
|
||||
const streamingBufferSeqRef = useRef(0)
|
||||
const [streamingBufferSeq, setStreamingBufferSeq] = useState(0)
|
||||
const [rendering, setRendering] = useState(false)
|
||||
const preview = useDocPreviewBinary(workspaceId, file)
|
||||
|
||||
useEffect(() => {
|
||||
if (streamingContent === undefined) return
|
||||
|
||||
let cancelled = false
|
||||
const controller = new AbortController()
|
||||
|
||||
const debounceTimer = setTimeout(async () => {
|
||||
if (cancelled) return
|
||||
|
||||
try {
|
||||
setRendering(true)
|
||||
|
||||
// boundary-raw-fetch: route returns binary PDF (read via response.arrayBuffer()), not JSON
|
||||
const response = await fetch(`/api/workspaces/${workspaceId}/pdf/preview`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ code: streamingContent }),
|
||||
signal: controller.signal,
|
||||
})
|
||||
if (!response.ok) {
|
||||
const err = await response.json().catch(() => ({ error: 'Preview failed' }))
|
||||
throw new Error(err.error || 'Preview failed')
|
||||
}
|
||||
|
||||
const buf = await response.arrayBuffer()
|
||||
if (cancelled) return
|
||||
|
||||
streamingBufferRef.current = buf
|
||||
streamingBufferSeqRef.current += 1
|
||||
setStreamingBuffer(buf)
|
||||
setStreamingBufferSeq(streamingBufferSeqRef.current)
|
||||
} catch (err) {
|
||||
if (!cancelled && !(err instanceof DOMException && err.name === 'AbortError')) {
|
||||
const msg = toError(err).message || 'Failed to render PDF'
|
||||
logger.info('Transient PDF streaming preview error (suppressed)', { error: msg })
|
||||
}
|
||||
} finally {
|
||||
if (!cancelled) setRendering(false)
|
||||
}
|
||||
}, 500)
|
||||
|
||||
return () => {
|
||||
cancelled = true
|
||||
clearTimeout(debounceTimer)
|
||||
controller.abort()
|
||||
}
|
||||
}, [streamingContent, workspaceId])
|
||||
|
||||
const staticSource = useMemo<PdfDocumentSource>(
|
||||
() => ({
|
||||
kind: 'url',
|
||||
url: `/api/files/serve/${encodeURIComponent(file.key)}?context=workspace`,
|
||||
}),
|
||||
[file.key]
|
||||
const bufferSource = useMemo<PdfDocumentSource | null>(
|
||||
() => (preview.data ? { kind: 'buffer', buffer: preview.data } : null),
|
||||
[preview.data]
|
||||
)
|
||||
|
||||
const streamingSource = useMemo<PdfDocumentSource | null>(
|
||||
() => (streamingBuffer ? { kind: 'buffer', buffer: streamingBuffer } : null),
|
||||
[streamingBuffer]
|
||||
)
|
||||
const error = resolvePreviewError(preview.error, null)
|
||||
if (error) return <PreviewError label='PDF' error={error} />
|
||||
|
||||
if (streamingContent !== undefined) {
|
||||
if (
|
||||
!streamingSource ||
|
||||
streamingSource.kind !== 'buffer' ||
|
||||
streamingSource.buffer.byteLength === 0
|
||||
) {
|
||||
return <div className='relative flex flex-1 overflow-hidden'>{PDF_PAGE_SKELETON}</div>
|
||||
}
|
||||
return <PdfViewerCore key={streamingBufferSeq} source={streamingSource} filename={file.name} />
|
||||
if (!bufferSource) {
|
||||
return <div className='relative flex flex-1 overflow-hidden'>{PDF_PAGE_SKELETON}</div>
|
||||
}
|
||||
|
||||
return <PdfViewerCore source={staticSource} filename={file.name} />
|
||||
return (
|
||||
<PdfViewerCore
|
||||
key={`${file.id}:${preview.dataUpdatedAt}`}
|
||||
source={bufferSource}
|
||||
filename={file.name}
|
||||
/>
|
||||
)
|
||||
})
|
||||
|
||||
function useBlobUrl(workspaceId: string, fileId: string, fileKey: string) {
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
export { resolveFileCategory } from './file-category'
|
||||
export type { PreviewMode } from './file-viewer'
|
||||
export { FileViewer, isPreviewable, isTextEditable } from './file-viewer'
|
||||
export { RICH_PREVIEWABLE_EXTENSIONS } from './preview-panel'
|
||||
|
||||
+10
-71
@@ -2,7 +2,6 @@
|
||||
|
||||
import { memo, useEffect, useState } from 'react'
|
||||
import { createLogger } from '@sim/logger'
|
||||
import { toError } from '@sim/utils/errors'
|
||||
import { Skeleton } from '@/components/emcn'
|
||||
import type { WorkspaceFileRecord } from '@/lib/uploads/contexts/workspace'
|
||||
import { PptxSandboxHost } from '@/app/workspace/[workspaceId]/files/components/file-viewer/pptx-sandbox-host'
|
||||
@@ -10,7 +9,7 @@ import {
|
||||
PreviewError,
|
||||
resolvePreviewError,
|
||||
} from '@/app/workspace/[workspaceId]/files/components/file-viewer/preview-shared'
|
||||
import { useWorkspaceFileBinary } from '@/hooks/queries/workspace-files'
|
||||
import { useDocPreviewBinary } from '@/app/workspace/[workspaceId]/files/components/file-viewer/use-doc-preview-binary'
|
||||
|
||||
const logger = createLogger('PptxPreview')
|
||||
|
||||
@@ -45,80 +44,24 @@ function pptxCacheKey(fileId: string, dataUpdatedAt: number, byteLength: number)
|
||||
export const PptxPreview = memo(function PptxPreview({
|
||||
file,
|
||||
workspaceId,
|
||||
streamingContent,
|
||||
}: {
|
||||
file: WorkspaceFileRecord
|
||||
workspaceId: string
|
||||
streamingContent?: string
|
||||
}) {
|
||||
const {
|
||||
data: fileData,
|
||||
error: fetchError,
|
||||
dataUpdatedAt,
|
||||
} = useWorkspaceFileBinary(workspaceId, file.id, file.key)
|
||||
const preview = useDocPreviewBinary(workspaceId, file)
|
||||
const fileData = preview.data
|
||||
const cacheKey = pptxCacheKey(file.id, preview.dataUpdatedAt, fileData?.byteLength ?? 0)
|
||||
|
||||
const cacheKey = pptxCacheKey(file.id, dataUpdatedAt, fileData?.byteLength ?? 0)
|
||||
|
||||
const [streamBuffer, setStreamBuffer] = useState<ArrayBuffer | null>(null)
|
||||
const [streamVersion, setStreamVersion] = useState(0)
|
||||
const [hasRendered, setHasRendered] = useState(false)
|
||||
const [renderError, setRenderError] = useState<string | null>(null)
|
||||
const isStreaming = streamingContent !== undefined
|
||||
|
||||
useEffect(() => {
|
||||
if (!isStreaming) return
|
||||
|
||||
let cancelled = false
|
||||
const controller = new AbortController()
|
||||
|
||||
const debounceTimer = setTimeout(async () => {
|
||||
if (cancelled) return
|
||||
try {
|
||||
// boundary-raw-fetch: route returns binary PPTX (read via response.arrayBuffer()), not JSON
|
||||
const response = await fetch(`/api/workspaces/${workspaceId}/pptx/preview`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ code: streamingContent }),
|
||||
signal: controller.signal,
|
||||
})
|
||||
if (!response.ok) {
|
||||
const err = await response.json().catch(() => ({ error: 'Preview failed' }))
|
||||
throw new Error(err.error || 'Preview failed')
|
||||
}
|
||||
if (cancelled) return
|
||||
const arrayBuffer = await response.arrayBuffer()
|
||||
if (cancelled) return
|
||||
setRenderError(null)
|
||||
setStreamBuffer(arrayBuffer)
|
||||
setStreamVersion((version) => version + 1)
|
||||
} catch (err) {
|
||||
if (!cancelled && !(err instanceof DOMException && err.name === 'AbortError')) {
|
||||
const msg = toError(err).message || 'Failed to render presentation'
|
||||
logger.info('Transient PPTX streaming preview error (suppressed)', { error: msg })
|
||||
}
|
||||
}
|
||||
}, 500)
|
||||
|
||||
return () => {
|
||||
cancelled = true
|
||||
clearTimeout(debounceTimer)
|
||||
controller.abort()
|
||||
}
|
||||
}, [isStreaming, streamingContent, workspaceId])
|
||||
|
||||
useEffect(() => {
|
||||
setRenderError(null)
|
||||
setHasRendered(false)
|
||||
if (!isStreaming) setStreamBuffer(null)
|
||||
}, [cacheKey, isStreaming])
|
||||
|
||||
const activeBuffer = isStreaming ? streamBuffer : fileData
|
||||
const activeRenderKey = isStreaming
|
||||
? `${file.id}:stream:${streamVersion}:${streamBuffer?.byteLength ?? 0}`
|
||||
: cacheKey
|
||||
}, [cacheKey])
|
||||
|
||||
function handleRenderStart() {
|
||||
if (!isStreaming) setRenderError(null)
|
||||
setRenderError(null)
|
||||
}
|
||||
|
||||
function handleRenderComplete() {
|
||||
@@ -126,27 +69,23 @@ export const PptxPreview = memo(function PptxPreview({
|
||||
}
|
||||
|
||||
function handleRenderError(message: string) {
|
||||
if (isStreaming) {
|
||||
logger.info('Transient PPTX streaming render error (suppressed)', { error: message })
|
||||
return
|
||||
}
|
||||
logger.error('PPTX render failed', { error: message })
|
||||
setRenderError(message || 'Failed to render presentation')
|
||||
}
|
||||
|
||||
const error = isStreaming ? null : resolvePreviewError(fetchError, renderError)
|
||||
const error = resolvePreviewError(preview.error, renderError)
|
||||
|
||||
if (error) return <PreviewError label='presentation' error={error} />
|
||||
|
||||
if (!activeBuffer) {
|
||||
if (!fileData) {
|
||||
return PPTX_SLIDE_SKELETON
|
||||
}
|
||||
|
||||
return (
|
||||
<div className='relative flex h-full min-h-0 flex-1 overflow-hidden bg-[var(--surface-1)]'>
|
||||
<PptxSandboxHost
|
||||
buffer={activeBuffer}
|
||||
requestId={activeRenderKey}
|
||||
buffer={fileData}
|
||||
requestId={cacheKey}
|
||||
onRenderStart={handleRenderStart}
|
||||
onRenderComplete={handleRenderComplete}
|
||||
onRenderError={handleRenderError}
|
||||
|
||||
+1
@@ -97,6 +97,7 @@ export const PptxSandboxHost = memo(function PptxSandboxHost({
|
||||
onRenderCompleteRef.current?.()
|
||||
} catch (error) {
|
||||
nextContainer.remove()
|
||||
// Aborted means a newer render superseded this one — don't report.
|
||||
if (controller.signal.aborted) return
|
||||
|
||||
const message = toError(error).message || 'Failed to render presentation'
|
||||
|
||||
+22
-3
@@ -35,6 +35,7 @@ import { cn } from '@/lib/core/utils/cn'
|
||||
import { extractTextContent } from '@/lib/core/utils/react-node-text'
|
||||
import { getFileExtension } from '@/lib/uploads/utils/file-utils'
|
||||
import { useScrollAnchor } from '@/hooks/use-scroll-anchor'
|
||||
import { useSmoothText } from '@/hooks/use-smooth-text'
|
||||
import { DataTable } from './data-table'
|
||||
import { ZoomablePreview } from './zoomable-preview'
|
||||
|
||||
@@ -183,6 +184,18 @@ function remarkCallouts() {
|
||||
const REMARK_PLUGINS = [remarkGfm, remarkBreaks, remarkMermaid, remarkCallouts]
|
||||
const REHYPE_PLUGINS = [rehypeSlug]
|
||||
|
||||
/**
|
||||
* Soft per-character fade for newly revealed markdown while streaming. Mirrors
|
||||
* the chat surface so a streamed file preview reveals with the same cadence;
|
||||
* paired with {@link useSmoothText}, which paces the reveal itself.
|
||||
*/
|
||||
const STREAM_ANIMATION = {
|
||||
animation: 'fadeIn',
|
||||
duration: 220,
|
||||
stagger: 0,
|
||||
sep: 'char',
|
||||
} as const
|
||||
|
||||
/**
|
||||
* Carries the contentRef and toggle handler from MarkdownPreview down to the
|
||||
* task-list renderers. Only present when the preview is interactive.
|
||||
@@ -866,16 +879,20 @@ const MarkdownPreview = memo(function MarkdownPreview({
|
||||
onCheckboxToggle?: (checkboxIndex: number, checked: boolean) => void
|
||||
}) {
|
||||
const { push: navigate } = useRouter()
|
||||
// Pace the reveal so streamed markdown builds at a steady cadence instead of
|
||||
// jumping per server chunk. `snapOnNonAppend` shows in-place rewrites (patch)
|
||||
// in full immediately so a diff never appears to retype from the top.
|
||||
const revealedContent = useSmoothText(content, isStreaming, { snapOnNonAppend: true })
|
||||
const { ref: autoScrollRef, spacerRef } = useScrollAnchor(
|
||||
isStreaming && !disableAutoScroll,
|
||||
content
|
||||
revealedContent
|
||||
)
|
||||
|
||||
const contentRef = useRef(content)
|
||||
contentRef.current = content
|
||||
|
||||
const { frontMatterData, markdownContent } = useMemo(() => {
|
||||
if (isStreaming) return { frontMatterData: null, markdownContent: content }
|
||||
if (isStreaming) return { frontMatterData: null, markdownContent: revealedContent }
|
||||
try {
|
||||
const parsed = matter(content)
|
||||
const hasFrontMatter = Object.keys(parsed.data).length > 0
|
||||
@@ -886,7 +903,7 @@ const MarkdownPreview = memo(function MarkdownPreview({
|
||||
} catch {
|
||||
return { frontMatterData: null, markdownContent: content }
|
||||
}
|
||||
}, [content, isStreaming])
|
||||
}, [content, revealedContent, isStreaming])
|
||||
|
||||
const ctxValue = useMemo(
|
||||
() => (onCheckboxToggle ? { contentRef, onToggle: onCheckboxToggle } : null),
|
||||
@@ -917,6 +934,8 @@ const MarkdownPreview = memo(function MarkdownPreview({
|
||||
{frontMatterData && <FrontMatterCard data={frontMatterData} />}
|
||||
<Streamdown
|
||||
mode={streamdownMode}
|
||||
animated={isStreaming ? STREAM_ANIMATION : false}
|
||||
isAnimating={isStreaming}
|
||||
remarkPlugins={REMARK_PLUGINS}
|
||||
rehypePlugins={REHYPE_PLUGINS}
|
||||
components={markdownComponents}
|
||||
|
||||
@@ -17,6 +17,13 @@ export function resolvePreviewError(
|
||||
fetchError: Error | null,
|
||||
renderError: string | null
|
||||
): string | null {
|
||||
// A doc whose compiled artifact never appeared (the binary query exhausted its
|
||||
// "still generating" polls) — usually a source that failed to compile or a
|
||||
// legacy file with no artifact. Give a clear, actionable message instead of a
|
||||
// generic fetch error.
|
||||
if (fetchError?.name === 'DocNotReadyError') {
|
||||
return "Couldn't generate this document preview. Re-run the file generation to rebuild it."
|
||||
}
|
||||
if (fetchError) return fetchError.message
|
||||
return renderError
|
||||
}
|
||||
|
||||
@@ -442,7 +442,9 @@ export const TextEditor = memo(function TextEditor({
|
||||
file.key,
|
||||
file.type === 'text/x-pptxgenjs' ||
|
||||
file.type === 'text/x-docxjs' ||
|
||||
file.type === 'text/x-pdflibjs'
|
||||
file.type === 'text/x-pdflibjs' ||
|
||||
file.type === 'text/x-python-pdf' ||
|
||||
file.type === 'text/x-python-xlsx'
|
||||
)
|
||||
|
||||
const updateContent = useUpdateWorkspaceFileContent()
|
||||
|
||||
+184
@@ -0,0 +1,184 @@
|
||||
/**
|
||||
* @vitest-environment node
|
||||
*/
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { resolveDocPreviewBinary, stepDocPreviewBinary } from './use-doc-preview-binary'
|
||||
|
||||
function buffer(byte: number): ArrayBuffer {
|
||||
return new Uint8Array([byte]).buffer
|
||||
}
|
||||
|
||||
describe('resolveDocPreviewBinary', () => {
|
||||
it('reports empty when nothing is committed and no binary has resolved', () => {
|
||||
const result = resolveDocPreviewBinary({
|
||||
data: undefined,
|
||||
isPlaceholderData: false,
|
||||
error: null,
|
||||
lastGood: null,
|
||||
hasCommittedContent: false,
|
||||
})
|
||||
|
||||
expect(result.state).toBe('empty')
|
||||
expect(result.data).toBeNull()
|
||||
expect(result.error).toBeNull()
|
||||
})
|
||||
|
||||
it('reports loading when committed but the first fetch has not resolved', () => {
|
||||
const result = resolveDocPreviewBinary({
|
||||
data: undefined,
|
||||
isPlaceholderData: false,
|
||||
error: null,
|
||||
lastGood: null,
|
||||
hasCommittedContent: true,
|
||||
})
|
||||
|
||||
expect(result.state).toBe('loading')
|
||||
expect(result.data).toBeNull()
|
||||
})
|
||||
|
||||
it('reports ready and advances the head on a fresh success', () => {
|
||||
const fresh = buffer(1)
|
||||
const result = resolveDocPreviewBinary({
|
||||
data: fresh,
|
||||
isPlaceholderData: false,
|
||||
error: null,
|
||||
lastGood: null,
|
||||
hasCommittedContent: true,
|
||||
})
|
||||
|
||||
expect(result.state).toBe('ready')
|
||||
expect(result.data).toBe(fresh)
|
||||
expect(result.lastGood).toBe(fresh)
|
||||
})
|
||||
|
||||
it('holds the previous binary as stale while a new version is fetching', () => {
|
||||
const previous = buffer(1)
|
||||
const result = resolveDocPreviewBinary({
|
||||
data: previous,
|
||||
isPlaceholderData: true,
|
||||
error: null,
|
||||
lastGood: previous,
|
||||
hasCommittedContent: true,
|
||||
})
|
||||
|
||||
expect(result.state).toBe('stale')
|
||||
expect(result.data).toBe(previous)
|
||||
})
|
||||
|
||||
it('falls back to the last good binary and suppresses the error after a failed refetch', () => {
|
||||
const previous = buffer(1)
|
||||
const result = resolveDocPreviewBinary({
|
||||
data: undefined,
|
||||
isPlaceholderData: false,
|
||||
error: new Error('boom'),
|
||||
lastGood: previous,
|
||||
hasCommittedContent: true,
|
||||
})
|
||||
|
||||
expect(result.state).toBe('stale')
|
||||
expect(result.data).toBe(previous)
|
||||
expect(result.error).toBeNull()
|
||||
})
|
||||
|
||||
it('surfaces the error only when there is no binary to fall back to', () => {
|
||||
const err = new Error('missing artifact')
|
||||
const result = resolveDocPreviewBinary({
|
||||
data: undefined,
|
||||
isPlaceholderData: false,
|
||||
error: err,
|
||||
lastGood: null,
|
||||
hasCommittedContent: true,
|
||||
})
|
||||
|
||||
expect(result.data).toBeNull()
|
||||
expect(result.error).toBe(err)
|
||||
})
|
||||
})
|
||||
|
||||
describe('stepDocPreviewBinary', () => {
|
||||
it('shows loading for a committed file whose first fetch has not resolved', () => {
|
||||
const step = stepDocPreviewBinary({
|
||||
fileChanged: false,
|
||||
data: undefined,
|
||||
isPlaceholderData: false,
|
||||
error: null,
|
||||
hasCommittedContent: true,
|
||||
prevHasResolvedForFile: false,
|
||||
prevLastGood: null,
|
||||
})
|
||||
|
||||
expect(step.resolved.state).toBe('loading')
|
||||
expect(step.hasResolvedForFile).toBe(false)
|
||||
expect(step.lastGood).toBeNull()
|
||||
})
|
||||
|
||||
it('advances the head and records resolution on a fresh success', () => {
|
||||
const fresh = buffer(1)
|
||||
const step = stepDocPreviewBinary({
|
||||
fileChanged: false,
|
||||
data: fresh,
|
||||
isPlaceholderData: false,
|
||||
error: null,
|
||||
hasCommittedContent: true,
|
||||
prevHasResolvedForFile: false,
|
||||
prevLastGood: null,
|
||||
})
|
||||
|
||||
expect(step.resolved.state).toBe('ready')
|
||||
expect(step.resolved.data).toBe(fresh)
|
||||
expect(step.hasResolvedForFile).toBe(true)
|
||||
expect(step.lastGood).toBe(fresh)
|
||||
})
|
||||
|
||||
it('ignores the prior-file placeholder on a file change (no cross-file bleed)', () => {
|
||||
const priorFileBytes = buffer(1)
|
||||
const step = stepDocPreviewBinary({
|
||||
fileChanged: true,
|
||||
data: priorFileBytes,
|
||||
isPlaceholderData: true,
|
||||
error: null,
|
||||
hasCommittedContent: true,
|
||||
prevHasResolvedForFile: true,
|
||||
prevLastGood: priorFileBytes,
|
||||
})
|
||||
|
||||
expect(step.resolved.state).toBe('loading')
|
||||
expect(step.resolved.data).toBeNull()
|
||||
expect(step.hasResolvedForFile).toBe(false)
|
||||
expect(step.lastGood).toBeNull()
|
||||
})
|
||||
|
||||
it('holds the previous version as stale during a same-file recompile', () => {
|
||||
const v1 = buffer(1)
|
||||
const step = stepDocPreviewBinary({
|
||||
fileChanged: false,
|
||||
data: v1,
|
||||
isPlaceholderData: true,
|
||||
error: null,
|
||||
hasCommittedContent: true,
|
||||
prevHasResolvedForFile: true,
|
||||
prevLastGood: v1,
|
||||
})
|
||||
|
||||
expect(step.resolved.state).toBe('stale')
|
||||
expect(step.resolved.data).toBe(v1)
|
||||
expect(step.hasResolvedForFile).toBe(true)
|
||||
})
|
||||
|
||||
it('keeps the last good binary and suppresses the error after a failed refetch', () => {
|
||||
const v1 = buffer(1)
|
||||
const step = stepDocPreviewBinary({
|
||||
fileChanged: false,
|
||||
data: undefined,
|
||||
isPlaceholderData: false,
|
||||
error: new Error('boom'),
|
||||
hasCommittedContent: true,
|
||||
prevHasResolvedForFile: true,
|
||||
prevLastGood: v1,
|
||||
})
|
||||
|
||||
expect(step.resolved.state).toBe('stale')
|
||||
expect(step.resolved.data).toBe(v1)
|
||||
expect(step.resolved.error).toBeNull()
|
||||
})
|
||||
})
|
||||
+161
@@ -0,0 +1,161 @@
|
||||
'use client'
|
||||
|
||||
import { useRef } from 'react'
|
||||
import type { WorkspaceFileRecord } from '@/lib/uploads/contexts/workspace'
|
||||
import { useWorkspaceFileBinary } from '@/hooks/queries/workspace-files'
|
||||
|
||||
export type DocPreviewState = 'empty' | 'loading' | 'ready' | 'stale'
|
||||
|
||||
export interface DocPreviewBinary {
|
||||
data: ArrayBuffer | null
|
||||
state: DocPreviewState
|
||||
error: Error | null
|
||||
dataUpdatedAt: number
|
||||
}
|
||||
|
||||
type DocPreviewFile = Pick<WorkspaceFileRecord, 'id' | 'key' | 'size' | 'updatedAt'>
|
||||
|
||||
interface ResolveDocPreviewArgs {
|
||||
data: ArrayBuffer | undefined
|
||||
isPlaceholderData: boolean
|
||||
error: Error | null
|
||||
lastGood: ArrayBuffer | null
|
||||
hasCommittedContent: boolean
|
||||
}
|
||||
|
||||
interface ResolvedDocPreview {
|
||||
data: ArrayBuffer | null
|
||||
state: DocPreviewState
|
||||
error: Error | null
|
||||
lastGood: ArrayBuffer | null
|
||||
}
|
||||
|
||||
/**
|
||||
* Pure resolution of which binary to render and the render state, given the binary
|
||||
* query's current data, whether it is keep-previous placeholder data, the last
|
||||
* successfully fetched binary, and whether the file has committed content.
|
||||
*
|
||||
* A fresh (non-placeholder) success advances the head and renders `ready`. A
|
||||
* placeholder result (a recompile fetching a new version) or a fall back to the
|
||||
* last good binary after an error renders `stale`. Only the absence of any binary
|
||||
* yields `loading` (committed, still fetching) or `empty` (nothing committed). An
|
||||
* error is surfaced only when there is no binary to render.
|
||||
*/
|
||||
export function resolveDocPreviewBinary({
|
||||
data,
|
||||
isPlaceholderData,
|
||||
error,
|
||||
lastGood,
|
||||
hasCommittedContent,
|
||||
}: ResolveDocPreviewArgs): ResolvedDocPreview {
|
||||
const fresh = data && !isPlaceholderData ? data : null
|
||||
const nextLastGood = fresh ?? lastGood
|
||||
const resolvedData = data ?? nextLastGood ?? null
|
||||
|
||||
let state: DocPreviewState
|
||||
if (resolvedData) {
|
||||
state = fresh ? 'ready' : 'stale'
|
||||
} else {
|
||||
state = hasCommittedContent ? 'loading' : 'empty'
|
||||
}
|
||||
|
||||
return {
|
||||
data: resolvedData,
|
||||
state,
|
||||
error: resolvedData ? null : error,
|
||||
lastGood: nextLastGood,
|
||||
}
|
||||
}
|
||||
|
||||
interface DocPreviewStepArgs {
|
||||
fileChanged: boolean
|
||||
data: ArrayBuffer | undefined
|
||||
isPlaceholderData: boolean
|
||||
error: Error | null
|
||||
hasCommittedContent: boolean
|
||||
prevHasResolvedForFile: boolean
|
||||
prevLastGood: ArrayBuffer | null
|
||||
}
|
||||
|
||||
interface DocPreviewStep {
|
||||
resolved: ResolvedDocPreview
|
||||
hasResolvedForFile: boolean
|
||||
lastGood: ArrayBuffer | null
|
||||
}
|
||||
|
||||
/**
|
||||
* Pure per-render step for {@link useDocPreviewBinary}: folds the prior last-good
|
||||
* binary and the "has a fresh binary resolved for this file yet" flag with the
|
||||
* current query result. On a file change the prior file's last-good and resolved
|
||||
* flag are dropped, and the keep-previous placeholder (which still holds the prior
|
||||
* file's bytes) is ignored until a fresh binary resolves for the new file.
|
||||
*/
|
||||
export function stepDocPreviewBinary({
|
||||
fileChanged,
|
||||
data,
|
||||
isPlaceholderData,
|
||||
error,
|
||||
hasCommittedContent,
|
||||
prevHasResolvedForFile,
|
||||
prevLastGood,
|
||||
}: DocPreviewStepArgs): DocPreviewStep {
|
||||
const lastGood = fileChanged ? null : prevLastGood
|
||||
const hasResolvedForFile =
|
||||
(fileChanged ? false : prevHasResolvedForFile) || (Boolean(data) && !isPlaceholderData)
|
||||
const placeholderFromPriorFile = isPlaceholderData && !hasResolvedForFile
|
||||
const resolved = resolveDocPreviewBinary({
|
||||
data: placeholderFromPriorFile ? undefined : data,
|
||||
isPlaceholderData,
|
||||
error,
|
||||
lastGood,
|
||||
hasCommittedContent,
|
||||
})
|
||||
return { resolved, hasResolvedForFile, lastGood: resolved.lastGood }
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolves the compiled binary to render for a generated or uploaded document and
|
||||
* retains the last successfully fetched binary as a fallback.
|
||||
*
|
||||
* A compiled-doc preview is a function of the file record (`size`, `updatedAt`)
|
||||
* and the binary serve route, never of the streaming tool session. While a
|
||||
* recompile is fetching (a new `updatedAt`) the previously fetched binary keeps
|
||||
* rendering; if a fetch errors after a prior success the last good binary is held
|
||||
* rather than dropping to an error. A skeleton (`empty`/`loading`) shows only when
|
||||
* no binary has ever resolved for the file. On a file switch the keep-previous
|
||||
* placeholder (which still holds the prior file's bytes) is ignored until a fresh
|
||||
* binary resolves for the new file, so one viewer never renders another file's content.
|
||||
*/
|
||||
export function useDocPreviewBinary(workspaceId: string, file: DocPreviewFile): DocPreviewBinary {
|
||||
const query = useWorkspaceFileBinary(workspaceId, file.id, file.key, {
|
||||
enabled: (file.size ?? 0) > 0,
|
||||
version: Number(new Date(file.updatedAt)) || file.size,
|
||||
})
|
||||
|
||||
const lastGoodRef = useRef<ArrayBuffer | null>(null)
|
||||
const fileIdRef = useRef(file.id)
|
||||
const hasResolvedForFileRef = useRef(false)
|
||||
const fileChanged = fileIdRef.current !== file.id
|
||||
if (fileChanged) {
|
||||
fileIdRef.current = file.id
|
||||
}
|
||||
|
||||
const step = stepDocPreviewBinary({
|
||||
fileChanged,
|
||||
data: query.data,
|
||||
isPlaceholderData: query.isPlaceholderData,
|
||||
error: (query.error as Error | null) ?? null,
|
||||
hasCommittedContent: (file.size ?? 0) > 0,
|
||||
prevHasResolvedForFile: hasResolvedForFileRef.current,
|
||||
prevLastGood: lastGoodRef.current,
|
||||
})
|
||||
hasResolvedForFileRef.current = step.hasResolvedForFile
|
||||
lastGoodRef.current = step.lastGood
|
||||
|
||||
return {
|
||||
data: step.resolved.data,
|
||||
state: step.resolved.state,
|
||||
error: step.resolved.error,
|
||||
dataUpdatedAt: query.dataUpdatedAt,
|
||||
}
|
||||
}
|
||||
+7
-161
@@ -1,19 +1,15 @@
|
||||
'use client'
|
||||
|
||||
import { memo, useCallback, useEffect, useMemo, useRef, useState } from 'react'
|
||||
import { memo, useEffect, useRef, useState } from 'react'
|
||||
import { createLogger } from '@sim/logger'
|
||||
import { toError } from '@sim/utils/errors'
|
||||
import type { WorkBook } from 'xlsx'
|
||||
import { Button, Skeleton } from '@/components/emcn'
|
||||
import { cn } from '@/lib/core/utils/cn'
|
||||
import type { WorkspaceFileRecord } from '@/lib/uploads/contexts/workspace'
|
||||
import {
|
||||
useUpdateWorkspaceFileContent,
|
||||
useWorkspaceFileBinary,
|
||||
} from '@/hooks/queries/workspace-files'
|
||||
import type { DataTableHandle } from './data-table'
|
||||
import { DataTable } from './data-table'
|
||||
import { PreviewError, resolvePreviewError } from './preview-shared'
|
||||
import { useDocPreviewBinary } from './use-doc-preview-binary'
|
||||
|
||||
const logger = createLogger('XlsxPreview')
|
||||
|
||||
@@ -54,37 +50,18 @@ const XLSX_SKELETON = (
|
||||
export const XlsxPreview = memo(function XlsxPreview({
|
||||
file,
|
||||
workspaceId,
|
||||
canEdit,
|
||||
onSaveStatusChange,
|
||||
saveRef,
|
||||
}: {
|
||||
file: WorkspaceFileRecord
|
||||
workspaceId: string
|
||||
canEdit: boolean
|
||||
onSaveStatusChange?: (status: 'idle' | 'saving' | 'saved' | 'error') => void
|
||||
saveRef?: React.MutableRefObject<(() => Promise<void>) | null>
|
||||
}) {
|
||||
const {
|
||||
data: fileData,
|
||||
isLoading,
|
||||
error: fetchError,
|
||||
} = useWorkspaceFileBinary(workspaceId, file.id, file.key)
|
||||
const preview = useDocPreviewBinary(workspaceId, file)
|
||||
const fileData = preview.data
|
||||
|
||||
const [sheetNames, setSheetNames] = useState<string[]>([])
|
||||
const [activeSheet, setActiveSheet] = useState(0)
|
||||
const [currentSheet, setCurrentSheet] = useState<XlsxSheet | null>(null)
|
||||
const [renderError, setRenderError] = useState<string | null>(null)
|
||||
const [isDirty, setIsDirty] = useState(false)
|
||||
const [isSaving, setIsSaving] = useState(false)
|
||||
const isSavingRef = useRef(false)
|
||||
const workbookRef = useRef<WorkBook | null>(null)
|
||||
const xlsxModuleRef = useRef<typeof import('xlsx') | null>(null)
|
||||
const dataTableRef = useRef<DataTableHandle>(null)
|
||||
const updateContent = useUpdateWorkspaceFileContent()
|
||||
const updateContentRef = useRef(updateContent)
|
||||
updateContentRef.current = updateContent
|
||||
const onSaveStatusChangeRef = useRef(onSaveStatusChange)
|
||||
onSaveStatusChangeRef.current = onSaveStatusChange
|
||||
|
||||
useEffect(() => {
|
||||
if (!fileData) return
|
||||
@@ -95,9 +72,7 @@ export const XlsxPreview = memo(function XlsxPreview({
|
||||
async function parse() {
|
||||
try {
|
||||
setRenderError(null)
|
||||
setIsDirty(false)
|
||||
const XLSX = await import('xlsx')
|
||||
xlsxModuleRef.current = XLSX
|
||||
const workbook = XLSX.read(new Uint8Array(data), { type: 'array' })
|
||||
if (!cancelled) {
|
||||
workbookRef.current = workbook
|
||||
@@ -157,122 +132,9 @@ export const XlsxPreview = memo(function XlsxPreview({
|
||||
}
|
||||
}, [sheetNames, activeSheet])
|
||||
|
||||
const handleCellChange = useCallback(
|
||||
(row: number, col: number, value: string) => {
|
||||
const wb = workbookRef.current
|
||||
const XLSX = xlsxModuleRef.current
|
||||
if (wb && XLSX) {
|
||||
const sheetName = sheetNames[activeSheet]
|
||||
const ws = wb.Sheets[sheetName]
|
||||
if (ws) {
|
||||
const cellAddr = XLSX.utils.encode_cell({ r: row + 1, c: col })
|
||||
const numValue = Number(value)
|
||||
ws[cellAddr] =
|
||||
value !== '' && !Number.isNaN(numValue) ? { t: 'n', v: numValue } : { t: 's', v: value }
|
||||
}
|
||||
}
|
||||
setCurrentSheet((prev) => {
|
||||
if (!prev) return prev
|
||||
const newRows = prev.rows.map((r, ri) =>
|
||||
ri === row ? r.map((v, ci) => (ci === col ? value : v)) : r
|
||||
)
|
||||
return { ...prev, rows: newRows }
|
||||
})
|
||||
setIsDirty(true)
|
||||
},
|
||||
[activeSheet, sheetNames]
|
||||
)
|
||||
|
||||
const handleHeaderChange = useCallback(
|
||||
(col: number, value: string) => {
|
||||
const wb = workbookRef.current
|
||||
const XLSX = xlsxModuleRef.current
|
||||
if (wb && XLSX) {
|
||||
const sheetName = sheetNames[activeSheet]
|
||||
const ws = wb.Sheets[sheetName]
|
||||
if (ws) {
|
||||
const cellAddr = XLSX.utils.encode_cell({ r: 0, c: col })
|
||||
ws[cellAddr] = { t: 's', v: value }
|
||||
}
|
||||
}
|
||||
setCurrentSheet((prev) => {
|
||||
if (!prev) return prev
|
||||
const newHeaders = prev.headers.map((h, i) => (i === col ? value : h))
|
||||
return { ...prev, headers: newHeaders }
|
||||
})
|
||||
setIsDirty(true)
|
||||
},
|
||||
[activeSheet, sheetNames]
|
||||
)
|
||||
|
||||
const handleSave = useCallback(async () => {
|
||||
dataTableRef.current?.commitEdit()
|
||||
const wb = workbookRef.current
|
||||
if (!wb || isSavingRef.current) return
|
||||
|
||||
try {
|
||||
isSavingRef.current = true
|
||||
setIsSaving(true)
|
||||
onSaveStatusChangeRef.current?.('saving')
|
||||
|
||||
const XLSX = await import('xlsx')
|
||||
const binary: number[] = XLSX.write(wb, { type: 'array', bookType: 'xlsx' })
|
||||
const bytes = new Uint8Array(binary)
|
||||
|
||||
const chunkSize = 8192
|
||||
const parts: string[] = []
|
||||
for (let i = 0; i < bytes.length; i += chunkSize) {
|
||||
parts.push(String.fromCharCode(...bytes.slice(i, i + chunkSize)))
|
||||
}
|
||||
const base64 = btoa(parts.join(''))
|
||||
|
||||
await updateContentRef.current.mutateAsync({
|
||||
workspaceId,
|
||||
fileId: file.id,
|
||||
content: base64,
|
||||
encoding: 'base64',
|
||||
})
|
||||
|
||||
setIsDirty(false)
|
||||
onSaveStatusChangeRef.current?.('saved')
|
||||
} catch (err) {
|
||||
logger.error('XLSX save failed', { error: toError(err).message })
|
||||
onSaveStatusChangeRef.current?.('error')
|
||||
} finally {
|
||||
isSavingRef.current = false
|
||||
setIsSaving(false)
|
||||
}
|
||||
}, [workspaceId, file.id])
|
||||
|
||||
useEffect(() => {
|
||||
if (!saveRef) return
|
||||
saveRef.current = handleSave
|
||||
return () => {
|
||||
if (saveRef.current === handleSave) saveRef.current = null
|
||||
}
|
||||
}, [handleSave, saveRef])
|
||||
|
||||
useEffect(() => {
|
||||
if (!canEdit) return
|
||||
const onKeyDown = (e: KeyboardEvent) => {
|
||||
if ((e.metaKey || e.ctrlKey) && e.key === 's') {
|
||||
e.preventDefault()
|
||||
handleSave()
|
||||
}
|
||||
}
|
||||
window.addEventListener('keydown', onKeyDown)
|
||||
return () => window.removeEventListener('keydown', onKeyDown)
|
||||
}, [canEdit, handleSave])
|
||||
|
||||
const editConfig = useMemo(
|
||||
() =>
|
||||
canEdit ? { onCellChange: handleCellChange, onHeaderChange: handleHeaderChange } : undefined,
|
||||
[canEdit, handleCellChange, handleHeaderChange]
|
||||
)
|
||||
|
||||
const error = resolvePreviewError(fetchError, renderError)
|
||||
const error = resolvePreviewError(preview.error, renderError)
|
||||
if (error) return <PreviewError label='spreadsheet' error={error} />
|
||||
if (isLoading || currentSheet === null) return XLSX_SKELETON
|
||||
if (!fileData || currentSheet === null) return XLSX_SKELETON
|
||||
|
||||
return (
|
||||
<div className='flex flex-1 flex-col overflow-hidden'>
|
||||
@@ -295,25 +157,9 @@ export const XlsxPreview = memo(function XlsxPreview({
|
||||
</Button>
|
||||
))}
|
||||
</div>
|
||||
{canEdit && isDirty && (
|
||||
<Button
|
||||
variant='primary'
|
||||
size='sm'
|
||||
onClick={handleSave}
|
||||
disabled={isSaving}
|
||||
className='mr-3'
|
||||
>
|
||||
{isSaving ? 'Saving…' : 'Save'}
|
||||
</Button>
|
||||
)}
|
||||
</div>
|
||||
<div className='flex-1 overflow-auto p-6'>
|
||||
<DataTable
|
||||
ref={dataTableRef}
|
||||
headers={currentSheet.headers}
|
||||
rows={currentSheet.rows}
|
||||
editConfig={editConfig}
|
||||
/>
|
||||
<DataTable headers={currentSheet.headers} rows={currentSheet.rows} />
|
||||
{currentSheet.truncated && (
|
||||
<p className='mt-3 text-center text-[12px] text-[var(--text-muted)]'>
|
||||
Showing first {XLSX_MAX_ROWS.toLocaleString()} rows. Download the file to view all data.
|
||||
|
||||
+44
-17
@@ -8,6 +8,7 @@ import { Credit } from '@/components/emcn/icons'
|
||||
import { ON_DEMAND_UNLIMITED } from '@/lib/billing/constants'
|
||||
import { formatCredits } from '@/lib/billing/credits/conversion'
|
||||
import { isBillingEnabled } from '@/app/workspace/[workspaceId]/settings/navigation'
|
||||
import { useMyMemberCredits } from '@/hooks/queries/organization'
|
||||
import { usePlanView } from '@/hooks/queries/plan-view'
|
||||
import { prefetchUpgradeBillingData, useSubscriptionData } from '@/hooks/queries/subscription'
|
||||
import { prefetchWorkspaceSettings } from '@/hooks/queries/workspace'
|
||||
@@ -30,6 +31,7 @@ function CreditsChipInner() {
|
||||
const router = useRouter()
|
||||
const queryClient = useQueryClient()
|
||||
const { workspaceId } = useParams<{ workspaceId: string }>()
|
||||
const { data: memberCredits, isLoading: memberLoading } = useMyMemberCredits(workspaceId)
|
||||
|
||||
const upgradeHref = `/workspace/${workspaceId}/upgrade`
|
||||
|
||||
@@ -43,22 +45,7 @@ function CreditsChipInner() {
|
||||
prefetchWorkspaceSettings(queryClient, workspaceId)
|
||||
}, [router, queryClient, upgradeHref, workspaceId])
|
||||
|
||||
if (isLoading || !hasData || !data?.data) return null
|
||||
if (!planView.showCredits) return null
|
||||
|
||||
const { usageLimit, currentUsage, creditBalance } = data.data
|
||||
|
||||
/**
|
||||
* Credits remaining = unused plan allowance plus any purchased credit balance.
|
||||
* Uncapped plans (limit at/above the on-demand threshold) render as ∞ via
|
||||
* `formatCredits`, so short-circuit instead of subtracting usage from it.
|
||||
*/
|
||||
const remainingCredits =
|
||||
usageLimit >= ON_DEMAND_UNLIMITED
|
||||
? ON_DEMAND_UNLIMITED
|
||||
: Math.max(0, usageLimit + creditBalance - currentUsage)
|
||||
|
||||
return (
|
||||
const renderChip = (dollars: number) => (
|
||||
<Chip
|
||||
aria-label='Credits remaining — upgrade plan'
|
||||
onClick={() => router.push(upgradeHref)}
|
||||
@@ -66,7 +53,47 @@ function CreditsChipInner() {
|
||||
onFocus={prefetchUpgrade}
|
||||
leftIcon={Credit}
|
||||
>
|
||||
{formatCredits(remainingCredits)}
|
||||
{formatCredits(dollars)}
|
||||
</Chip>
|
||||
)
|
||||
|
||||
// Wait for the per-member cap result before rendering: until it resolves,
|
||||
// `limitDollars` is null and a capped member would briefly see the larger
|
||||
// pooled number. Disabled (no workspace) → not loading, so non-org users are
|
||||
// unaffected; cached after the first load (30s staleTime), so it's a one-time
|
||||
// wait, not a per-navigation one.
|
||||
if (memberLoading) return null
|
||||
|
||||
/**
|
||||
* Pooled/plan remaining (dollars): unused plan allowance plus any purchased
|
||||
* credit balance. Null when the plan-based chip wouldn't show on its own (data
|
||||
* not ready, or the plan isn't credit-metered). `ON_DEMAND_UNLIMITED` means
|
||||
* effectively unbounded — rendered as ∞ — so short-circuit instead of
|
||||
* subtracting usage from the sentinel.
|
||||
*/
|
||||
const pooledData = !isLoading && hasData && planView.showCredits ? (data?.data ?? null) : null
|
||||
const pooledRemaining =
|
||||
pooledData === null
|
||||
? null
|
||||
: pooledData.usageLimit >= ON_DEMAND_UNLIMITED
|
||||
? ON_DEMAND_UNLIMITED
|
||||
: Math.max(0, pooledData.usageLimit + pooledData.creditBalance - pooledData.currentUsage)
|
||||
|
||||
/**
|
||||
* A per-member cap is the authoritative personal remaining, but the actor gate
|
||||
* blocks on the pooled cap first — so show the tighter of the two, or a member
|
||||
* could see credits left while every action 402s on org/plan usage. Clamp at 0.
|
||||
* Fall back to personal alone when pooled isn't available/shown, so a capped
|
||||
* member still sees a balance even where the plan chip would be hidden.
|
||||
*/
|
||||
const limitDollars = memberCredits?.limitDollars ?? null
|
||||
if (limitDollars !== null) {
|
||||
const personalRemaining = Math.max(0, limitDollars - (memberCredits?.usedDollars ?? 0))
|
||||
return renderChip(
|
||||
pooledRemaining === null ? personalRemaining : Math.min(personalRemaining, pooledRemaining)
|
||||
)
|
||||
}
|
||||
|
||||
if (pooledRemaining === null) return null
|
||||
return renderChip(pooledRemaining)
|
||||
}
|
||||
|
||||
+45
-31
@@ -7,9 +7,24 @@ import type { ToolCallData } from '../../../../types'
|
||||
import { getAgentIcon } from '../../utils'
|
||||
import { ToolCallItem } from './tool-call-item'
|
||||
|
||||
/**
|
||||
* A subagent group nested inside another agent's output. Carries the same shape
|
||||
* as a top-level group so {@link AgentGroup} can render it recursively, which is
|
||||
* how deterministic parent/child nesting (e.g. Deploy inside Workflow) is drawn.
|
||||
*/
|
||||
export interface NestedAgentGroup {
|
||||
id: string
|
||||
agentName: string
|
||||
agentLabel: string
|
||||
items: AgentGroupItem[]
|
||||
isDelegating: boolean
|
||||
isOpen: boolean
|
||||
}
|
||||
|
||||
export type AgentGroupItem =
|
||||
| { type: 'text'; content: string }
|
||||
| { type: 'tool'; data: ToolCallData }
|
||||
| { type: 'agent_group'; group: NestedAgentGroup }
|
||||
|
||||
interface AgentGroupProps {
|
||||
agentName: string
|
||||
@@ -17,7 +32,6 @@ interface AgentGroupProps {
|
||||
items: AgentGroupItem[]
|
||||
isDelegating?: boolean
|
||||
isStreaming?: boolean
|
||||
autoCollapse?: boolean
|
||||
defaultExpanded?: boolean
|
||||
}
|
||||
|
||||
@@ -27,7 +41,8 @@ function isToolDone(status: ToolCallData['status']): boolean {
|
||||
status === 'error' ||
|
||||
status === 'cancelled' ||
|
||||
status === 'skipped' ||
|
||||
status === 'rejected'
|
||||
status === 'rejected' ||
|
||||
status === 'interrupted'
|
||||
)
|
||||
}
|
||||
|
||||
@@ -37,7 +52,6 @@ export function AgentGroup({
|
||||
items,
|
||||
isDelegating = false,
|
||||
isStreaming = false,
|
||||
autoCollapse = false,
|
||||
defaultExpanded = false,
|
||||
}: AgentGroupProps) {
|
||||
const AgentIcon = getAgentIcon(agentName)
|
||||
@@ -46,43 +60,29 @@ export function AgentGroup({
|
||||
(item): item is Extract<AgentGroupItem, { type: 'tool' }> => item.type === 'tool'
|
||||
)
|
||||
const allDone = toolItems.length > 0 && toolItems.every((t) => isToolDone(t.data.status))
|
||||
// Only a live turn can be delegating. Once the turn is terminal (complete,
|
||||
// errored, or stopped) no subagent should spin — even one aborted before its
|
||||
// first tool call, where `allDone` is false because there are no tools yet.
|
||||
const showDelegatingSpinner = isStreaming && isDelegating && !allDone
|
||||
|
||||
const [expanded, setExpanded] = useState(defaultExpanded || !allDone)
|
||||
const didAutoCollapseRef = useRef(allDone)
|
||||
const wasAutoExpandedRef = useRef(defaultExpanded)
|
||||
|
||||
useEffect(() => {
|
||||
if (defaultExpanded) {
|
||||
wasAutoExpandedRef.current = true
|
||||
setExpanded(true)
|
||||
return
|
||||
}
|
||||
|
||||
if (wasAutoExpandedRef.current && allDone) {
|
||||
wasAutoExpandedRef.current = false
|
||||
setExpanded(false)
|
||||
}
|
||||
}, [defaultExpanded, allDone])
|
||||
|
||||
useEffect(() => {
|
||||
if (!autoCollapse || didAutoCollapseRef.current) return
|
||||
didAutoCollapseRef.current = true
|
||||
setExpanded(false)
|
||||
}, [autoCollapse])
|
||||
// Expand only while the turn is live and the group is still open or working.
|
||||
// Once the turn ends (isStreaming false) — or a subagent closes mid-turn — the
|
||||
// group auto-collapses, so finished subagent blocks never stay expanded. A
|
||||
// manual toggle pins the choice for the rest of the message.
|
||||
const autoExpanded = isStreaming && (defaultExpanded || !allDone)
|
||||
const [manualExpanded, setManualExpanded] = useState<boolean | null>(null)
|
||||
const expanded = manualExpanded ?? autoExpanded
|
||||
|
||||
return (
|
||||
<div className='flex flex-col gap-1.5'>
|
||||
{hasItems ? (
|
||||
<button
|
||||
type='button'
|
||||
onClick={() => {
|
||||
wasAutoExpandedRef.current = false
|
||||
setExpanded((prev) => !prev)
|
||||
}}
|
||||
onClick={() => setManualExpanded(!expanded)}
|
||||
className='flex cursor-pointer items-center gap-2'
|
||||
>
|
||||
<div className='flex size-[16px] flex-shrink-0 items-center justify-center'>
|
||||
{isDelegating ? (
|
||||
{showDelegatingSpinner ? (
|
||||
<PillsRing className='size-[15px] text-[var(--text-icon)]' animate />
|
||||
) : (
|
||||
<AgentIcon className='size-[16px] text-[var(--text-icon)]' />
|
||||
@@ -99,7 +99,7 @@ export function AgentGroup({
|
||||
) : (
|
||||
<div className='flex items-center gap-2'>
|
||||
<div className='flex size-[16px] flex-shrink-0 items-center justify-center'>
|
||||
{isDelegating ? (
|
||||
{showDelegatingSpinner ? (
|
||||
<PillsRing className='size-[15px] text-[var(--text-icon)]' animate />
|
||||
) : (
|
||||
<AgentIcon className='size-[16px] text-[var(--text-icon)]' />
|
||||
@@ -125,6 +125,20 @@ export function AgentGroup({
|
||||
/>
|
||||
)
|
||||
}
|
||||
if (item.type === 'agent_group') {
|
||||
return (
|
||||
<div key={item.group.id} className='pl-6'>
|
||||
<AgentGroup
|
||||
agentName={item.group.agentName}
|
||||
agentLabel={item.group.agentLabel}
|
||||
items={item.group.items}
|
||||
isDelegating={item.group.isDelegating}
|
||||
isStreaming={isStreaming}
|
||||
defaultExpanded={item.group.isOpen}
|
||||
/>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
return (
|
||||
<span
|
||||
key={`text-${idx}`}
|
||||
|
||||
+1
-1
@@ -1,3 +1,3 @@
|
||||
export type { AgentGroupItem } from './agent-group'
|
||||
export type { AgentGroupItem, NestedAgentGroup } from './agent-group'
|
||||
export { AgentGroup } from './agent-group'
|
||||
export { CircleStop } from './tool-call-item'
|
||||
|
||||
+18
@@ -42,6 +42,21 @@ export function CircleStop({ className }: { className?: string }) {
|
||||
)
|
||||
}
|
||||
|
||||
function Hyphen({ className }: { className?: string }) {
|
||||
return (
|
||||
<svg
|
||||
width='16'
|
||||
height='16'
|
||||
viewBox='0 0 16 16'
|
||||
fill='none'
|
||||
xmlns='http://www.w3.org/2000/svg'
|
||||
className={className}
|
||||
>
|
||||
<path d='M4 8H12' stroke='currentColor' strokeWidth='1.25' strokeLinecap='round' />
|
||||
</svg>
|
||||
)
|
||||
}
|
||||
|
||||
function StatusIcon({ status, toolName }: { status: ToolCallStatus; toolName: string }) {
|
||||
if (status === 'executing') {
|
||||
return <PillsRing className='size-[15px] text-[var(--text-tertiary)]' animate />
|
||||
@@ -49,6 +64,9 @@ function StatusIcon({ status, toolName }: { status: ToolCallStatus; toolName: st
|
||||
if (status === 'cancelled') {
|
||||
return <CircleStop className='size-[15px] text-[var(--text-tertiary)]' />
|
||||
}
|
||||
if (status === 'interrupted') {
|
||||
return <Hyphen className='size-[15px] text-[var(--text-tertiary)]' />
|
||||
}
|
||||
const Icon = getToolIcon(toolName)
|
||||
if (Icon) {
|
||||
return <Icon className='size-[15px] text-[var(--text-tertiary)]' />
|
||||
|
||||
+22
@@ -0,0 +1,22 @@
|
||||
/**
|
||||
* @vitest-environment node
|
||||
*/
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { sanitizeChatDisplayContent } from './chat-sanitize'
|
||||
|
||||
describe('sanitizeChatDisplayContent', () => {
|
||||
it('unwraps workspace resource tags from inline code spans', () => {
|
||||
const content =
|
||||
'`I updated <workspace_resource>{"type":"workflow","id":"wf-1","title":"Workflow"}</workspace_resource>.`'
|
||||
|
||||
expect(sanitizeChatDisplayContent(content)).toBe(
|
||||
'I updated <workspace_resource>{"type":"workflow","id":"wf-1","title":"Workflow"}</workspace_resource>.'
|
||||
)
|
||||
})
|
||||
|
||||
it('removes hidden internal references wrapped in inline code', () => {
|
||||
const content = 'Read `internal/tool-results/read-1.md` and found the issue.'
|
||||
|
||||
expect(sanitizeChatDisplayContent(content)).toBe('Read and found the issue.')
|
||||
})
|
||||
})
|
||||
+52
-10
@@ -18,6 +18,8 @@ import {
|
||||
SpecialTags,
|
||||
} from '@/app/workspace/[workspaceId]/home/components/message-content/components/special-tags'
|
||||
import type { MothershipResource } from '@/app/workspace/[workspaceId]/home/types'
|
||||
import { useSmoothText } from '@/hooks/use-smooth-text'
|
||||
import { sanitizeChatDisplayContent } from './chat-sanitize'
|
||||
|
||||
const LANG_ALIASES: Record<string, string> = {
|
||||
js: 'javascript',
|
||||
@@ -47,6 +49,19 @@ const PROSE_CLASSES = cn(
|
||||
'prose-table:my-0'
|
||||
)
|
||||
|
||||
/**
|
||||
* Soft fade for newly revealed text. Paired with {@link useSmoothText}, which
|
||||
* paces the reveal: `sep: 'char'` fades each character as the pacer exposes it
|
||||
* (so a growing trailing word never re-animates), and `stagger: 0` keeps the
|
||||
* cadence driven by the pacer rather than an overlapping per-token delay ramp.
|
||||
*/
|
||||
const STREAM_ANIMATION = {
|
||||
animation: 'fadeIn',
|
||||
duration: 220,
|
||||
stagger: 0,
|
||||
sep: 'char',
|
||||
} as const
|
||||
|
||||
function startsInlineWord(value: string): boolean {
|
||||
return /^[A-Za-z0-9_(]/.test(value)
|
||||
}
|
||||
@@ -58,7 +73,7 @@ function endsInlineWord(value: string): boolean {
|
||||
function nextInlineSegmentLabel(segment?: ContentSegment): string {
|
||||
if (!segment) return ''
|
||||
if (segment.type === 'text' || segment.type === 'thinking') return segment.content
|
||||
if (segment.type === 'workspace_resource') return segment.data.title || segment.data.id
|
||||
if (segment.type === 'workspace_resource') return segment.data.title || segment.data.id || ''
|
||||
return ''
|
||||
}
|
||||
|
||||
@@ -168,10 +183,15 @@ const MARKDOWN_COMPONENTS = {
|
||||
e.preventDefault()
|
||||
const match = href.match(/^#wsres-(\w+)-(.+)$/)
|
||||
if (match) {
|
||||
const linkText = e.currentTarget.textContent || match[2]
|
||||
const type = match[1]
|
||||
const ref = match[2]
|
||||
const linkText = e.currentTarget.textContent || ref
|
||||
window.dispatchEvent(
|
||||
new CustomEvent('wsres-click', {
|
||||
detail: { type: match[1], id: match[2], title: linkText },
|
||||
detail:
|
||||
type === 'file'
|
||||
? { type, path: ref, title: linkText }
|
||||
: { type, id: ref, title: linkText },
|
||||
})
|
||||
)
|
||||
}
|
||||
@@ -270,16 +290,27 @@ function ChatContentInner({
|
||||
const onWorkspaceResourceSelectRef = useRef(onWorkspaceResourceSelect)
|
||||
onWorkspaceResourceSelectRef.current = onWorkspaceResourceSelect
|
||||
|
||||
const displayContent = useMemo(() => sanitizeChatDisplayContent(content), [content])
|
||||
const streamedContent = useSmoothText(displayContent, isStreaming)
|
||||
|
||||
useEffect(() => {
|
||||
const handler = (e: Event) => {
|
||||
const { type, id, title } = (e as CustomEvent).detail
|
||||
onWorkspaceResourceSelectRef.current?.({ type, id, title: title || id })
|
||||
const { type, id, path, title } = (e as CustomEvent).detail
|
||||
onWorkspaceResourceSelectRef.current?.({
|
||||
type,
|
||||
id: id ?? '',
|
||||
path,
|
||||
title: title || id || path || '',
|
||||
})
|
||||
}
|
||||
window.addEventListener('wsres-click', handler)
|
||||
return () => window.removeEventListener('wsres-click', handler)
|
||||
}, [])
|
||||
|
||||
const parsed = useMemo(() => parseSpecialTags(content, isStreaming), [content, isStreaming])
|
||||
const parsed = useMemo(
|
||||
() => parseSpecialTags(streamedContent, isStreaming),
|
||||
[streamedContent, isStreaming]
|
||||
)
|
||||
const hasSpecialContent = parsed.hasPendingTag || parsed.segments.some((s) => s.type !== 'text')
|
||||
|
||||
if (hasSpecialContent) {
|
||||
@@ -305,10 +336,14 @@ function ChatContentInner({
|
||||
const s = parsed.segments[i]
|
||||
const nextSegment = parsed.segments[i + 1]
|
||||
if (s.type === 'workspace_resource') {
|
||||
const label = s.data.title || s.data.id
|
||||
// Files are addressed by their encoded VFS path (copied verbatim from the tag);
|
||||
// workflows/tables/KBs by id. The angle-bracket link destination keeps the path
|
||||
// intact through markdown parsing (tolerates parens) without re-encoding it.
|
||||
const ref = s.data.type === 'file' ? (s.data.path ?? s.data.id ?? '') : (s.data.id ?? '')
|
||||
const label = s.data.title || ref
|
||||
pendingMarkdown = appendInlineReferenceMarkdown(
|
||||
pendingMarkdown,
|
||||
`[${label}](#wsres-${s.data.type}-${s.data.id})`,
|
||||
`[${label}](<#wsres-${s.data.type}-${ref}>)`,
|
||||
nextSegment
|
||||
)
|
||||
} else if (s.type === 'text' || s.type === 'thinking') {
|
||||
@@ -331,6 +366,8 @@ function ChatContentInner({
|
||||
>
|
||||
<Streamdown
|
||||
mode={isStreaming ? undefined : 'static'}
|
||||
animated={isStreaming ? STREAM_ANIMATION : false}
|
||||
isAnimating={isStreaming}
|
||||
components={MARKDOWN_COMPONENTS}
|
||||
>
|
||||
{group.markdown}
|
||||
@@ -353,8 +390,13 @@ function ChatContentInner({
|
||||
|
||||
return (
|
||||
<div className={cn(PROSE_CLASSES, '[&>:first-child]:mt-0 [&>:last-child]:mb-0')}>
|
||||
<Streamdown mode={isStreaming ? undefined : 'static'} components={MARKDOWN_COMPONENTS}>
|
||||
{content}
|
||||
<Streamdown
|
||||
mode={isStreaming ? undefined : 'static'}
|
||||
animated={isStreaming ? STREAM_ANIMATION : false}
|
||||
isAnimating={isStreaming}
|
||||
components={MARKDOWN_COMPONENTS}
|
||||
>
|
||||
{streamedContent}
|
||||
</Streamdown>
|
||||
</div>
|
||||
)
|
||||
|
||||
+12
@@ -0,0 +1,12 @@
|
||||
const HIDDEN_INLINE_REFERENCE_PATTERN =
|
||||
/`[^`\n]*(?:internal\/tool-results\/|internal\/blocktips\/|components\/integrations\/[^`\n]*README)[^`\n]*`/g
|
||||
const WORKSPACE_RESOURCE_CODE_SPAN_PATTERN =
|
||||
/`([^`\n]*<workspace_resource>[\s\S]*?<\/workspace_resource>[^`\n]*)`/g
|
||||
|
||||
export function sanitizeChatDisplayContent(content: string): string {
|
||||
return content
|
||||
.replace(WORKSPACE_RESOURCE_CODE_SPAN_PATTERN, '$1')
|
||||
.replace(HIDDEN_INLINE_REFERENCE_PATTERN, '')
|
||||
.replace(/`(\s*<workspace_resource>)/g, '$1')
|
||||
.replace(/(<\/workspace_resource>\s*)`/g, '$1')
|
||||
}
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
export type { AgentGroupItem } from './agent-group'
|
||||
export type { AgentGroupItem, NestedAgentGroup } from './agent-group'
|
||||
export { AgentGroup, CircleStop } from './agent-group'
|
||||
export { ChatContent } from './chat-content'
|
||||
export { Options } from './options'
|
||||
|
||||
+33
-12
@@ -9,6 +9,7 @@ import {
|
||||
ExpandableContent,
|
||||
SecretReveal,
|
||||
} from '@/components/emcn'
|
||||
import { canonicalWorkspaceFilePath } from '@/lib/copilot/vfs/path-utils'
|
||||
import { cn } from '@/lib/core/utils/cn'
|
||||
import { OAUTH_PROVIDERS } from '@/lib/oauth/oauth'
|
||||
import { ContextMentionIcon } from '@/app/workspace/[workspaceId]/home/components/context-mention-icon'
|
||||
@@ -77,7 +78,8 @@ export type WorkspaceResourceTagType = (typeof WORKSPACE_RESOURCE_TAG_TYPES)[num
|
||||
|
||||
export interface WorkspaceResourceTagData {
|
||||
type: WorkspaceResourceTagType
|
||||
id: string
|
||||
id?: string
|
||||
path?: string
|
||||
title?: string
|
||||
}
|
||||
|
||||
@@ -169,12 +171,20 @@ function isMothershipErrorTagData(value: unknown): value is MothershipErrorTagDa
|
||||
|
||||
function isWorkspaceResourceTagData(value: unknown): value is WorkspaceResourceTagData {
|
||||
if (!isRecord(value)) return false
|
||||
return (
|
||||
typeof value.type === 'string' &&
|
||||
(WORKSPACE_RESOURCE_TAG_TYPES as readonly string[]).includes(value.type) &&
|
||||
typeof value.id === 'string' &&
|
||||
value.id.trim().length > 0
|
||||
)
|
||||
if (
|
||||
typeof value.type !== 'string' ||
|
||||
!(WORKSPACE_RESOURCE_TAG_TYPES as readonly string[]).includes(value.type)
|
||||
) {
|
||||
return false
|
||||
}
|
||||
if (value.title !== undefined && typeof value.title !== 'string') return false
|
||||
if (value.path !== undefined && typeof value.path !== 'string') return false
|
||||
if (value.id !== undefined && typeof value.id !== 'string') return false
|
||||
|
||||
const id = typeof value.id === 'string' ? value.id.trim() : ''
|
||||
const path = typeof value.path === 'string' ? value.path.trim() : ''
|
||||
if (value.type === 'file') return id.length > 0 || path.length > 0
|
||||
return id.length > 0
|
||||
}
|
||||
|
||||
export function parseJsonTagBody<T>(
|
||||
@@ -491,11 +501,11 @@ function toMothershipResourceType(type: WorkspaceResourceTagType): MothershipRes
|
||||
function toChatMessageContext(data: WorkspaceResourceTagData, label: string): ChatMessageContext {
|
||||
switch (data.type) {
|
||||
case 'workflow':
|
||||
return { kind: 'workflow', label, workflowId: data.id }
|
||||
return { kind: 'workflow', label, workflowId: data.id ?? '' }
|
||||
case 'table':
|
||||
return { kind: 'table', label, tableId: data.id }
|
||||
return { kind: 'table', label, tableId: data.id ?? '' }
|
||||
case 'file':
|
||||
return { kind: 'file', label, fileId: data.id }
|
||||
return { kind: 'file', label, fileId: data.id ?? data.path ?? '' }
|
||||
}
|
||||
}
|
||||
|
||||
@@ -513,6 +523,14 @@ export function WorkspaceResourceDisplay({
|
||||
const { data: knowledgeBases = [] } = useKnowledgeBasesQuery(workspaceId)
|
||||
|
||||
const resource = useMemo<MothershipResource>(() => {
|
||||
const fileFromPath =
|
||||
data.type === 'file' && data.path
|
||||
? files.find(
|
||||
(file) =>
|
||||
canonicalWorkspaceFilePath({ folderPath: file.folderPath, name: file.name }) ===
|
||||
data.path
|
||||
)
|
||||
: undefined
|
||||
const title =
|
||||
data.type === 'workflow'
|
||||
? (workflows.find((workflow) => workflow.id === data.id)?.name ??
|
||||
@@ -522,16 +540,19 @@ export function WorkspaceResourceDisplay({
|
||||
fallbackWorkspaceResourceTitle(data.type))
|
||||
: data.type === 'file'
|
||||
? (files.find((file) => file.id === data.id)?.name ??
|
||||
fileFromPath?.name ??
|
||||
data.title ??
|
||||
fallbackWorkspaceResourceTitle(data.type))
|
||||
: (knowledgeBases.find((knowledgeBase) => knowledgeBase.id === data.id)?.name ??
|
||||
fallbackWorkspaceResourceTitle(data.type))
|
||||
|
||||
return {
|
||||
type: toMothershipResourceType(data.type),
|
||||
id: data.id,
|
||||
id: data.id ?? fileFromPath?.id ?? data.path ?? '',
|
||||
title,
|
||||
...(data.type === 'file' && data.path ? { path: data.path } : {}),
|
||||
}
|
||||
}, [data.id, data.type, files, knowledgeBases, tables, workflows])
|
||||
}, [data.id, data.path, data.title, data.type, files, knowledgeBases, tables, workflows])
|
||||
|
||||
const context = toChatMessageContext(data, resource.title)
|
||||
|
||||
|
||||
+3
-18
@@ -1,8 +1,7 @@
|
||||
'use client'
|
||||
|
||||
import { useEffect, useLayoutEffect, useRef, useState } from 'react'
|
||||
import { ChevronDown, Expandable, ExpandableContent } from '@/components/emcn'
|
||||
import { BrainIcon } from '@/components/icons'
|
||||
import { Blimp, ChevronDown, Expandable, ExpandableContent } from '@/components/emcn'
|
||||
import { cn } from '@/lib/core/utils/cn'
|
||||
|
||||
interface ThinkingBlockProps {
|
||||
@@ -10,7 +9,6 @@ interface ThinkingBlockProps {
|
||||
isActive: boolean
|
||||
isStreaming?: boolean
|
||||
startedAt?: number
|
||||
endedAt?: number
|
||||
}
|
||||
|
||||
const MIN_VISIBLE_THINKING_MS = 3000
|
||||
@@ -20,7 +18,6 @@ export function ThinkingBlock({
|
||||
isActive,
|
||||
isStreaming = false,
|
||||
startedAt,
|
||||
endedAt,
|
||||
}: ThinkingBlockProps) {
|
||||
// Start collapsed so the `Expandable` plays its height-open animation
|
||||
// when `expanded` flips to true below — otherwise the panel mounts
|
||||
@@ -78,18 +75,6 @@ export function ThinkingBlock({
|
||||
|
||||
if (!thresholdReached) return null
|
||||
|
||||
const elapsedMs =
|
||||
startedAt !== undefined && endedAt !== undefined && endedAt >= startedAt
|
||||
? endedAt - startedAt
|
||||
: undefined
|
||||
const elapsedSeconds =
|
||||
elapsedMs !== undefined ? Math.max(1, Math.round(elapsedMs / 1000)) : undefined
|
||||
const label = isActive
|
||||
? 'Thinking'
|
||||
: elapsedSeconds !== undefined
|
||||
? `Thought for ${elapsedSeconds}s`
|
||||
: 'Thought'
|
||||
|
||||
return (
|
||||
<div className='flex flex-col gap-1.5'>
|
||||
<button
|
||||
@@ -98,9 +83,9 @@ export function ThinkingBlock({
|
||||
className='flex cursor-pointer items-center gap-2'
|
||||
>
|
||||
<div className='flex size-[16px] flex-shrink-0 items-center justify-center'>
|
||||
<BrainIcon className='size-[14px] text-[var(--text-icon)]' />
|
||||
<Blimp className='size-[16px] text-[var(--text-icon)]' />
|
||||
</div>
|
||||
<span className='text-[var(--text-body)] text-sm'>{label}</span>
|
||||
<span className='text-[var(--text-body)] text-sm'>Mothership</span>
|
||||
<ChevronDown
|
||||
className={cn(
|
||||
'h-[7px] w-[9px] text-[var(--text-icon)] transition-transform duration-150',
|
||||
|
||||
+201
@@ -0,0 +1,201 @@
|
||||
/**
|
||||
* @vitest-environment node
|
||||
*/
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import type { ContentBlock } from '../../types'
|
||||
import { parseBlocks, shouldSmoothTextSegment } from './message-content'
|
||||
|
||||
function subagentStart(name: string, spanId: string, parentSpanId: string): ContentBlock {
|
||||
return { type: 'subagent', content: name, spanId, parentSpanId, timestamp: 1 }
|
||||
}
|
||||
|
||||
function subagentToolCall(
|
||||
id: string,
|
||||
name: string,
|
||||
spanId: string,
|
||||
calledBy: string
|
||||
): ContentBlock {
|
||||
return {
|
||||
type: 'tool_call',
|
||||
toolCall: { id, name, status: 'success', calledBy },
|
||||
spanId,
|
||||
timestamp: 1,
|
||||
}
|
||||
}
|
||||
|
||||
function mainText(content: string): ContentBlock {
|
||||
return { type: 'text', content, timestamp: 1 }
|
||||
}
|
||||
|
||||
function mainToolCall(id: string, name: string): ContentBlock {
|
||||
return { type: 'tool_call', toolCall: { id, name, status: 'success' }, timestamp: 1 }
|
||||
}
|
||||
|
||||
describe('parseBlocks span-identity tree', () => {
|
||||
it('nests a deploy subagent inside the workflow subagent that spawned it', () => {
|
||||
const blocks: ContentBlock[] = [
|
||||
subagentStart('workflow', 'S1', 'main'),
|
||||
subagentToolCall('t1', 'create_workflow', 'S1', 'workflow'),
|
||||
subagentStart('deploy', 'S2', 'S1'),
|
||||
subagentToolCall('t2', 'check_deployment_status', 'S2', 'deploy'),
|
||||
]
|
||||
|
||||
const segments = parseBlocks(blocks)
|
||||
|
||||
expect(segments).toHaveLength(1)
|
||||
const workflow = segments[0]
|
||||
expect(workflow.type).toBe('agent_group')
|
||||
if (workflow.type !== 'agent_group') throw new Error('expected workflow group')
|
||||
expect(workflow.agentName).toBe('workflow')
|
||||
|
||||
const nested = workflow.items.find((item) => item.type === 'agent_group')
|
||||
expect(nested).toBeDefined()
|
||||
if (!nested || nested.type !== 'agent_group') throw new Error('expected nested deploy group')
|
||||
expect(nested.group.agentName).toBe('deploy')
|
||||
// Deploy's own tool nests under deploy, not under workflow.
|
||||
expect(nested.group.items.some((item) => item.type === 'tool')).toBe(true)
|
||||
})
|
||||
|
||||
it('keeps two top-level subagents as siblings', () => {
|
||||
const blocks: ContentBlock[] = [
|
||||
subagentStart('workflow', 'S1', 'main'),
|
||||
subagentStart('research', 'S3', 'main'),
|
||||
]
|
||||
|
||||
const segments = parseBlocks(blocks)
|
||||
const groups = segments.filter((s) => s.type === 'agent_group')
|
||||
expect(groups).toHaveLength(2)
|
||||
})
|
||||
|
||||
it('creates distinct groups for repeated deploy invocations (no collision)', () => {
|
||||
const blocks: ContentBlock[] = [
|
||||
subagentStart('deploy', 'S2', 'main'),
|
||||
subagentToolCall('t1', 'deploy_api', 'S2', 'deploy'),
|
||||
subagentStart('deploy', 'S4', 'main'),
|
||||
subagentToolCall('t2', 'deploy_api', 'S4', 'deploy'),
|
||||
]
|
||||
|
||||
const segments = parseBlocks(blocks)
|
||||
const groups = segments.filter((s) => s.type === 'agent_group')
|
||||
expect(groups).toHaveLength(2)
|
||||
})
|
||||
|
||||
it('shows the delegating spinner while a span subagent is open with no output, and clears it once content arrives', () => {
|
||||
const openOnly = parseBlocks([subagentStart('deploy', 'S2', 'main')])
|
||||
expect(openOnly).toHaveLength(1)
|
||||
if (openOnly[0].type !== 'agent_group') throw new Error('expected group')
|
||||
expect(openOnly[0].isDelegating).toBe(true)
|
||||
|
||||
const withContent = parseBlocks([
|
||||
subagentStart('deploy', 'S2', 'main'),
|
||||
{ type: 'subagent_text', content: 'working on it', spanId: 'S2', timestamp: 2 },
|
||||
])
|
||||
if (withContent[0].type !== 'agent_group') throw new Error('expected group')
|
||||
expect(withContent[0].isDelegating).toBe(false)
|
||||
})
|
||||
|
||||
it('prunes an empty nested subagent that started and ended without output', () => {
|
||||
const blocks: ContentBlock[] = [
|
||||
subagentStart('workflow', 'S1', 'main'),
|
||||
subagentToolCall('t1', 'create_workflow', 'S1', 'workflow'),
|
||||
subagentStart('deploy', 'S2', 'S1'),
|
||||
{ type: 'subagent_end', spanId: 'S2', parentSpanId: 'S1', timestamp: 3 },
|
||||
]
|
||||
const segments = parseBlocks(blocks)
|
||||
expect(segments).toHaveLength(1)
|
||||
if (segments[0].type !== 'agent_group') throw new Error('expected workflow group')
|
||||
// The empty, ended deploy group is pruned; only the workflow tool remains.
|
||||
expect(segments[0].items.some((item) => item.type === 'agent_group')).toBe(false)
|
||||
expect(segments[0].items.some((item) => item.type === 'tool')).toBe(true)
|
||||
})
|
||||
|
||||
it('interleaves mothership tools with main text instead of clustering them at the top', () => {
|
||||
const blocks: ContentBlock[] = [
|
||||
mainText('Let me search.'),
|
||||
mainToolCall('t1', 'grep'),
|
||||
subagentStart('research', 'S1', 'main'),
|
||||
{ type: 'subagent_text', content: 'looking', spanId: 'S1', timestamp: 2 },
|
||||
{ type: 'subagent_end', spanId: 'S1', parentSpanId: 'main', timestamp: 3 },
|
||||
mainText('Found it, now finding files.'),
|
||||
mainToolCall('t2', 'glob'),
|
||||
]
|
||||
|
||||
const segments = parseBlocks(blocks)
|
||||
|
||||
// Order is preserved chronologically: the second mothership tool stays below
|
||||
// the research subagent and the trailing text rather than jumping back up
|
||||
// into the first group.
|
||||
const shape = segments.map((s) => (s.type === 'agent_group' ? s.agentName : s.type))
|
||||
expect(shape).toEqual(['text', 'mothership', 'research', 'text', 'mothership'])
|
||||
|
||||
// The two mothership tools land in two distinct groups, one each.
|
||||
const mothershipGroups = segments.filter(
|
||||
(s) => s.type === 'agent_group' && s.agentName === 'mothership'
|
||||
)
|
||||
expect(mothershipGroups).toHaveLength(2)
|
||||
const [first, second] = mothershipGroups
|
||||
if (first.type !== 'agent_group' || second.type !== 'agent_group') {
|
||||
throw new Error('expected mothership groups')
|
||||
}
|
||||
expect(first.items).toHaveLength(1)
|
||||
expect(second.items).toHaveLength(1)
|
||||
expect(first.items[0].type === 'tool' && first.items[0].data.toolName).toBe('grep')
|
||||
expect(second.items[0].type === 'tool' && second.items[0].data.toolName).toBe('glob')
|
||||
})
|
||||
|
||||
it('absorbs the dispatch tool of a nested file subagent from its parent span group', () => {
|
||||
const blocks: ContentBlock[] = [
|
||||
subagentStart('workflow', 'S1', 'main'),
|
||||
subagentToolCall('t1', 'workspace_file', 'S1', 'workflow'),
|
||||
{ type: 'subagent', content: 'file', spanId: 'S2', parentSpanId: 'S1', timestamp: 2 },
|
||||
{ type: 'subagent_text', content: 'writing', spanId: 'S2', timestamp: 3 },
|
||||
]
|
||||
|
||||
const segments = parseBlocks(blocks)
|
||||
expect(segments).toHaveLength(1)
|
||||
const workflow = segments[0]
|
||||
if (workflow.type !== 'agent_group') throw new Error('expected workflow group')
|
||||
|
||||
// The workspace_file dispatch tool is absorbed (not shown as a sibling tool);
|
||||
// only the nested file subagent remains under workflow.
|
||||
expect(workflow.items.some((item) => item.type === 'tool')).toBe(false)
|
||||
const nested = workflow.items.find((item) => item.type === 'agent_group')
|
||||
if (!nested || nested.type !== 'agent_group') throw new Error('expected nested file group')
|
||||
expect(nested.group.agentName).toBe('file')
|
||||
})
|
||||
|
||||
it('falls back to legacy flat grouping when blocks have no span identity', () => {
|
||||
const blocks: ContentBlock[] = [
|
||||
{ type: 'subagent', content: 'workflow', parentToolCallId: 'tc-1', timestamp: 1 },
|
||||
{
|
||||
type: 'tool_call',
|
||||
toolCall: { id: 't1', name: 'create_workflow', status: 'success', calledBy: 'workflow' },
|
||||
parentToolCallId: 'tc-1',
|
||||
timestamp: 1,
|
||||
},
|
||||
]
|
||||
|
||||
const segments = parseBlocks(blocks)
|
||||
const groups = segments.filter((s) => s.type === 'agent_group')
|
||||
expect(groups).toHaveLength(1)
|
||||
if (groups[0].type !== 'agent_group') throw new Error('expected group')
|
||||
expect(groups[0].agentName).toBe('workflow')
|
||||
})
|
||||
})
|
||||
|
||||
describe('shouldSmoothTextSegment', () => {
|
||||
it('only smooths the trailing text segment of a live stream', () => {
|
||||
expect(shouldSmoothTextSegment({ isStreaming: true, segmentIndex: 0, segmentCount: 2 })).toBe(
|
||||
false
|
||||
)
|
||||
expect(shouldSmoothTextSegment({ isStreaming: true, segmentIndex: 1, segmentCount: 2 })).toBe(
|
||||
true
|
||||
)
|
||||
})
|
||||
|
||||
it('never smooths completed messages', () => {
|
||||
expect(shouldSmoothTextSegment({ isStreaming: false, segmentIndex: 0, segmentCount: 1 })).toBe(
|
||||
false
|
||||
)
|
||||
})
|
||||
})
|
||||
+282
-11
@@ -76,6 +76,10 @@ function isToolResultRead(params?: Record<string, unknown>): boolean {
|
||||
return typeof path === 'string' && path.startsWith('internal/tool-results/')
|
||||
}
|
||||
|
||||
function isHiddenToolCall(toolName: string | undefined): boolean {
|
||||
return isToolHiddenInUi(toolName)
|
||||
}
|
||||
|
||||
function formatToolName(name: string): string {
|
||||
return stripVersionSuffix(name)
|
||||
.split('_')
|
||||
@@ -93,7 +97,8 @@ function isToolDone(status: ToolCallData['status']): boolean {
|
||||
status === 'error' ||
|
||||
status === 'cancelled' ||
|
||||
status === 'skipped' ||
|
||||
status === 'rejected'
|
||||
status === 'rejected' ||
|
||||
status === 'interrupted'
|
||||
)
|
||||
}
|
||||
|
||||
@@ -146,13 +151,270 @@ function toToolData(tc: NonNullable<ContentBlock['toolCall']>): ToolCallData {
|
||||
}
|
||||
}
|
||||
|
||||
const SPAN_ROOT = 'main'
|
||||
|
||||
function createAgentGroupSegment(name: string, idKey: string, ordinal: number): AgentGroupSegment {
|
||||
return {
|
||||
type: 'agent_group',
|
||||
id: `agent-${idKey}-${ordinal}`,
|
||||
agentName: name,
|
||||
agentLabel: resolveAgentLabel(name),
|
||||
items: [],
|
||||
isDelegating: false,
|
||||
isOpen: false,
|
||||
}
|
||||
}
|
||||
|
||||
function appendTextItem(group: AgentGroupSegment, content: string): void {
|
||||
const lastItem = group.items[group.items.length - 1]
|
||||
if (lastItem?.type === 'text') {
|
||||
lastItem.content += content
|
||||
} else {
|
||||
group.items.push({ type: 'text', content })
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Deterministic span-identity grouping. Every subagent-scoped block carries the
|
||||
* stable `spanId` of the run that produced it and a `parentSpanId` linking it to
|
||||
* its caller. Groups are keyed by `spanId` and nested under their parent's group
|
||||
* via `parentSpanId`, producing a real tree (e.g. Deploy inside Workflow) with
|
||||
* no name/tool-call reverse lookups. Delegation tool_calls are absorbed — the
|
||||
* subagent span is the canonical representation of the nested agent.
|
||||
*/
|
||||
function parseBlocksWithSpanTree(blocks: ContentBlock[]): MessageSegment[] {
|
||||
const segments: MessageSegment[] = []
|
||||
const groupsBySpanId = new Map<string, AgentGroupSegment>()
|
||||
|
||||
const tailMothershipGroup = (): AgentGroupSegment | null => {
|
||||
const last = segments[segments.length - 1]
|
||||
return last?.type === 'agent_group' && last.agentName === 'mothership' ? last : null
|
||||
}
|
||||
|
||||
// Top-level (mothership) tool calls render in a collapsible group. Reuse that
|
||||
// group only while it is still the most recent segment so consecutive tools
|
||||
// stay together; once any other segment (main text, a spawned subagent,
|
||||
// thinking, etc.) breaks the run, the next tool opens a fresh group below it
|
||||
// instead of jumping back up into the original one. This keeps the mothership's
|
||||
// tools and prose interleaved in the order they actually happened.
|
||||
const ensureMothership = (): AgentGroupSegment => {
|
||||
const existing = tailMothershipGroup()
|
||||
if (existing) return existing
|
||||
const group = createAgentGroupSegment('mothership', 'mothership', segments.length)
|
||||
segments.push(group)
|
||||
return group
|
||||
}
|
||||
|
||||
// When a subagent spawns, drop the dispatch tool that triggered it (e.g.
|
||||
// workspace_file -> file) from whichever container it landed in so it does not
|
||||
// render as a separate entry beside the agent group.
|
||||
const absorbDispatchTool = (toolName: string, parentSpanId: string | undefined): void => {
|
||||
const container =
|
||||
parentSpanId && parentSpanId !== SPAN_ROOT
|
||||
? groupsBySpanId.get(parentSpanId)
|
||||
: tailMothershipGroup()
|
||||
if (!container) return
|
||||
const last = container.items[container.items.length - 1]
|
||||
if (last?.type === 'tool' && last.data.toolName === toolName) {
|
||||
container.items.pop()
|
||||
}
|
||||
}
|
||||
|
||||
const attachSpanGroup = (group: AgentGroupSegment, parentSpanId: string | undefined): void => {
|
||||
if (parentSpanId && parentSpanId !== SPAN_ROOT) {
|
||||
const parent = groupsBySpanId.get(parentSpanId)
|
||||
if (parent) {
|
||||
parent.items.push({ type: 'agent_group', group })
|
||||
return
|
||||
}
|
||||
}
|
||||
segments.push(group)
|
||||
}
|
||||
|
||||
const ensureSpanGroup = (
|
||||
name: string,
|
||||
spanId: string,
|
||||
parentSpanId: string | undefined,
|
||||
ordinal: number
|
||||
): AgentGroupSegment => {
|
||||
const existing = groupsBySpanId.get(spanId)
|
||||
if (existing) return existing
|
||||
const group = createAgentGroupSegment(name, spanId, ordinal)
|
||||
groupsBySpanId.set(spanId, group)
|
||||
attachSpanGroup(group, parentSpanId)
|
||||
return group
|
||||
}
|
||||
|
||||
const flushMainText = (content: string) => {
|
||||
const last = segments[segments.length - 1]
|
||||
if (last?.type === 'text') {
|
||||
last.content += content
|
||||
} else {
|
||||
segments.push({ type: 'text', content })
|
||||
}
|
||||
}
|
||||
|
||||
for (let i = 0; i < blocks.length; i++) {
|
||||
const block = blocks[i]
|
||||
|
||||
if (block.type === 'subagent_text' || block.type === 'subagent_thinking') {
|
||||
if (!block.content || !block.spanId) continue
|
||||
let g = groupsBySpanId.get(block.spanId)
|
||||
// Out-of-order safety: content can arrive before its subagent-start block
|
||||
// (live streaming across resume legs). Create the span group on demand,
|
||||
// nested via parentSpanId, instead of dropping the content.
|
||||
if (!g && block.subagent) {
|
||||
g = ensureSpanGroup(block.subagent, block.spanId, block.parentSpanId, i)
|
||||
}
|
||||
if (!g) continue
|
||||
g.isDelegating = false
|
||||
appendTextItem(g, block.content)
|
||||
continue
|
||||
}
|
||||
|
||||
if (block.type === 'thinking') {
|
||||
if (!block.content?.trim()) continue
|
||||
const last = segments[segments.length - 1]
|
||||
if (last?.type === 'thinking' && last.endedAt === undefined) {
|
||||
last.content += block.content
|
||||
if (block.endedAt !== undefined) last.endedAt = block.endedAt
|
||||
} else {
|
||||
segments.push({
|
||||
type: 'thinking',
|
||||
id: `thinking-${i}`,
|
||||
content: block.content,
|
||||
startedAt: block.timestamp,
|
||||
endedAt: block.endedAt,
|
||||
})
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
if (block.type === 'text') {
|
||||
if (!block.content) continue
|
||||
if (block.subagent && block.spanId) {
|
||||
let g = groupsBySpanId.get(block.spanId)
|
||||
// Out-of-order safety: see subagent_text branch above.
|
||||
if (!g) g = ensureSpanGroup(block.subagent, block.spanId, block.parentSpanId, i)
|
||||
if (g) {
|
||||
g.isDelegating = false
|
||||
appendTextItem(g, block.content)
|
||||
continue
|
||||
}
|
||||
}
|
||||
flushMainText(block.content)
|
||||
continue
|
||||
}
|
||||
|
||||
if (block.type === 'subagent') {
|
||||
if (!block.content || !block.spanId) continue
|
||||
// Absorb a trailing dispatch tool (e.g. workspace_file -> file) so it does
|
||||
// not render as a separate entry alongside the agent group.
|
||||
const dispatchToolName = SUBAGENT_DISPATCH_TOOLS[block.content]
|
||||
if (dispatchToolName) absorbDispatchTool(dispatchToolName, block.parentSpanId)
|
||||
const g = ensureSpanGroup(block.content, block.spanId, block.parentSpanId, i)
|
||||
// Show the working/delegating spinner from span open until the agent
|
||||
// emits its first content or tool (or ends). The legacy path derived this
|
||||
// from the dispatch tool_call, which the span path absorbs, so we set it
|
||||
// here. It is cleared in the subagent_text/subagent_thinking, scoped text,
|
||||
// tool_call, and subagent_end branches.
|
||||
g.isDelegating = true
|
||||
g.isOpen = true
|
||||
continue
|
||||
}
|
||||
|
||||
if (block.type === 'tool_call') {
|
||||
if (!block.toolCall) continue
|
||||
const tc = block.toolCall
|
||||
if (isHiddenToolCall(tc.name)) continue
|
||||
if (tc.name === ReadTool.id && isToolResultRead(tc.params)) continue
|
||||
// Delegation tools are represented by their subagent span group; absorb.
|
||||
if (SUBAGENT_KEYS.has(tc.name)) continue
|
||||
const tool = toToolData(tc)
|
||||
if (block.spanId) {
|
||||
let g = groupsBySpanId.get(block.spanId)
|
||||
// Out-of-order safety: a subagent's tool can stream before its
|
||||
// subagent-start block (live streaming across resume legs). Create the
|
||||
// span group on demand (nested via parentSpanId) so the tool nests
|
||||
// under its agent instead of leaking to the top-level mothership flow.
|
||||
if (!g && tc.calledBy) {
|
||||
g = ensureSpanGroup(tc.calledBy, block.spanId, block.parentSpanId, i)
|
||||
}
|
||||
if (g) {
|
||||
g.isDelegating = false
|
||||
g.items.push({ type: 'tool', data: tool })
|
||||
continue
|
||||
}
|
||||
}
|
||||
ensureMothership().items.push({ type: 'tool', data: tool })
|
||||
continue
|
||||
}
|
||||
|
||||
if (block.type === 'options') {
|
||||
if (!block.options?.length) continue
|
||||
segments.push({ type: 'options', items: block.options })
|
||||
continue
|
||||
}
|
||||
|
||||
if (block.type === 'subagent_end') {
|
||||
if (block.spanId) {
|
||||
const g = groupsBySpanId.get(block.spanId)
|
||||
if (g) {
|
||||
g.isOpen = false
|
||||
g.isDelegating = false
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
if (block.type === 'stopped') {
|
||||
segments.push({ type: 'stopped' })
|
||||
}
|
||||
}
|
||||
|
||||
// Recursively drop empty, closed, non-delegating nested groups so a subagent
|
||||
// that started and ended without emitting anything does not leave a stray
|
||||
// header row. The top-level filter below covers top-level groups.
|
||||
const pruneEmptyNested = (items: AgentGroupItem[]): AgentGroupItem[] =>
|
||||
items.filter((item) => {
|
||||
if (item.type !== 'agent_group') return true
|
||||
item.group.items = pruneEmptyNested(item.group.items)
|
||||
return item.group.items.length > 0 || item.group.isOpen || item.group.isDelegating
|
||||
})
|
||||
for (const segment of segments) {
|
||||
if (segment.type === 'agent_group') {
|
||||
segment.items = pruneEmptyNested(segment.items)
|
||||
}
|
||||
}
|
||||
|
||||
return segments.filter(
|
||||
(segment) =>
|
||||
segment.type !== 'agent_group' ||
|
||||
segment.items.length > 0 ||
|
||||
segment.isDelegating ||
|
||||
segment.isOpen
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Groups content blocks into agent-scoped segments.
|
||||
* Dispatch tool_calls (name matches a subagent key, no calledBy) are absorbed
|
||||
* into the agent header. Inner tool_calls are nested underneath their agent.
|
||||
* Orphan tool_calls (no calledBy, not a dispatch) group under "Mothership".
|
||||
*
|
||||
* New backends stamp every subagent block with deterministic span identity; in
|
||||
* that case {@link parseBlocksWithSpanTree} builds a real nested tree. The
|
||||
* legacy flat heuristics below are retained for transcripts persisted before
|
||||
* span identity existed.
|
||||
*/
|
||||
function parseBlocks(blocks: ContentBlock[]): MessageSegment[] {
|
||||
export function parseBlocks(blocks: ContentBlock[]): MessageSegment[] {
|
||||
if (blocks.some((block) => Boolean(block.spanId))) {
|
||||
return parseBlocksWithSpanTree(blocks)
|
||||
}
|
||||
return parseBlocksLegacy(blocks)
|
||||
}
|
||||
|
||||
function parseBlocksLegacy(blocks: ContentBlock[]): MessageSegment[] {
|
||||
const segments: MessageSegment[] = []
|
||||
const groupsByKey = new Map<string, AgentGroupSegment>()
|
||||
let activeGroupKey: string | null = null
|
||||
@@ -397,6 +659,18 @@ export function assistantMessageHasRenderableContent(
|
||||
return segments.length > 0
|
||||
}
|
||||
|
||||
export function shouldSmoothTextSegment({
|
||||
isStreaming,
|
||||
segmentIndex,
|
||||
segmentCount,
|
||||
}: {
|
||||
isStreaming: boolean
|
||||
segmentIndex: number
|
||||
segmentCount: number
|
||||
}): boolean {
|
||||
return isStreaming && segmentIndex === segmentCount - 1
|
||||
}
|
||||
|
||||
interface MessageContentProps {
|
||||
blocks: ContentBlock[]
|
||||
fallbackContent: string
|
||||
@@ -457,7 +731,11 @@ function MessageContentInner({
|
||||
<ChatContent
|
||||
key={`text-${i}`}
|
||||
content={segment.content}
|
||||
isStreaming={isStreaming}
|
||||
isStreaming={shouldSmoothTextSegment({
|
||||
isStreaming,
|
||||
segmentIndex: i,
|
||||
segmentCount: segments.length,
|
||||
})}
|
||||
onOptionSelect={onOptionSelect}
|
||||
onWorkspaceResourceSelect={onWorkspaceResourceSelect}
|
||||
/>
|
||||
@@ -479,19 +757,13 @@ function MessageContentInner({
|
||||
<ThinkingBlock
|
||||
content={segment.content}
|
||||
isActive={isActive}
|
||||
isStreaming={isStreaming}
|
||||
startedAt={segment.startedAt}
|
||||
endedAt={segment.endedAt}
|
||||
isStreaming={isStreaming}
|
||||
/>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
case 'agent_group': {
|
||||
const toolItems = segment.items.filter((item) => item.type === 'tool')
|
||||
const allToolsDone =
|
||||
toolItems.length === 0 ||
|
||||
toolItems.every((t) => t.type === 'tool' && isToolDone(t.data.status))
|
||||
const hasFollowingText = segments.slice(i + 1).some((s) => s.type === 'text')
|
||||
return (
|
||||
<div key={segment.id} className={isStreaming ? 'animate-stream-fade-in' : undefined}>
|
||||
<AgentGroup
|
||||
@@ -501,7 +773,6 @@ function MessageContentInner({
|
||||
items={segment.items}
|
||||
isDelegating={segment.isDelegating}
|
||||
isStreaming={isStreaming}
|
||||
autoCollapse={!segment.isOpen && allToolsDone && hasFollowingText}
|
||||
defaultExpanded={segment.isOpen}
|
||||
/>
|
||||
</div>
|
||||
|
||||
@@ -20,7 +20,7 @@ import {
|
||||
Wrench,
|
||||
} from '@/components/emcn'
|
||||
import { Calendar, Table as TableIcon } from '@/components/emcn/icons'
|
||||
import { AgentIcon } from '@/components/icons'
|
||||
import { AgentIcon, ImageIcon, TTSIcon, VideoIcon } from '@/components/icons'
|
||||
|
||||
export type IconComponent = ComponentType<SVGProps<SVGSVGElement>>
|
||||
|
||||
@@ -58,6 +58,11 @@ const TOOL_ICONS: Record<string, IconComponent> = {
|
||||
context_compaction: Asterisk,
|
||||
open_resource: Eye,
|
||||
file: File,
|
||||
media: VideoIcon,
|
||||
generate_image: ImageIcon,
|
||||
generate_video: VideoIcon,
|
||||
generate_audio: TTSIcon,
|
||||
ffmpeg: Wrench,
|
||||
}
|
||||
|
||||
export function getAgentIcon(name: string): IconComponent {
|
||||
|
||||
+55
-34
@@ -21,18 +21,20 @@ import {
|
||||
markRunToolManuallyStopped,
|
||||
reportManualRunToolStop,
|
||||
} from '@/lib/copilot/tools/client/run-tool-execution'
|
||||
import { canonicalWorkspaceFilePath } from '@/lib/copilot/vfs/path-utils'
|
||||
import { triggerFileDownload } from '@/lib/uploads/client/download'
|
||||
import { getFileExtension, getMimeTypeFromExtension } from '@/lib/uploads/utils/file-utils'
|
||||
import {
|
||||
FileViewer,
|
||||
type PreviewMode,
|
||||
resolveFileCategory,
|
||||
} from '@/app/workspace/[workspaceId]/files/components/file-viewer'
|
||||
import { GenericResourceContent } from '@/app/workspace/[workspaceId]/home/components/mothership-view/components/resource-content/components/generic-resource-content'
|
||||
import {
|
||||
RESOURCE_TAB_ICON_BUTTON_CLASS,
|
||||
RESOURCE_TAB_ICON_CLASS,
|
||||
} from '@/app/workspace/[workspaceId]/home/components/mothership-view/components/resource-tabs/resource-tab-controls'
|
||||
import { hasRenderableFilePreviewContent } from '@/app/workspace/[workspaceId]/home/hooks/use-file-preview-sessions'
|
||||
import { hasRenderableFilePreviewContent } from '@/app/workspace/[workspaceId]/home/hooks/preview'
|
||||
import type {
|
||||
GenericResourceData,
|
||||
MothershipResource,
|
||||
@@ -115,39 +117,28 @@ export const ResourceContent = memo(function ResourceContent({
|
||||
}, [workspaceId, streamFileName])
|
||||
|
||||
const disableStreamingAutoScroll = previewSession?.operation === 'patch'
|
||||
const rawPreviewText = previewSession?.previewText
|
||||
const streamingPreviewText =
|
||||
previewSession &&
|
||||
typeof rawPreviewText === 'string' &&
|
||||
const isTextPreview =
|
||||
!!previewSession && resolveFileCategory(null, previewSession.fileName) === 'text-editable'
|
||||
const textStreamingContent =
|
||||
isTextPreview &&
|
||||
typeof previewSession?.previewText === 'string' &&
|
||||
hasRenderableFilePreviewContent(previewSession)
|
||||
? rawPreviewText
|
||||
: undefined
|
||||
const pendingOrStreamingFilePreviewText =
|
||||
previewSession?.fileId === resource.id &&
|
||||
typeof rawPreviewText === 'string' &&
|
||||
hasRenderableFilePreviewContent(previewSession)
|
||||
? rawPreviewText
|
||||
? previewSession.previewText
|
||||
: undefined
|
||||
|
||||
if (resource.id === 'streaming-file') {
|
||||
return (
|
||||
<div className='flex h-full flex-col overflow-hidden'>
|
||||
{streamingPreviewText !== undefined ? (
|
||||
<FileViewer
|
||||
file={syntheticFile}
|
||||
workspaceId={workspaceId}
|
||||
canEdit={false}
|
||||
previewMode={previewMode ?? 'preview'}
|
||||
streamingContent={streamingPreviewText}
|
||||
streamingMode='replace'
|
||||
disableStreamingAutoScroll={disableStreamingAutoScroll}
|
||||
previewContextKey={previewContextKey}
|
||||
/>
|
||||
) : (
|
||||
<div className='flex h-full items-center justify-center'>
|
||||
<p className='text-[13px] text-[var(--text-muted)]'>Processing file…</p>
|
||||
</div>
|
||||
)}
|
||||
<FileViewer
|
||||
file={syntheticFile}
|
||||
workspaceId={workspaceId}
|
||||
canEdit={false}
|
||||
previewMode={previewMode ?? 'preview'}
|
||||
streamingContent={textStreamingContent}
|
||||
streamingMode='replace'
|
||||
disableStreamingAutoScroll={disableStreamingAutoScroll}
|
||||
previewContextKey={previewContextKey}
|
||||
/>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -162,8 +153,11 @@ export const ResourceContent = memo(function ResourceContent({
|
||||
key={resource.id}
|
||||
workspaceId={workspaceId}
|
||||
fileId={resource.id}
|
||||
filePath={resource.path}
|
||||
previewMode={previewMode}
|
||||
streamingContent={pendingOrStreamingFilePreviewText}
|
||||
streamingContent={
|
||||
previewSession?.fileId === resource.id ? textStreamingContent : undefined
|
||||
}
|
||||
streamingMode='replace'
|
||||
disableStreamingAutoScroll={disableStreamingAutoScroll}
|
||||
previewContextKey={previewContextKey}
|
||||
@@ -218,7 +212,13 @@ export function ResourceActions({ workspaceId, resource }: ResourceActionsProps)
|
||||
case 'workflow':
|
||||
return <EmbeddedWorkflowActions workspaceId={workspaceId} workflowId={resource.id} />
|
||||
case 'file':
|
||||
return <EmbeddedFileActions workspaceId={workspaceId} fileId={resource.id} />
|
||||
return (
|
||||
<EmbeddedFileActions
|
||||
workspaceId={workspaceId}
|
||||
fileId={resource.id}
|
||||
filePath={resource.path}
|
||||
/>
|
||||
)
|
||||
case 'knowledgebase':
|
||||
return (
|
||||
<EmbeddedKnowledgeBaseActions workspaceId={workspaceId} knowledgeBaseId={resource.id} />
|
||||
@@ -426,12 +426,22 @@ const fileLogger = createLogger('EmbeddedFileActions')
|
||||
interface EmbeddedFileActionsProps {
|
||||
workspaceId: string
|
||||
fileId: string
|
||||
filePath?: string
|
||||
}
|
||||
|
||||
function EmbeddedFileActions({ workspaceId, fileId }: EmbeddedFileActionsProps) {
|
||||
function EmbeddedFileActions({ workspaceId, fileId, filePath }: EmbeddedFileActionsProps) {
|
||||
const router = useRouter()
|
||||
const { data: files = [] } = useWorkspaceFiles(workspaceId)
|
||||
const file = useMemo(() => files.find((f) => f.id === fileId), [files, fileId])
|
||||
const file = useMemo(
|
||||
() =>
|
||||
files.find(
|
||||
(f) =>
|
||||
f.id === fileId ||
|
||||
(filePath &&
|
||||
canonicalWorkspaceFilePath({ folderPath: f.folderPath, name: f.name }) === filePath)
|
||||
),
|
||||
[files, fileId, filePath]
|
||||
)
|
||||
|
||||
const handleDownload = async () => {
|
||||
if (!file) return
|
||||
@@ -443,7 +453,7 @@ function EmbeddedFileActions({ workspaceId, fileId }: EmbeddedFileActionsProps)
|
||||
}
|
||||
|
||||
const handleOpenInFiles = () => {
|
||||
router.push(`/workspace/${workspaceId}/files/${encodeURIComponent(fileId)}`)
|
||||
router.push(`/workspace/${workspaceId}/files/${encodeURIComponent(file?.id ?? fileId)}`)
|
||||
}
|
||||
|
||||
return (
|
||||
@@ -521,6 +531,7 @@ function EmbeddedWorkflow({ workspaceId, workflowId }: EmbeddedWorkflowProps) {
|
||||
interface EmbeddedFileProps {
|
||||
workspaceId: string
|
||||
fileId: string
|
||||
filePath?: string
|
||||
previewMode?: PreviewMode
|
||||
streamingContent?: string
|
||||
streamingMode?: 'append' | 'replace'
|
||||
@@ -531,6 +542,7 @@ interface EmbeddedFileProps {
|
||||
function EmbeddedFile({
|
||||
workspaceId,
|
||||
fileId,
|
||||
filePath,
|
||||
previewMode,
|
||||
streamingContent,
|
||||
streamingMode,
|
||||
@@ -539,7 +551,16 @@ function EmbeddedFile({
|
||||
}: EmbeddedFileProps) {
|
||||
const { canEdit } = useUserPermissionsContext()
|
||||
const { data: files = [], isLoading, isFetching } = useWorkspaceFiles(workspaceId)
|
||||
const file = useMemo(() => files.find((f) => f.id === fileId), [files, fileId])
|
||||
const file = useMemo(
|
||||
() =>
|
||||
files.find(
|
||||
(f) =>
|
||||
f.id === fileId ||
|
||||
(filePath &&
|
||||
canonicalWorkspaceFilePath({ folderPath: f.folderPath, name: f.name }) === filePath)
|
||||
),
|
||||
[files, fileId, filePath]
|
||||
)
|
||||
|
||||
if (isLoading || (isFetching && !file)) return LOADING_SKELETON
|
||||
|
||||
|
||||
+1
-1
@@ -6,7 +6,7 @@ import { cn } from '@/lib/core/utils/cn'
|
||||
import { getFileExtension } from '@/lib/uploads/utils/file-utils'
|
||||
import type { PreviewMode } from '@/app/workspace/[workspaceId]/files/components/file-viewer'
|
||||
import { RICH_PREVIEWABLE_EXTENSIONS } from '@/app/workspace/[workspaceId]/files/components/file-viewer'
|
||||
import { hasRenderableFilePreviewContent } from '@/app/workspace/[workspaceId]/home/hooks/use-file-preview-sessions'
|
||||
import { hasRenderableFilePreviewContent } from '@/app/workspace/[workspaceId]/home/hooks/preview'
|
||||
import type {
|
||||
GenericResourceData,
|
||||
MothershipResource,
|
||||
|
||||
@@ -13,7 +13,7 @@ import {
|
||||
} from 'react'
|
||||
import { createLogger } from '@sim/logger'
|
||||
import { useParams } from 'next/navigation'
|
||||
import { Button, Paperclip, Slash, Tooltip } from '@/components/emcn'
|
||||
import { Button, Paperclip, Slash, Tooltip, toast } from '@/components/emcn'
|
||||
import { getMothershipAttachmentPreviewUrl } from '@/lib/copilot/chat/attachment-preview'
|
||||
import { SIM_RESOURCE_DRAG_TYPE, SIM_RESOURCES_DRAG_TYPE } from '@/lib/copilot/resource-types'
|
||||
import { cn } from '@/lib/core/utils/cn'
|
||||
@@ -402,8 +402,20 @@ export const UserInput = forwardRef<UserInputHandle, UserInputProps>(function Us
|
||||
valueRef.current = converted
|
||||
}
|
||||
|
||||
function handleUsageLimitExceeded() {
|
||||
navigateToSettings({ section: 'billing' })
|
||||
function handleUsageLimitExceeded(message?: string, isMemberLimit?: boolean) {
|
||||
// A per-member cap can only be raised by an org admin, so don't offer Upgrade
|
||||
// (the member can't act on it) — the message already tells them to ask an admin.
|
||||
toast.error(
|
||||
message || 'You are out of credits.',
|
||||
isMemberLimit
|
||||
? undefined
|
||||
: {
|
||||
action: {
|
||||
label: 'Upgrade',
|
||||
onClick: () => navigateToSettings({ section: 'billing' }),
|
||||
},
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
const {
|
||||
@@ -414,6 +426,7 @@ export const UserInput = forwardRef<UserInputHandle, UserInputProps>(function Us
|
||||
} = useSpeechToText({
|
||||
onTranscript: handleTranscript,
|
||||
onUsageLimitExceeded: handleUsageLimitExceeded,
|
||||
workspaceId,
|
||||
})
|
||||
|
||||
const toggleListening = useCallback(() => {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
'use client'
|
||||
|
||||
import { useCallback, useEffect, useRef, useState } from 'react'
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from 'react'
|
||||
import { createLogger } from '@sim/logger'
|
||||
import { useParams, useRouter } from 'next/navigation'
|
||||
import { usePostHog } from 'posthog-js/react'
|
||||
@@ -8,6 +8,12 @@ import { Button } from '@/components/emcn'
|
||||
import { PanelLeft } from '@/components/emcn/icons'
|
||||
import { requestJson } from '@/lib/api/client/request'
|
||||
import { createWorkflowContract } from '@/lib/api/contracts'
|
||||
import { canonicalWorkspaceFilePath } from '@/lib/copilot/vfs/path-utils'
|
||||
import {
|
||||
buildWorkflowAliasWorkflowEntries,
|
||||
resolveWorkflowAliasPath,
|
||||
resolveWorkspacePlanAliasPath,
|
||||
} from '@/lib/copilot/vfs/workflow-aliases'
|
||||
import {
|
||||
LandingPromptStorage,
|
||||
type LandingWorkflowSeed,
|
||||
@@ -19,10 +25,13 @@ import {
|
||||
} from '@/lib/mothership/events'
|
||||
import { captureEvent } from '@/lib/posthog/client'
|
||||
import { persistImportedWorkflow } from '@/lib/workflows/operations/import-export'
|
||||
import { useFolders } from '@/hooks/queries/folders'
|
||||
import {
|
||||
useMarkMothershipChatRead,
|
||||
useMothershipChatHistory,
|
||||
} from '@/hooks/queries/mothership-chats'
|
||||
import { useWorkflows } from '@/hooks/queries/workflows'
|
||||
import { useWorkspaceFiles } from '@/hooks/queries/workspace-files'
|
||||
import { useOAuthReturnRouter } from '@/hooks/use-oauth-return'
|
||||
import type { ChatContext } from '@/stores/panel'
|
||||
import {
|
||||
@@ -50,6 +59,9 @@ export function Home({ chatId, userName, userId, initialResourceId = null }: Hom
|
||||
const { workspaceId } = useParams<{ workspaceId: string }>()
|
||||
const router = useRouter()
|
||||
const firstName = userName?.split(' ')[0] ?? ''
|
||||
const { data: workspaceFiles = [] } = useWorkspaceFiles(workspaceId)
|
||||
const { data: workflows = [] } = useWorkflows(workspaceId)
|
||||
const { data: folders = [] } = useFolders(workspaceId)
|
||||
const posthog = usePostHog()
|
||||
const posthogRef = useRef(posthog)
|
||||
posthogRef.current = posthog
|
||||
@@ -291,10 +303,61 @@ export function Home({ chatId, userName, userId, initialResourceId = null }: Hom
|
||||
removeResource(resolved.type, resolved.id)
|
||||
}
|
||||
|
||||
const workflowAliasEntries = useMemo(
|
||||
() =>
|
||||
buildWorkflowAliasWorkflowEntries(
|
||||
workflows.map((workflow) => ({
|
||||
id: workflow.id,
|
||||
name: workflow.name,
|
||||
folderId: workflow.folderId ?? null,
|
||||
})),
|
||||
folders.map((folder) => ({
|
||||
folderId: folder.id,
|
||||
folderName: folder.name,
|
||||
parentId: folder.parentId ?? null,
|
||||
}))
|
||||
),
|
||||
[folders, workflows]
|
||||
)
|
||||
|
||||
const resolveFileResource = useCallback(
|
||||
(resource: MothershipResource): MothershipResource => {
|
||||
if (resource.type !== 'file') return resource
|
||||
|
||||
const reference = (resource.path || resource.id).trim()
|
||||
const workspacePlanAlias = resolveWorkspacePlanAliasPath(reference)
|
||||
const workflowAlias = workspacePlanAlias
|
||||
? null
|
||||
: resolveWorkflowAliasPath(reference, workflowAliasEntries)
|
||||
const alias = workspacePlanAlias || workflowAlias
|
||||
const targetPath = alias && alias.kind !== 'plans_dir' ? alias.backingPath : reference
|
||||
|
||||
const file = workspaceFiles.find((candidate) => {
|
||||
const candidatePath = canonicalWorkspaceFilePath({
|
||||
folderPath: candidate.folderPath,
|
||||
name: candidate.name,
|
||||
})
|
||||
return (
|
||||
candidate.id === reference || candidatePath === reference || candidatePath === targetPath
|
||||
)
|
||||
})
|
||||
|
||||
if (!file) return resource
|
||||
return {
|
||||
...resource,
|
||||
id: file.id,
|
||||
title: resource.title || file.name,
|
||||
path: alias ? reference : resource.path,
|
||||
}
|
||||
},
|
||||
[workflowAliasEntries, workspaceFiles]
|
||||
)
|
||||
|
||||
function handleWorkspaceResourceSelect(resource: MothershipResource) {
|
||||
const wasAdded = addResource(resource)
|
||||
const resolvedResource = resolveFileResource(resource)
|
||||
const wasAdded = addResource(resolvedResource)
|
||||
if (!wasAdded) {
|
||||
setActiveResourceId(resource.id)
|
||||
setActiveResourceId(resolvedResource.id)
|
||||
}
|
||||
handleResourceEvent()
|
||||
}
|
||||
|
||||
+137
@@ -0,0 +1,137 @@
|
||||
/**
|
||||
* @vitest-environment node
|
||||
*/
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import type { FilePreviewSession } from '@/lib/copilot/request/session'
|
||||
import { deriveFilePreviewSession } from './apply-file-preview-phase'
|
||||
|
||||
const NOW = '2026-06-08T00:00:00.000Z'
|
||||
|
||||
function session(overrides: Partial<FilePreviewSession>): FilePreviewSession {
|
||||
return {
|
||||
schemaVersion: 1,
|
||||
id: 'tool-1',
|
||||
streamId: 'stream-1',
|
||||
toolCallId: 'tool-1',
|
||||
status: 'streaming',
|
||||
fileName: 'deck.pptx',
|
||||
previewText: '',
|
||||
previewVersion: 1,
|
||||
updatedAt: NOW,
|
||||
...overrides,
|
||||
}
|
||||
}
|
||||
|
||||
describe('deriveFilePreviewSession', () => {
|
||||
it('starts a pending session keyed to the tool call', () => {
|
||||
const next = deriveFilePreviewSession(
|
||||
undefined,
|
||||
{ previewPhase: 'file_preview_start', toolCallId: 'tool-1', toolName: 'workspace_file' },
|
||||
'stream-1',
|
||||
NOW
|
||||
)
|
||||
expect(next.status).toBe('pending')
|
||||
expect(next.id).toBe('tool-1')
|
||||
expect(next.previewVersion).toBe(0)
|
||||
expect(next.streamId).toBe('stream-1')
|
||||
})
|
||||
|
||||
it('captures target identity (fileId, name, kind, operation)', () => {
|
||||
const next = deriveFilePreviewSession(
|
||||
session({ status: 'pending' }),
|
||||
{
|
||||
previewPhase: 'file_preview_target',
|
||||
toolCallId: 'tool-1',
|
||||
toolName: 'workspace_file',
|
||||
operation: 'append',
|
||||
target: { kind: 'file_id', fileId: 'file-9', fileName: 'deck.pptx' },
|
||||
},
|
||||
'stream-1',
|
||||
NOW
|
||||
)
|
||||
expect(next.fileId).toBe('file-9')
|
||||
expect(next.targetKind).toBe('file_id')
|
||||
expect(next.operation).toBe('append')
|
||||
expect(next.status).toBe('pending')
|
||||
})
|
||||
|
||||
it('appends delta content and advances the version monotonically when none supplied', () => {
|
||||
const prev = session({ previewText: 'slide one', previewVersion: 2 })
|
||||
const next = deriveFilePreviewSession(
|
||||
prev,
|
||||
{
|
||||
previewPhase: 'file_preview_content',
|
||||
toolCallId: 'tool-1',
|
||||
toolName: 'workspace_file',
|
||||
content: ' slide two',
|
||||
contentMode: 'delta',
|
||||
fileName: 'deck.pptx',
|
||||
},
|
||||
'stream-1',
|
||||
NOW
|
||||
)
|
||||
expect(next.status).toBe('streaming')
|
||||
expect(next.previewText).toBe('slide one slide two')
|
||||
expect(next.previewVersion).toBe(3)
|
||||
})
|
||||
|
||||
it('appends delta content and uses the supplied version verbatim', () => {
|
||||
const prev = session({ previewText: 'slide one', previewVersion: 2 })
|
||||
const next = deriveFilePreviewSession(
|
||||
prev,
|
||||
{
|
||||
previewPhase: 'file_preview_content',
|
||||
toolCallId: 'tool-1',
|
||||
toolName: 'workspace_file',
|
||||
content: ' slide two',
|
||||
contentMode: 'delta',
|
||||
previewVersion: 9,
|
||||
fileName: 'deck.pptx',
|
||||
},
|
||||
'stream-1',
|
||||
NOW
|
||||
)
|
||||
expect(next.previewText).toBe('slide one slide two')
|
||||
expect(next.previewVersion).toBe(9)
|
||||
})
|
||||
|
||||
it('replaces text on a snapshot and carries forward prior fileId', () => {
|
||||
const prev = session({ previewText: 'old', fileId: 'file-9', previewVersion: 4 })
|
||||
const next = deriveFilePreviewSession(
|
||||
prev,
|
||||
{
|
||||
previewPhase: 'file_preview_content',
|
||||
toolCallId: 'tool-1',
|
||||
toolName: 'workspace_file',
|
||||
content: 'fresh snapshot',
|
||||
contentMode: 'snapshot',
|
||||
previewVersion: 5,
|
||||
fileName: 'deck.pptx',
|
||||
},
|
||||
undefined,
|
||||
NOW
|
||||
)
|
||||
expect(next.previewText).toBe('fresh snapshot')
|
||||
expect(next.fileId).toBe('file-9')
|
||||
expect(next.streamId).toBe('stream-1')
|
||||
})
|
||||
|
||||
it('marks completion with completedAt and a resolved version', () => {
|
||||
const prev = session({ previewText: 'final', previewVersion: 7, fileId: 'file-9' })
|
||||
const next = deriveFilePreviewSession(
|
||||
prev,
|
||||
{
|
||||
previewPhase: 'file_preview_complete',
|
||||
toolCallId: 'tool-1',
|
||||
toolName: 'workspace_file',
|
||||
fileId: 'file-9',
|
||||
},
|
||||
'stream-1',
|
||||
NOW
|
||||
)
|
||||
expect(next.status).toBe('complete')
|
||||
expect(next.completedAt).toBe(NOW)
|
||||
expect(next.previewVersion).toBe(7)
|
||||
expect(next.fileId).toBe('file-9')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,93 @@
|
||||
import type { SyntheticFilePreviewPayload } from '@/lib/copilot/request/session'
|
||||
import type {
|
||||
FilePreviewSession,
|
||||
FilePreviewTargetKind,
|
||||
} from '@/lib/copilot/request/session/file-preview-session-contract'
|
||||
|
||||
function toTargetKind(value: string | undefined): FilePreviewTargetKind | undefined {
|
||||
return value === 'new_file' || value === 'file_id' ? value : undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* Derives the next {@link FilePreviewSession} from a synthetic preview-phase
|
||||
* payload and the prior session for that tool call. Pure: identity fields prefer
|
||||
* the payload then fall back to the previous session, content accumulates by
|
||||
* delta/snapshot, and `previewVersion` advances monotonically. `now` is injected
|
||||
* so the result is deterministic; the caller supplies a single timestamp used for
|
||||
* both `updatedAt` and (on completion) `completedAt`.
|
||||
*/
|
||||
export function deriveFilePreviewSession(
|
||||
prev: FilePreviewSession | undefined,
|
||||
payload: SyntheticFilePreviewPayload,
|
||||
streamId: string | undefined,
|
||||
now: string
|
||||
): FilePreviewSession {
|
||||
const id = payload.toolCallId
|
||||
|
||||
let targetKind = prev?.targetKind
|
||||
let fileId = prev?.fileId
|
||||
let fileName = prev?.fileName ?? ''
|
||||
let operation = prev?.operation
|
||||
let edit = prev?.edit
|
||||
|
||||
if (payload.previewPhase === 'file_preview_target') {
|
||||
targetKind = toTargetKind(payload.target.kind) ?? targetKind
|
||||
fileId = payload.target.fileId ?? fileId
|
||||
fileName = payload.target.fileName ?? fileName
|
||||
operation = payload.operation ?? operation
|
||||
} else if (payload.previewPhase === 'file_preview_content') {
|
||||
targetKind = toTargetKind(payload.targetKind) ?? targetKind
|
||||
fileId = payload.fileId ?? fileId
|
||||
fileName = payload.fileName ?? fileName
|
||||
operation = payload.operation ?? operation
|
||||
edit = payload.edit ?? edit
|
||||
} else if (payload.previewPhase === 'file_preview_edit_meta') {
|
||||
edit = payload.edit ?? edit
|
||||
} else if (payload.previewPhase === 'file_preview_complete') {
|
||||
fileId = payload.fileId ?? fileId
|
||||
}
|
||||
|
||||
const base: FilePreviewSession = {
|
||||
schemaVersion: 1,
|
||||
id,
|
||||
streamId: streamId ?? prev?.streamId ?? '',
|
||||
toolCallId: id,
|
||||
status: prev?.status ?? 'pending',
|
||||
fileName,
|
||||
...(fileId ? { fileId } : {}),
|
||||
...(targetKind ? { targetKind } : {}),
|
||||
...(operation ? { operation } : {}),
|
||||
...(edit ? { edit } : {}),
|
||||
previewText: prev?.previewText ?? '',
|
||||
previewVersion: prev?.previewVersion ?? 0,
|
||||
updatedAt: now,
|
||||
...(prev?.completedAt ? { completedAt: prev.completedAt } : {}),
|
||||
}
|
||||
|
||||
switch (payload.previewPhase) {
|
||||
case 'file_preview_start':
|
||||
case 'file_preview_target':
|
||||
case 'file_preview_edit_meta':
|
||||
return base
|
||||
|
||||
case 'file_preview_content': {
|
||||
const previewText =
|
||||
payload.contentMode === 'delta'
|
||||
? (prev?.previewText ?? '') + payload.content
|
||||
: payload.content
|
||||
const previewVersion =
|
||||
typeof payload.previewVersion === 'number' && Number.isFinite(payload.previewVersion)
|
||||
? payload.previewVersion
|
||||
: (prev?.previewVersion ?? 0) + 1
|
||||
return { ...base, status: 'streaming', previewText, previewVersion }
|
||||
}
|
||||
|
||||
case 'file_preview_complete':
|
||||
return {
|
||||
...base,
|
||||
status: 'complete',
|
||||
previewVersion: payload.previewVersion ?? prev?.previewVersion ?? 0,
|
||||
completedAt: now,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
export { useFilePreviewController } from './use-file-preview-controller'
|
||||
export {
|
||||
hasRenderableFilePreviewContent,
|
||||
shouldReplaceSession,
|
||||
} from './use-file-preview-sessions'
|
||||
+412
@@ -0,0 +1,412 @@
|
||||
'use client'
|
||||
|
||||
import {
|
||||
type Dispatch,
|
||||
type MutableRefObject,
|
||||
type SetStateAction,
|
||||
useCallback,
|
||||
useRef,
|
||||
} from 'react'
|
||||
import { useQueryClient } from '@tanstack/react-query'
|
||||
import type { SyntheticFilePreviewPayload } from '@/lib/copilot/request/session'
|
||||
import type { FilePreviewSession } from '@/lib/copilot/request/session/file-preview-session-contract'
|
||||
import { invalidateResourceQueries } from '@/app/workspace/[workspaceId]/home/components/mothership-view/components/resource-registry'
|
||||
import { deriveFilePreviewSession } from '@/app/workspace/[workspaceId]/home/hooks/preview/apply-file-preview-phase'
|
||||
import {
|
||||
buildCompletedPreviewSessions,
|
||||
type FilePreviewSessionsState,
|
||||
hasRenderableFilePreviewContent,
|
||||
INITIAL_FILE_PREVIEW_SESSIONS_STATE,
|
||||
reduceFilePreviewSessions,
|
||||
useFilePreviewSessions,
|
||||
} from '@/app/workspace/[workspaceId]/home/hooks/preview/use-file-preview-sessions'
|
||||
import type { MothershipResource } from '@/app/workspace/[workspaceId]/home/types'
|
||||
import { workspaceFilesKeys } from '@/hooks/queries/workspace-files'
|
||||
|
||||
interface FilePreviewControllerDeps {
|
||||
workspaceId: string
|
||||
setResources: Dispatch<SetStateAction<MothershipResource[]>>
|
||||
setActiveResourceId: Dispatch<SetStateAction<string | null>>
|
||||
activeResourceIdRef: MutableRefObject<string | null>
|
||||
}
|
||||
|
||||
function asPayloadRecord(value: unknown): Record<string, unknown> | undefined {
|
||||
return value && typeof value === 'object' && !Array.isArray(value)
|
||||
? (value as Record<string, unknown>)
|
||||
: undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* Owns the file-preview state machine for the mothership chat: the preview-session
|
||||
* store, the streaming-to-committed resource handoff bookkeeping, and translation
|
||||
* of synthetic preview-phase events into session updates and resource chrome. The
|
||||
* viewer renders committed binaries (see `useDocPreviewBinary`); this controller
|
||||
* decides which resource is active and when a generated file is promoted from the
|
||||
* synthetic `streaming-file` placeholder to its real workspace file.
|
||||
*/
|
||||
export function useFilePreviewController({
|
||||
workspaceId,
|
||||
setResources,
|
||||
setActiveResourceId,
|
||||
activeResourceIdRef,
|
||||
}: FilePreviewControllerDeps) {
|
||||
const queryClient = useQueryClient()
|
||||
|
||||
const previewActivationOwnerRef = useRef<Map<string, string | null>>(new Map())
|
||||
const completedPreviewResourceHandoffRef = useRef<
|
||||
Map<string, { sessionId: string; suppressActivation: boolean }>
|
||||
>(new Map())
|
||||
|
||||
const rememberPreviewActivationOwner = useCallback(
|
||||
(session: FilePreviewSession) => {
|
||||
if (!session.fileId || previewActivationOwnerRef.current.has(session.id)) {
|
||||
return
|
||||
}
|
||||
previewActivationOwnerRef.current.set(session.id, activeResourceIdRef.current)
|
||||
},
|
||||
[activeResourceIdRef]
|
||||
)
|
||||
|
||||
const shouldAutoActivatePreviewSession = useCallback(
|
||||
(session: FilePreviewSession) => {
|
||||
if (!session.fileId) {
|
||||
return false
|
||||
}
|
||||
const currentActiveResourceId = activeResourceIdRef.current
|
||||
const activationOwnerId = previewActivationOwnerRef.current.get(session.id)
|
||||
return (
|
||||
currentActiveResourceId === null ||
|
||||
currentActiveResourceId === session.fileId ||
|
||||
currentActiveResourceId === 'streaming-file' ||
|
||||
currentActiveResourceId === activationOwnerId
|
||||
)
|
||||
},
|
||||
[activeResourceIdRef]
|
||||
)
|
||||
|
||||
const seedCompletedPreviewContentCache = useCallback(
|
||||
(fileId: string, previewText: string) => {
|
||||
queryClient.setQueriesData<string>(
|
||||
{ queryKey: workspaceFilesKeys.content(workspaceId, fileId, 'text') },
|
||||
previewText
|
||||
)
|
||||
|
||||
const activeFiles = queryClient.getQueryData<Array<{ id: string; key: string }>>(
|
||||
workspaceFilesKeys.list(workspaceId, 'active')
|
||||
)
|
||||
const fileKey = activeFiles?.find((file) => file.id === fileId)?.key
|
||||
if (fileKey) {
|
||||
queryClient.setQueryData(
|
||||
[...workspaceFilesKeys.content(workspaceId, fileId, 'text'), fileKey],
|
||||
previewText
|
||||
)
|
||||
}
|
||||
},
|
||||
[queryClient, workspaceId]
|
||||
)
|
||||
|
||||
const {
|
||||
previewSession,
|
||||
previewSessionsById,
|
||||
activePreviewSessionId,
|
||||
hydratePreviewSessions,
|
||||
upsertPreviewSession,
|
||||
completePreviewSession,
|
||||
removePreviewSession,
|
||||
resetPreviewSessions,
|
||||
} = useFilePreviewSessions()
|
||||
const previewSessionRef = useRef(previewSession)
|
||||
previewSessionRef.current = previewSession
|
||||
const previewSessionsRef = useRef(previewSessionsById)
|
||||
previewSessionsRef.current = previewSessionsById
|
||||
const activePreviewSessionIdRef = useRef(activePreviewSessionId)
|
||||
activePreviewSessionIdRef.current = activePreviewSessionId
|
||||
const latestPreviewTargetToolCallIdRef = useRef<string | null>(null)
|
||||
const previewSessionsStateRef = useRef<FilePreviewSessionsState>({
|
||||
activeSessionId: activePreviewSessionId,
|
||||
sessions: previewSessionsById,
|
||||
})
|
||||
previewSessionsStateRef.current = {
|
||||
activeSessionId: activePreviewSessionId,
|
||||
sessions: previewSessionsById,
|
||||
}
|
||||
|
||||
const syncPreviewSessionRefs = useCallback((nextState: FilePreviewSessionsState) => {
|
||||
previewSessionsStateRef.current = nextState
|
||||
previewSessionsRef.current = nextState.sessions
|
||||
activePreviewSessionIdRef.current = nextState.activeSessionId
|
||||
previewSessionRef.current =
|
||||
nextState.activeSessionId !== null
|
||||
? (nextState.sessions[nextState.activeSessionId] ?? null)
|
||||
: null
|
||||
}, [])
|
||||
|
||||
const applyPreviewSessionUpdate = useCallback(
|
||||
(session: FilePreviewSession, options?: { activate?: boolean }) => {
|
||||
const nextState = reduceFilePreviewSessions(previewSessionsStateRef.current, {
|
||||
type: 'upsert',
|
||||
session,
|
||||
...(options?.activate === false ? { activate: false } : {}),
|
||||
})
|
||||
syncPreviewSessionRefs(nextState)
|
||||
upsertPreviewSession(session, options)
|
||||
return nextState
|
||||
},
|
||||
[syncPreviewSessionRefs, upsertPreviewSession]
|
||||
)
|
||||
|
||||
const applyCompletedPreviewSession = useCallback(
|
||||
(session: FilePreviewSession) => {
|
||||
const nextState = reduceFilePreviewSessions(previewSessionsStateRef.current, {
|
||||
type: 'complete',
|
||||
session,
|
||||
})
|
||||
syncPreviewSessionRefs(nextState)
|
||||
completePreviewSession(session)
|
||||
return nextState
|
||||
},
|
||||
[completePreviewSession, syncPreviewSessionRefs]
|
||||
)
|
||||
|
||||
const reconcileTerminalPreviewSessions = useCallback(() => {
|
||||
const completedAt = new Date().toISOString()
|
||||
const completedSessions = buildCompletedPreviewSessions(
|
||||
previewSessionsStateRef.current.sessions,
|
||||
completedAt
|
||||
)
|
||||
|
||||
for (const session of completedSessions) {
|
||||
applyCompletedPreviewSession(session)
|
||||
}
|
||||
}, [applyCompletedPreviewSession])
|
||||
|
||||
const removePreviewSessionImmediate = useCallback(
|
||||
(sessionId: string) => {
|
||||
const nextState = reduceFilePreviewSessions(previewSessionsStateRef.current, {
|
||||
type: 'remove',
|
||||
sessionId,
|
||||
})
|
||||
syncPreviewSessionRefs(nextState)
|
||||
removePreviewSession(sessionId)
|
||||
return nextState
|
||||
},
|
||||
[removePreviewSession, syncPreviewSessionRefs]
|
||||
)
|
||||
|
||||
const resetEphemeralPreviewState = useCallback(
|
||||
(options?: { removeStreamingResource?: boolean }) => {
|
||||
previewActivationOwnerRef.current.clear()
|
||||
completedPreviewResourceHandoffRef.current.clear()
|
||||
latestPreviewTargetToolCallIdRef.current = null
|
||||
syncPreviewSessionRefs(INITIAL_FILE_PREVIEW_SESSIONS_STATE)
|
||||
resetPreviewSessions()
|
||||
if (options?.removeStreamingResource) {
|
||||
setResources((current) => current.filter((resource) => resource.id !== 'streaming-file'))
|
||||
}
|
||||
},
|
||||
[resetPreviewSessions, setResources, syncPreviewSessionRefs]
|
||||
)
|
||||
|
||||
const promoteFileResource = useCallback(
|
||||
(fileId: string, title: string) => {
|
||||
setResources((current) => {
|
||||
const withoutStreaming = current.filter((resource) => resource.id !== 'streaming-file')
|
||||
if (
|
||||
withoutStreaming.some((resource) => resource.type === 'file' && resource.id === fileId)
|
||||
) {
|
||||
return withoutStreaming
|
||||
}
|
||||
return [...withoutStreaming, { type: 'file', id: fileId, title }]
|
||||
})
|
||||
},
|
||||
[setResources]
|
||||
)
|
||||
|
||||
const syncPreviewResourceChrome = useCallback(
|
||||
(session: FilePreviewSession, options?: { activate?: boolean }) => {
|
||||
if (session.targetKind === 'new_file') {
|
||||
setResources((current) => {
|
||||
const existing = current.find((resource) => resource.id === 'streaming-file')
|
||||
if (existing) {
|
||||
return current.map((resource) =>
|
||||
resource.id === 'streaming-file'
|
||||
? { ...resource, title: session.fileName || 'Writing file...' }
|
||||
: resource
|
||||
)
|
||||
}
|
||||
return [
|
||||
...current,
|
||||
{ type: 'file', id: 'streaming-file', title: session.fileName || 'Writing file...' },
|
||||
]
|
||||
})
|
||||
setActiveResourceId('streaming-file')
|
||||
return
|
||||
}
|
||||
|
||||
if (session.fileId && hasRenderableFilePreviewContent(session)) {
|
||||
promoteFileResource(session.fileId, session.fileName || 'File')
|
||||
if (options?.activate !== false) {
|
||||
setActiveResourceId(session.fileId)
|
||||
}
|
||||
}
|
||||
},
|
||||
[promoteFileResource, setActiveResourceId, setResources]
|
||||
)
|
||||
|
||||
const seedPreviewSessions = useCallback(
|
||||
(sessions: FilePreviewSession[]) => {
|
||||
if (sessions.length === 0) {
|
||||
return
|
||||
}
|
||||
|
||||
const nextState = reduceFilePreviewSessions(previewSessionsStateRef.current, {
|
||||
type: 'hydrate',
|
||||
sessions,
|
||||
})
|
||||
syncPreviewSessionRefs(nextState)
|
||||
hydratePreviewSessions(sessions)
|
||||
const active =
|
||||
nextState.activeSessionId !== null
|
||||
? (nextState.sessions[nextState.activeSessionId] ?? null)
|
||||
: null
|
||||
if (active) {
|
||||
syncPreviewResourceChrome(active, {
|
||||
activate: active.targetKind === 'new_file' || shouldAutoActivatePreviewSession(active),
|
||||
})
|
||||
}
|
||||
},
|
||||
[
|
||||
hydratePreviewSessions,
|
||||
shouldAutoActivatePreviewSession,
|
||||
syncPreviewResourceChrome,
|
||||
syncPreviewSessionRefs,
|
||||
]
|
||||
)
|
||||
|
||||
const onPreviewPhase = useCallback(
|
||||
(payload: SyntheticFilePreviewPayload, streamId: string | undefined) => {
|
||||
const id = payload.toolCallId
|
||||
const prevSession = previewSessionsRef.current[id]
|
||||
const nextSession = deriveFilePreviewSession(
|
||||
prevSession,
|
||||
payload,
|
||||
streamId,
|
||||
new Date().toISOString()
|
||||
)
|
||||
|
||||
if (payload.previewPhase === 'file_preview_start') {
|
||||
latestPreviewTargetToolCallIdRef.current = id
|
||||
applyPreviewSessionUpdate(nextSession)
|
||||
return
|
||||
}
|
||||
|
||||
if (payload.previewPhase === 'file_preview_target') {
|
||||
latestPreviewTargetToolCallIdRef.current = id
|
||||
rememberPreviewActivationOwner(nextSession)
|
||||
const nextState = applyPreviewSessionUpdate(nextSession)
|
||||
const activePreview =
|
||||
nextState.activeSessionId !== null
|
||||
? (nextState.sessions[nextState.activeSessionId] ?? null)
|
||||
: null
|
||||
if (activePreview?.id === nextSession.id) {
|
||||
syncPreviewResourceChrome(activePreview, {
|
||||
activate:
|
||||
activePreview.targetKind === 'new_file' ||
|
||||
shouldAutoActivatePreviewSession(activePreview),
|
||||
})
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
if (payload.previewPhase === 'file_preview_edit_meta') {
|
||||
applyPreviewSessionUpdate(nextSession)
|
||||
return
|
||||
}
|
||||
|
||||
if (payload.previewPhase === 'file_preview_content') {
|
||||
applyPreviewSessionUpdate(nextSession)
|
||||
if (!prevSession || !hasRenderableFilePreviewContent(prevSession)) {
|
||||
syncPreviewResourceChrome(nextSession, {
|
||||
activate:
|
||||
nextSession.targetKind === 'new_file' ||
|
||||
shouldAutoActivatePreviewSession(nextSession),
|
||||
})
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
const resultData = asPayloadRecord(payload.output)
|
||||
const outputData = asPayloadRecord(resultData?.data) ?? resultData
|
||||
const wasRenderableBeforeComplete =
|
||||
prevSession !== undefined && hasRenderableFilePreviewContent(prevSession)
|
||||
const nextState = applyCompletedPreviewSession(nextSession)
|
||||
const fileId = nextSession.fileId
|
||||
|
||||
if (fileId && resultData?.success === true && outputData?.id === fileId) {
|
||||
const fileName = (outputData.name as string) ?? nextSession.fileName ?? 'File'
|
||||
promoteFileResource(fileId, fileName)
|
||||
const completedExt = fileName.includes('.')
|
||||
? (fileName.split('.').pop()?.toLowerCase() ?? '')
|
||||
: ''
|
||||
const isCompiledDocPreview = ['docx', 'pptx', 'pdf', 'xlsx'].includes(completedExt)
|
||||
const shouldActivateOnComplete =
|
||||
(isCompiledDocPreview ||
|
||||
(!wasRenderableBeforeComplete && hasRenderableFilePreviewContent(nextSession))) &&
|
||||
shouldAutoActivatePreviewSession(nextSession)
|
||||
if (shouldActivateOnComplete) {
|
||||
setActiveResourceId(fileId)
|
||||
}
|
||||
completedPreviewResourceHandoffRef.current.set(fileId, {
|
||||
sessionId: nextSession.id,
|
||||
suppressActivation: !shouldActivateOnComplete,
|
||||
})
|
||||
if (hasRenderableFilePreviewContent(nextSession)) {
|
||||
seedCompletedPreviewContentCache(fileId, nextSession.previewText)
|
||||
}
|
||||
invalidateResourceQueries(queryClient, workspaceId, 'file', fileId)
|
||||
} else {
|
||||
const activePreview =
|
||||
nextState.activeSessionId !== null
|
||||
? (nextState.sessions[nextState.activeSessionId] ?? null)
|
||||
: null
|
||||
if (activePreview) {
|
||||
syncPreviewResourceChrome(activePreview, {
|
||||
activate:
|
||||
activePreview.targetKind === 'new_file' ||
|
||||
shouldAutoActivatePreviewSession(activePreview),
|
||||
})
|
||||
}
|
||||
}
|
||||
},
|
||||
[
|
||||
applyCompletedPreviewSession,
|
||||
applyPreviewSessionUpdate,
|
||||
promoteFileResource,
|
||||
queryClient,
|
||||
rememberPreviewActivationOwner,
|
||||
seedCompletedPreviewContentCache,
|
||||
setActiveResourceId,
|
||||
shouldAutoActivatePreviewSession,
|
||||
syncPreviewResourceChrome,
|
||||
workspaceId,
|
||||
]
|
||||
)
|
||||
|
||||
return {
|
||||
previewSession,
|
||||
previewSessionRef,
|
||||
previewSessionsRef,
|
||||
activePreviewSessionIdRef,
|
||||
latestPreviewTargetToolCallIdRef,
|
||||
previewActivationOwnerRef,
|
||||
completedPreviewResourceHandoffRef,
|
||||
shouldAutoActivatePreviewSession,
|
||||
applyPreviewSessionUpdate,
|
||||
removePreviewSessionImmediate,
|
||||
reconcileTerminalPreviewSessions,
|
||||
resetEphemeralPreviewState,
|
||||
promoteFileResource,
|
||||
seedPreviewSessions,
|
||||
onPreviewPhase,
|
||||
}
|
||||
}
|
||||
+3
-1
@@ -7,9 +7,10 @@ import {
|
||||
buildCompletedPreviewSessions,
|
||||
hasRenderableFilePreviewContent,
|
||||
INITIAL_FILE_PREVIEW_SESSIONS_STATE,
|
||||
pickActiveSessionId,
|
||||
reduceFilePreviewSessions,
|
||||
shouldReplaceSession,
|
||||
} from '@/app/workspace/[workspaceId]/home/hooks/use-file-preview-sessions'
|
||||
} from '@/app/workspace/[workspaceId]/home/hooks/preview/use-file-preview-sessions'
|
||||
|
||||
function createSession(
|
||||
overrides: Partial<FilePreviewSession> & Pick<FilePreviewSession, 'id' | 'toolCallId'>
|
||||
@@ -275,6 +276,7 @@ describe('reduceFilePreviewSessions', () => {
|
||||
})
|
||||
|
||||
expect(afterEmptyUpsert.activeSessionId).toBe('preview-1')
|
||||
expect(pickActiveSessionId(afterEmptyUpsert.sessions, null)).toBe('preview-2')
|
||||
|
||||
const afterContent = reduceFilePreviewSessions(afterEmptyUpsert, {
|
||||
type: 'upsert',
|
||||
@@ -0,0 +1,124 @@
|
||||
/**
|
||||
* @vitest-environment node
|
||||
*/
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { MothershipStreamV1EventType } from '@/lib/copilot/generated/mothership-stream-v1'
|
||||
import type { PersistedStreamEventEnvelope } from '@/lib/copilot/request/session/contract'
|
||||
import type { StreamLoopContext } from './stream-context'
|
||||
|
||||
const handlers = vi.hoisted(() => ({
|
||||
handleSessionEvent: vi.fn(),
|
||||
handleTextEvent: vi.fn(),
|
||||
handleToolEvent: vi.fn(),
|
||||
handleResourceEvent: vi.fn(),
|
||||
handleRunEvent: vi.fn(),
|
||||
handleSpanEvent: vi.fn(),
|
||||
handleErrorEvent: vi.fn(),
|
||||
handleCompleteEvent: vi.fn(),
|
||||
}))
|
||||
|
||||
vi.mock('@/app/workspace/[workspaceId]/home/hooks/stream/handle-session-event', () => ({
|
||||
handleSessionEvent: handlers.handleSessionEvent,
|
||||
}))
|
||||
vi.mock('@/app/workspace/[workspaceId]/home/hooks/stream/handle-text-event', () => ({
|
||||
handleTextEvent: handlers.handleTextEvent,
|
||||
}))
|
||||
vi.mock('@/app/workspace/[workspaceId]/home/hooks/stream/handle-tool-event', () => ({
|
||||
handleToolEvent: handlers.handleToolEvent,
|
||||
}))
|
||||
vi.mock('@/app/workspace/[workspaceId]/home/hooks/stream/handle-resource-event', () => ({
|
||||
handleResourceEvent: handlers.handleResourceEvent,
|
||||
}))
|
||||
vi.mock('@/app/workspace/[workspaceId]/home/hooks/stream/handle-run-event', () => ({
|
||||
handleRunEvent: handlers.handleRunEvent,
|
||||
}))
|
||||
vi.mock('@/app/workspace/[workspaceId]/home/hooks/stream/handle-span-event', () => ({
|
||||
handleSpanEvent: handlers.handleSpanEvent,
|
||||
}))
|
||||
vi.mock('@/app/workspace/[workspaceId]/home/hooks/stream/handle-error-event', () => ({
|
||||
handleErrorEvent: handlers.handleErrorEvent,
|
||||
}))
|
||||
vi.mock('@/app/workspace/[workspaceId]/home/hooks/stream/handle-complete-event', () => ({
|
||||
handleCompleteEvent: handlers.handleCompleteEvent,
|
||||
}))
|
||||
|
||||
import { dispatchStreamEvent } from './dispatch-stream-event'
|
||||
|
||||
function makeCtx(): StreamLoopContext {
|
||||
return {
|
||||
state: {} as StreamLoopContext['state'],
|
||||
deps: {} as StreamLoopContext['deps'],
|
||||
ops: {
|
||||
resolveScopedSubagent: vi.fn(() => 'sub-agent'),
|
||||
} as unknown as StreamLoopContext['ops'],
|
||||
}
|
||||
}
|
||||
|
||||
function event(type: string, scope?: Record<string, unknown>): PersistedStreamEventEnvelope {
|
||||
return {
|
||||
type,
|
||||
payload: {},
|
||||
scope,
|
||||
seq: 1,
|
||||
stream: { streamId: 's' },
|
||||
ts: '2026-01-01T00:00:00Z',
|
||||
v: 1,
|
||||
} as unknown as PersistedStreamEventEnvelope
|
||||
}
|
||||
|
||||
describe('dispatchStreamEvent', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
})
|
||||
|
||||
it('routes every event type to exactly its handler', () => {
|
||||
const ctx = makeCtx()
|
||||
dispatchStreamEvent(ctx, event(MothershipStreamV1EventType.session))
|
||||
dispatchStreamEvent(ctx, event(MothershipStreamV1EventType.text))
|
||||
dispatchStreamEvent(ctx, event(MothershipStreamV1EventType.tool))
|
||||
dispatchStreamEvent(ctx, event(MothershipStreamV1EventType.resource))
|
||||
dispatchStreamEvent(ctx, event(MothershipStreamV1EventType.run))
|
||||
dispatchStreamEvent(ctx, event(MothershipStreamV1EventType.span))
|
||||
dispatchStreamEvent(ctx, event(MothershipStreamV1EventType.error))
|
||||
dispatchStreamEvent(ctx, event(MothershipStreamV1EventType.complete))
|
||||
|
||||
expect(handlers.handleSessionEvent).toHaveBeenCalledTimes(1)
|
||||
expect(handlers.handleTextEvent).toHaveBeenCalledTimes(1)
|
||||
expect(handlers.handleToolEvent).toHaveBeenCalledTimes(1)
|
||||
expect(handlers.handleResourceEvent).toHaveBeenCalledTimes(1)
|
||||
expect(handlers.handleRunEvent).toHaveBeenCalledTimes(1)
|
||||
expect(handlers.handleSpanEvent).toHaveBeenCalledTimes(1)
|
||||
expect(handlers.handleErrorEvent).toHaveBeenCalledTimes(1)
|
||||
expect(handlers.handleCompleteEvent).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('computes and passes per-event scope to scoped handlers', () => {
|
||||
const ctx = makeCtx()
|
||||
dispatchStreamEvent(
|
||||
ctx,
|
||||
event(MothershipStreamV1EventType.text, { spanId: 'span-9', agentId: 'agent-x' })
|
||||
)
|
||||
expect(ctx.ops.resolveScopedSubagent).toHaveBeenCalledWith('agent-x', undefined, 'span-9')
|
||||
const call = handlers.handleTextEvent.mock.calls[0]
|
||||
expect(call[0]).toBe(ctx)
|
||||
const scope = call[2]
|
||||
expect(scope.scopedSpanId).toBe('span-9')
|
||||
expect(scope.scopedAgentId).toBe('agent-x')
|
||||
expect(scope.scopedSubagent).toBe('sub-agent')
|
||||
expect(scope.spanIdentity).toEqual({ spanId: 'span-9' })
|
||||
})
|
||||
|
||||
it('invokes ctx-only handlers (session/run/complete) without a scope argument', () => {
|
||||
const ctx = makeCtx()
|
||||
const sessionEvent = event(MothershipStreamV1EventType.session)
|
||||
dispatchStreamEvent(ctx, sessionEvent)
|
||||
expect(handlers.handleSessionEvent).toHaveBeenCalledWith(ctx, sessionEvent)
|
||||
expect(handlers.handleSessionEvent.mock.calls[0]).toHaveLength(2)
|
||||
})
|
||||
|
||||
it('ignores unknown event types without throwing', () => {
|
||||
const ctx = makeCtx()
|
||||
expect(() => dispatchStreamEvent(ctx, event('totally-unknown'))).not.toThrow()
|
||||
expect(handlers.handleSessionEvent).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,82 @@
|
||||
import { MothershipStreamV1EventType } from '@/lib/copilot/generated/mothership-stream-v1'
|
||||
import type { PersistedStreamEventEnvelope } from '@/lib/copilot/request/session/contract'
|
||||
import { handleCompleteEvent } from '@/app/workspace/[workspaceId]/home/hooks/stream/handle-complete-event'
|
||||
import { handleErrorEvent } from '@/app/workspace/[workspaceId]/home/hooks/stream/handle-error-event'
|
||||
import { handleResourceEvent } from '@/app/workspace/[workspaceId]/home/hooks/stream/handle-resource-event'
|
||||
import { handleRunEvent } from '@/app/workspace/[workspaceId]/home/hooks/stream/handle-run-event'
|
||||
import { handleSessionEvent } from '@/app/workspace/[workspaceId]/home/hooks/stream/handle-session-event'
|
||||
import { handleSpanEvent } from '@/app/workspace/[workspaceId]/home/hooks/stream/handle-span-event'
|
||||
import { handleTextEvent } from '@/app/workspace/[workspaceId]/home/hooks/stream/handle-text-event'
|
||||
import { handleToolEvent } from '@/app/workspace/[workspaceId]/home/hooks/stream/handle-tool-event'
|
||||
import type {
|
||||
StreamEventScope,
|
||||
StreamLoopContext,
|
||||
} from '@/app/workspace/[workspaceId]/home/hooks/stream/stream-context'
|
||||
|
||||
function computeEventScope(
|
||||
ctx: StreamLoopContext,
|
||||
parsed: PersistedStreamEventEnvelope
|
||||
): StreamEventScope {
|
||||
const scopedParentToolCallId =
|
||||
typeof parsed.scope?.parentToolCallId === 'string' ? parsed.scope.parentToolCallId : undefined
|
||||
const scopedAgentId = typeof parsed.scope?.agentId === 'string' ? parsed.scope.agentId : undefined
|
||||
const scopedSpanId = typeof parsed.scope?.spanId === 'string' ? parsed.scope.spanId : undefined
|
||||
const scopedParentSpanId =
|
||||
typeof parsed.scope?.parentSpanId === 'string' ? parsed.scope.parentSpanId : undefined
|
||||
const scopedSubagent = ctx.ops.resolveScopedSubagent(
|
||||
scopedAgentId,
|
||||
scopedParentToolCallId,
|
||||
scopedSpanId
|
||||
)
|
||||
const spanIdentity: { spanId?: string; parentSpanId?: string } = {
|
||||
...(scopedSpanId ? { spanId: scopedSpanId } : {}),
|
||||
...(scopedParentSpanId ? { parentSpanId: scopedParentSpanId } : {}),
|
||||
}
|
||||
return {
|
||||
scopedSubagent,
|
||||
scopedParentToolCallId,
|
||||
scopedAgentId,
|
||||
scopedSpanId,
|
||||
scopedParentSpanId,
|
||||
spanIdentity,
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Routes a parsed stream event to its handler. Per-event subagent/span scope is
|
||||
* resolved once here and passed to the handlers that nest blocks by it. The
|
||||
* caller's transport loop owns staleness, cursor dedup, and `streamId`/
|
||||
* `streamRequestId` updates; this function only mutates the supplied context.
|
||||
*/
|
||||
export function dispatchStreamEvent(
|
||||
ctx: StreamLoopContext,
|
||||
parsed: PersistedStreamEventEnvelope
|
||||
): void {
|
||||
const scope = computeEventScope(ctx, parsed)
|
||||
switch (parsed.type) {
|
||||
case MothershipStreamV1EventType.session:
|
||||
handleSessionEvent(ctx, parsed)
|
||||
break
|
||||
case MothershipStreamV1EventType.text:
|
||||
handleTextEvent(ctx, parsed, scope)
|
||||
break
|
||||
case MothershipStreamV1EventType.tool:
|
||||
handleToolEvent(ctx, parsed, scope)
|
||||
break
|
||||
case MothershipStreamV1EventType.resource:
|
||||
handleResourceEvent(ctx, parsed)
|
||||
break
|
||||
case MothershipStreamV1EventType.run:
|
||||
handleRunEvent(ctx, parsed)
|
||||
break
|
||||
case MothershipStreamV1EventType.span:
|
||||
handleSpanEvent(ctx, parsed, scope)
|
||||
break
|
||||
case MothershipStreamV1EventType.error:
|
||||
handleErrorEvent(ctx, parsed, scope)
|
||||
break
|
||||
case MothershipStreamV1EventType.complete:
|
||||
handleCompleteEvent(ctx, parsed)
|
||||
break
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
import { MothershipStreamV1CompletionStatus } from '@/lib/copilot/generated/mothership-stream-v1'
|
||||
import type { PersistedStreamEventEnvelope } from '@/lib/copilot/request/session/contract'
|
||||
import type { StreamLoopContext } from '@/app/workspace/[workspaceId]/home/hooks/stream/stream-context'
|
||||
import {
|
||||
asPayloadRecord,
|
||||
finalizeResidualToolCalls,
|
||||
} from '@/app/workspace/[workspaceId]/home/hooks/stream/stream-helpers'
|
||||
|
||||
type CompleteEvent = Extract<PersistedStreamEventEnvelope, { type: 'complete' }>
|
||||
|
||||
export function handleCompleteEvent(ctx: StreamLoopContext, parsed: CompleteEvent): void {
|
||||
const { state, ops } = ctx
|
||||
state.sawCompleteEvent = true
|
||||
ops.stampBlockEnd(state.blocks[state.blocks.length - 1])
|
||||
const completeResponse = asPayloadRecord(parsed.payload.response)
|
||||
if (completeResponse === undefined || !('async_pause' in completeResponse)) {
|
||||
finalizeResidualToolCalls(
|
||||
state.blocks,
|
||||
parsed.payload.status === MothershipStreamV1CompletionStatus.cancelled
|
||||
? 'cancelled'
|
||||
: 'complete'
|
||||
)
|
||||
ops.flush()
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user