mirror of
https://github.com/simstudioai/sim.git
synced 2026-09-01 14:59:19 +08:00
feat(secrets): record which secrets each run resolves, and surface it per secret (#6823)
* feat(secrets): record which secrets each run resolves and surface it per secret
Redaction stops a value at a boundary but cannot stop code that never emits it —
a Function block can print a key one character at a time and nothing ever matches
the secret. That is undecidable in general, so this adds the other half of the
posture: attribution.
Every run now records which configured secrets it actually resolved, under whose
identity, through which surface (workflow, Sim agent, MCP). The data already
existed in ResolvedSecretTraceRegistry.addActiveEntry and was persisted only for
paused runs; this persists it for every terminal path.
Execution logs cannot answer this. They store the whole available encrypted
environment rather than what a run referenced, they evidence a secret only where
value-matching redaction happened to fire, and they expire under
logRetentionHours — while "who has touched this key" outlives any single run.
- secret_usage: per-UTC-day rollup keyed by workspace, secret, scope, owner,
source, workflow, actor. A one-minute schedule touching three secrets would
otherwise write thousands of rows a day, which is also why this is not
audit_log. workflow_id/actor_user_id use '' sentinels rather than null so the
unique key works on Postgres 14 without NULLS NOT DISTINCT, and are not FKs:
they are historical facts, and an onDelete would rewrite a key column.
- secret_owner_user_id is part of the key. Two people can hold a personal secret
under one name and a shared personal secret resolves for a caller who does not
own it, so name and scope alone do not identify a secret. It is NOT the actor:
a scheduled run resolves the workflow owner's personal slice under the
workspace's execution actor.
- Direct environment reads are now detected in JS (TypeScript AST), Python
(tokenizer-checked) and shell (quote/heredoc-scanned), so a secret read as
environmentVariables['K'] or $K enters the run's provenance instead of going
unredacted. Each detector prescans for names that are actually configured
secrets before paying for a lex or quote-frame pass.
- Copilot integration tool calls are covered: resolveCopilotEnvReferences
substitutes {{SECRET}} into user-only params, which is a real use.
- See usage lives behind a credential-admin gate, using the same predicate that
reveals the value; members get a disabled chip explaining why.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore(audit): register the secret-usage route in the validation baseline
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): keep rollup metadata with its run, and stop shadowed bindings faking usage
Review round 1.
- record.ts: last_execution_id/last_trigger were assigned unconditionally while
last_used_at was chosen by greatest(), so two runs completing out of order split
one row between them — the newer run's timestamp beside the older run's execution
id, making "View log" open a run the row does not describe. Both are now guarded
on the timestamp actually advancing, so the row's metadata always belongs to the
run that owns its timestamp.
- javascript.ts: a local binding named environmentVariables (declaration, parameter,
destructured binding, or bare reassignment) made reads off the user's own object
look like mounted-secret reads. Any such binding now disables detection for the
file; the AST already had parent pointers, so this is a kind check during the
existing walk.
- python.ts: same class of bug with no parser available, so the rule is an allowlist
— every mention of the binding must be a literal subscript or .get(), otherwise
detection is off for the file. This also subsumes the cross-line attribute case
(other.\n environmentVariables['K']), which the previous space-and-tab look-behind
missed.
Under-reporting is the safe direction here: a trail that claims a use that never
happened is worse than one that misses a use.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore(db): format the generated migration snapshot
CI runs lint:check across every workspace; the drizzle-kit output in packages/db
had never been through biome, so the branch was green locally (where lint had
only been run inside apps/sim) and red on CI. Whitespace only — both files are
byte-for-byte identical once parsed, and drizzle-kit still reports no pending
schema diff against the reformatted snapshot.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): detect every rebinding of the environment identifier, not just declarations
Review round 2. A bare `for (environmentVariables of rows)` has no declaration to
key off, so the previous check missed it and reads of the loop value were still
recorded as secret usage.
Rather than extend the hand-rolled node-kind list, this reuses the pair the same
file already applies to reject a placeholder in a write position:
isDeclarationIdentifier covers declarations, parameters, destructured bindings and
imports, and isWriteIdentifier covers every assignment operator, ++/--,
destructuring targets, and for-in / for-of initializers.
That also closes four forms neither the review nor the original check named:
logical (||=) and nullish (??=) assignment, and object and array destructuring
assignment. Six of the eight added cases fail against the previous check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): apply the rebinding rule to shell, and say when a run's log is gone
Review round 3, plus the docs that were left claiming the old behavior.
- shell.ts: a script that writes a configured name (API_KEY=local, export/local/
readonly, read, for, unset) expands its own value from that point on, not the
mounted secret, so recording it claimed a use that never happened. Every mention
of the name must now be a `$NAME` / `${NAME}` expansion, matching the allowlist
shape the Python detector already uses. Applied per name rather than per file:
JavaScript and Python shadow one object holding every secret, whereas rebinding
one shell variable says nothing about the rest.
- The usage trail deliberately outlives execution logs, so a row routinely names a
run whose log has been pruned. The read now left-joins workflow_execution_logs on
its unique execution_id and reports availability, and the panel renders the chip
disabled with the platform tooltip instead of linking into an empty Logs view.
Three states: no run to link, a run whose log is gone, and a live link.
- Docs said a direct environmentVariables/$KEY read does not activate masking,
which this branch changes. Corrected in credentials.mdx, function.mdx and the
logging FAQ, and the recognition limits are now written down: runtime-built
names, reassigned bindings, and reads that cannot be told apart from text.
Added a "See usage" section covering who can see it and why an empty trail
means "nothing recognized" rather than "never used".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): writing a name is not reading it, and a bare mention is not a rebinding
Review round 4.
- javascript.ts / python.ts: `environmentVariables.API_KEY = 'x'` and
`delete environmentVariables.API_KEY` touch the name without ever reading the
mounted value, but the detectors matched the member access and recorded a use
that never happened. JavaScript now asks the same isWriteIdentifier the
placeholder rewriter uses (its parameter is widened to ts.Node — the body
already walked generic nodes, so this is a type change, not a behaviour one)
plus a delete check; Python excludes a subscript followed by `=` and a `del`
target.
- shell.ts: requiring every mention of a name to be an expansion also fired on
text that binds nothing — a comment naming the key, or `echo "API_KEY=$API_KEY"`
where the literal is an argument rather than an assignment — and dropping those
cost masking on a genuine read. It now looks for actual writes: an assignment at
command-word position, a binding builtin, `printf -v`, or a `for` target.
The two directions are not symmetric, which is why this errs toward detecting
the read: missing a write records a use of a secret the script only had in its
environment, a misleading audit row and nothing more, since masking still
searches for the real value and will not find it. Over-detecting a write
suppresses masking on a value that does reach the log.
This also makes the code match what the docs already described — skipping after
a rebinding, not after any mention.
13 tests added; 11 fail against the previous code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): an update reads before it stores, and a del target may be parenthesized
Review round 5. The first of these is a regression from round 4.
- javascript.ts: reusing isWriteIdentifier to answer "is this a read" was wrong.
That predicate answers the rewriter's question — is this a target the
substitution must refuse — so it treats every assignment operator alike, which
is correct there and wrong here: `+=`, `||=`, `??=`, `++` and `--` all load the
current value before storing, so they are genuine reads and were silently
losing their masking. Only a plain `=` stores without reading. Replaced with a
purpose-named predicate, and isWriteIdentifier's parameter is narrowed back to
ts.Identifier now that nothing else needs it widened.
A test committed last round asserted the wrong behaviour for `+=`; it has been
corrected rather than left to pin the bug.
- python.ts: `del (environmentVariables['K'])` slipped past a check that looked
only at the characters immediately before the match. It now isolates the
enclosing logical line and tests whether that is a del statement, which also
covers `del((x))`, `del(x)`, `del a, x`, and a del after a semicolon.
12 tests added or corrected; 10 fail against the previous code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): stop excluding Python writes, which kept leaking in the unsafe direction
Review round 6. Greptile found that `del environmentVariables[environmentVariables['K']]`
had its inner access — which computes a key, so it is a genuine read — skipped
along with the delete, leaving that value unmasked.
The narrow fix was another textual rule. Instead this removes the write and delete
exclusions from the Python detector entirely, because they were optimizing the
wrong direction.
`resolvedSecretNames` feeds `outputSecretMatcher`, an exact-value matcher over the
output. Naming a secret the code never read costs nothing there: the matcher scans
for a value that does not appear. Failing to name one that was read leaves it
unmasked. The two error directions are therefore not comparable, and the
exclusions bought only audit-trail tidiness while every heuristic they needed has
so far leaked into the dangerous side — first a parenthesized target, now a nested
read. A `del` or an assignment is reported like any other access.
JavaScript keeps its exclusion: a real AST answers the question per node, with no
text to misread, and it has produced no such hole.
Net 30 lines removed from python.ts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): report recognized reads instead of proving they are not reads
Review round 7. Greptile flagged both directions at once — false usage from
reporting a write target, and unmasked secrets from the file-wide shadow flag —
so I traced what the signal actually drives before choosing.
The chain: the compiler's names feed outputSecretPlaintextsByName and the
exact-value matcher, NOT context.resolvedSecretNames, which starts empty. After
execution activateOutputSecretProvenance scans the output and adds only names
whose plaintext actually appeared; those become __resolvedSecretNames, which
tools/index.ts turns into recordResolved calls, which is what the usage trail
reads.
So a compile-time false positive produces no usage row on the ordinary path — it
only hands the matcher a value the code never emits. It does produce one on the
!projection.safe fallback, where the system already over-approximates by design.
A false negative, by contrast, keeps the value out of the matcher entirely, so a
genuinely read secret is never masked on any path.
That asymmetry decides it, so every "prove this is not a read" mechanism is gone:
- javascript.ts: the file-wide shadow flag. A helper declaring its own
environmentVariables discarded genuine reads of the mounted binding everywhere
else in the file — Greptile's security finding, and real.
- python.ts: the allowlist requiring every mention to be a subscript or .get().
Same hole: passing the dict to a function suppressed unrelated reads.
- shell.ts: the rebinding check. It had the same hole in a form nobody flagged —
`echo "$API_KEY"; API_KEY=local` dropped the first read, which is of the real
secret.
What stays is the question of whether the text is code at all — strings, comments,
single quotes, quoted heredocs — plus the receiver check that `other.environment
Variables['K']` is a different object, and JavaScript's node-precise write/delete
exclusion, which cannot suppress a read elsewhere.
Net 215 lines removed across the three detectors and their tests. Docs updated:
the rule is now stated as reporting rather than proving, and that See usage may
occasionally list a secret the code had available but did not read.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(secrets): drop the last write-vs-read special case
`environmentVariables` is a plain object deserialized from the run payload
(route.ts:206), not a handle on the stored secret. Assigning to it changes
nothing outside the sandbox and is discarded when the run ends, so separating a
write from a read bought almost nothing while leaving JavaScript as the one
language still trying to prove a read is not a read.
Every language now follows the same rule: report a recognized read of a
configured secret name. The only exclusions left are facts rather than
inferences — the text is not executable (string, comment, single quote, quoted
heredoc), the receiver is a different object, or the name is not statically
knowable.
Docs note that assigning to the binding does not edit the secret.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(secrets): ship only the fields the trail actually shows
Five fields crossed the API and reached no reader: usageDate, firstUsedAt,
actorEmail, workflowId and actorUserId. The panel renders the timestamp, the
trigger, what used the secret, the actor's name, the run count and the run link;
everything else was projected, serialized and discarded.
first_used_at is dropped from the table as well. Nothing read it, and inside a
per-day bucket "first used that day" says nothing next to "last used that day" —
so it was a column written on every run for no question anyone asks. The upsert
loses its least() with it. Migration regenerated; the identifier columns behind
the joins stay, they simply are not returned.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): report referenced code secrets, not only ones that surface in output
The Function route activated a secret's provenance — and therefore its usage row
and downstream masking — only when the exact value appeared in the result,
stdout, or error. That gate made the trail miss silent use entirely: a key that
authenticates an API call and is never echoed reported nothing, and so did the
founding scenario of this feature, a key exfiltrated character by character. The
innocent run that echoed a key got a row; the run worth catching did not.
Activation now follows the referenced set the compiler already computes: resolved
{{KEY}} bindings plus recognized direct reads, filtered to configured values —
the same set the unsafe-projection fallback already activated. An extra name only
hands the output matcher a value that never appears; configured-but-unreferenced
values are still never included. The output-scan activation path and its surface
helper are deleted rather than kept alongside.
One old test pinned the gate ("does not activate a referenced secret that does
not cross the Function result"); it now asserts the reverse, with the reasoning
attached. Two new tests pin the char-split exfiltration and the silent API-call
case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): shell escaping is backslash parity, not adjacency
Review round 8. `\\$API_KEY` is an escaped backslash followed by a LIVE expansion
— bash prints `\` plus the value — while `\$API_KEY` is an escaped dollar and
stays literal. Checking only the character adjacent to `$` read every even run as
escaped, dropping a real read from usage and masking alike; verified against
bash before fixing.
The scanner now counts the run of backslashes before the `$` and skips only odd
runs, the same parity rule logicalLineEndAfterContinuations in this file already
applies to line continuations. Six-case parity table added; the three even-run
cases fail against the previous check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): recognize destructured environment reads
Review round 9. `const { API_KEY } = environmentVariables` delivers the value by
name with no property- or element-access node in the AST, so the member-access
walk missed it entirely — and a missed read leaves an emitted value unmasked,
the dangerous direction.
The AST walk now also recognizes the declaration form (shorthand, renames,
defaults, string-literal keys), the assignment form ({ KEY } = env), and a
...rest element — which names no key but takes every value, so it reports every
configured name; the alternative left `const { ...all } = env; return all`
entirely unmasked. A computed key stays unrecognized, the same runtime-name
boundary as a computed subscript, and a receiver that is not the bare identifier
is not attributed.
Nine cases added; the six positive ones fail against the previous walk.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): one receiver rule for destructured reads, parentheses included
Review round 10. Two accurate findings, folded into a generalization instead of
two more special cases:
- A parameter default (function f({ API_KEY } = environmentVariables)) and a
binding-element default are the same by-name delivery as a variable
declaration. The detector now keys on the ObjectBindingPattern itself and
checks its parent's initializer, so every declaration position follows one
rule instead of per-kind arms.
- Parentheses group without changing the receiver, so (environmentVariables) is
unwrapped before the identifier check — in the destructuring arm AND the
member-access arm, which had the same hole unreported.
Declined the for-of-over-array-literal finding: the receiver there is a
container, not the environment object, and following data flow through
containers has no fixed point — the same documented boundary as aliasing and
computed keys. A test pins the boundary so it reads as chosen, not missed.
Eight cases added; the seven receiver-rule cases fail against the previous code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(secrets): a dot in prose is not a qualifier, and a literal computed key is a subscript
Review round 11. Both findings were implementation-narrower-than-rule, fixed by
consulting authorities the detectors already had rather than adding new ones:
- python.ts: the receiver walk crosses whitespace so a parenthesized `other.` on
a previous line is seen — but it landed on a comment's final period
(`# Load the value.`) and discarded the genuine read on the next line. The
landing position is now checked against the same lexer ranges that filter the
candidates, which is also why the receiver check moves after lexing.
- javascript.ts: `const { ['API_KEY']: key } = environmentVariables` is the
element-access rule in pattern position, so a computed key holding a string
literal resolves like a literal subscript; any other computed key keeps the
runtime-name boundary a computed subscript already has.
Eight cases added; the comment-period case and all three literal-computed-key
cases fail against the previous code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Vikhyath Mondreti
committed by
GitHub
GitHub
parent
ff232799a7
commit
521348b529
@@ -94,7 +94,7 @@ import { FAQ } from '@/components/ui/faq'
|
||||
<FAQ items={[
|
||||
{ question: "How long are run logs retained?", answer: "Free plans retain logs for 7 days — after that, logs are archived to cloud storage and deleted from the database. Pro, Team, and Enterprise plans retain logs indefinitely with no automatic cleanup." },
|
||||
{ question: "What data is captured in each run log?", answer: "Each log entry includes the run ID, workflow ID, trigger type, start and end timestamps, total duration in milliseconds, cost breakdown (total cost, token counts, and per-model breakdowns), run data with trace spans, final output, and any associated files. The log details sidebar lets you inspect block-level inputs and outputs." },
|
||||
{ question: "Are saved secrets visible in logs?", answer: "When a value saved under Secrets is successfully substituted through {{KEY}}, exact, case-sensitive occurrences are masked throughout the log-facing copy, including the live block-log display, Logs Overview input and output, Trace, log-read APIs, and the Logs block's Get Run Details output. This is not a general redactor: hardcoded or directly read values do not activate masking by themselves, and encoded, hashed, or transformed values are not matched. Functional execution responses, streams, and callbacks remain unchanged. See Execution log protection under Secrets for details." },
|
||||
{ question: "Are saved secrets visible in logs?", answer: "When a value saved under Secrets is successfully substituted through {{KEY}}, exact, case-sensitive occurrences are masked throughout the log-facing copy, including the live block-log display, Logs Overview input and output, Trace, log-read APIs, and the Logs block's Get Run Details output. Direct reads such as environmentVariables['KEY'] or shell $KEY also activate masking when Sim can recognize the read in the code beforehand; a name built at runtime, a reassigned binding, or a hardcoded literal is not recognized. This is not a general redactor: encoded, hashed, or transformed values are not matched. Functional execution responses, streams, and callbacks remain unchanged. See Execution log protection under Secrets for details." },
|
||||
{ question: "What is a workflow snapshot?", answer: "A frozen copy of the workflow's structure (blocks, connections, and configuration) captured at run time, so you can see the exact state behind a particular run — useful for debugging workflows that have been modified since." },
|
||||
{ question: "Can I access logs programmatically?", answer: "Yes. The External API provides endpoints to query logs with filtering by workflow, time range, trigger type, duration, cost, and model. You can also set up webhook, email, or Slack notifications for real-time alerts when runs complete." },
|
||||
{ question: "What does Live mode do on the Logs page?", answer: "It refreshes the Logs page in real time so new entries appear as they are recorded — useful during deployments or when monitoring active workflows." },
|
||||
|
||||
@@ -71,8 +71,22 @@ When a saved secret is successfully substituted through a `{{KEY}}` reference, S
|
||||
|
||||
Secret resolution and functional workflow behavior are unchanged: blocks, tools, and downstream steps receive the real runtime value. Stored functional execution data, workflow execution responses, streams, callbacks, block state, and snapshots are not rewritten. Log-facing views and read APIs receive a separate protected copy, so the Logs Overview **Workflow Input** and **Workflow Output** are masked without changing the underlying workflow result. Model requests receive another protected projection: exact secret values known to the run are replaced with `{{KEY}}` before model-visible messages, prompts, tool arguments, or tool continuations leave Sim.
|
||||
|
||||
Code that reads a secret straight off the runtime environment — `environmentVariables['KEY']`, `environmentVariables.KEY`, or `const { KEY } = environmentVariables` in JavaScript, `environmentVariables['KEY']` or `environmentVariables.get('KEY')` in Python, `$KEY` or `${KEY}` in shell — also activates masking, provided Sim can see the read in the code before it runs. A hardcoded literal never does: Sim has no way to know it came from a secret.
|
||||
|
||||
<Callout type="warn">
|
||||
Execution-log masking is activated only when Sim successfully resolves a value from **Settings → Secrets** through `{{KEY}}`. A hardcoded literal, direct `environmentVariables['KEY']` read, or shell `$KEY` read does not activate log masking by itself. Model-bound projection also checks the run's authorized secret catalog, including direct reads, but both protections match only exact values. Encoded, hashed, fragmented, or otherwise transformed versions are not matched. Do not deliberately return or print secrets.
|
||||
Direct reads are found by reading the code, not by running it, so recognition stops where the code stops being readable ahead of time. An unrecognized read is not masked, and does not appear under **See usage**.
|
||||
|
||||
Where a read is recognized, Sim reports it rather than trying to prove it is not one. Code that shadows the environment binding with its own object, overwrites a variable before reading it, or assigns to the name instead of reading it is still reported. Naming a secret costs only an exact value the code never emits; failing to name one leaves it unmasked. **See usage** can therefore occasionally list a secret the code had available but did not read.
|
||||
|
||||
Assigning to the injected binding does not change the stored secret — it is an ordinary object built from the run's payload and discarded when the run ends. Edit a secret under **Settings → Secrets**.
|
||||
|
||||
A read is **not** recognized when:
|
||||
|
||||
- **The name is built at runtime.** `environmentVariables[keyName]`, `$@`, `${!indirect}`, `eval`, `printenv`, or a sourced file hide which secret is being read.
|
||||
- **The read is of a different object.** `other.environmentVariables['KEY']` reads something that merely shares the name.
|
||||
- **The read cannot be told apart from text.** A `$KEY` inside single quotes or a quoted heredoc (`<<'EOF'`) never expands, and Sim treats anything its scanner cannot place as not running.
|
||||
|
||||
Both masking and model-bound projection match only exact values in either case. Encoded, hashed, fragmented, or otherwise transformed versions are not matched, and a value assembled or emitted piece by piece cannot be matched at all — determining whether arbitrary code will eventually reveal a value is not decidable in general. Treat these as a safety net, not a boundary: do not deliberately return, print, or transmit secrets.
|
||||
</Callout>
|
||||
|
||||
### Copilot code execution
|
||||
@@ -105,9 +119,22 @@ From here you can:
|
||||
- View the **Key** and edit the **Value**
|
||||
- Edit the **Description** — an optional note telling teammates what the secret is for. Workspace secrets only; a personal secret is not shared, so it has none
|
||||
- Manage **Members** — invite teammates by email and assign them an **Admin** or **Member** role
|
||||
- Open **See usage** — where this secret has actually been used
|
||||
|
||||
Click **Save** to apply changes, or **Back** to return to the list.
|
||||
|
||||
### See usage
|
||||
|
||||
**See usage** lists the runs that resolved this secret: when it was last used, what used it (a workflow, the Sim agent, or an MCP server), how it was triggered, who it resolved under, and a link to the most recent run in Logs. Rows are grouped by day, so a workflow on a schedule reads as one row per day rather than thousands.
|
||||
|
||||
This answers the question worth asking before rotating a key: who has been using it, inside what, and how recently.
|
||||
|
||||
Only people who can read the value can see it — a Credential Admin on a workspace secret, or the owner of a personal one. For everyone else the action is visible but disabled, because the trail names workflows, people, and run IDs, which is the same information masking withholds. Two people who each hold a personal secret under the same name see only their own runs.
|
||||
|
||||
<Callout>
|
||||
Usage is recorded independently of execution logs, so it outlives them: logs expire under your workspace's retention setting, while the record of who touched a credential does not. It records what a run resolved, subject to the recognition limits under [Execution log protection](#execution-log-protection) — a read Sim cannot attribute is left out rather than guessed at, so treat an empty trail as "nothing recognized," not proof a secret was never used.
|
||||
</Callout>
|
||||
|
||||
## Workspace vs. Personal
|
||||
|
||||
| | Workspace | Personal |
|
||||
|
||||
@@ -279,10 +279,13 @@ packages, and 10 managed CLI tools.
|
||||
|
||||
When a Function block is used as an Agent tool, its code can read every workspace
|
||||
secret by default — both `{{MY_SECRET}}` and `environmentVariables['MY_SECRET']`.
|
||||
Use `{{MY_SECRET}}` when the value may appear in execution logs: a successful
|
||||
double-brace substitution activates [execution-trace masking](/platform/credentials#execution-log-protection),
|
||||
while direct `environmentVariables['MY_SECRET']` access alone does not activate
|
||||
it by itself.
|
||||
Prefer `{{MY_SECRET}}` when the value may appear in execution logs. A successful
|
||||
double-brace substitution always activates
|
||||
[execution-trace masking](/platform/credentials#execution-log-protection). A direct
|
||||
`environmentVariables['MY_SECRET']` read activates it too, but only when Sim can
|
||||
recognize the read in the code beforehand — a name built at runtime, or a file that
|
||||
reassigns `environmentVariables` itself, is not recognized. See
|
||||
[the recognition limits](/platform/credentials#execution-log-protection).
|
||||
|
||||
To narrow that, set **Secret access** to *Selected secrets* in the block's
|
||||
tool configuration and pick the names the code may read. Two things change:
|
||||
|
||||
@@ -2441,6 +2441,50 @@ describe('Function Execute API Route', () => {
|
||||
expect(sandboxRequest.privateInputs[0].content).toContain('$UNRELATED `touch /tmp/nope`')
|
||||
})
|
||||
|
||||
/**
|
||||
* The founding scenario of the usage trail: code that reads a secret and emits it only in
|
||||
* transformed form. No output ever matches the value, so an output-gated report said
|
||||
* "never used" for exactly the run an admin needs to see. A referenced secret reports
|
||||
* whether or not its value surfaces.
|
||||
*/
|
||||
it('reports a secret exfiltrated character by character', async () => {
|
||||
mockExecuteInIsolatedVM.mockResolvedValueOnce({
|
||||
result: 's|e|c|r|e|t|-|v|a|l|u|e|-|1|2|3|4',
|
||||
stdout: '',
|
||||
})
|
||||
const response = await POST(
|
||||
createMockRequest(
|
||||
'POST',
|
||||
{
|
||||
code: "const k = '{{API_KEY}}'; return k.split('').join('|')",
|
||||
envVars: { API_KEY: 'secret-value-1234' },
|
||||
},
|
||||
{ 'x-sim-request-private-tool-metadata': 'resolved-secret-names-v1' }
|
||||
)
|
||||
)
|
||||
|
||||
expect(response.status).toBe(200)
|
||||
expect((await response.json()).__resolvedSecretNames).toEqual(['API_KEY'])
|
||||
})
|
||||
|
||||
/** The ordinary silent use: the key authenticates a call and never appears in output. */
|
||||
it('reports a secret used without appearing in the output', async () => {
|
||||
mockExecuteInIsolatedVM.mockResolvedValueOnce({ result: { status: 200 }, stdout: '' })
|
||||
const response = await POST(
|
||||
createMockRequest(
|
||||
'POST',
|
||||
{
|
||||
code: "await fetch('https://api.example.com', { headers: { auth: environmentVariables['API_KEY'] } }); return { status: 200 }",
|
||||
envVars: { API_KEY: 'secret-value-1234' },
|
||||
},
|
||||
{ 'x-sim-request-private-tool-metadata': 'resolved-secret-names-v1' }
|
||||
)
|
||||
)
|
||||
|
||||
expect(response.status).toBe(200)
|
||||
expect((await response.json()).__resolvedSecretNames).toEqual(['API_KEY'])
|
||||
})
|
||||
|
||||
it('does not report a reference when validation rejects before code resolution', async () => {
|
||||
const response = await POST(
|
||||
createMockRequest(
|
||||
@@ -2469,7 +2513,12 @@ describe('Function Execute API Route', () => {
|
||||
expect(mockExecuteInSandbox).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('reports exact secret values returned through placeholders without inferring direct environment reads', async () => {
|
||||
/**
|
||||
* A direct read is a factual reference to the environment binding, not the value-coincidence
|
||||
* inference #6374 removed — that one claimed a secret because its plaintext happened to equal
|
||||
* an unrelated output. Reporting it is what activates execution-log masking for the value.
|
||||
*/
|
||||
it('reports secrets reached through placeholders and through direct environment reads', async () => {
|
||||
mockExecuteInIsolatedVM.mockResolvedValueOnce({
|
||||
result: 'secret-valueother-secret',
|
||||
stdout: '',
|
||||
@@ -2507,14 +2556,14 @@ describe('Function Execute API Route', () => {
|
||||
|
||||
expect(envData.__resolvedSecretNames).toEqual(['ENV_ONLY', 'SHARED'])
|
||||
expect(directData.output.result).toBe('secret-value')
|
||||
expect(directData.__resolvedSecretNames).toEqual([])
|
||||
expect(directData.__resolvedSecretNames).toEqual(['API_KEY'])
|
||||
})
|
||||
|
||||
it.each([
|
||||
{ name: 'numeric', secret: '123', result: 123 },
|
||||
{ name: 'boolean', secret: 'true', result: true },
|
||||
])(
|
||||
'preserves a typed $name value returned through legacy direct environment access without inferred provenance',
|
||||
'preserves a typed $name value returned through a direct environment read while reporting it',
|
||||
async ({ secret, result }) => {
|
||||
mockExecuteInIsolatedVM.mockResolvedValueOnce({ result, stdout: '' })
|
||||
|
||||
@@ -2532,12 +2581,13 @@ describe('Function Execute API Route', () => {
|
||||
)
|
||||
const data = await response.json()
|
||||
|
||||
/** The typed value survives: a secret this short is never substitutable. */
|
||||
expect(data.output.result).toBe(result)
|
||||
expect(data.__resolvedSecretNames).toEqual([])
|
||||
expect(data.__resolvedSecretNames).toEqual(['API_KEY'])
|
||||
}
|
||||
)
|
||||
|
||||
it('reports placeholder output without inferring provenance from legacy shell environment access', async () => {
|
||||
it('reports placeholder output and a shell environment expansion alike', async () => {
|
||||
envFlagsMock.isRemoteSandboxEnabled = true
|
||||
mockExecuteShellInSandbox.mockResolvedValueOnce({
|
||||
result: null,
|
||||
@@ -2582,7 +2632,7 @@ describe('Function Execute API Route', () => {
|
||||
|
||||
expect(referencedData.__resolvedSecretNames).toEqual(['API_KEY'])
|
||||
expect(directData.output.stdout).toBe('secret-value')
|
||||
expect(directData.__resolvedSecretNames).toEqual([])
|
||||
expect(directData.__resolvedSecretNames).toEqual(['API_KEY'])
|
||||
})
|
||||
|
||||
it('returns nonzero shell stderr as a visible 422 error and diagnostic output', async () => {
|
||||
@@ -2672,7 +2722,14 @@ describe('Function Execute API Route', () => {
|
||||
expect((await response.json()).__resolvedSecretNames).toEqual(['__proto__'])
|
||||
})
|
||||
|
||||
it('does not activate a referenced secret that does not cross the Function result', async () => {
|
||||
/**
|
||||
* Previously asserted the inverse: a referenced secret whose value stayed out of the
|
||||
* result reported nothing. That gate made the trail miss silent use — the ordinary
|
||||
* API-call case and the transformed-exfiltration case alike — so activation now follows
|
||||
* the referenced set. The value never appearing costs nothing downstream; the masking
|
||||
* matcher simply never fires on it.
|
||||
*/
|
||||
it('activates a referenced secret even when its value never crosses the result', async () => {
|
||||
mockExecuteInIsolatedVM.mockResolvedValueOnce({ result: 'safe-result', stdout: '' })
|
||||
|
||||
const response = await POST(
|
||||
@@ -2686,7 +2743,7 @@ describe('Function Execute API Route', () => {
|
||||
)
|
||||
)
|
||||
|
||||
expect((await response.json()).__resolvedSecretNames).toEqual([])
|
||||
expect((await response.json()).__resolvedSecretNames).toEqual(['API_KEY'])
|
||||
})
|
||||
|
||||
it.concurrent('should resolve tag variables with <tag_name> syntax', async () => {
|
||||
|
||||
@@ -103,7 +103,6 @@ import {
|
||||
} from '@/executor/utils/reference-validation'
|
||||
import {
|
||||
createResolvedSecretMatcher,
|
||||
projectResolvedSecretContent,
|
||||
type ResolvedSecretMatcher,
|
||||
scanResolvedSecretString,
|
||||
} from '@/executor/utils/resolved-secret-content-projection'
|
||||
@@ -1164,7 +1163,7 @@ async function functionJsonResponse<T>(
|
||||
fileKeys: context.fileKeys,
|
||||
}
|
||||
if (context.includePrivateResolvedSecretNames) {
|
||||
activateOutputSecretProvenance(getFunctionResultProvenanceSurface(body), context)
|
||||
activateReferencedSecretProvenance(context)
|
||||
}
|
||||
const response = NextResponse.json(await compactFunctionRouteBody(responseBody, context), init)
|
||||
return appendPrivateResolvedSecretNames(
|
||||
@@ -1174,54 +1173,19 @@ async function functionJsonResponse<T>(
|
||||
)
|
||||
}
|
||||
|
||||
function getFunctionResultProvenanceSurface(body: unknown): unknown {
|
||||
const record = toRecord(body)
|
||||
const output = toRecord(record.output)
|
||||
const debug = toRecord(record.debug)
|
||||
return [
|
||||
Object.hasOwn(record, 'error') ? record.error : undefined,
|
||||
Object.hasOwn(output, 'result') ? output.result : undefined,
|
||||
Object.hasOwn(output, 'stdout') ? output.stdout : undefined,
|
||||
Object.hasOwn(debug, 'lineContent') ? debug.lineContent : undefined,
|
||||
Object.hasOwn(debug, 'stack') ? debug.stack : undefined,
|
||||
]
|
||||
}
|
||||
|
||||
function activateOutputSecretProvenance(
|
||||
body: unknown,
|
||||
context: FunctionRouteExecutionContext
|
||||
): void {
|
||||
if (!context.outputSecretMatcher) {
|
||||
activateCompiledSecretProvenance(context)
|
||||
return
|
||||
}
|
||||
|
||||
const matchedPlaintexts = new Set<string>()
|
||||
const projection = projectResolvedSecretContent(
|
||||
body,
|
||||
context.outputSecretMatcher,
|
||||
MAX_SANDBOX_OUTPUT_BYTES,
|
||||
{
|
||||
onMatch: (plaintext) => matchedPlaintexts.add(plaintext),
|
||||
}
|
||||
)
|
||||
if (!projection.safe) {
|
||||
activateCompiledSecretProvenance(context)
|
||||
return
|
||||
}
|
||||
for (const plaintext of matchedPlaintexts) {
|
||||
for (const name of context.outputSecretNamesByScanLiteral.get(plaintext) ?? []) {
|
||||
context.resolvedSecretNames.add(name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Conservatively activates only secrets whose placeholders were compiled for this invocation.
|
||||
* This fallback is used when the bounded output classifier cannot inspect a result; it never
|
||||
* considers configured-but-unused environment values and never mutates the functional result.
|
||||
* Activates every secret this invocation's code referenced — compiled `{{KEY}}` bindings and
|
||||
* recognized direct reads, filtered to configured environment values.
|
||||
*
|
||||
* Deliberately not gated on the value appearing in the output. Gating it was backwards for
|
||||
* both consumers of these names: a run that used a key silently — an ordinary API call, or a
|
||||
* value exfiltrated in transformed form — reported nothing, so the usage trail missed exactly
|
||||
* the runs it exists to catch, while downstream masking never learned a value the code
|
||||
* demonstrably held. The referenced set errs toward reporting instead: an extra name only
|
||||
* hands the matcher a value that never appears. Configured-but-unreferenced values are never
|
||||
* included, and the functional result is never mutated.
|
||||
*/
|
||||
function activateCompiledSecretProvenance(context: FunctionRouteExecutionContext): void {
|
||||
function activateReferencedSecretProvenance(context: FunctionRouteExecutionContext): void {
|
||||
for (const name of context.outputSecretPlaintextsByName.keys()) {
|
||||
context.resolvedSecretNames.add(name)
|
||||
}
|
||||
@@ -1309,11 +1273,10 @@ function getPrivateResolvedSecretNames(context: FunctionRouteExecutionContext):
|
||||
|
||||
async function appendResolvedSecretNames(
|
||||
response: NextResponse,
|
||||
context: FunctionRouteExecutionContext,
|
||||
provenanceValue: unknown
|
||||
context: FunctionRouteExecutionContext
|
||||
): Promise<NextResponse> {
|
||||
if (!context.includePrivateResolvedSecretNames) return response
|
||||
activateOutputSecretProvenance(provenanceValue, context)
|
||||
activateReferencedSecretProvenance(context)
|
||||
return appendPrivateResolvedSecretNames(
|
||||
response,
|
||||
getPrivateResolvedSecretNames(context),
|
||||
@@ -2122,7 +2085,7 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
}))
|
||||
)
|
||||
} catch {
|
||||
activateCompiledSecretProvenance(routeContext)
|
||||
activateReferencedSecretProvenance(routeContext)
|
||||
}
|
||||
}
|
||||
resolvedCode = compilation.code
|
||||
@@ -2230,11 +2193,7 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
executionTime,
|
||||
})
|
||||
if (fileExportResponse) {
|
||||
return appendResolvedSecretNames(
|
||||
fileExportResponse,
|
||||
routeContext,
|
||||
cleanStdout(shellStdout)
|
||||
)
|
||||
return appendResolvedSecretNames(fileExportResponse, routeContext)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2414,7 +2373,7 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
executionTime,
|
||||
})
|
||||
if (fileExportResponse) {
|
||||
return appendResolvedSecretNames(fileExportResponse, routeContext, cleanStdout(stdout))
|
||||
return appendResolvedSecretNames(fileExportResponse, routeContext)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2505,7 +2464,7 @@ export const POST = withRouteHandler(async (req: NextRequest) => {
|
||||
executionTime,
|
||||
})
|
||||
if (fileExportResponse) {
|
||||
return appendResolvedSecretNames(fileExportResponse, routeContext, cleanStdout(stdout))
|
||||
return appendResolvedSecretNames(fileExportResponse, routeContext)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
/**
|
||||
* @vitest-environment node
|
||||
*/
|
||||
import { authMockFns, createMockRequest } from '@sim/testing'
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
|
||||
const mocks = vi.hoisted(() => ({ listUsage: vi.fn() }))
|
||||
|
||||
vi.mock('@/lib/secrets/application/use-cases', () => ({
|
||||
listSecretUsageUseCase: {
|
||||
operation: { id: 'secrets.usage' },
|
||||
execute: mocks.listUsage,
|
||||
},
|
||||
}))
|
||||
|
||||
import { GET } from '@/app/api/secrets/usage/route'
|
||||
|
||||
const url =
|
||||
'http://localhost/api/secrets/usage?workspaceId=workspace-1&name=API_KEY&scope=workspace'
|
||||
|
||||
describe('GET /api/secrets/usage', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
authMockFns.mockGetSession.mockResolvedValue({
|
||||
user: { id: 'admin-1' },
|
||||
session: { id: 'session-1' },
|
||||
})
|
||||
})
|
||||
|
||||
/**
|
||||
* The presenter calls `toISOString()` on values the database layer produces. A mocked db
|
||||
* returns no rows, so nothing here exercised that until a real query handed back a driver
|
||||
* string and the route 500'd on `toISOString is not a function`. This pins the shape the
|
||||
* presenter is entitled to assume.
|
||||
*/
|
||||
it('serializes the timestamps an entry carries', async () => {
|
||||
mocks.listUsage.mockResolvedValue({
|
||||
entries: [
|
||||
{
|
||||
id: 'usage-1',
|
||||
useCount: 4,
|
||||
lastUsedAt: new Date('2026-03-14T09:30:00.000Z'),
|
||||
source: 'workflow',
|
||||
workflowName: 'Nightly sync',
|
||||
actorName: 'Ada',
|
||||
lastExecutionId: 'execution-1',
|
||||
lastExecutionAvailable: true,
|
||||
lastTrigger: 'schedule',
|
||||
},
|
||||
],
|
||||
})
|
||||
|
||||
const response = await GET(createMockRequest('GET', undefined, {}, url))
|
||||
|
||||
expect(response.status).toBe(200)
|
||||
expect(await response.json()).toEqual({
|
||||
entries: [
|
||||
expect.objectContaining({
|
||||
id: 'usage-1',
|
||||
lastUsedAt: '2026-03-14T09:30:00.000Z',
|
||||
}),
|
||||
],
|
||||
})
|
||||
})
|
||||
|
||||
it('returns an empty trail for a secret that has never been used', async () => {
|
||||
mocks.listUsage.mockResolvedValue({ entries: [] })
|
||||
|
||||
const response = await GET(createMockRequest('GET', undefined, {}, url))
|
||||
|
||||
expect(response.status).toBe(200)
|
||||
expect(await response.json()).toEqual({ entries: [] })
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,31 @@
|
||||
import { getSecretUsageContract } from '@/lib/api/contracts/secrets'
|
||||
import {
|
||||
defineInternalJsonRoute,
|
||||
internalOrchestrationErrorPolicy,
|
||||
internalRateLimits,
|
||||
internalSessionAuth,
|
||||
} from '@/lib/api/server/routes'
|
||||
import { secretOperations } from '@/lib/secrets/application/operations'
|
||||
import { listSecretUsageUseCase } from '@/lib/secrets/application/use-cases'
|
||||
|
||||
/** GET /api/secrets/usage — One secret's usage trail, for the credential detail panel. */
|
||||
export const GET = defineInternalJsonRoute({
|
||||
contract: getSecretUsageContract,
|
||||
auth: internalSessionAuth,
|
||||
operation: secretOperations.usage,
|
||||
rateLimit: internalRateLimits.none({ reason: 'Preserve existing internal behavior' }),
|
||||
errorPolicy: internalOrchestrationErrorPolicy,
|
||||
mapInput: ({ query }) => ({
|
||||
workspaceId: query.workspaceId,
|
||||
name: query.name,
|
||||
scope: query.scope,
|
||||
limit: query.limit,
|
||||
}),
|
||||
useCase: listSecretUsageUseCase,
|
||||
present: ({ entries }) => ({
|
||||
entries: entries.map((entry) => ({
|
||||
...entry,
|
||||
lastUsedAt: entry.lastUsedAt.toISOString(),
|
||||
})),
|
||||
}),
|
||||
})
|
||||
+58
-16
@@ -17,6 +17,12 @@ export interface ActivityLogEntry {
|
||||
description: ReactNode
|
||||
actor: ReactNode
|
||||
details?: ReactNode
|
||||
/**
|
||||
* Row action (typically a `Chip`/`ChipLink`) in a trailing column after every
|
||||
* data column. The column appears as soon as any entry supplies one, and the
|
||||
* header reserves the same width so the Actor column stays aligned.
|
||||
*/
|
||||
trailing?: ReactNode
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -32,14 +38,19 @@ const EVENT_COLUMN_WIDTH_CLASS = {
|
||||
|
||||
type EventColumnWidth = keyof typeof EVENT_COLUMN_WIDTH_CLASS
|
||||
|
||||
/** Trailing row-action column, wide enough for a chip without wrapping its label. */
|
||||
const TRAILING_COLUMN_WIDTH_CLASS = 'w-[100px]'
|
||||
|
||||
const ROW_CLASS = 'flex w-full items-center gap-3 px-3 py-2 text-left'
|
||||
|
||||
function ActivityLogRow({
|
||||
entry,
|
||||
eventColumn,
|
||||
hasTrailingColumn,
|
||||
}: {
|
||||
entry: ActivityLogEntry
|
||||
eventColumn: EventColumnWidth
|
||||
hasTrailingColumn: boolean
|
||||
}) {
|
||||
const [expanded, setExpanded] = useState(false)
|
||||
const expandable = entry.details != null
|
||||
@@ -85,21 +96,40 @@ function ActivityLogRow({
|
||||
expanded && 'bg-[var(--surface-2)]'
|
||||
)}
|
||||
>
|
||||
{expandable ? (
|
||||
<button
|
||||
type='button'
|
||||
aria-expanded={expanded}
|
||||
className={ROW_CLASS}
|
||||
onClick={() => setExpanded(!expanded)}
|
||||
>
|
||||
{cells}
|
||||
</button>
|
||||
) : (
|
||||
// A row with nothing to expand is inert content, not a disabled control:
|
||||
// browsers suppress pointer events over a disabled button AND its
|
||||
// descendants, which would swallow the hover tooltips inside the cells.
|
||||
<div className={ROW_CLASS}>{cells}</div>
|
||||
)}
|
||||
{/*
|
||||
The trailing action is a SIBLING of the expand button, never a child: a link
|
||||
or button nested inside another button is invalid, and the inner control's
|
||||
click would toggle the row on its way up.
|
||||
*/}
|
||||
<div className={cn('flex items-center', hasTrailingColumn && 'gap-3 pr-3')}>
|
||||
{expandable ? (
|
||||
<button
|
||||
type='button'
|
||||
aria-expanded={expanded}
|
||||
className={cn(ROW_CLASS, 'min-w-0 flex-1', hasTrailingColumn && 'pr-0')}
|
||||
onClick={() => setExpanded(!expanded)}
|
||||
>
|
||||
{cells}
|
||||
</button>
|
||||
) : (
|
||||
// A row with nothing to expand is inert content, not a disabled control:
|
||||
// browsers suppress pointer events over a disabled button AND its
|
||||
// descendants, which would swallow the hover tooltips inside the cells.
|
||||
<div className={cn(ROW_CLASS, 'min-w-0 flex-1', hasTrailingColumn && 'pr-0')}>
|
||||
{cells}
|
||||
</div>
|
||||
)}
|
||||
{hasTrailingColumn && (
|
||||
<span
|
||||
className={cn(
|
||||
TRAILING_COLUMN_WIDTH_CLASS,
|
||||
'flex flex-shrink-0 items-center justify-end'
|
||||
)}
|
||||
>
|
||||
{entry.trailing}
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
{expandable && expanded && (
|
||||
<div className='px-3 pb-2'>
|
||||
<div className='flex flex-col gap-1.5 rounded-lg border border-[var(--border-1)] bg-[var(--surface-3)] p-3 text-small'>
|
||||
@@ -140,6 +170,8 @@ export function ActivityLog({
|
||||
emptyState,
|
||||
footer,
|
||||
}: ActivityLogProps) {
|
||||
const hasTrailingColumn = entries.some((entry) => entry.trailing != null)
|
||||
|
||||
return (
|
||||
<div className='flex flex-col'>
|
||||
<div className='flex items-center gap-3 px-3 pb-1 text-[var(--text-tertiary)] text-caption'>
|
||||
@@ -149,6 +181,11 @@ export function ActivityLog({
|
||||
</span>
|
||||
<span className='min-w-0 flex-1'>{descriptionLabel}</span>
|
||||
<span className='w-[160px] flex-shrink-0 text-right'>Actor</span>
|
||||
{/* Row actions carry no header, but the column must still be reserved
|
||||
here or every label above would sit left of the data below it. */}
|
||||
{hasTrailingColumn && (
|
||||
<span className={cn(TRAILING_COLUMN_WIDTH_CLASS, 'flex-shrink-0')} aria-hidden />
|
||||
)}
|
||||
</div>
|
||||
|
||||
{entries.length === 0 ? (
|
||||
@@ -156,7 +193,12 @@ export function ActivityLog({
|
||||
) : (
|
||||
<div className='flex flex-col gap-0.5'>
|
||||
{entries.map((entry) => (
|
||||
<ActivityLogRow key={entry.id} entry={entry} eventColumn={eventColumn} />
|
||||
<ActivityLogRow
|
||||
key={entry.id}
|
||||
entry={entry}
|
||||
eventColumn={eventColumn}
|
||||
hasTrailingColumn={hasTrailingColumn}
|
||||
/>
|
||||
))}
|
||||
{footer}
|
||||
</div>
|
||||
|
||||
+1
@@ -0,0 +1 @@
|
||||
export { SecretUsagePanel } from './secret-usage-panel'
|
||||
+125
@@ -0,0 +1,125 @@
|
||||
'use client'
|
||||
|
||||
import { useMemo } from 'react'
|
||||
import { ChipLink } from '@sim/emcn'
|
||||
import { formatDateTime } from '@sim/utils/formatting'
|
||||
import { SettingsActionChip } from '@/components/settings/settings-header'
|
||||
import type { SecretUsageEntryPayload, SecretUsageScope } from '@/lib/api/contracts'
|
||||
import { FloatingOverflowText } from '@/app/workspace/[workspaceId]/components'
|
||||
import { DELETED_WORKFLOW_LABEL, TriggerBadge } from '@/app/workspace/[workspaceId]/logs/utils'
|
||||
import {
|
||||
ActivityLog,
|
||||
type ActivityLogEntry,
|
||||
} from '@/app/workspace/[workspaceId]/settings/components/activity-log'
|
||||
import { SettingsEmptyState } from '@/app/workspace/[workspaceId]/settings/components/settings-empty-state'
|
||||
import { useSecretUsage } from '@/hooks/queries/credentials'
|
||||
|
||||
/**
|
||||
* The disabled twin of the View log chip, for a run whose log has been pruned. Routed through
|
||||
* the shared settings chip so it carries the platform's disabled tooltip treatment — including
|
||||
* the pointer-events handling a disabled button needs for the tooltip to fire at all.
|
||||
*/
|
||||
const EXPIRED_LOG_ACTION = {
|
||||
id: 'view-log',
|
||||
text: 'View log',
|
||||
disabled: true,
|
||||
tooltip: 'This run\u2019s log is past your workspace\u2019s retention window',
|
||||
onSelect: () => {},
|
||||
} as const
|
||||
|
||||
interface SecretUsagePanelProps {
|
||||
workspaceId: string
|
||||
secretName: string
|
||||
scope: SecretUsageScope
|
||||
}
|
||||
|
||||
/** What used the secret, in the reader's terms rather than the storage enum's. */
|
||||
function usedBy(entry: SecretUsageEntryPayload): string {
|
||||
if (entry.source === 'copilot') return 'Sim agent'
|
||||
if (entry.source === 'mcp') return 'MCP server'
|
||||
return entry.workflowName ?? DELETED_WORKFLOW_LABEL
|
||||
}
|
||||
|
||||
/**
|
||||
* One secret's usage trail.
|
||||
*
|
||||
* Rows are per-day buckets, so the timestamp is the most recent use in that day and the run
|
||||
* count only appears when it is above one — a "1 run" on every row is noise, not data. The
|
||||
* trigger badge is the Logs page's own, so a row reads the same here as at the run it links to.
|
||||
*/
|
||||
export function SecretUsagePanel({ workspaceId, secretName, scope }: SecretUsagePanelProps) {
|
||||
const { data, isPending, isError } = useSecretUsage({ workspaceId, name: secretName, scope })
|
||||
|
||||
const entries = useMemo<ActivityLogEntry[]>(
|
||||
() =>
|
||||
(data?.entries ?? []).map((entry) => ({
|
||||
id: entry.id,
|
||||
timestamp: formatDateTime(new Date(entry.lastUsedAt)),
|
||||
event: <TriggerBadge trigger={entry.lastTrigger ?? entry.source} />,
|
||||
/**
|
||||
* Plain text, so the name sits flush under its column header — a chip's own
|
||||
* padding would indent it out of line with every other column.
|
||||
*/
|
||||
description: (
|
||||
<span className='flex min-w-0 items-center gap-2'>
|
||||
<FloatingOverflowText label={usedBy(entry)} className='block truncate' />
|
||||
{entry.useCount > 1 && (
|
||||
<span className='flex-shrink-0 text-[var(--text-muted)]'>{entry.useCount} runs</span>
|
||||
)}
|
||||
</span>
|
||||
),
|
||||
actor: entry.actorName ?? 'Unknown',
|
||||
/**
|
||||
* Three states, not two. A row with no execution id never had a run to link (Sim agent
|
||||
* and MCP resolutions have none). A row whose run has since been pruned — usage
|
||||
* outlives logs on purpose — keeps the chip but disables it, so the reader learns the
|
||||
* log expired instead of clicking into an empty Logs view.
|
||||
*
|
||||
* `border` is the outline-only variant: a bare chip renders as unadorned
|
||||
* `--text-body` text at `text-sm`, which next to the Actor cell's `--text-secondary`
|
||||
* `text-small` reads as one more data column rather than a control. The outline is
|
||||
* the lightest chrome that says "this is a button" without adding a fill to every row.
|
||||
* The negative margin lets the 30px pill overhang the row's text line instead of
|
||||
* growing it, so a row with a link is the same height as one without.
|
||||
*/
|
||||
trailing: entry.lastExecutionId ? (
|
||||
entry.lastExecutionAvailable ? (
|
||||
<ChipLink
|
||||
href={`/workspace/${workspaceId}/logs?executionId=${entry.lastExecutionId}`}
|
||||
variant='border'
|
||||
className='-my-1'
|
||||
>
|
||||
View log
|
||||
</ChipLink>
|
||||
) : (
|
||||
<span className='-my-1 inline-flex'>
|
||||
<SettingsActionChip action={EXPIRED_LOG_ACTION} />
|
||||
</span>
|
||||
)
|
||||
) : undefined,
|
||||
})),
|
||||
[data?.entries, workspaceId]
|
||||
)
|
||||
|
||||
if (isError) {
|
||||
return (
|
||||
<SettingsEmptyState variant='inline' tone='error'>
|
||||
Could not load usage.
|
||||
</SettingsEmptyState>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<ActivityLog
|
||||
entries={entries}
|
||||
eventLabel='Trigger'
|
||||
descriptionLabel='Used by'
|
||||
eventColumn='compact'
|
||||
emptyState={
|
||||
<SettingsEmptyState variant='inline'>
|
||||
{isPending ? 'Loading…' : 'This secret has not been used yet.'}
|
||||
</SettingsEmptyState>
|
||||
}
|
||||
/>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
'use client'
|
||||
|
||||
import { ChipLink } from '@sim/emcn'
|
||||
import { ArrowLeft } from '@sim/emcn/icons'
|
||||
import { useParams } from 'next/navigation'
|
||||
import { CredentialDetailLayout } from '@/app/workspace/[workspaceId]/components/credential-detail'
|
||||
import { SettingsEmptyState } from '@/app/workspace/[workspaceId]/settings/components/settings-empty-state'
|
||||
|
||||
/**
|
||||
* Serves both the route transition into a secret and the in-page Suspense boundary the
|
||||
* detail's `useQueryState` needs, so the chrome never flashes between the two.
|
||||
*/
|
||||
export default function SecretDetailLoading() {
|
||||
const { workspaceId } = useParams<{ workspaceId: string }>()
|
||||
|
||||
return (
|
||||
<CredentialDetailLayout
|
||||
back={
|
||||
<ChipLink href={`/workspace/${workspaceId}/settings/secrets`} leftIcon={ArrowLeft}>
|
||||
Secrets
|
||||
</ChipLink>
|
||||
}
|
||||
>
|
||||
<SettingsEmptyState variant='inline'>Loading…</SettingsEmptyState>
|
||||
</CredentialDetailLayout>
|
||||
)
|
||||
}
|
||||
@@ -1,4 +1,6 @@
|
||||
import { Suspense } from 'react'
|
||||
import type { Metadata } from 'next'
|
||||
import SecretDetailLoading from '@/app/workspace/[workspaceId]/settings/secrets/[credentialId]/loading'
|
||||
import { SecretDetail } from '@/app/workspace/[workspaceId]/settings/secrets/[credentialId]/secret-detail'
|
||||
|
||||
export const metadata: Metadata = {
|
||||
@@ -11,5 +13,9 @@ export default async function SecretDetailPage({
|
||||
params: Promise<{ workspaceId: string; credentialId: string }>
|
||||
}) {
|
||||
const { workspaceId, credentialId } = await params
|
||||
return <SecretDetail workspaceId={workspaceId} credentialId={credentialId} />
|
||||
return (
|
||||
<Suspense fallback={<SecretDetailLoading />}>
|
||||
<SecretDetail workspaceId={workspaceId} credentialId={credentialId} />
|
||||
</Suspense>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
import { parseAsStringLiteral } from 'nuqs/server'
|
||||
|
||||
/**
|
||||
* `secret-view` deep-links a secret to its usage view, opened from the detail header.
|
||||
* Mirrors `fork-view` on the Forks tab: usage is its own destination, not a section that
|
||||
* expands inside the secret it belongs to.
|
||||
*/
|
||||
export const secretDetailViewParam = {
|
||||
key: 'secret-view',
|
||||
parser: parseAsStringLiteral(['usage'] as const),
|
||||
} as const
|
||||
|
||||
/** Opening the usage view is a destination → push to history; clear on close. */
|
||||
export const secretDetailViewUrlKeys = {
|
||||
history: 'push',
|
||||
clearOnDefault: true,
|
||||
} as const
|
||||
+106
-12
@@ -2,8 +2,11 @@
|
||||
|
||||
import { useState } from 'react'
|
||||
import { Chip, ChipCopyInput, ChipLink, ChipTextarea } from '@sim/emcn'
|
||||
import { ArrowLeft, Key, Send } from '@sim/emcn/icons'
|
||||
import { ArrowLeft, Clock, Key, Send } from '@sim/emcn/icons'
|
||||
import { useQueryState } from 'nuqs'
|
||||
import { SaveDiscardChips } from '@/components/settings/save-discard-actions'
|
||||
import { SettingsActionChips } from '@/components/settings/settings-header'
|
||||
import { isApiClientError } from '@/lib/api/client/errors'
|
||||
import { ResourceTile } from '@/app/workspace/[workspaceId]/components'
|
||||
import {
|
||||
AddPeopleModal,
|
||||
@@ -17,6 +20,11 @@ import {
|
||||
import { SecretValueField } from '@/app/workspace/[workspaceId]/settings/components/secrets/components/secret-value-field'
|
||||
import { useSecretValue } from '@/app/workspace/[workspaceId]/settings/components/secrets/hooks/use-secret-value'
|
||||
import { SettingsEmptyState } from '@/app/workspace/[workspaceId]/settings/components/settings-empty-state'
|
||||
import { SecretUsagePanel } from '@/app/workspace/[workspaceId]/settings/secrets/[credentialId]/components/secret-usage-panel'
|
||||
import {
|
||||
secretDetailViewParam,
|
||||
secretDetailViewUrlKeys,
|
||||
} from '@/app/workspace/[workspaceId]/settings/secrets/[credentialId]/search-params'
|
||||
import { useWorkspaceCredential } from '@/hooks/queries/credentials'
|
||||
|
||||
interface SecretDetailProps {
|
||||
@@ -27,11 +35,15 @@ interface SecretDetailProps {
|
||||
export function SecretDetail({ workspaceId, credentialId }: SecretDetailProps) {
|
||||
const secretsHref = `/workspace/${workspaceId}/settings/secrets`
|
||||
|
||||
const { data: credential = null, isPending } = useWorkspaceCredential(credentialId)
|
||||
const { data: credential = null, isPending, error } = useWorkspaceCredential(credentialId)
|
||||
const isAdmin = credential?.role === 'admin'
|
||||
const isPersonal = credential?.type === 'env_personal'
|
||||
|
||||
const [isShareModalOpen, setIsShareModalOpen] = useState(false)
|
||||
const [view, setView] = useQueryState(secretDetailViewParam.key, {
|
||||
...secretDetailViewParam.parser,
|
||||
...secretDetailViewUrlKeys,
|
||||
})
|
||||
|
||||
const valueField = useSecretValue({ workspaceId, credential })
|
||||
|
||||
@@ -58,22 +70,58 @@ export function SecretDetail({ workspaceId, credentialId }: SecretDetailProps) {
|
||||
|
||||
const canEditValue = valueField.canEdit && !valueField.isConflicted
|
||||
|
||||
const actions =
|
||||
credential && (isWorkspaceSecretAdmin || canEditValue) ? (
|
||||
<>
|
||||
{isWorkspaceSecretAdmin && (
|
||||
<Chip leftIcon={Send} onClick={() => setIsShareModalOpen(true)}>
|
||||
Share
|
||||
</Chip>
|
||||
)}
|
||||
/**
|
||||
* Usage names workflows, people, and run ids — the same slice value masking withholds — so
|
||||
* it is gated on the same predicate that reveals the value: admin of a workspace secret,
|
||||
* owner of a personal one. `canEdit` is exactly that predicate. Deliberately not
|
||||
* `canEditValue`: a personal secret shadowed by a workspace variable is read-only, but it
|
||||
* is still its owner's to audit.
|
||||
*
|
||||
* A member who cannot see it gets a disabled chip rather than no chip, so the capability is
|
||||
* discoverable and the reason is stated instead of silently missing.
|
||||
*/
|
||||
const canViewUsage = valueField.canEdit
|
||||
|
||||
const actions = credential ? (
|
||||
<>
|
||||
<SettingsActionChips
|
||||
actions={[
|
||||
{
|
||||
id: 'usage',
|
||||
text: 'See usage',
|
||||
icon: Clock,
|
||||
onSelect: () => void setView('usage'),
|
||||
disabled: !canViewUsage,
|
||||
...(canViewUsage
|
||||
? {}
|
||||
: {
|
||||
tooltip: isPersonal
|
||||
? 'Only the owner of this secret can see its usage'
|
||||
: 'Only admins of this secret can see its usage',
|
||||
}),
|
||||
},
|
||||
...(isWorkspaceSecretAdmin
|
||||
? [
|
||||
{
|
||||
id: 'share',
|
||||
text: 'Share',
|
||||
icon: Send,
|
||||
onSelect: () => setIsShareModalOpen(true),
|
||||
},
|
||||
]
|
||||
: []),
|
||||
]}
|
||||
/>
|
||||
{(isWorkspaceSecretAdmin || canEditValue) && (
|
||||
<SaveDiscardChips
|
||||
dirty={form.isDirty}
|
||||
saving={form.isSaving}
|
||||
onSave={form.save}
|
||||
onDiscard={form.discard}
|
||||
/>
|
||||
</>
|
||||
) : null
|
||||
)}
|
||||
</>
|
||||
) : null
|
||||
|
||||
if (isPending && !credential) {
|
||||
return (
|
||||
@@ -83,6 +131,23 @@ export function SecretDetail({ workspaceId, credentialId }: SecretDetailProps) {
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* A failed load is not a missing secret. Every outcome used to render "Secret not found",
|
||||
* so a permission failure or an unreachable API was indistinguishable from a deleted
|
||||
* credential — and the one message sent you looking for the wrong problem.
|
||||
*/
|
||||
if (error && !(isApiClientError(error) && error.status === 404)) {
|
||||
return (
|
||||
<CredentialDetailLayout back={back} actions={actions}>
|
||||
<SettingsEmptyState variant='inline' tone='error'>
|
||||
{isApiClientError(error) && error.status === 403
|
||||
? 'You do not have access to this secret.'
|
||||
: 'Could not load this secret.'}
|
||||
</SettingsEmptyState>
|
||||
</CredentialDetailLayout>
|
||||
)
|
||||
}
|
||||
|
||||
if (!credential) {
|
||||
return (
|
||||
<CredentialDetailLayout back={back} actions={actions}>
|
||||
@@ -91,6 +156,35 @@ export function SecretDetail({ workspaceId, credentialId }: SecretDetailProps) {
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Usage is a destination reached from the header, the same shape as the Forks tab's
|
||||
* "See activity" — it replaces the secret rather than expanding inside it, so the two
|
||||
* readings never compete for the same column. Back returns with `replace`, since opening
|
||||
* already pushed.
|
||||
*/
|
||||
if (canViewUsage && view === 'usage') {
|
||||
return (
|
||||
<CredentialDetailLayout
|
||||
back={
|
||||
<Chip leftIcon={ArrowLeft} onClick={() => void setView(null, { history: 'replace' })}>
|
||||
{credential.envKey || credential.displayName}
|
||||
</Chip>
|
||||
}
|
||||
>
|
||||
<CredentialDetailHeading
|
||||
leading={<ResourceTile icon={Clock} />}
|
||||
title='Usage'
|
||||
subtitle={credential.envKey || credential.displayName}
|
||||
/>
|
||||
<SecretUsagePanel
|
||||
workspaceId={workspaceId}
|
||||
secretName={credential.envKey || ''}
|
||||
scope={isPersonal ? 'personal' : 'workspace'}
|
||||
/>
|
||||
</CredentialDetailLayout>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<>
|
||||
<CredentialDetailLayout back={back} actions={actions}>
|
||||
|
||||
@@ -598,6 +598,7 @@ async function executeWebhookJobInternal(
|
||||
personalDecrypted: secretEnvironment.personalDecrypted,
|
||||
workspaceDecrypted: secretEnvironment.workspaceDecrypted,
|
||||
decryptionFailures: secretEnvironment.decryptionFailures,
|
||||
personalOwners: secretEnvironment.personalOwners,
|
||||
scope: secretScope,
|
||||
})
|
||||
} catch (error) {
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
---
|
||||
slug: secret-provenance
|
||||
title: 'Tracking Secrets Through an Agent Run'
|
||||
description: 'How Sim tracks the secrets an agent run actually uses, carries that provenance across tools and storage, and redacts values at every egress boundary.'
|
||||
description: 'How Sim tracks the secrets an agent run actually uses, carries that provenance across tools and storage, redacts values at every egress boundary, and records which run touched which credential.'
|
||||
date: 2026-08-08
|
||||
updated: 2026-08-12
|
||||
updated: 2026-08-18
|
||||
authors:
|
||||
- vik
|
||||
readingTime: 7
|
||||
@@ -26,6 +26,8 @@ faq:
|
||||
a: "Across both. Durable content such as workspace files, table cells, knowledge documents, and agent memory carries encrypted provenance so a later run does not mistake secret-bearing data for clean data."
|
||||
- q: "Is the model treated as an internal component or an egress boundary?"
|
||||
a: "An egress boundary. Model-bound content leaves Sim's infrastructure, may be retained by the provider, and can be echoed in a later turn, so it is projected before the request is sent."
|
||||
- q: "Can code inside a workflow defeat redaction and print a secret anyway?"
|
||||
a: "Code that never emits the value can avoid a value matcher — printing a key one character at a time produces no output that matches it. Deciding this in general is undecidable, so Sim pairs redaction with attribution: every run records which secrets it resolved, under which identity, so misuse is visible even when the value never appears."
|
||||
---
|
||||
|
||||
Secrets usually do not leak when an application first reads them. They leak after they have been copied into an error, passed to another service, or written somewhere that outlives the request.
|
||||
@@ -84,6 +86,14 @@ Eight characters is not a universal definition of a secret. It is the point at w
|
||||
|
||||
There is another boundary to the guarantee: matching uses exact bytes. A hash, signature, or re-encoding derived from a secret no longer contains those bytes and will not be caught by the same matcher.
|
||||
|
||||
## Recording What a Run Used
|
||||
|
||||
Projection stops a value at a boundary. It cannot stop code that never produces the value: a few lines in a Function block can print a key one character at a time, and no output ever matches the secret it came from. That is not a gap in the matcher. Deciding whether arbitrary code will eventually reveal a value is undecidable in general, which is why [ShellCheck](https://www.shellcheck.net/wiki/SC2154) says the same about tracking indirect references.
|
||||
|
||||
So the second posture is attribution rather than prevention. Every run records which configured secrets it actually resolved, under whose identity, and through which surface. Execution logs cannot answer that: they persist the environment a run *could* have read rather than the subset it referenced, and they expire on the workspace's retention window.
|
||||
|
||||
Redaction keeps the value out of the record. The trail says whose hands it passed through.
|
||||
|
||||
## Taint Tracking in Reverse
|
||||
|
||||
Seen through the lens of information-flow security, this is an old idea pointed in a different direction. Dorothy Denning's [lattice model](https://dl.acm.org/doi/10.1145/360051.360056) described how data can be classified and constrained as it moves between security levels. Myers and Liskov's [decentralized label model](https://www.cs.cornell.edu/andru/papers/iflow-sosp97/paper.html) added controlled declassification: releasing labeled data only after transforming it into a safe form.
|
||||
|
||||
@@ -455,6 +455,7 @@ export class WorkflowBlockHandler implements BlockHandler {
|
||||
personalDecrypted: ownerEnv.personalDecrypted,
|
||||
workspaceDecrypted: ownerEnv.workspaceDecrypted,
|
||||
decryptionFailures: ownerEnv.decryptionFailures,
|
||||
personalOwners: ownerEnv.personalOwners,
|
||||
scope: { userId: loadUserId, workspaceId: sourceWorkspaceId },
|
||||
})
|
||||
if (ctx.resolvedSecretTraceRegistry) {
|
||||
|
||||
@@ -612,6 +612,98 @@ describe('ResolvedSecretTraceRegistry', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('getResolvedSecretUsage', () => {
|
||||
it('reports only the secrets a run actually resolved, with their scope', async () => {
|
||||
const registry = await createResolvedSecretTraceRegistry({
|
||||
personalEncrypted: { PERSONAL_KEY: 'personal-encrypted' },
|
||||
workspaceEncrypted: { WORKSPACE_KEY: 'workspace-encrypted', UNUSED: 'unused-encrypted' },
|
||||
personalDecrypted: { PERSONAL_KEY: 'personal-secret' },
|
||||
workspaceDecrypted: { WORKSPACE_KEY: 'workspace-secret', UNUSED: 'unused-secret' },
|
||||
personalOwners: { PERSONAL_KEY: 'owner-1' },
|
||||
})
|
||||
|
||||
expect(registry.recordResolved('PERSONAL_KEY', 'personal-secret')).toBe(true)
|
||||
expect(registry.recordResolved('WORKSPACE_KEY', 'workspace-secret')).toBe(true)
|
||||
|
||||
expect(registry.getResolvedSecretUsage()).toEqual([
|
||||
{ name: 'PERSONAL_KEY', scope: 'personal', ownerUserId: 'owner-1' },
|
||||
{ name: 'WORKSPACE_KEY', scope: 'workspace', ownerUserId: null },
|
||||
])
|
||||
})
|
||||
|
||||
/**
|
||||
* A personal secret shared into the workspace resolves for someone who does not own it.
|
||||
* The trail is read per owner, so it has to be filed under the sharer or it would show up
|
||||
* under the borrower's own same-named secret.
|
||||
*/
|
||||
it('attributes a shared personal secret to its owner, not the resolving caller', async () => {
|
||||
const registry = await createResolvedSecretTraceRegistry({
|
||||
personalEncrypted: { SHARED_KEY: 'personal-encrypted' },
|
||||
workspaceEncrypted: {},
|
||||
personalDecrypted: { SHARED_KEY: 'shared-secret' },
|
||||
workspaceDecrypted: {},
|
||||
personalOwners: { SHARED_KEY: 'sharer-1' },
|
||||
scope: { userId: 'borrower-1', workspaceId: 'workspace-1' },
|
||||
})
|
||||
|
||||
expect(registry.recordResolved('SHARED_KEY', 'shared-secret')).toBe(true)
|
||||
expect(registry.getResolvedSecretUsage()).toEqual([
|
||||
{ name: 'SHARED_KEY', scope: 'personal', ownerUserId: 'sharer-1' },
|
||||
])
|
||||
})
|
||||
|
||||
/**
|
||||
* Recording an unattributed personal row would surface it under every other user's
|
||||
* secret of the same name, so it is dropped instead.
|
||||
*/
|
||||
it('drops a personal secret whose owner is unknown', async () => {
|
||||
const registry = await createResolvedSecretTraceRegistry({
|
||||
personalEncrypted: { PERSONAL_KEY: 'personal-encrypted' },
|
||||
workspaceEncrypted: {},
|
||||
personalDecrypted: { PERSONAL_KEY: 'personal-secret' },
|
||||
workspaceDecrypted: {},
|
||||
})
|
||||
|
||||
expect(registry.recordResolved('PERSONAL_KEY', 'personal-secret')).toBe(true)
|
||||
expect(registry.getResolvedSecretUsage()).toEqual([])
|
||||
})
|
||||
|
||||
it('is empty when a configured secret was never resolved', async () => {
|
||||
const registry = await createResolvedSecretTraceRegistry({
|
||||
personalEncrypted: {},
|
||||
workspaceEncrypted: { API_KEY: 'workspace-encrypted' },
|
||||
personalDecrypted: {},
|
||||
workspaceDecrypted: { API_KEY: 'workspace-secret' },
|
||||
})
|
||||
|
||||
expect(registry.getResolvedSecretUsage()).toEqual([])
|
||||
})
|
||||
|
||||
/**
|
||||
* An imported entry is a secret a sub-run or tool call already recorded against its own
|
||||
* execution; counting it again here would double it.
|
||||
*/
|
||||
it('omits entries adopted from imported provenance', async () => {
|
||||
const registry = await createResolvedSecretTraceRegistry({
|
||||
personalEncrypted: {},
|
||||
workspaceEncrypted: {},
|
||||
personalDecrypted: {},
|
||||
workspaceDecrypted: {},
|
||||
})
|
||||
|
||||
await registry.importProvenance(
|
||||
{
|
||||
version: 1,
|
||||
complete: true,
|
||||
entries: [{ name: 'CROSSED_KEY', encryptedValue: 'crossed-encrypted' }],
|
||||
},
|
||||
{ trusted: true }
|
||||
)
|
||||
|
||||
expect(registry.getResolvedSecretUsage()).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
it('ignores empty decryption failures but fails closed for a resolved value outside the catalog', async () => {
|
||||
const registry = await createResolvedSecretTraceRegistry({
|
||||
personalEncrypted: { FAILED: 'failed-ciphertext' },
|
||||
|
||||
@@ -185,10 +185,36 @@ const PROVENANCE_PROPERTY_NAMES = new Set(['version', 'complete', 'entries', 'sc
|
||||
const PROVENANCE_ENTRY_PROPERTY_NAMES = new Set(['encryptedValue', 'name'])
|
||||
const PROVENANCE_SCOPE_PROPERTY_NAMES = new Set(['userId', 'workspaceId'])
|
||||
|
||||
/** Which environment a catalog entry's value came from, when that is known. */
|
||||
export type ResolvedSecretScope = 'workspace' | 'personal'
|
||||
|
||||
/** One secret a run resolved, identified the way the usage trail is keyed. */
|
||||
export interface ResolvedSecretUsageEntry {
|
||||
name: string
|
||||
scope: ResolvedSecretScope
|
||||
/** The owning user for a personal secret; null for a workspace one. */
|
||||
ownerUserId: string | null
|
||||
}
|
||||
|
||||
export interface ResolvedSecretTraceCatalogEntry {
|
||||
name: string
|
||||
plaintext: string
|
||||
encryptedValue: string
|
||||
/**
|
||||
* Optional because only a run's own effective catalog knows it. Entries adopted from an
|
||||
* imported provenance envelope carry a name but no scope, and are deliberately left
|
||||
* unattributed — the sub-run or tool call they crossed from records its own usage, so
|
||||
* attributing them here would double-count.
|
||||
*/
|
||||
scope?: ResolvedSecretScope
|
||||
/**
|
||||
* Whose personal environment a `personal` entry came from. Required to tell two people's
|
||||
* same-named personal secrets apart, and NOT the same as the run's actor: a personal
|
||||
* secret shared with the workspace resolves for a caller who does not own it, and a
|
||||
* scheduled run resolves the workflow owner's personal slice under a different actor.
|
||||
* Unset for workspace entries, which the workspace itself owns.
|
||||
*/
|
||||
ownerUserId?: string
|
||||
}
|
||||
|
||||
export interface ResolvedSecretTraceMatch {
|
||||
@@ -302,6 +328,8 @@ export interface CreateResolvedSecretTraceRegistryOptions {
|
||||
personalDecrypted: Record<string, string>
|
||||
workspaceDecrypted: Record<string, string>
|
||||
decryptionFailures?: readonly string[]
|
||||
/** `envKey` → owning user, from the environment snapshot; only personal keys appear. */
|
||||
personalOwners?: Record<string, string>
|
||||
restoredProvenance?: unknown
|
||||
restoredCheckpointVersion?: unknown
|
||||
restoreTrusted?: boolean
|
||||
@@ -537,13 +565,28 @@ function buildEffectiveCatalogEntry(
|
||||
name: string,
|
||||
encryptedValue: string
|
||||
): ResolvedSecretTraceCatalogEntry | undefined {
|
||||
const plaintext = hasOwn(options.workspaceDecrypted, name)
|
||||
const fromWorkspace = hasOwn(options.workspaceDecrypted, name)
|
||||
const plaintext = fromWorkspace
|
||||
? options.workspaceDecrypted[name]
|
||||
: options.personalDecrypted[name]
|
||||
if (plaintext === undefined || (plaintext.length === 0 && failedNames.has(name))) {
|
||||
return undefined
|
||||
}
|
||||
return { name, plaintext, encryptedValue }
|
||||
/**
|
||||
* Scope follows the value that actually won, matching the workspace-shadows-personal
|
||||
* precedence the merged environment applies. A name present in both must not be
|
||||
* attributed to the personal secret it shadowed.
|
||||
*/
|
||||
if (fromWorkspace) return { name, plaintext, encryptedValue, scope: 'workspace' }
|
||||
|
||||
const ownerUserId = options.personalOwners?.[name]
|
||||
return {
|
||||
name,
|
||||
plaintext,
|
||||
encryptedValue,
|
||||
scope: 'personal',
|
||||
...(ownerUserId ? { ownerUserId } : {}),
|
||||
}
|
||||
}
|
||||
|
||||
function* iterateEffectiveCatalogEntries(
|
||||
@@ -1461,6 +1504,36 @@ export class ResolvedSecretTraceRegistry {
|
||||
return this.buildMatches(this.activeEntries.values())
|
||||
}
|
||||
|
||||
/**
|
||||
* Names the configured secrets this run actually resolved, for the usage trail.
|
||||
*
|
||||
* Only named entries from the run's own effective catalog qualify: an anonymous entry has
|
||||
* no name to attribute, and a named entry adopted from an imported envelope has no scope
|
||||
* because the sub-run it crossed from records its own usage. Deduplicated by name, scope,
|
||||
* and owner, since one secret can be activated at many input paths.
|
||||
*
|
||||
* A personal entry with no known owner is dropped rather than recorded unattributed: the
|
||||
* trail is read per owner, so an ownerless row would surface under someone else's
|
||||
* same-named secret.
|
||||
*
|
||||
* Carries no plaintext or ciphertext — the caller persists this, and a usage trail must
|
||||
* never become a second place a secret's value lives.
|
||||
*/
|
||||
getResolvedSecretUsage(): ReadonlyArray<ResolvedSecretUsageEntry> {
|
||||
const usage = new Map<string, ResolvedSecretUsageEntry>()
|
||||
for (const entry of this.activeEntries.values()) {
|
||||
if (entry.anonymous || !entry.scope) continue
|
||||
if (entry.scope === 'personal' && !entry.ownerUserId) continue
|
||||
const ownerUserId = entry.scope === 'personal' ? (entry.ownerUserId as string) : null
|
||||
usage.set(`${entry.scope}\u0000${ownerUserId ?? ''}\u0000${entry.name}`, {
|
||||
name: entry.name,
|
||||
scope: entry.scope,
|
||||
ownerUserId,
|
||||
})
|
||||
}
|
||||
return [...usage.values()]
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns committed literals that must be removed before content can cross into a model.
|
||||
* Only entries activated by an exact resolver or trusted provenance boundary participate;
|
||||
|
||||
@@ -8,10 +8,12 @@ import {
|
||||
createCredentialDraftContract,
|
||||
createWorkspaceCredentialContract,
|
||||
deleteWorkspaceCredentialContract,
|
||||
getSecretUsageContract,
|
||||
getWorkspaceCredentialContract,
|
||||
listWorkspaceCredentialMembersContract,
|
||||
listWorkspaceCredentialsContract,
|
||||
removeWorkspaceCredentialMemberContract,
|
||||
type SecretUsageScope,
|
||||
updateWorkspaceCredentialContract,
|
||||
upsertWorkspaceCredentialMemberContract,
|
||||
type WorkspaceCredential,
|
||||
@@ -303,3 +305,33 @@ export function useRemoveWorkspaceCredentialMember() {
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* The trail is written by every run that resolves the secret, so it goes stale quickly. A
|
||||
* short window keeps "last used" meaningful without refetching on every panel interaction.
|
||||
*/
|
||||
export const SECRET_USAGE_STALE_TIME = 30 * 1000
|
||||
|
||||
interface SecretUsageParams {
|
||||
workspaceId?: string
|
||||
name?: string
|
||||
scope?: SecretUsageScope
|
||||
}
|
||||
|
||||
/** Reads one secret's usage trail. Only credential admins are authorized server-side. */
|
||||
export function useSecretUsage({ workspaceId, name, scope }: SecretUsageParams, enabled = true) {
|
||||
return useQuery({
|
||||
queryKey: workspaceCredentialKeys.usage(workspaceId, name, scope),
|
||||
queryFn: ({ signal }) =>
|
||||
requestJson(getSecretUsageContract, {
|
||||
query: {
|
||||
workspaceId: workspaceId as string,
|
||||
name: name as string,
|
||||
scope: scope as SecretUsageScope,
|
||||
},
|
||||
signal,
|
||||
}),
|
||||
enabled: Boolean(workspaceId && name && scope) && enabled,
|
||||
staleTime: SECRET_USAGE_STALE_TIME,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -21,4 +21,16 @@ export const workspaceCredentialKeys = {
|
||||
[...workspaceCredentialKeys.details(), credentialId ?? 'none'] as const,
|
||||
members: (credentialId?: string) =>
|
||||
[...workspaceCredentialKeys.detail(credentialId), 'members'] as const,
|
||||
/**
|
||||
* Keyed by name and scope rather than credential id: the usage trail is recorded against
|
||||
* the secret's name, so it survives a credential row being recreated for the same key.
|
||||
*/
|
||||
usage: (workspaceId?: string, name?: string, scope?: string) =>
|
||||
[
|
||||
...workspaceCredentialKeys.all,
|
||||
'usage',
|
||||
workspaceId ?? 'none',
|
||||
scope ?? 'all',
|
||||
name ?? '',
|
||||
] as const,
|
||||
}
|
||||
|
||||
@@ -22,6 +22,7 @@ export * from './pinned-items'
|
||||
export * from './primitives'
|
||||
export * from './sandboxes'
|
||||
export * from './secret-mount-policy'
|
||||
export * from './secrets'
|
||||
export * from './selectors'
|
||||
export * from './skills'
|
||||
export * from './storage-transfer'
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
import { z } from 'zod'
|
||||
import { defineRouteContract } from '@/lib/api/contracts/types'
|
||||
|
||||
const SECRET_USAGE_DEFAULT_LIMIT = 100
|
||||
const SECRET_USAGE_MAX_LIMIT = 500
|
||||
|
||||
export const secretUsageScopeSchema = z.enum(['workspace', 'personal'])
|
||||
|
||||
export const secretUsageQuerySchema = z.object({
|
||||
workspaceId: z.string().min(1, 'workspaceId is required'),
|
||||
name: z.string().min(1, 'Secret name is required'),
|
||||
scope: secretUsageScopeSchema,
|
||||
limit: z.coerce
|
||||
.number()
|
||||
.int()
|
||||
.min(1, 'limit must be at least 1')
|
||||
.max(SECRET_USAGE_MAX_LIMIT, `limit cannot exceed ${SECRET_USAGE_MAX_LIMIT}`)
|
||||
.default(SECRET_USAGE_DEFAULT_LIMIT),
|
||||
})
|
||||
|
||||
export const secretUsageEntrySchema = z.object({
|
||||
id: z.string(),
|
||||
useCount: z.number().int().nonnegative(),
|
||||
lastUsedAt: z.string(),
|
||||
source: z.enum(['workflow', 'copilot', 'mcp']),
|
||||
workflowName: z.string().nullable(),
|
||||
actorName: z.string().nullable(),
|
||||
lastExecutionId: z.string().nullable(),
|
||||
/** False once that run's log has aged out of the workspace's retention window. */
|
||||
lastExecutionAvailable: z.boolean(),
|
||||
lastTrigger: z.string().nullable(),
|
||||
})
|
||||
|
||||
export const getSecretUsageContract = defineRouteContract({
|
||||
method: 'GET',
|
||||
path: '/api/secrets/usage',
|
||||
query: secretUsageQuerySchema,
|
||||
response: {
|
||||
mode: 'json',
|
||||
schema: z.object({
|
||||
entries: z.array(secretUsageEntrySchema),
|
||||
}),
|
||||
},
|
||||
})
|
||||
|
||||
export type SecretUsageScope = z.output<typeof secretUsageScopeSchema>
|
||||
export type SecretUsageQuery = z.input<typeof secretUsageQuerySchema>
|
||||
export type SecretUsageEntryPayload = z.output<typeof secretUsageEntrySchema>
|
||||
@@ -22,6 +22,7 @@ export async function createCopilotEnvironmentContext(
|
||||
personalDecrypted: environment.personalDecrypted,
|
||||
workspaceDecrypted: environment.workspaceDecrypted,
|
||||
decryptionFailures: environment.decryptionFailures,
|
||||
personalOwners: environment.personalOwners,
|
||||
scope: { userId, workspaceId },
|
||||
})
|
||||
|
||||
|
||||
@@ -14,8 +14,9 @@ const { getToolEntry, isKnownTool, isSimExecuted, isClientExecuted } = vi.hoiste
|
||||
isClientExecuted: vi.fn(),
|
||||
}))
|
||||
|
||||
const { executeAppTool } = vi.hoisted(() => ({
|
||||
const { executeAppTool, recordSecretUsage } = vi.hoisted(() => ({
|
||||
executeAppTool: vi.fn(),
|
||||
recordSecretUsage: vi.fn(),
|
||||
}))
|
||||
|
||||
vi.mock('./router', () => ({
|
||||
@@ -29,6 +30,8 @@ vi.mock('@/tools', () => ({
|
||||
executeTool: executeAppTool,
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/secrets/usage/record', () => ({ recordSecretUsage }))
|
||||
|
||||
import { clearHandlers, executeTool, registerHandler } from './executor'
|
||||
|
||||
const toolExecutorLogger = vi.mocked(loggerMock.createLogger).mock.results[
|
||||
@@ -410,4 +413,95 @@ describe('copilot tool executor fallback', () => {
|
||||
}
|
||||
)
|
||||
})
|
||||
|
||||
/**
|
||||
* An integration tool resolves `{{SECRET}}` into its user-only params, which is a real use
|
||||
* of a workspace secret. This is the branch that carries it — a gateway call Go resolves to
|
||||
* `slack_send` is not in the copilot catalog, so it lands here rather than on a handler.
|
||||
*/
|
||||
it('records the secrets an integration tool call resolved', async () => {
|
||||
isKnownTool.mockReturnValue(false)
|
||||
isSimExecuted.mockReturnValue(false)
|
||||
const registry = new ResolvedSecretTraceRegistry([
|
||||
{ name: 'SLACK_TOKEN', plaintext: 'xoxb-value', encryptedValue: 'enc', scope: 'workspace' },
|
||||
])
|
||||
registry.recordResolved('SLACK_TOKEN', 'xoxb-value')
|
||||
executeAppTool.mockResolvedValue({ success: true })
|
||||
|
||||
await executeTool(
|
||||
'slack_send',
|
||||
{ token: '{{SLACK_TOKEN}}' },
|
||||
{
|
||||
userId: 'user-1',
|
||||
workflowId: '',
|
||||
workspaceId: 'ws-1',
|
||||
copilotToolExecution: true,
|
||||
resolvedSecretTraceRegistry: registry,
|
||||
}
|
||||
)
|
||||
|
||||
expect(recordSecretUsage).toHaveBeenCalledWith(
|
||||
[{ name: 'SLACK_TOKEN', scope: 'workspace', ownerUserId: null }],
|
||||
{
|
||||
workspaceId: 'ws-1',
|
||||
source: 'copilot',
|
||||
actorUserId: 'user-1',
|
||||
trigger: 'copilot',
|
||||
}
|
||||
)
|
||||
})
|
||||
|
||||
/** A failed call still resolved the secret, so the trail must not lose it. */
|
||||
it('records usage even when the integration tool throws', async () => {
|
||||
isKnownTool.mockReturnValue(false)
|
||||
isSimExecuted.mockReturnValue(false)
|
||||
const registry = new ResolvedSecretTraceRegistry([
|
||||
{ name: 'SLACK_TOKEN', plaintext: 'xoxb-value', encryptedValue: 'enc', scope: 'workspace' },
|
||||
])
|
||||
registry.recordResolved('SLACK_TOKEN', 'xoxb-value')
|
||||
executeAppTool.mockRejectedValue(new Error('provider rejected the call'))
|
||||
|
||||
await expect(
|
||||
executeTool(
|
||||
'slack_send',
|
||||
{},
|
||||
{
|
||||
userId: 'user-1',
|
||||
workflowId: '',
|
||||
workspaceId: 'ws-1',
|
||||
resolvedSecretTraceRegistry: registry,
|
||||
}
|
||||
)
|
||||
).rejects.toThrow('provider rejected the call')
|
||||
|
||||
expect(recordSecretUsage).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
/**
|
||||
* `function_execute` records its own mounted secrets in the copilot handler. If it also
|
||||
* recorded here, every Sim agent code run would count each secret twice.
|
||||
*/
|
||||
it('does not record from the handler branch, which owns its own accounting', async () => {
|
||||
isKnownTool.mockReturnValue(true)
|
||||
isSimExecuted.mockReturnValue(true)
|
||||
isClientExecuted.mockReturnValue(false)
|
||||
const registry = new ResolvedSecretTraceRegistry([
|
||||
{ name: 'API_KEY', plaintext: 'a-value', encryptedValue: 'enc', scope: 'workspace' },
|
||||
])
|
||||
registry.recordResolved('API_KEY', 'a-value')
|
||||
registerHandler('function_execute', async () => ({ success: true }))
|
||||
|
||||
await executeTool(
|
||||
'function_execute',
|
||||
{ code: 'return 1' },
|
||||
{
|
||||
userId: 'user-1',
|
||||
workflowId: '',
|
||||
workspaceId: 'ws-1',
|
||||
resolvedSecretTraceRegistry: registry,
|
||||
}
|
||||
)
|
||||
|
||||
expect(recordSecretUsage).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -3,6 +3,7 @@ import { type PermissionType, permissionSatisfies } from '@sim/platform-authz/wo
|
||||
import { toError } from '@sim/utils/errors'
|
||||
import { projectToolErrorMessageForCopilot } from '@/lib/copilot/request/tools/resolved-secret-result'
|
||||
import { DEFAULT_EXECUTION_TIMEOUT_MS } from '@/lib/execution/constants'
|
||||
import { recordSecretUsage } from '@/lib/secrets/usage/record'
|
||||
import { executeTool as executeAppTool } from '@/tools'
|
||||
import { getToolEntry, isClientExecuted, isKnownTool, isSimExecuted } from './router'
|
||||
import type { ToolExecutionContext, ToolExecutionResult, ToolHandler } from './types'
|
||||
@@ -76,9 +77,13 @@ export async function executeTool(
|
||||
}
|
||||
: {}),
|
||||
}
|
||||
return Object.keys(options).length > 0
|
||||
? executeAppTool(toolId, appParams, options)
|
||||
: executeAppTool(toolId, appParams)
|
||||
try {
|
||||
return await (Object.keys(options).length > 0
|
||||
? executeAppTool(toolId, appParams, options)
|
||||
: executeAppTool(toolId, appParams))
|
||||
} finally {
|
||||
recordAppToolSecretUsage(context)
|
||||
}
|
||||
}
|
||||
|
||||
if (context.abortSignal?.aborted) {
|
||||
@@ -135,6 +140,29 @@ function normalizeToolParams(
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Records the secrets an integration tool call resolved.
|
||||
*
|
||||
* `resolveCopilotEnvReferences` in `@/tools` substitutes `{{SECRET}}` into a tool's
|
||||
* `user-only` params — an API key reaching Slack or Stripe is as real a use as one read in
|
||||
* sandboxed code, and without this the trail reports "never used" for it. Every tool call
|
||||
* gets its own registry (`forkForInputPaths([])` returns one with no active entries), so this
|
||||
* counts only what THIS call resolved rather than everything earlier in the turn.
|
||||
*
|
||||
* Only the `executeAppTool` branch reaches here. `function_execute` takes the registered-handler
|
||||
* branch and records its own mounted secrets, so the two never count the same resolution twice.
|
||||
*/
|
||||
function recordAppToolSecretUsage(context: ToolExecutionContext): void {
|
||||
const registry = context.resolvedSecretTraceRegistry
|
||||
if (!registry || !context.workspaceId) return
|
||||
recordSecretUsage(registry.getResolvedSecretUsage(), {
|
||||
workspaceId: context.workspaceId,
|
||||
source: 'copilot',
|
||||
actorUserId: context.userId,
|
||||
trigger: 'copilot',
|
||||
})
|
||||
}
|
||||
|
||||
function buildAppToolParams(
|
||||
params: Record<string, unknown>,
|
||||
context: ToolExecutionContext
|
||||
|
||||
@@ -19,6 +19,7 @@ import {
|
||||
PRIVATE_SECRET_PROVENANCE_FIELD,
|
||||
} from '@/lib/execution/private-tool-metadata'
|
||||
import { MAX_PLAN_REQUIRED } from '@/lib/execution/remote-sandbox/workspace-sandboxes'
|
||||
import { recordSecretUsage } from '@/lib/secrets/usage/record'
|
||||
import { getColumnId } from '@/lib/table/column-keys'
|
||||
import { TABLE_LIMITS } from '@/lib/table/constants'
|
||||
import { formatCsvCell, neutralizeCsvFormula, toCsvRow } from '@/lib/table/export-format'
|
||||
@@ -659,9 +660,15 @@ export async function executeFunctionExecute(
|
||||
let mountedRegistry: ResolvedSecretTraceRegistry | undefined
|
||||
let crossingValue: unknown
|
||||
|
||||
/**
|
||||
* Hoisted so the usage trail in `finally` attributes the run to the same identity the mount
|
||||
* authorized against. Deriving it a second time down there let the two disagree whenever
|
||||
* `secretActorUserId` was explicitly null.
|
||||
*/
|
||||
const secretActorUserId =
|
||||
context.secretActorUserId === undefined ? context.userId : context.secretActorUserId
|
||||
|
||||
try {
|
||||
const secretActorUserId =
|
||||
context.secretActorUserId === undefined ? context.userId : context.secretActorUserId
|
||||
let mounted: MaterializedCopilotCodeSecrets = { envVars: {}, catalogEntries: [] }
|
||||
if (requestedNames.length > 0) {
|
||||
if (!secretActorUserId) {
|
||||
@@ -764,6 +771,21 @@ export async function executeFunctionExecute(
|
||||
crossingValue
|
||||
)
|
||||
}
|
||||
/**
|
||||
* Copilot-run code is a real read of a workspace secret and has to appear in the trail;
|
||||
* without this an admin reviewing a secret sees "never used" for one someone read through
|
||||
* Mothership. Read from the registry rather than `requestedNames` so only names the code
|
||||
* actually resolved are counted. The headless inbox runner reaches the same handler, so
|
||||
* it is covered here too.
|
||||
*/
|
||||
if (mountedRegistry && context.workspaceId) {
|
||||
recordSecretUsage(mountedRegistry.getResolvedSecretUsage(), {
|
||||
workspaceId: context.workspaceId,
|
||||
source: 'copilot',
|
||||
actorUserId: secretActorUserId ?? null,
|
||||
trigger: 'copilot',
|
||||
})
|
||||
}
|
||||
completePendingActivation?.()
|
||||
}
|
||||
}
|
||||
|
||||
@@ -107,11 +107,42 @@ describe('materializeCopilotCodeSecrets', () => {
|
||||
).resolves.toEqual({
|
||||
envVars: { API_KEY: 'plain:personal-cipher' },
|
||||
catalogEntries: [
|
||||
{ name: 'API_KEY', plaintext: 'plain:personal-cipher', encryptedValue: 'personal-cipher' },
|
||||
{
|
||||
name: 'API_KEY',
|
||||
plaintext: 'plain:personal-cipher',
|
||||
encryptedValue: 'personal-cipher',
|
||||
scope: 'personal',
|
||||
ownerUserId: 'user-1',
|
||||
},
|
||||
],
|
||||
})
|
||||
})
|
||||
|
||||
it('scopes a workspace-authorized secret to the workspace', async () => {
|
||||
mockCheckWorkspaceAccess.mockResolvedValue({
|
||||
exists: true,
|
||||
hasAccess: true,
|
||||
canWrite: true,
|
||||
canAdmin: true,
|
||||
})
|
||||
queueSources({ workspace: { API_KEY: 'workspace-cipher' } })
|
||||
|
||||
const result = await materializeCopilotCodeSecrets({
|
||||
actorUserId: 'user-1',
|
||||
workspaceId: 'workspace-1',
|
||||
requestedNames: ['API_KEY'],
|
||||
})
|
||||
|
||||
expect(result.catalogEntries).toEqual([
|
||||
{
|
||||
name: 'API_KEY',
|
||||
plaintext: 'plain:workspace-cipher',
|
||||
encryptedValue: 'workspace-cipher',
|
||||
scope: 'workspace',
|
||||
},
|
||||
])
|
||||
})
|
||||
|
||||
it('mounts an own __proto__ secret as data without mutating record prototypes', async () => {
|
||||
queueSources({ personal: Object.fromEntries([['__proto__', 'personal-cipher']]) })
|
||||
|
||||
@@ -339,6 +370,14 @@ describe('materializeCopilotCodeSecrets', () => {
|
||||
})
|
||||
|
||||
expect(result.envVars).toEqual({ SHARED_KEY: 'plain:shared-cipher' })
|
||||
/**
|
||||
* The usage trail is read per owner, so a borrowed secret has to be filed under the
|
||||
* sharer. Attributing it to the actor would surface it under the actor's own
|
||||
* same-named secret and hide it from the person who can actually rotate it.
|
||||
*/
|
||||
expect(result.catalogEntries).toEqual([
|
||||
expect.objectContaining({ name: 'SHARED_KEY', scope: 'personal', ownerUserId: 'owner-2' }),
|
||||
])
|
||||
})
|
||||
|
||||
it('uses the current encrypted value on every call so rotation is observed', async () => {
|
||||
|
||||
@@ -9,7 +9,10 @@ import {
|
||||
import { decryptSecret } from '@/lib/core/security/encryption'
|
||||
import { setRecordValue } from '@/lib/core/utils/records'
|
||||
import { checkWorkspaceAccess } from '@/lib/workspaces/permissions/utils'
|
||||
import type { ResolvedSecretTraceCatalogEntry } from '@/executor/utils/resolved-secret-trace-registry'
|
||||
import type {
|
||||
ResolvedSecretScope,
|
||||
ResolvedSecretTraceCatalogEntry,
|
||||
} from '@/executor/utils/resolved-secret-trace-registry'
|
||||
|
||||
export { MAX_SECRET_MOUNT_NAME_LENGTH, MAX_SECRET_MOUNT_NAMES }
|
||||
|
||||
@@ -31,6 +34,15 @@ interface CredentialAccessRow {
|
||||
interface AuthorizedEncryptedSecret {
|
||||
name: string
|
||||
encryptedValue: string
|
||||
/** Which environment authorized this value, so the usage trail can attribute it. */
|
||||
scope: ResolvedSecretScope
|
||||
/**
|
||||
* Whose personal environment a `personal` value came from — the actor for their own
|
||||
* secret, the sharer for one shared with them. Never the actor by default: the trail is
|
||||
* read per owner, so attributing a shared secret to its borrower would file the row under
|
||||
* a secret the borrower does not have.
|
||||
*/
|
||||
ownerUserId?: string
|
||||
}
|
||||
|
||||
export interface MaterializedCopilotCodeSecrets {
|
||||
@@ -239,7 +251,7 @@ export async function materializeCopilotCodeSecrets(params: {
|
||||
overLimit.push(name)
|
||||
continue
|
||||
}
|
||||
authorizedSources.push({ name, encryptedValue: workspaceValue })
|
||||
authorizedSources.push({ name, encryptedValue: workspaceValue, scope: 'workspace' })
|
||||
continue
|
||||
}
|
||||
|
||||
@@ -249,7 +261,12 @@ export async function materializeCopilotCodeSecrets(params: {
|
||||
continue
|
||||
}
|
||||
if (ownPersonalValue !== undefined) {
|
||||
authorizedSources.push({ name, encryptedValue: ownPersonalValue })
|
||||
authorizedSources.push({
|
||||
name,
|
||||
encryptedValue: ownPersonalValue,
|
||||
scope: 'personal',
|
||||
ownerUserId: params.actorUserId,
|
||||
})
|
||||
continue
|
||||
}
|
||||
|
||||
@@ -267,7 +284,13 @@ export async function materializeCopilotCodeSecrets(params: {
|
||||
}
|
||||
const sharedPersonalValue = sharedPersonal?.encryptedValue ?? undefined
|
||||
if (sharedPersonalValue !== undefined) {
|
||||
authorizedSources.push({ name, encryptedValue: sharedPersonalValue })
|
||||
authorizedSources.push({
|
||||
name,
|
||||
encryptedValue: sharedPersonalValue,
|
||||
scope: 'personal',
|
||||
/** Non-null by the `authorizedSharedPersonalRows` filter above. */
|
||||
ownerUserId: sharedPersonal?.envOwnerUserId as string,
|
||||
})
|
||||
continue
|
||||
}
|
||||
|
||||
@@ -279,12 +302,18 @@ export async function materializeCopilotCodeSecrets(params: {
|
||||
}
|
||||
if (unavailable.length > 0) throw unavailableError(unavailable)
|
||||
|
||||
let decryptedEntries: Array<{ name: string; plaintext: string; encryptedValue: string }>
|
||||
let decryptedEntries: ResolvedSecretTraceCatalogEntry[]
|
||||
try {
|
||||
decryptedEntries = await Promise.all(
|
||||
authorizedSources.map(async ({ name, encryptedValue }) => {
|
||||
authorizedSources.map(async ({ name, encryptedValue, scope, ownerUserId }) => {
|
||||
const { decrypted } = await decryptSecret(encryptedValue)
|
||||
return { name, plaintext: decrypted, encryptedValue }
|
||||
return {
|
||||
name,
|
||||
plaintext: decrypted,
|
||||
encryptedValue,
|
||||
scope,
|
||||
...(ownerUserId ? { ownerUserId } : {}),
|
||||
}
|
||||
})
|
||||
)
|
||||
} catch {
|
||||
|
||||
@@ -1078,3 +1078,471 @@ describe('code placeholder compiler', () => {
|
||||
).resolves.toEqual(['DELIMITER'])
|
||||
})
|
||||
})
|
||||
|
||||
describe('direct environment reads', () => {
|
||||
it('reports a JavaScript read that never used a placeholder', async () => {
|
||||
const compiled = await compileCodePlaceholders({
|
||||
code: [
|
||||
'const a = environmentVariables.API_KEY',
|
||||
"const b = environmentVariables['OTHER_KEY']",
|
||||
'return { a, b }',
|
||||
].join('\n'),
|
||||
language: CodeLanguage.JavaScript,
|
||||
environmentVariables: { API_KEY: 'a-value', OTHER_KEY: 'b-value', UNUSED: 'c-value' },
|
||||
})
|
||||
|
||||
expect(compiled.resolvedSecretNames).toEqual(['API_KEY', 'OTHER_KEY'])
|
||||
})
|
||||
|
||||
it('merges direct reads with placeholder resolutions in source order', async () => {
|
||||
const compiled = await compileCodePlaceholders({
|
||||
code: [
|
||||
'const a = environmentVariables.FIRST',
|
||||
'const b = "{{SECOND}}"',
|
||||
'return { a, b }',
|
||||
].join('\n'),
|
||||
language: CodeLanguage.JavaScript,
|
||||
environmentVariables: { FIRST: 'one', SECOND: 'two' },
|
||||
})
|
||||
|
||||
expect(compiled.resolvedSecretNames).toEqual(['FIRST', 'SECOND'])
|
||||
})
|
||||
|
||||
it('ignores an identifier that is not a configured secret', async () => {
|
||||
const compiled = await compileCodePlaceholders({
|
||||
code: 'return environmentVariables.NOT_A_SECRET',
|
||||
language: CodeLanguage.JavaScript,
|
||||
environmentVariables: { API_KEY: 'a-value' },
|
||||
})
|
||||
|
||||
expect(compiled.resolvedSecretNames).toEqual([])
|
||||
})
|
||||
|
||||
/** A computed key is the boundary: statically unresolvable, so deliberately unreported. */
|
||||
it('does not guess a computed subscript', async () => {
|
||||
const compiled = await compileCodePlaceholders({
|
||||
code: ['const name = "API_KEY"', 'return environmentVariables[name]'].join('\n'),
|
||||
language: CodeLanguage.JavaScript,
|
||||
environmentVariables: { API_KEY: 'a-value' },
|
||||
})
|
||||
|
||||
expect(compiled.resolvedSecretNames).toEqual([])
|
||||
})
|
||||
|
||||
it('does not read off an unrelated object with a matching property', async () => {
|
||||
const compiled = await compileCodePlaceholders({
|
||||
code: ['const other = { API_KEY: 1 }', 'return other.API_KEY'].join('\n'),
|
||||
language: CodeLanguage.JavaScript,
|
||||
environmentVariables: { API_KEY: 'a-value' },
|
||||
})
|
||||
|
||||
expect(compiled.resolvedSecretNames).toEqual([])
|
||||
})
|
||||
|
||||
/**
|
||||
* Copilot mounts a secret only for an explicit placeholder. Analysis must not widen that,
|
||||
* or an identifier in a string would pull a value into agent-authored code.
|
||||
*/
|
||||
it('never reports a direct read to the Copilot mount analyzer', async () => {
|
||||
const names = await analyzeCodePlaceholders(
|
||||
'return environmentVariables.API_KEY',
|
||||
CodeLanguage.JavaScript
|
||||
)
|
||||
|
||||
expect(names).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('direct environment reads in Python', () => {
|
||||
it('reports subscript and get() reads', async () => {
|
||||
const compiled = await compileCodePlaceholders({
|
||||
code: [
|
||||
"a = environmentVariables['API_KEY']",
|
||||
"b = environmentVariables.get('OTHER_KEY')",
|
||||
'return {"a": a, "b": b}',
|
||||
].join('\n'),
|
||||
language: CodeLanguage.Python,
|
||||
environmentVariables: { API_KEY: 'a-value', OTHER_KEY: 'b-value', UNUSED: 'c-value' },
|
||||
})
|
||||
|
||||
expect(compiled.resolvedSecretNames).toEqual(['API_KEY', 'OTHER_KEY'])
|
||||
})
|
||||
|
||||
it('ignores a read written inside a string or comment', async () => {
|
||||
const compiled = await compileCodePlaceholders({
|
||||
code: [
|
||||
'doc = "environmentVariables[\'API_KEY\']"',
|
||||
"# environmentVariables['API_KEY']",
|
||||
'return doc',
|
||||
].join('\n'),
|
||||
language: CodeLanguage.Python,
|
||||
environmentVariables: { API_KEY: 'a-value' },
|
||||
})
|
||||
|
||||
expect(compiled.resolvedSecretNames).toEqual([])
|
||||
})
|
||||
|
||||
it('does not match a longer identifier that merely ends in the binding name', async () => {
|
||||
const compiled = await compileCodePlaceholders({
|
||||
code: "return myenvironmentVariables['API_KEY']",
|
||||
language: CodeLanguage.Python,
|
||||
environmentVariables: { API_KEY: 'a-value' },
|
||||
})
|
||||
|
||||
expect(compiled.resolvedSecretNames).toEqual([])
|
||||
})
|
||||
|
||||
it('does not guess a computed subscript', async () => {
|
||||
const compiled = await compileCodePlaceholders({
|
||||
code: ['name = "API_KEY"', 'return environmentVariables[name]'].join('\n'),
|
||||
language: CodeLanguage.Python,
|
||||
environmentVariables: { API_KEY: 'a-value' },
|
||||
})
|
||||
|
||||
expect(compiled.resolvedSecretNames).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('direct environment reads in shell', () => {
|
||||
it('reports bare and braced parameter expansions', async () => {
|
||||
const compiled = await compileCodePlaceholders({
|
||||
// biome-ignore lint/suspicious/noTemplateCurlyInString: shell parameter expansion, not a JS template
|
||||
code: ['echo "$API_KEY"', 'curl -H "Authorization: ${OTHER_KEY}"'].join('\n'),
|
||||
language: CodeLanguage.Shell,
|
||||
environmentVariables: { API_KEY: 'a-value', OTHER_KEY: 'b-value', UNUSED: 'c-value' },
|
||||
})
|
||||
|
||||
expect(compiled.resolvedSecretNames).toEqual(['API_KEY', 'OTHER_KEY'])
|
||||
})
|
||||
|
||||
it('keeps the name when a default is supplied', async () => {
|
||||
const compiled = await compileCodePlaceholders({
|
||||
// biome-ignore lint/suspicious/noTemplateCurlyInString: shell parameter expansion, not a JS template
|
||||
code: 'echo "${API_KEY:-fallback}"',
|
||||
language: CodeLanguage.Shell,
|
||||
environmentVariables: { API_KEY: 'a-value' },
|
||||
})
|
||||
|
||||
expect(compiled.resolvedSecretNames).toEqual(['API_KEY'])
|
||||
})
|
||||
|
||||
/** Single quotes suppress expansion, so the secret is never read. */
|
||||
it('ignores a single-quoted expansion', async () => {
|
||||
const compiled = await compileCodePlaceholders({
|
||||
code: "echo '$API_KEY'",
|
||||
language: CodeLanguage.Shell,
|
||||
environmentVariables: { API_KEY: 'a-value' },
|
||||
})
|
||||
|
||||
expect(compiled.resolvedSecretNames).toEqual([])
|
||||
})
|
||||
|
||||
it('ignores an escaped expansion and positional parameters', async () => {
|
||||
const compiled = await compileCodePlaceholders({
|
||||
code: ['echo "\\$API_KEY"', 'echo "$1 $@ $$"'].join('\n'),
|
||||
language: CodeLanguage.Shell,
|
||||
environmentVariables: { API_KEY: 'a-value' },
|
||||
})
|
||||
|
||||
expect(compiled.resolvedSecretNames).toEqual([])
|
||||
})
|
||||
|
||||
/** A quoted delimiter makes the body literal, so nothing in it expands. */
|
||||
it('ignores an expansion inside a quoted heredoc but reports an unquoted one', async () => {
|
||||
const quoted = await compileCodePlaceholders({
|
||||
code: ["cat <<'EOF'", '$API_KEY', 'EOF'].join('\n'),
|
||||
language: CodeLanguage.Shell,
|
||||
environmentVariables: { API_KEY: 'a-value' },
|
||||
})
|
||||
const unquoted = await compileCodePlaceholders({
|
||||
code: ['cat <<EOF', '$API_KEY', 'EOF'].join('\n'),
|
||||
language: CodeLanguage.Shell,
|
||||
environmentVariables: { API_KEY: 'a-value' },
|
||||
})
|
||||
|
||||
expect(quoted.resolvedSecretNames).toEqual([])
|
||||
expect(unquoted.resolvedSecretNames).toEqual(['API_KEY'])
|
||||
})
|
||||
|
||||
it('ignores a shell variable that is not a configured secret', async () => {
|
||||
const compiled = await compileCodePlaceholders({
|
||||
code: 'echo "$PATH $HOME"',
|
||||
language: CodeLanguage.Shell,
|
||||
environmentVariables: { API_KEY: 'a-value' },
|
||||
})
|
||||
|
||||
expect(compiled.resolvedSecretNames).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
const directReadEnv = { API_KEY: 'a-value' }
|
||||
const directReadNames = async (code: string, language: CodeLanguage) =>
|
||||
(await compileCodePlaceholders({ code, language, environmentVariables: directReadEnv }))
|
||||
.resolvedSecretNames
|
||||
|
||||
describe('direct environment read edge cases', () => {
|
||||
it('ignores a Python attribute on an unrelated object with the same name', async () => {
|
||||
expect(
|
||||
await directReadNames("return other.environmentVariables['API_KEY']", CodeLanguage.Python)
|
||||
).toEqual([])
|
||||
})
|
||||
|
||||
it('reads a Python subscript split across lines', async () => {
|
||||
expect(
|
||||
await directReadNames("x = environmentVariables[\n 'API_KEY'\n]", CodeLanguage.Python)
|
||||
).toEqual(['API_KEY'])
|
||||
})
|
||||
|
||||
/** The whole f-string is one token, so the read is missed — a miss, never a false claim. */
|
||||
it('does not report a Python f-string read', async () => {
|
||||
expect(
|
||||
await directReadNames('x = f"{environmentVariables[\'API_KEY\']}"', CodeLanguage.Python)
|
||||
).toEqual([])
|
||||
})
|
||||
|
||||
it('ignores a commented-out shell expansion', async () => {
|
||||
expect(await directReadNames('# echo "$API_KEY"', CodeLanguage.Shell)).toEqual([])
|
||||
})
|
||||
|
||||
it('does not prefix-match a longer shell name', async () => {
|
||||
expect(await directReadNames('echo "$API_KEYS"', CodeLanguage.Shell)).toEqual([])
|
||||
})
|
||||
|
||||
it('does not guess a shell indirect expansion', async () => {
|
||||
// biome-ignore lint/suspicious/noTemplateCurlyInString: shell parameter expansion, not a JS template
|
||||
expect(await directReadNames('echo "${!API_KEY}"', CodeLanguage.Shell)).toEqual([])
|
||||
})
|
||||
|
||||
it('ignores a double-quoted expansion nested in single quotes', async () => {
|
||||
expect(await directReadNames(`echo '"$API_KEY"'`, CodeLanguage.Shell)).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('touching a configured name reports it, whatever the code does with it', () => {
|
||||
/**
|
||||
* `environmentVariables` is a plain object deserialized from the run payload, not a handle on
|
||||
* the stored secret: writing to it changes nothing outside the sandbox and is discarded when
|
||||
* the run ends. Telling a write apart from a read therefore buys almost nothing, so the
|
||||
* detector does not try — the same rule every language here follows.
|
||||
*/
|
||||
it.each([
|
||||
['property assignment', "environmentVariables.API_KEY = 'x'"],
|
||||
['subscript assignment', "environmentVariables['API_KEY'] = 'x'"],
|
||||
['compound assignment', "environmentVariables.API_KEY += 'x'"],
|
||||
['delete', 'delete environmentVariables.API_KEY'],
|
||||
])('javascript: %s', async (_label, code) => {
|
||||
expect(await directReadNames(code, CodeLanguage.JavaScript)).toEqual(['API_KEY'])
|
||||
})
|
||||
|
||||
it.each([
|
||||
['subscript assignment', "environmentVariables['API_KEY'] = 'x'"],
|
||||
['del', "del environmentVariables['API_KEY']"],
|
||||
['nested read inside a del', "del environmentVariables[environmentVariables['API_KEY']]"],
|
||||
])('python: %s', async (_label, code) => {
|
||||
expect(await directReadNames(code, CodeLanguage.Python)).toEqual(['API_KEY'])
|
||||
})
|
||||
})
|
||||
|
||||
describe('a shadowing local does not suppress reads', () => {
|
||||
/**
|
||||
* A read off a shadowing local is reported rather than discarded. Dropping it was file-wide,
|
||||
* so a helper with its own `environmentVariables` silently removed genuine reads elsewhere in
|
||||
* the source — and a dropped read never reaches the output matcher, leaving a real secret
|
||||
* unmasked. Reporting one costs only an exact value the code never emits.
|
||||
*/
|
||||
it('reports a read that a helper-scoped binding would previously have discarded', async () => {
|
||||
const code = [
|
||||
'function helper(environmentVariables) { return environmentVariables.API_KEY }',
|
||||
'return helper({}) + environmentVariables.API_KEY',
|
||||
].join('\n')
|
||||
expect(await directReadNames(code, CodeLanguage.JavaScript)).toEqual(['API_KEY'])
|
||||
})
|
||||
|
||||
it.each([
|
||||
[
|
||||
'const declaration',
|
||||
"const environmentVariables = { API_KEY: 'x' }\nreturn environmentVariables.API_KEY",
|
||||
],
|
||||
['bare for-of target', 'for (environmentVariables of rows) log(environmentVariables.API_KEY)'],
|
||||
[
|
||||
'bare reassignment',
|
||||
"environmentVariables = { API_KEY: 'x' }\nreturn environmentVariables.API_KEY",
|
||||
],
|
||||
])('javascript reports despite a shadow: %s', async (_label, code) => {
|
||||
expect(await directReadNames(code, CodeLanguage.JavaScript)).toEqual(['API_KEY'])
|
||||
})
|
||||
|
||||
it.each([
|
||||
['assignment', "environmentVariables = {'API_KEY': 'x'}\nk = environmentVariables['API_KEY']"],
|
||||
[
|
||||
'def parameter',
|
||||
"def read(environmentVariables):\n return environmentVariables['API_KEY']",
|
||||
],
|
||||
])('python reports despite a shadow: %s', async (_label, code) => {
|
||||
expect(await directReadNames(code, CodeLanguage.Python)).toEqual(['API_KEY'])
|
||||
})
|
||||
})
|
||||
|
||||
describe('a dot in prose is not a qualifier', () => {
|
||||
/**
|
||||
* The receiver walk crosses whitespace so a parenthesized `other.` on a previous line is
|
||||
* seen — but a comment or string can also end in a period, and landing on THAT dot must not
|
||||
* discard the read below it. The lexer decides which dots are code.
|
||||
*/
|
||||
/** The read sits at line start, so the walk reaches the previous line's final character. */
|
||||
it.each([
|
||||
['comment ending in a period', "# Load the value.\nenvironmentVariables['API_KEY']"],
|
||||
['docstring ending in a period', '"""Reads the key."""\nenvironmentVariables[\'API_KEY\']'],
|
||||
['string ending in a period', "s = 'done.'\nenvironmentVariables['API_KEY']"],
|
||||
])('%s', async (_label, code) => {
|
||||
expect(await directReadNames(code, CodeLanguage.Python)).toEqual(['API_KEY'])
|
||||
})
|
||||
|
||||
it('still discards a real cross-line attribute access', async () => {
|
||||
expect(
|
||||
await directReadNames(
|
||||
"k = (other.\n environmentVariables['API_KEY'])",
|
||||
CodeLanguage.Python
|
||||
)
|
||||
).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('literal computed pattern keys resolve like subscripts', () => {
|
||||
it.each([
|
||||
['string literal', "const { ['API_KEY']: key } = environmentVariables\nreturn key"],
|
||||
['template literal', 'const { [`API_KEY`]: key } = environmentVariables\nreturn key'],
|
||||
['parenthesized literal', "const { [('API_KEY')]: key } = environmentVariables\nreturn key"],
|
||||
])('%s', async (_label, code) => {
|
||||
expect(await directReadNames(code, CodeLanguage.JavaScript)).toEqual(['API_KEY'])
|
||||
})
|
||||
|
||||
/** A non-literal computed key stays the runtime-name boundary a computed subscript has. */
|
||||
it('does not attribute an identifier computed key', async () => {
|
||||
expect(
|
||||
await directReadNames(
|
||||
'const { [k]: v } = environmentVariables\nreturn v',
|
||||
CodeLanguage.JavaScript
|
||||
)
|
||||
).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('destructured environment reads are reads', () => {
|
||||
/**
|
||||
* `const { API_KEY } = environmentVariables` delivers the value by name with no property-
|
||||
* or element-access node in the AST, so the member-access walk alone missed it — and a
|
||||
* missed read leaves an emitted value unmasked.
|
||||
*/
|
||||
it.each([
|
||||
['shorthand', 'const { API_KEY } = environmentVariables\nreturn API_KEY'],
|
||||
['renamed', 'const { API_KEY: key } = environmentVariables\nreturn key'],
|
||||
['with a default', "const { API_KEY = 'x' } = environmentVariables\nreturn API_KEY"],
|
||||
['string-literal key', "const { 'API_KEY': key } = environmentVariables\nreturn key"],
|
||||
['assignment form', 'let key\n;({ API_KEY: key } = environmentVariables)\nreturn key'],
|
||||
])('%s', async (_label, code) => {
|
||||
expect(await directReadNames(code, CodeLanguage.JavaScript)).toEqual(['API_KEY'])
|
||||
})
|
||||
|
||||
it.each([
|
||||
[
|
||||
'parameter default',
|
||||
'function f({ API_KEY } = environmentVariables) { return API_KEY }\nreturn f()',
|
||||
],
|
||||
[
|
||||
'arrow parameter default',
|
||||
'const f = ({ API_KEY } = environmentVariables) => API_KEY\nreturn f()',
|
||||
],
|
||||
[
|
||||
'binding-element default',
|
||||
'const { config: { API_KEY } = environmentVariables } = payload\nreturn API_KEY',
|
||||
],
|
||||
['parenthesized initializer', 'const { API_KEY } = (environmentVariables)\nreturn API_KEY'],
|
||||
[
|
||||
'double-parenthesized initializer',
|
||||
'const { API_KEY } = ((environmentVariables))\nreturn API_KEY',
|
||||
],
|
||||
['parenthesized member access', 'return (environmentVariables).API_KEY'],
|
||||
['parenthesized subscript', "return (environmentVariables)['API_KEY']"],
|
||||
])('receiver rule covers: %s', async (_label, code) => {
|
||||
expect(await directReadNames(code, CodeLanguage.JavaScript)).toEqual(['API_KEY'])
|
||||
})
|
||||
|
||||
/**
|
||||
* The environment wrapped in a container and read back out is data flow, not a receiver —
|
||||
* the same documented boundary as an alias (`const e = environmentVariables`) or a computed
|
||||
* key. Attribution stops where the receiver stops being demonstrably the environment
|
||||
* object; following containers has no fixed point.
|
||||
*/
|
||||
it('does not follow the environment through an array literal', async () => {
|
||||
expect(
|
||||
await directReadNames(
|
||||
'for (const { API_KEY } of [environmentVariables]) log(API_KEY)',
|
||||
CodeLanguage.JavaScript
|
||||
)
|
||||
).toEqual([])
|
||||
})
|
||||
|
||||
it('reports every configured name for a rest grab', async () => {
|
||||
const compiled = await compileCodePlaceholders({
|
||||
code: 'const { ...all } = environmentVariables\nreturn all',
|
||||
language: CodeLanguage.JavaScript,
|
||||
environmentVariables: { API_KEY: 'a-value-123456', OTHER_KEY: 'b-value-123456' },
|
||||
})
|
||||
expect(compiled.resolvedSecretNames).toEqual(['API_KEY', 'OTHER_KEY'])
|
||||
})
|
||||
|
||||
it.each([
|
||||
['different receiver', 'const { API_KEY } = other\nreturn API_KEY'],
|
||||
['computed key', 'const { [k]: v } = environmentVariables\nreturn v'],
|
||||
['unconfigured name', 'const { NOT_CONFIGURED } = environmentVariables\nreturn NOT_CONFIGURED'],
|
||||
])('does not attribute: %s', async (_label, code) => {
|
||||
expect(await directReadNames(code, CodeLanguage.JavaScript)).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('shell backslash escaping is parity, not presence', () => {
|
||||
/**
|
||||
* `\\$KEY` is an escaped backslash followed by a LIVE expansion — bash prints `\` plus the
|
||||
* value — while `\$KEY` is an escaped dollar and stays literal. Checking only the adjacent
|
||||
* character read the even case as escaped and dropped a real read from usage and masking.
|
||||
*/
|
||||
it.each([
|
||||
['no backslash', 'echo "$API_KEY"', ['API_KEY']],
|
||||
['one (escaped dollar)', 'echo "\\$API_KEY"', []],
|
||||
['two (escaped backslash, live expansion)', 'echo "\\\\$API_KEY"', ['API_KEY']],
|
||||
['three (escaped both)', 'echo "\\\\\\$API_KEY"', []],
|
||||
['four (two literal backslashes, live expansion)', 'echo "\\\\\\\\$API_KEY"', ['API_KEY']],
|
||||
['unquoted even run', 'echo \\\\$API_KEY', ['API_KEY']],
|
||||
])('%s', async (_label, code, expected) => {
|
||||
expect(await directReadNames(code, CodeLanguage.Shell)).toEqual(expected)
|
||||
})
|
||||
})
|
||||
|
||||
describe('shell true positives survive the fail-closed rule', () => {
|
||||
it.each([
|
||||
['bare unquoted', 'echo $API_KEY'],
|
||||
['assignment', 'export FOO=$API_KEY'],
|
||||
['inside double quotes', 'curl -H "Authorization: Bearer $API_KEY"'],
|
||||
['command substitution', 'X=$(echo $API_KEY)'],
|
||||
['backticks', 'X=`echo $API_KEY`'],
|
||||
// biome-ignore lint/suspicious/noTemplateCurlyInString: shell parameter expansion, not a JS template
|
||||
['braced', 'echo ${API_KEY}'],
|
||||
['second line', 'set -e\necho $API_KEY'],
|
||||
['trailing comment on another line', 'echo $API_KEY # note'],
|
||||
])('%s', async (_label, code) => {
|
||||
expect(await directReadNames(code, CodeLanguage.Shell)).toEqual(['API_KEY'])
|
||||
})
|
||||
})
|
||||
|
||||
describe('python true positives survive the dot guard', () => {
|
||||
it.each([
|
||||
['subscript', "x = environmentVariables['API_KEY']"],
|
||||
['get', "x = environmentVariables.get('API_KEY')"],
|
||||
['in a call', "print(environmentVariables['API_KEY'])"],
|
||||
['after open paren', "x = str(environmentVariables['API_KEY'])"],
|
||||
['double quotes', 'x = environmentVariables["API_KEY"]'],
|
||||
])('%s', async (_label, code) => {
|
||||
expect(await directReadNames(code, CodeLanguage.Python)).toEqual(['API_KEY'])
|
||||
})
|
||||
})
|
||||
|
||||
@@ -23,6 +23,132 @@ const SENTINEL_ALPHABET = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQ
|
||||
interface DecodedJavaScriptSyntax {
|
||||
identifierNames: string[]
|
||||
values: string[]
|
||||
environmentReads: DirectEnvironmentRead[]
|
||||
/** Offset of a `...rest` grab off the environment object, which takes every value at once. */
|
||||
environmentRestReadOffset?: number
|
||||
}
|
||||
|
||||
export interface DirectEnvironmentRead {
|
||||
name: string
|
||||
offset: number
|
||||
}
|
||||
|
||||
/** The runtime identifier the sandbox prologue binds the environment to. */
|
||||
const ENVIRONMENT_VARIABLES_IDENTIFIER = 'environmentVariables'
|
||||
|
||||
/**
|
||||
* Names a statically visible read off the runtime environment object, covering
|
||||
* `environmentVariables.NAME`, `environmentVariables['NAME']`, and their optional-chained
|
||||
* forms. A computed subscript is deliberately not resolved — see
|
||||
* {@link CodePlaceholderCompilationContext.recordDirectEnvironmentRead}.
|
||||
*/
|
||||
/** Parentheses group; they never change which object an expression evaluates to. */
|
||||
function unwrapParentheses(node: ts.Expression): ts.Expression {
|
||||
let current = node
|
||||
while (ts.isParenthesizedExpression(current)) current = current.expression
|
||||
return current
|
||||
}
|
||||
|
||||
/** Whether this expression is, after grouping, the bare runtime environment identifier. */
|
||||
function isEnvironmentReceiver(node: ts.Expression): boolean {
|
||||
const unwrapped = unwrapParentheses(node)
|
||||
return ts.isIdentifier(unwrapped) && unwrapped.text === ENVIRONMENT_VARIABLES_IDENTIFIER
|
||||
}
|
||||
|
||||
function directEnvironmentRead(node: ts.Node): DirectEnvironmentRead | undefined {
|
||||
if (ts.isPropertyAccessExpression(node)) {
|
||||
if (!isEnvironmentReceiver(node.expression)) return undefined
|
||||
return node.name.text ? { name: node.name.text, offset: node.getStart() } : undefined
|
||||
}
|
||||
if (ts.isElementAccessExpression(node)) {
|
||||
if (!isEnvironmentReceiver(node.expression)) return undefined
|
||||
const argument = node.argumentExpression
|
||||
if (!ts.isStringLiteralLike(argument) || !argument.text) return undefined
|
||||
return { name: argument.text, offset: node.getStart() }
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
interface DestructuredEnvironmentReads {
|
||||
reads: DirectEnvironmentRead[]
|
||||
restOffset?: number
|
||||
}
|
||||
|
||||
/**
|
||||
* Names read off the environment object through destructuring, which the member-access walk
|
||||
* cannot see: `const { API_KEY } = environmentVariables` contains no property- or
|
||||
* element-access node, yet delivers the value by name exactly like a subscript.
|
||||
*
|
||||
* Covers the declaration form (renames, defaults, string-literal keys) and the assignment
|
||||
* form `({ API_KEY } = environmentVariables)`. A `...rest` element is returned separately:
|
||||
* it names no key but takes every value, so the caller reports every configured name — the
|
||||
* alternative leaves `const { ...all } = environmentVariables; return all` entirely unmasked.
|
||||
* A computed key (`{ [k]: v }`) stays unrecognized, the same runtime-name boundary as a
|
||||
* computed subscript, and an initializer that is not the bare identifier (`other.env…`,
|
||||
* `environmentVariables ?? {}`) is not attributed.
|
||||
*/
|
||||
function destructuredEnvironmentReads(node: ts.Node): DestructuredEnvironmentReads | undefined {
|
||||
let pattern: ts.ObjectBindingPattern | ts.ObjectLiteralExpression | undefined
|
||||
if (ts.isObjectBindingPattern(node)) {
|
||||
/**
|
||||
* One receiver rule wherever the pattern sits: a variable declaration, a parameter
|
||||
* default (`function f({ KEY } = environmentVariables)`), or a binding element's own
|
||||
* default all hang the initializer off the pattern's parent, so checking the parent's
|
||||
* initializer covers every declaration position without per-kind cases.
|
||||
*/
|
||||
const parent = node.parent
|
||||
const initializer =
|
||||
ts.isVariableDeclaration(parent) || ts.isParameter(parent) || ts.isBindingElement(parent)
|
||||
? parent.initializer
|
||||
: undefined
|
||||
if (initializer === undefined || !isEnvironmentReceiver(initializer)) return undefined
|
||||
pattern = node
|
||||
} else if (
|
||||
ts.isBinaryExpression(node) &&
|
||||
node.operatorToken.kind === ts.SyntaxKind.EqualsToken &&
|
||||
isEnvironmentReceiver(node.right) &&
|
||||
ts.isObjectLiteralExpression(node.left)
|
||||
) {
|
||||
pattern = node.left
|
||||
}
|
||||
if (!pattern) return undefined
|
||||
|
||||
const result: DestructuredEnvironmentReads = { reads: [] }
|
||||
const record = (name: ts.PropertyName | ts.Identifier, offset: number): void => {
|
||||
/**
|
||||
* A computed key holding a string literal — `{ ['API_KEY']: key }` — is the element-access
|
||||
* rule in pattern position, so it resolves like a literal subscript; a computed key
|
||||
* holding anything else stays the runtime-name boundary a computed subscript already has.
|
||||
*/
|
||||
if (ts.isComputedPropertyName(name)) {
|
||||
const key = unwrapParentheses(name.expression)
|
||||
if (ts.isStringLiteralLike(key) && key.text) result.reads.push({ name: key.text, offset })
|
||||
return
|
||||
}
|
||||
if (ts.isIdentifier(name) || ts.isStringLiteralLike(name)) {
|
||||
if (name.text) result.reads.push({ name: name.text, offset })
|
||||
}
|
||||
}
|
||||
if (ts.isObjectBindingPattern(pattern)) {
|
||||
for (const element of pattern.elements) {
|
||||
if (element.dotDotDotToken) {
|
||||
result.restOffset = element.getStart()
|
||||
continue
|
||||
}
|
||||
record(element.propertyName ?? (element.name as ts.Identifier), element.getStart())
|
||||
}
|
||||
} else {
|
||||
for (const property of pattern.properties) {
|
||||
if (ts.isSpreadAssignment(property)) {
|
||||
result.restOffset = property.getStart()
|
||||
} else if (ts.isShorthandPropertyAssignment(property)) {
|
||||
record(property.name, property.getStart())
|
||||
} else if (ts.isPropertyAssignment(property)) {
|
||||
record(property.name, property.getStart())
|
||||
}
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
interface AnnexBHtmlCommentRange {
|
||||
@@ -41,6 +167,8 @@ function collectDecodedSyntax(code: string): DecodedJavaScriptSyntax {
|
||||
)
|
||||
const identifierNames: string[] = []
|
||||
const values: string[] = []
|
||||
const environmentReads: DirectEnvironmentRead[] = []
|
||||
let environmentRestReadOffset: number | undefined
|
||||
const visit = (node: ts.Node): void => {
|
||||
const isTemplateToken =
|
||||
node.kind === ts.SyntaxKind.TemplateHead ||
|
||||
@@ -53,10 +181,40 @@ function collectDecodedSyntax(code: string): DecodedJavaScriptSyntax {
|
||||
const rawText: unknown = Reflect.get(node, 'rawText')
|
||||
if (typeof rawText === 'string' && rawText) values.push(rawText)
|
||||
}
|
||||
/** Kind-checked inline: this visitor runs for every node in the file, and a call per
|
||||
* node to re-test the same two kinds is measurable on a large source. */
|
||||
if (
|
||||
node.kind === ts.SyntaxKind.PropertyAccessExpression ||
|
||||
node.kind === ts.SyntaxKind.ElementAccessExpression
|
||||
) {
|
||||
const environmentRead = directEnvironmentRead(node)
|
||||
if (environmentRead) environmentReads.push(environmentRead)
|
||||
} else if (
|
||||
node.kind === ts.SyntaxKind.ObjectBindingPattern ||
|
||||
node.kind === ts.SyntaxKind.BinaryExpression
|
||||
) {
|
||||
const destructured = destructuredEnvironmentReads(node)
|
||||
if (destructured) {
|
||||
environmentReads.push(...destructured.reads)
|
||||
if (destructured.restOffset !== undefined) {
|
||||
environmentRestReadOffset ??= destructured.restOffset
|
||||
}
|
||||
}
|
||||
}
|
||||
ts.forEachChild(node, visit)
|
||||
}
|
||||
visit(sourceFile)
|
||||
return { identifierNames, values }
|
||||
/**
|
||||
* A local binding that shadows the runtime environment name is deliberately NOT used to
|
||||
* discard these reads.
|
||||
*
|
||||
* Doing so was file-wide, so a helper declaring its own `environmentVariables` silently
|
||||
* dropped genuine reads of the mounted binding everywhere else in the source, and a dropped
|
||||
* read never reaches the output matcher — leaving a real secret unmasked. Reporting a read
|
||||
* off a shadowing local costs far less: the matcher is given that secret's exact value, the
|
||||
* code never emits it, and nothing matches.
|
||||
*/
|
||||
return { identifierNames, values, environmentReads, environmentRestReadOffset }
|
||||
}
|
||||
|
||||
function collectForbiddenSentinels(
|
||||
@@ -503,6 +661,20 @@ export async function compileJavaScriptPlaceholders(
|
||||
...input,
|
||||
reservedNames: [...(input.reservedNames ?? []), ...decodedSyntax.identifierNames],
|
||||
})
|
||||
/**
|
||||
* Recorded before the no-placeholder early return below: code that only reads the
|
||||
* environment directly has no `{{NAME}}` occurrence at all, and that is exactly the case
|
||||
* this exists to cover.
|
||||
*/
|
||||
for (const read of decodedSyntax.environmentReads) {
|
||||
context.recordDirectEnvironmentRead(read.name, read.offset)
|
||||
}
|
||||
if (decodedSyntax.environmentRestReadOffset !== undefined) {
|
||||
/** `...rest` delivers every configured value at once, so every configured name is a read. */
|
||||
for (const name of Object.keys(input.environmentVariables ?? {})) {
|
||||
context.recordDirectEnvironmentRead(name, decodedSyntax.environmentRestReadOffset)
|
||||
}
|
||||
}
|
||||
if (context.occurrences.length === 0) {
|
||||
const sourceFile = ts.createSourceFile(
|
||||
'user-code.js',
|
||||
|
||||
@@ -7,6 +7,7 @@ import {
|
||||
type SourceEdit,
|
||||
} from '@/lib/execution/code-placeholders/shared'
|
||||
import type {
|
||||
CodePlaceholderCompilationContext,
|
||||
CodePlaceholderOccurrence,
|
||||
CompiledCodePlaceholders,
|
||||
InternalCompileCodePlaceholdersInput,
|
||||
@@ -563,13 +564,92 @@ function classifyPythonBarePlaceholder(
|
||||
return 'value'
|
||||
}
|
||||
|
||||
/**
|
||||
* Matches the two ways Python code reaches the runtime environment by a literal name:
|
||||
* `environmentVariables['NAME']` and `environmentVariables.get('NAME')`. Attribute access is
|
||||
* absent because the binding is a plain dict, where `environmentVariables.NAME` raises.
|
||||
*/
|
||||
const PYTHON_DIRECT_ENVIRONMENT_READ =
|
||||
/environmentVariables\s*(?:\[\s*(['"])([A-Za-z0-9_]+)\1\s*\]|\.\s*get\s*\(\s*(['"])([A-Za-z0-9_]+)\3)/g
|
||||
|
||||
/**
|
||||
* Reports environment reads that bypass `{{NAME}}`, skipping any match that the lexer places
|
||||
* inside a string or comment — the same authority the placeholder rewriter uses to decide
|
||||
* what is real code.
|
||||
*
|
||||
* `lex` is a thunk, not a result: code with no placeholders returned without lexing at all
|
||||
* before this existed, and the overwhelmingly common case is code that never mentions
|
||||
* `environmentVariables`. Scanning for that with a regex first keeps the lexer off the path
|
||||
* entirely unless there is something to classify.
|
||||
*/
|
||||
function recordPythonDirectEnvironmentReads(
|
||||
code: string,
|
||||
lex: () => PythonLexResult,
|
||||
context: CodePlaceholderCompilationContext
|
||||
): void {
|
||||
const matches: RegExpExecArray[] = []
|
||||
PYTHON_DIRECT_ENVIRONMENT_READ.lastIndex = 0
|
||||
let match: RegExpExecArray | null
|
||||
while ((match = PYTHON_DIRECT_ENVIRONMENT_READ.exec(code)) !== null) {
|
||||
if (isIdentifierCharacter(code[match.index - 1])) continue
|
||||
if (!context.tracksDirectEnvironmentRead(match[2] ?? match[4] ?? '')) continue
|
||||
matches.push(match)
|
||||
}
|
||||
if (matches.length === 0) return
|
||||
|
||||
const lexed = lex()
|
||||
const ignoredRanges: Array<[number, number]> = [
|
||||
...lexed.comments,
|
||||
...lexed.strings.map((token): [number, number] => [token.start, token.end]),
|
||||
]
|
||||
|
||||
/**
|
||||
* A write or `del` target is reported like any other access, deliberately.
|
||||
*
|
||||
* `resolvedSecretNames` feeds an exact-value matcher over the output. Naming a secret the
|
||||
* code never read costs nothing there — the matcher scans for a value that does not appear
|
||||
* — while failing to name one that was read leaves it unmasked. Telling the two apart in
|
||||
* Python means textual heuristics, and every one of them has so far leaked in the second,
|
||||
* dangerous direction: a nested read inside a `del`, a parenthesized target. So this stops
|
||||
* trying, and errs toward reporting.
|
||||
*
|
||||
* JavaScript keeps its own write/delete exclusion because a real AST answers the question
|
||||
* per node, with no text to misread.
|
||||
*/
|
||||
for (const candidate of matches) {
|
||||
if (isOffsetInRanges(candidate.index, ignoredRanges)) continue
|
||||
/**
|
||||
* `other.environmentVariables['K']` reads a different object that merely shares the name,
|
||||
* so it is not the mounted binding at all. This is the receiver check the JavaScript side
|
||||
* gets from the AST, and unlike a scope or rebinding rule it cannot suppress a genuine
|
||||
* read: it only rejects an access whose receiver is demonstrably something else.
|
||||
*
|
||||
* Whitespace and line continuations are skipped, so a `.` left on a previous line inside
|
||||
* parentheses reads the same as one written adjacently — but the dot counts as a
|
||||
* qualifier only when it is code. A comment or string on the previous line can end in a
|
||||
* period (`# Load the value.`), and discarding on that would drop a genuine read, so the
|
||||
* landing position is checked against the same lexer ranges that filter the candidates.
|
||||
* This is why the receiver check runs after lexing rather than in the collection loop.
|
||||
*/
|
||||
let previous = candidate.index - 1
|
||||
while (previous >= 0 && /[\s\\]/.test(code[previous])) previous -= 1
|
||||
if (code[previous] === '.' && !isOffsetInRanges(previous, ignoredRanges)) continue
|
||||
const name = candidate[2] ?? candidate[4]
|
||||
if (name) context.recordDirectEnvironmentRead(name, candidate.index)
|
||||
}
|
||||
}
|
||||
|
||||
export async function compilePythonPlaceholders(
|
||||
input: InternalCompileCodePlaceholdersInput
|
||||
): Promise<CompiledCodePlaceholders> {
|
||||
const context = createCodePlaceholderCompilationContext(input, { identifierSuffix: '__' })
|
||||
if (context.occurrences.length === 0) return context.finish(input.code)
|
||||
if (context.occurrences.length === 0) {
|
||||
recordPythonDirectEnvironmentReads(input.code, () => lexPython(input.code), context)
|
||||
return context.finish(input.code)
|
||||
}
|
||||
|
||||
const lexed = lexPython(input.code)
|
||||
recordPythonDirectEnvironmentReads(input.code, () => lexed, context)
|
||||
const edits: SourceEdit[] = []
|
||||
const consumed = new Set<CodePlaceholderOccurrence>()
|
||||
let compilationSentinel: string | undefined
|
||||
|
||||
@@ -120,6 +120,40 @@ export function createCodePlaceholderCompilationContext(
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The gate {@link recordDirectEnvironmentRead} applies, exposed so a scanner can drop
|
||||
* candidates before classifying them. Deciding whether an expansion really runs costs a
|
||||
* lex or a quote-frame pass over the whole document, and the overwhelming majority of
|
||||
* `$VAR` / `environmentVariables[...]` reads in real code name something that is not a
|
||||
* configured secret — so answering "would this even be recorded" first keeps those passes
|
||||
* off the common path entirely.
|
||||
*/
|
||||
const tracksDirectEnvironmentRead = (name: string): boolean =>
|
||||
!input.analysisOnly && Object.hasOwn(environmentVariables, name)
|
||||
|
||||
/**
|
||||
* Records a secret the code reads straight off the runtime environment —
|
||||
* `environmentVariables.NAME` / `environmentVariables['NAME']` in JavaScript and Python,
|
||||
* `$NAME` in shell — rather than through a `{{NAME}}` placeholder.
|
||||
*
|
||||
* Those reads reach the same value but were invisible to this compiler, so nothing
|
||||
* downstream knew the secret was live: it never entered the run's active provenance, and
|
||||
* execution-log masking is activated by that entry. A direct read was therefore a secret
|
||||
* the logs would not redact. Reporting it here fixes that at the source, because
|
||||
* `resolvedSecretNames` is already the channel the runtime boundary reads back.
|
||||
*
|
||||
* Deliberately inert under `analysisOnly`. That mode drives Copilot's secret mount, whose
|
||||
* policy is that code receives a value only for an explicit `{{NAME}}` reference; widening
|
||||
* it here would mount secrets on the strength of an identifier appearing in a string.
|
||||
*/
|
||||
const recordDirectEnvironmentRead = (name: string, offset: number): void => {
|
||||
if (!tracksDirectEnvironmentRead(name)) return
|
||||
const currentOffset = resolvedSecretNameOffsets.get(name)
|
||||
if (currentOffset === undefined || offset < currentOffset) {
|
||||
resolvedSecretNameOffsets.set(name, offset)
|
||||
}
|
||||
}
|
||||
|
||||
const resolveValue = (
|
||||
occurrence: CodePlaceholderOccurrence
|
||||
): ResolvedCodePlaceholderValueOccurrence | undefined => {
|
||||
@@ -147,6 +181,8 @@ export function createCodePlaceholderCompilationContext(
|
||||
occurrences,
|
||||
hasValue,
|
||||
resolveValue,
|
||||
recordDirectEnvironmentRead,
|
||||
tracksDirectEnvironmentRead,
|
||||
runtimeBindingFor(kind) {
|
||||
const existing = runtimeBindingByKind.get(kind)
|
||||
if (existing) return existing
|
||||
|
||||
@@ -6,6 +6,7 @@ import {
|
||||
type SourceEdit,
|
||||
} from '@/lib/execution/code-placeholders/shared'
|
||||
import type {
|
||||
CodePlaceholderCompilationContext,
|
||||
CodePlaceholderOccurrence,
|
||||
CompiledCodePlaceholders,
|
||||
InternalCompileCodePlaceholdersInput,
|
||||
@@ -593,10 +594,88 @@ function collectShellOccurrenceContexts(
|
||||
return contexts
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the character at `index` is escaped: an odd run of backslashes immediately before it.
|
||||
*
|
||||
* Parity, not presence — `\\$KEY` is an escaped backslash followed by a live expansion, so
|
||||
* checking only the adjacent character reads a real read as escaped and drops it from usage
|
||||
* and masking alike. The same rule already decides line continuations in
|
||||
* {@link logicalLineEndAfterContinuations}.
|
||||
*/
|
||||
function isBackslashEscaped(code: string, index: number): boolean {
|
||||
let backslashes = 0
|
||||
let cursor = index - 1
|
||||
while (cursor >= 0 && code[cursor] === '\\') {
|
||||
backslashes += 1
|
||||
cursor -= 1
|
||||
}
|
||||
return backslashes % 2 === 1
|
||||
}
|
||||
|
||||
/** `$NAME` and `${NAME}` — including `${NAME:-default}`, whose name still ends at `:`. */
|
||||
const SHELL_PARAMETER_EXPANSION = /\$(?:\{\s*([A-Za-z_][A-Za-z0-9_]*)|([A-Za-z_][A-Za-z0-9_]*))/g
|
||||
|
||||
function recordShellDirectEnvironmentReads(
|
||||
code: string,
|
||||
context: CodePlaceholderCompilationContext
|
||||
): void {
|
||||
/**
|
||||
* The regex runs before anything else so a script with no expansion at all — or none naming
|
||||
* a configured secret — costs one scan and returns, rather than paying for the heredoc and
|
||||
* quote passes below. This function runs ahead of the no-placeholder early return, so that
|
||||
* cheap path has to stay cheap.
|
||||
*/
|
||||
const matches: RegExpExecArray[] = []
|
||||
SHELL_PARAMETER_EXPANSION.lastIndex = 0
|
||||
let match: RegExpExecArray | null
|
||||
while ((match = SHELL_PARAMETER_EXPANSION.exec(code)) !== null) {
|
||||
if (isBackslashEscaped(code, match.index)) continue
|
||||
const name = match[1] ?? match[2]
|
||||
if (name && context.tracksDirectEnvironmentRead(name)) matches.push(match)
|
||||
}
|
||||
if (matches.length === 0) return
|
||||
|
||||
/**
|
||||
* A heredoc with a quoted delimiter (`<<'EOF'`) is literal, so nothing in its body expands.
|
||||
* The frame scanner below models quoting within a line, not heredoc bodies, so those are
|
||||
* excluded up front — otherwise a `$NAME` printed verbatim would be reported as a read that
|
||||
* never happened, and a usage trail must not claim uses that did not occur.
|
||||
*/
|
||||
const literalHeredocBodies = collectHeredocs(code)
|
||||
.filter((heredoc) => heredoc.quoted)
|
||||
.map((heredoc): [number, number] => [heredoc.bodyStart, heredoc.bodyEnd])
|
||||
|
||||
const candidates: CodePlaceholderOccurrence[] = []
|
||||
for (const candidate of matches) {
|
||||
if (isOffsetInRanges(candidate.index, literalHeredocBodies)) continue
|
||||
candidates.push({
|
||||
start: candidate.index,
|
||||
end: candidate.index + candidate[0].length,
|
||||
raw: candidate[0],
|
||||
name: (candidate[1] ?? candidate[2]) as string,
|
||||
})
|
||||
}
|
||||
if (candidates.length === 0) return
|
||||
|
||||
const contexts = collectShellOccurrenceContexts(code, candidates, 0, code.length, false)
|
||||
for (const candidate of candidates) {
|
||||
const shellContext = contexts.get(candidate)
|
||||
/**
|
||||
* No context means the scanner never reached this offset — it skipped the region as a
|
||||
* comment. Absence is therefore evidence the expansion does not run, not permission to
|
||||
* record it, so this reads as an allowlist rather than a denylist. Single quotes suppress
|
||||
* expansion outright.
|
||||
*/
|
||||
if (!shellContext || shellContext.quote === 'single') continue
|
||||
context.recordDirectEnvironmentRead(candidate.name, candidate.start)
|
||||
}
|
||||
}
|
||||
|
||||
export async function compileShellPlaceholders(
|
||||
input: InternalCompileCodePlaceholdersInput
|
||||
): Promise<CompiledCodePlaceholders> {
|
||||
const context = createCodePlaceholderCompilationContext(input)
|
||||
recordShellDirectEnvironmentReads(input.code, context)
|
||||
if (context.occurrences.length === 0) return context.finish(input.code)
|
||||
|
||||
const validateShellValue = <T extends { value: string } | undefined>(
|
||||
|
||||
@@ -66,6 +66,10 @@ export interface CodePlaceholderCompilationContext {
|
||||
occurrence: CodePlaceholderOccurrence
|
||||
): ResolvedCodePlaceholderValueOccurrence | undefined
|
||||
resolve(occurrence: CodePlaceholderOccurrence): ResolvedCodePlaceholderOccurrence | undefined
|
||||
/** Reports a secret read straight off the runtime environment, without a `{{NAME}}` placeholder. */
|
||||
recordDirectEnvironmentRead(name: string, offset: number): void
|
||||
/** Whether {@link recordDirectEnvironmentRead} would keep this name, so a scanner can skip work. */
|
||||
tracksDirectEnvironmentRead(name: string): boolean
|
||||
runtimeBindingFor(kind: CodePlaceholderRuntimeBinding['kind']): CodePlaceholderRuntimeBinding
|
||||
registerInternalIdentifier(identifier: string): void
|
||||
createPrivateInput(content: string): CodePlaceholderPrivateInput
|
||||
|
||||
@@ -47,6 +47,9 @@ const { decryptSecretMock } = vi.hoisted(() => ({
|
||||
decryptSecretMock: vi.fn(async (encryptedValue: string) => ({ decrypted: encryptedValue })),
|
||||
}))
|
||||
|
||||
const { recordSecretUsageMock } = vi.hoisted(() => ({ recordSecretUsageMock: vi.fn() }))
|
||||
vi.mock('@/lib/secrets/usage/record', () => ({ recordSecretUsage: recordSecretUsageMock }))
|
||||
|
||||
vi.mock('drizzle-orm', () => ({
|
||||
eq: dbMocks.eq,
|
||||
and: dbMocks.and,
|
||||
@@ -137,6 +140,7 @@ function createSecretRegistry(
|
||||
return {
|
||||
isComplete: () => complete,
|
||||
getActiveMatches: () => matches,
|
||||
getResolvedSecretUsage: () => [{ name: 'API_KEY', scope: 'workspace' as const }],
|
||||
exportProvenance: () => ({ version: 1, complete, entries: [] }),
|
||||
exportCheckpointProvenance: () => ({ version: 1, complete, entries: [] }),
|
||||
} as unknown as ResolvedSecretTraceRegistry
|
||||
@@ -1804,3 +1808,63 @@ describe('LoggingSession progress-marker write path', () => {
|
||||
expect(dbChainMockFns.execute).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
})
|
||||
|
||||
describe('secret usage trail', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
resetDbChainMock()
|
||||
dbChainMockFns.limit.mockResolvedValue([])
|
||||
completeWorkflowExecutionMock.mockResolvedValue({})
|
||||
releaseExecutionSlotMock.mockResolvedValue(undefined)
|
||||
})
|
||||
|
||||
async function startSession(executionId: string) {
|
||||
const session = new LoggingSession('workflow-1', executionId, 'schedule', 'req-usage')
|
||||
session.setResolvedSecretTraceRegistry(createSecretRegistry([]))
|
||||
await session.start({
|
||||
userId: 'user-1',
|
||||
actorUserId: 'actor-1',
|
||||
workspaceId: 'workspace-1',
|
||||
skipLogCreation: true,
|
||||
})
|
||||
return session
|
||||
}
|
||||
|
||||
it('records what a completed run resolved, against the run actor', async () => {
|
||||
const session = await startSession('execution-usage-complete')
|
||||
|
||||
await session.complete({})
|
||||
|
||||
expect(recordSecretUsageMock).toHaveBeenCalledWith(
|
||||
[{ name: 'API_KEY', scope: 'workspace' }],
|
||||
expect.objectContaining({
|
||||
workspaceId: 'workspace-1',
|
||||
source: 'workflow',
|
||||
actorUserId: 'actor-1',
|
||||
workflowId: 'workflow-1',
|
||||
executionId: 'execution-usage-complete',
|
||||
trigger: 'schedule',
|
||||
})
|
||||
)
|
||||
})
|
||||
|
||||
it('records a failed run, which resolved the secret just the same', async () => {
|
||||
const session = await startSession('execution-usage-error')
|
||||
|
||||
await session.completeWithError({ error: new Error('boom') })
|
||||
|
||||
expect(recordSecretUsageMock).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
/**
|
||||
* A paused run resumes and completes later. Recording at the pause as well would count every
|
||||
* human-in-the-loop run twice.
|
||||
*/
|
||||
it('does not record a pause, which the resume will record', async () => {
|
||||
const session = await startSession('execution-usage-pause')
|
||||
|
||||
await session.completeWithPause({})
|
||||
|
||||
expect(recordSecretUsageMock).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -41,6 +41,7 @@ import type {
|
||||
TraceSpan,
|
||||
WorkflowState,
|
||||
} from '@/lib/logs/types'
|
||||
import { recordSecretUsage } from '@/lib/secrets/usage/record'
|
||||
import type { SerializableExecutionState } from '@/executor/execution/types'
|
||||
import type { BlockLog } from '@/executor/types'
|
||||
import { projectResolvedSecretDiagnosticError } from '@/executor/utils/resolved-secret-content-projection'
|
||||
@@ -208,6 +209,8 @@ export class LoggingSession {
|
||||
private correlation?: NonNullable<ExecutionTrigger['data']>['correlation']
|
||||
private trustedExecutionCorrelation?: NonNullable<ExecutionTrigger['data']>['correlation']
|
||||
private actorUserId: string | null = null
|
||||
/** Held directly rather than read off `environment`, which a caller may never build. */
|
||||
private workspaceId?: string
|
||||
private billingAttribution?: BillingAttributionSnapshot
|
||||
private isResume = false
|
||||
private completed = false
|
||||
@@ -632,6 +635,34 @@ export class LoggingSession {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Writes the run's secret-usage trail.
|
||||
*
|
||||
* Here rather than at resolution time because this is the one funnel every terminal path
|
||||
* reaches, and because a per-resolution write would put a database round trip in the
|
||||
* executor's hot path. A paused run is skipped: its registry is persisted with the
|
||||
* resumable snapshot, and the resume's own terminal completion records the usage, so
|
||||
* counting here as well would double every human-in-the-loop run.
|
||||
*
|
||||
* A hard worker kill records nothing. That is the same gap the execution log row itself
|
||||
* has — it stays `running` — and it is not worth a hot-path write to close.
|
||||
*/
|
||||
private recordResolvedSecretUsage(finalizationPath: ExecutionFinalizationPath): void {
|
||||
if (finalizationPath === 'paused') return
|
||||
|
||||
if (!this.workspaceId) return
|
||||
|
||||
const usage = this.resolvedSecretTraceRegistry?.getResolvedSecretUsage() ?? []
|
||||
recordSecretUsage(usage, {
|
||||
workspaceId: this.workspaceId,
|
||||
source: 'workflow',
|
||||
actorUserId: this.actorUserId,
|
||||
workflowId: this.workflowId,
|
||||
executionId: this.executionId,
|
||||
trigger: this.triggerType,
|
||||
})
|
||||
}
|
||||
|
||||
private async completeExecutionWithFinalization(params: {
|
||||
endedAt: string
|
||||
totalDurationMs: number
|
||||
@@ -689,6 +720,7 @@ export class LoggingSession {
|
||||
billingAttribution: this.billingAttribution,
|
||||
})
|
||||
this.persistedCompletionStatus = completedLog.persistedStatus
|
||||
this.recordResolvedSecretUsage(params.finalizationPath)
|
||||
|
||||
/**
|
||||
* Pause persistence releases only after the resumable snapshot is durable.
|
||||
@@ -775,6 +807,7 @@ export class LoggingSession {
|
||||
workflowState,
|
||||
} = params
|
||||
this.actorUserId = billingAttribution?.actorUserId ?? actorUserId ?? userId ?? null
|
||||
this.workspaceId = workspaceId
|
||||
this.billingAttribution = billingAttribution
|
||||
if (!this.resolvedSecretTraceRegistry) {
|
||||
const scopeUserId = userId ?? this.actorUserId
|
||||
|
||||
@@ -11,6 +11,7 @@ import {
|
||||
getEffectiveEnvironmentSnapshot,
|
||||
} from '@/lib/environment/utils'
|
||||
import type { McpServerConfig } from '@/lib/mcp/types'
|
||||
import { recordSecretUsage } from '@/lib/secrets/usage/record'
|
||||
import { resolveEnvVarReferences } from '@/executor/utils/reference-validation'
|
||||
import {
|
||||
createIncompleteResolvedSecretTraceRegistry,
|
||||
@@ -75,6 +76,7 @@ export async function resolveMcpConfigEnvVars(
|
||||
personalDecrypted: env.personalDecrypted,
|
||||
workspaceDecrypted: env.workspaceDecrypted,
|
||||
decryptionFailures: env.decryptionFailures,
|
||||
personalOwners: env.personalOwners,
|
||||
scope,
|
||||
})
|
||||
} catch (error) {
|
||||
@@ -127,6 +129,19 @@ export async function resolveMcpConfigEnvVars(
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* MCP server config resolves outside any workflow run, so no execution completion will
|
||||
* record it. Without this a secret used only to reach an MCP server reads as never used.
|
||||
*/
|
||||
if (workspaceId) {
|
||||
recordSecretUsage(resolvedSecretTraceRegistry.getResolvedSecretUsage(), {
|
||||
workspaceId,
|
||||
source: 'mcp',
|
||||
actorUserId: userId,
|
||||
trigger: 'mcp',
|
||||
})
|
||||
}
|
||||
|
||||
return {
|
||||
config: resolvedConfig,
|
||||
missingVars: allMissingVars,
|
||||
|
||||
@@ -21,6 +21,16 @@ export const secretOperations = {
|
||||
workspaceApiKey: 'deny',
|
||||
principalKinds: HUMAN_API_PRINCIPAL_KINDS,
|
||||
}),
|
||||
/**
|
||||
* Reading a secret's usage trail names who ran what with it. The use case narrows this to
|
||||
* the same people who may read the value itself; the operation only sets the floor.
|
||||
*/
|
||||
usage: defineWorkspaceOperation({
|
||||
id: 'secrets.usage',
|
||||
minimumRole: 'read',
|
||||
workspaceApiKey: 'deny',
|
||||
principalKinds: HUMAN_API_PRINCIPAL_KINDS,
|
||||
}),
|
||||
} as const
|
||||
|
||||
export type SecretOperation = (typeof secretOperations)[keyof typeof secretOperations]
|
||||
|
||||
@@ -3,7 +3,11 @@
|
||||
*/
|
||||
import type { Principal } from '@sim/auth/principal'
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import type { DeleteSecretInput, SetSecretInput } from '@/lib/secrets/application/use-cases'
|
||||
import type {
|
||||
DeleteSecretInput,
|
||||
ListSecretUsageInput,
|
||||
SetSecretInput,
|
||||
} from '@/lib/secrets/application/use-cases'
|
||||
|
||||
const { mocks } = vi.hoisted(() => ({
|
||||
mocks: {
|
||||
@@ -16,6 +20,7 @@ const { mocks } = vi.hoisted(() => ({
|
||||
setPersonal: vi.fn(),
|
||||
deletePersonal: vi.fn(),
|
||||
listCredentials: vi.fn(),
|
||||
secretUsage: vi.fn(),
|
||||
audit: vi.fn(),
|
||||
},
|
||||
}))
|
||||
@@ -46,6 +51,9 @@ vi.mock('@/lib/credentials/environment', () => ({
|
||||
vi.mock('@/lib/credentials/queries', () => ({
|
||||
listVisibleWorkspaceCredentials: mocks.listCredentials,
|
||||
}))
|
||||
vi.mock('@/lib/secrets/usage/queries', () => ({
|
||||
getSecretUsage: mocks.secretUsage,
|
||||
}))
|
||||
vi.mock('@/lib/credentials/secret-values', () => ({
|
||||
deletePersonalSecret: mocks.deletePersonal,
|
||||
deleteWorkspaceSecret: vi.fn(),
|
||||
@@ -53,7 +61,11 @@ vi.mock('@/lib/credentials/secret-values', () => ({
|
||||
setWorkspaceSecret: mocks.setWorkspace,
|
||||
}))
|
||||
|
||||
import { deleteSecretUseCase, setSecretUseCase } from '@/lib/secrets/application/use-cases'
|
||||
import {
|
||||
deleteSecretUseCase,
|
||||
listSecretUsageUseCase,
|
||||
setSecretUseCase,
|
||||
} from '@/lib/secrets/application/use-cases'
|
||||
|
||||
const workspace = {
|
||||
workspaceId: 'workspace-1',
|
||||
@@ -102,6 +114,7 @@ describe('secret application use cases', () => {
|
||||
mocks.personalMetadata.mockResolvedValue(null)
|
||||
mocks.deletePersonal.mockResolvedValue(true)
|
||||
mocks.listCredentials.mockResolvedValue({ data: [secret], nextCursorKeys: null })
|
||||
mocks.secretUsage.mockResolvedValue({ entries: [] })
|
||||
})
|
||||
|
||||
it('rejects workspace keys before resolving or reading secret state', async () => {
|
||||
@@ -313,3 +326,76 @@ describe('secret application use cases', () => {
|
||||
expect(result).toEqual({ name: personalSecret.envKey, scope: 'personal' })
|
||||
})
|
||||
})
|
||||
|
||||
describe('listSecretUsageUseCase', () => {
|
||||
const execute = listSecretUsageUseCase.execute as (args: {
|
||||
principal: Principal
|
||||
input: ListSecretUsageInput
|
||||
}) => Promise<unknown>
|
||||
|
||||
const workspaceInput: ListSecretUsageInput = {
|
||||
workspaceId: workspace.workspaceId,
|
||||
name: 'STRIPE_API_KEY',
|
||||
scope: 'workspace',
|
||||
limit: 100,
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
mocks.loadContext.mockResolvedValue(workspace)
|
||||
mocks.resolvePermission.mockResolvedValue('write')
|
||||
mocks.secretUsage.mockResolvedValue({ entries: [] })
|
||||
})
|
||||
|
||||
/**
|
||||
* The trail names workflows, people, and run ids. A Member who may use the secret but not
|
||||
* read it must not get that back — it is a slice of exactly what value masking withholds.
|
||||
*/
|
||||
it('denies a credential member who is not an admin of the key', async () => {
|
||||
mocks.workspaceAccess.mockResolvedValue({ hasAccess: true, canWrite: true, canAdmin: false })
|
||||
mocks.keyAccess.mockResolvedValue({
|
||||
knownKeys: new Set(['STRIPE_API_KEY']),
|
||||
adminKeys: new Set(),
|
||||
})
|
||||
|
||||
await expect(execute({ principal: session, input: workspaceInput })).rejects.toThrow(
|
||||
'Credential admin permission required to view this secret usage'
|
||||
)
|
||||
expect(mocks.secretUsage).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('allows a credential admin of that key', async () => {
|
||||
mocks.workspaceAccess.mockResolvedValue({ hasAccess: true, canWrite: true, canAdmin: false })
|
||||
mocks.keyAccess.mockResolvedValue({
|
||||
knownKeys: new Set(['STRIPE_API_KEY']),
|
||||
adminKeys: new Set(['STRIPE_API_KEY']),
|
||||
})
|
||||
|
||||
await expect(execute({ principal: session, input: workspaceInput })).resolves.toMatchObject({
|
||||
entries: [],
|
||||
})
|
||||
expect(mocks.secretUsage).toHaveBeenCalledWith({
|
||||
workspaceId: workspace.workspaceId,
|
||||
secretName: 'STRIPE_API_KEY',
|
||||
secretScope: 'workspace',
|
||||
secretOwnerUserId: '',
|
||||
limit: 100,
|
||||
})
|
||||
})
|
||||
|
||||
it('allows a workspace admin without a per-key grant', async () => {
|
||||
mocks.workspaceAccess.mockResolvedValue({ hasAccess: true, canWrite: true, canAdmin: true })
|
||||
mocks.keyAccess.mockResolvedValue({ knownKeys: new Set(), adminKeys: new Set() })
|
||||
|
||||
await expect(execute({ principal: session, input: workspaceInput })).resolves.toBeDefined()
|
||||
})
|
||||
|
||||
/** A personal secret is only ever the caller's own namespace, so there is nothing to gate. */
|
||||
it('reads a personal secret without a credential-admin check', async () => {
|
||||
await expect(
|
||||
execute({ principal: session, input: { ...workspaceInput, scope: 'personal' } })
|
||||
).resolves.toBeDefined()
|
||||
expect(mocks.workspaceAccess).not.toHaveBeenCalled()
|
||||
expect(mocks.keyAccess).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -19,6 +19,7 @@ import {
|
||||
setWorkspaceSecret,
|
||||
} from '@/lib/credentials/secret-values'
|
||||
import { secretOperations } from '@/lib/secrets/application/operations'
|
||||
import { getSecretUsage } from '@/lib/secrets/usage/queries'
|
||||
import { loadActiveWorkspaceContext } from '@/lib/uploads/contexts/workspace'
|
||||
import { checkWorkspaceAccess } from '@/lib/workspaces/permissions/utils'
|
||||
|
||||
@@ -355,3 +356,74 @@ export const deleteSecretUseCase = defineAuthorizedWorkspaceUseCase({
|
||||
metadata: { scope: input.scope, name: input.name },
|
||||
}),
|
||||
})
|
||||
|
||||
export interface ListSecretUsageInput {
|
||||
workspaceId: string
|
||||
name: string
|
||||
scope: SecretScope
|
||||
limit: number
|
||||
}
|
||||
|
||||
/**
|
||||
* Gates the usage trail behind the same permission that reveals the value.
|
||||
*
|
||||
* The trail names workflows, people, and run ids. Someone who may use a secret but not read
|
||||
* it has no claim on that, and letting a Member enumerate who else uses a key would hand back
|
||||
* a slice of exactly what the value masking withholds. Workspace secrets therefore require
|
||||
* workspace-admin or credential-admin on that key — the same predicate
|
||||
* `maskWorkspaceEnvForViewer` applies — while a personal secret is only ever the caller's own.
|
||||
*/
|
||||
async function requireSecretUsageReadAccess(params: {
|
||||
workspaceId: string
|
||||
name: string
|
||||
scope: SecretScope
|
||||
userId: string
|
||||
}): Promise<void> {
|
||||
if (params.scope === 'personal') return
|
||||
|
||||
const [workspaceAccess, keyAccess] = await Promise.all([
|
||||
checkWorkspaceAccess(params.workspaceId, params.userId),
|
||||
getWorkspaceEnvKeyAdminAccess({
|
||||
workspaceId: params.workspaceId,
|
||||
envKeys: [params.name],
|
||||
userId: params.userId,
|
||||
}),
|
||||
])
|
||||
|
||||
if (!workspaceAccess.canAdmin && !keyAccess.adminKeys.has(params.name)) {
|
||||
throw new ForbiddenOperationError(
|
||||
'SECRET_ADMIN_ACCESS_REQUIRED',
|
||||
'Credential admin permission required to view this secret usage'
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
export const listSecretUsageUseCase = defineAuthorizedWorkspaceUseCase({
|
||||
operation: secretOperations.usage,
|
||||
resolveContext: ({ input }: { input: ListSecretUsageInput }) =>
|
||||
resolveWorkspaceContext(input.workspaceId),
|
||||
authorizationOptions,
|
||||
async execute({ principal, input, context }) {
|
||||
const userId = principalUserId(principal)
|
||||
await requireSecretUsageReadAccess({
|
||||
workspaceId: context.workspaceId,
|
||||
name: input.name,
|
||||
scope: input.scope,
|
||||
userId,
|
||||
})
|
||||
|
||||
return getSecretUsage({
|
||||
workspaceId: context.workspaceId,
|
||||
secretName: input.name,
|
||||
secretScope: input.scope,
|
||||
/**
|
||||
* A personal trail is only ever the caller's own. Scoping the read to their id is what
|
||||
* enforces that — two people can hold a personal `OPENAI_KEY`, and a name-and-scope
|
||||
* filter alone would hand each of them the other's workflows, actors, and run links.
|
||||
* A workspace secret has no owner, so it reads under the storage sentinel.
|
||||
*/
|
||||
secretOwnerUserId: input.scope === 'personal' ? userId : '',
|
||||
limit: input.limit,
|
||||
})
|
||||
},
|
||||
})
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
import { db } from '@sim/db'
|
||||
import { secretUsage, user, workflow, workflowExecutionLogs } from '@sim/db/schema'
|
||||
import { and, desc, eq } from 'drizzle-orm'
|
||||
import type { ResolvedSecretScope } from '@/executor/utils/resolved-secret-trace-registry'
|
||||
|
||||
export interface SecretUsageEntry {
|
||||
id: string
|
||||
useCount: number
|
||||
lastUsedAt: Date
|
||||
source: 'workflow' | 'copilot' | 'mcp'
|
||||
workflowName: string | null
|
||||
actorName: string | null
|
||||
lastExecutionId: string | null
|
||||
/**
|
||||
* Whether that run's log still exists. Usage outlives logs by design — the trail is not
|
||||
* bound by `logRetentionHours` — so a row routinely names a run whose log has since been
|
||||
* pruned, and the UI has to say so rather than link into an empty view.
|
||||
*/
|
||||
lastExecutionAvailable: boolean
|
||||
lastTrigger: string | null
|
||||
}
|
||||
|
||||
export interface SecretUsagePage {
|
||||
entries: SecretUsageEntry[]
|
||||
}
|
||||
|
||||
interface SecretUsageQuery {
|
||||
workspaceId: string
|
||||
secretName: string
|
||||
secretScope: ResolvedSecretScope
|
||||
/** The owning user for a personal secret; empty for a workspace one. */
|
||||
secretOwnerUserId: string
|
||||
limit: number
|
||||
}
|
||||
|
||||
/**
|
||||
* Reads one secret's usage trail, newest bucket first.
|
||||
*
|
||||
* The filter is the `(workspaceId, secretName, secretScope, secretOwnerUserId)` prefix that
|
||||
* the `secret_usage_secret_recent_idx` index covers, so the ordered page is an index read.
|
||||
* The owner is part of it, not an afterthought: two people can hold personal secrets under
|
||||
* one name, and without it each would read the other's runs as their own.
|
||||
*/
|
||||
export async function getSecretUsage(query: SecretUsageQuery): Promise<SecretUsagePage> {
|
||||
const rows = await db
|
||||
.select({
|
||||
id: secretUsage.id,
|
||||
useCount: secretUsage.useCount,
|
||||
lastUsedAt: secretUsage.lastUsedAt,
|
||||
source: secretUsage.source,
|
||||
workflowName: workflow.name,
|
||||
actorName: user.name,
|
||||
lastExecutionId: secretUsage.lastExecutionId,
|
||||
/** At most one row: `execution_id` carries a unique index, so this cannot fan out. */
|
||||
lastExecutionLogId: workflowExecutionLogs.id,
|
||||
lastTrigger: secretUsage.lastTrigger,
|
||||
})
|
||||
.from(secretUsage)
|
||||
.leftJoin(workflow, eq(workflow.id, secretUsage.workflowId))
|
||||
.leftJoin(user, eq(user.id, secretUsage.actorUserId))
|
||||
.leftJoin(
|
||||
workflowExecutionLogs,
|
||||
eq(workflowExecutionLogs.executionId, secretUsage.lastExecutionId)
|
||||
)
|
||||
.where(
|
||||
and(
|
||||
eq(secretUsage.workspaceId, query.workspaceId),
|
||||
eq(secretUsage.secretName, query.secretName),
|
||||
eq(secretUsage.secretScope, query.secretScope),
|
||||
eq(secretUsage.secretOwnerUserId, query.secretOwnerUserId)
|
||||
)
|
||||
)
|
||||
.orderBy(desc(secretUsage.lastUsedAt))
|
||||
.limit(query.limit)
|
||||
|
||||
return {
|
||||
entries: rows.map(({ lastExecutionLogId, ...row }) => ({
|
||||
...row,
|
||||
lastExecutionAvailable: lastExecutionLogId !== null,
|
||||
})),
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,137 @@
|
||||
/**
|
||||
* @vitest-environment node
|
||||
*/
|
||||
import { dbChainMockFns, resetDbChainMock } from '@sim/testing'
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { recordSecretUsage } from '@/lib/secrets/usage/record'
|
||||
|
||||
/** `recordSecretUsage` is fire-and-forget, so tests await the microtask it queues. */
|
||||
const flush = () => new Promise((resolve) => setImmediate(resolve))
|
||||
|
||||
describe('recordSecretUsage', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
resetDbChainMock()
|
||||
})
|
||||
|
||||
it('writes one statement for every secret a run resolved', async () => {
|
||||
recordSecretUsage(
|
||||
[
|
||||
{ name: 'API_KEY', scope: 'workspace', ownerUserId: null },
|
||||
{ name: 'MY_TOKEN', scope: 'personal', ownerUserId: 'owner-1' },
|
||||
],
|
||||
{
|
||||
workspaceId: 'workspace-1',
|
||||
source: 'workflow',
|
||||
actorUserId: 'user-1',
|
||||
workflowId: 'workflow-1',
|
||||
executionId: 'execution-1',
|
||||
trigger: 'schedule',
|
||||
}
|
||||
)
|
||||
await flush()
|
||||
|
||||
expect(dbChainMockFns.insert).toHaveBeenCalledTimes(1)
|
||||
const rows = dbChainMockFns.values.mock.calls[0]?.[0]
|
||||
expect(rows).toHaveLength(2)
|
||||
expect(rows[0]).toMatchObject({
|
||||
workspaceId: 'workspace-1',
|
||||
secretName: 'API_KEY',
|
||||
secretScope: 'workspace',
|
||||
source: 'workflow',
|
||||
workflowId: 'workflow-1',
|
||||
actorUserId: 'user-1',
|
||||
secretOwnerUserId: '',
|
||||
useCount: 1,
|
||||
lastExecutionId: 'execution-1',
|
||||
lastTrigger: 'schedule',
|
||||
})
|
||||
/**
|
||||
* The owner is stored, not the actor: a scheduled run resolves the workflow owner's
|
||||
* personal slice under the workspace's execution actor, and filing the row under the
|
||||
* actor would hide it from the person whose secret it actually is.
|
||||
*/
|
||||
expect(rows[1]).toMatchObject({
|
||||
secretName: 'MY_TOKEN',
|
||||
secretScope: 'personal',
|
||||
secretOwnerUserId: 'owner-1',
|
||||
actorUserId: 'user-1',
|
||||
})
|
||||
})
|
||||
|
||||
it('buckets by UTC day rather than the server calendar', async () => {
|
||||
vi.useFakeTimers()
|
||||
try {
|
||||
/** 00:30 UTC — a server behind UTC would bucket this as the previous day. */
|
||||
vi.setSystemTime(new Date('2026-03-14T00:30:00.000Z'))
|
||||
recordSecretUsage([{ name: 'API_KEY', scope: 'workspace', ownerUserId: null }], {
|
||||
workspaceId: 'workspace-1',
|
||||
source: 'workflow',
|
||||
actorUserId: 'user-1',
|
||||
})
|
||||
await vi.runAllTimersAsync()
|
||||
} finally {
|
||||
vi.useRealTimers()
|
||||
}
|
||||
await flush()
|
||||
|
||||
expect(dbChainMockFns.values.mock.calls[0]?.[0][0]).toMatchObject({ usageDate: '2026-03-14' })
|
||||
})
|
||||
|
||||
it('increments the existing bucket instead of inserting a duplicate', async () => {
|
||||
recordSecretUsage([{ name: 'API_KEY', scope: 'workspace', ownerUserId: null }], {
|
||||
workspaceId: 'workspace-1',
|
||||
source: 'workflow',
|
||||
actorUserId: 'user-1',
|
||||
})
|
||||
await flush()
|
||||
|
||||
const conflict = dbChainMockFns.onConflictDoUpdate.mock.calls[0]?.[0]
|
||||
/** Every column of the day bucket, or two runs would collide into one row. */
|
||||
expect(conflict?.target).toHaveLength(8)
|
||||
const set = JSON.stringify(conflict?.set)
|
||||
expect(set).toContain(' + 1')
|
||||
/** Out-of-order completions must not walk the most recent timestamp backwards. */
|
||||
expect(set).toContain('greatest(')
|
||||
})
|
||||
|
||||
it('writes a Copilot run without a workflow', async () => {
|
||||
recordSecretUsage([{ name: 'API_KEY', scope: 'workspace', ownerUserId: null }], {
|
||||
workspaceId: 'workspace-1',
|
||||
source: 'copilot',
|
||||
actorUserId: 'user-1',
|
||||
trigger: 'copilot',
|
||||
})
|
||||
await flush()
|
||||
|
||||
/** Empty rather than null: the unique bucket key has to stay null-free on Postgres 14. */
|
||||
expect(dbChainMockFns.values.mock.calls[0]?.[0][0]).toMatchObject({
|
||||
source: 'copilot',
|
||||
workflowId: '',
|
||||
})
|
||||
})
|
||||
|
||||
it('does not touch the database when a run resolved nothing', async () => {
|
||||
recordSecretUsage([], {
|
||||
workspaceId: 'workspace-1',
|
||||
source: 'workflow',
|
||||
actorUserId: 'user-1',
|
||||
})
|
||||
await flush()
|
||||
|
||||
expect(dbChainMockFns.insert).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('never rejects when the write fails', async () => {
|
||||
dbChainMockFns.onConflictDoUpdate.mockRejectedValueOnce(new Error('constraint violation'))
|
||||
|
||||
expect(() =>
|
||||
recordSecretUsage([{ name: 'API_KEY', scope: 'workspace', ownerUserId: null }], {
|
||||
workspaceId: 'workspace-1',
|
||||
source: 'workflow',
|
||||
actorUserId: 'user-1',
|
||||
})
|
||||
).not.toThrow()
|
||||
await flush()
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,130 @@
|
||||
import { db } from '@sim/db'
|
||||
import { secretUsage } from '@sim/db/schema'
|
||||
import { createLogger } from '@sim/logger'
|
||||
import { generateShortId } from '@sim/utils/id'
|
||||
import { sql } from 'drizzle-orm'
|
||||
import type { ResolvedSecretScope } from '@/executor/utils/resolved-secret-trace-registry'
|
||||
|
||||
const logger = createLogger('SecretUsage')
|
||||
|
||||
/** Which surface resolved the secret. Mirrors the `secret_usage_source` enum. */
|
||||
export type SecretUsageSource = 'workflow' | 'copilot' | 'mcp'
|
||||
|
||||
export interface SecretUsageContext {
|
||||
workspaceId: string
|
||||
source: SecretUsageSource
|
||||
/** Whose access authorized the resolution; the run's actor. */
|
||||
actorUserId: string | null
|
||||
/** Absent for a Copilot run, which has no workflow. */
|
||||
workflowId?: string | null
|
||||
executionId?: string | null
|
||||
trigger?: string | null
|
||||
}
|
||||
|
||||
export interface ResolvedSecretUsage {
|
||||
name: string
|
||||
scope: ResolvedSecretScope
|
||||
/** The owning user of a personal secret; null for a workspace one. */
|
||||
ownerUserId: string | null
|
||||
}
|
||||
|
||||
/**
|
||||
* The UTC day a usage row buckets into.
|
||||
*
|
||||
* Explicitly UTC rather than the server's local calendar: rows are aggregated by this value
|
||||
* and read back by workspaces in every timezone, so a server-local bucket would shift the
|
||||
* boundary with the deployment region and split one day's usage across two rows.
|
||||
*/
|
||||
function utcDayBucket(at: Date): string {
|
||||
return at.toISOString().slice(0, 10)
|
||||
}
|
||||
|
||||
/**
|
||||
* Records which configured secrets a run resolved.
|
||||
*
|
||||
* Fire-and-forget and never throwing, matching `recordAudit` in `packages/audit/src/log.ts`:
|
||||
* a run must not fail because its usage trail could not be written, and this is called from
|
||||
* execution-completion paths that are already committing their result.
|
||||
*
|
||||
* One statement regardless of how many secrets a run touched. The upsert increments an
|
||||
* existing day bucket rather than inserting, which is what keeps a workflow on a one-minute
|
||||
* schedule from writing thousands of rows a day.
|
||||
*/
|
||||
export function recordSecretUsage(
|
||||
usage: readonly ResolvedSecretUsage[],
|
||||
context: SecretUsageContext
|
||||
): void {
|
||||
if (usage.length === 0) return
|
||||
|
||||
upsertSecretUsage(usage, context).catch((error) => {
|
||||
logger.error('Failed to record secret usage', {
|
||||
error,
|
||||
workspaceId: context.workspaceId,
|
||||
source: context.source,
|
||||
secretCount: usage.length,
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
async function upsertSecretUsage(
|
||||
usage: readonly ResolvedSecretUsage[],
|
||||
context: SecretUsageContext
|
||||
): Promise<void> {
|
||||
const now = new Date()
|
||||
const usageDate = utcDayBucket(now)
|
||||
|
||||
const rows = usage.map((entry) => ({
|
||||
id: generateShortId(),
|
||||
workspaceId: context.workspaceId,
|
||||
secretName: entry.name,
|
||||
secretScope: entry.scope,
|
||||
/** Empty sentinel for a workspace secret, matching the null-free bucket key. */
|
||||
secretOwnerUserId: entry.ownerUserId ?? '',
|
||||
source: context.source,
|
||||
/** Empty sentinel, never null — the unique bucket key must stay null-free. */
|
||||
workflowId: context.workflowId ?? '',
|
||||
actorUserId: context.actorUserId ?? '',
|
||||
usageDate,
|
||||
useCount: 1,
|
||||
lastUsedAt: now,
|
||||
lastExecutionId: context.executionId ?? null,
|
||||
lastTrigger: context.trigger ?? null,
|
||||
}))
|
||||
|
||||
await db
|
||||
.insert(secretUsage)
|
||||
.values(rows)
|
||||
.onConflictDoUpdate({
|
||||
target: [
|
||||
secretUsage.workspaceId,
|
||||
secretUsage.secretName,
|
||||
secretUsage.secretScope,
|
||||
secretUsage.secretOwnerUserId,
|
||||
secretUsage.source,
|
||||
secretUsage.workflowId,
|
||||
secretUsage.actorUserId,
|
||||
secretUsage.usageDate,
|
||||
],
|
||||
set: {
|
||||
useCount: sql`${secretUsage.useCount} + 1`,
|
||||
/**
|
||||
* `greatest` rather than a bare assignment: concurrent runs finishing out of order
|
||||
* must not walk the most recent timestamp backwards.
|
||||
*/
|
||||
lastUsedAt: sql`greatest(${secretUsage.lastUsedAt}, excluded.last_used_at)`,
|
||||
/**
|
||||
* The run that owns `last_used_at` has to own the metadata beside it. Assigning
|
||||
* these unconditionally while the timestamp is chosen by `greatest` lets two runs
|
||||
* completing out of order split one row between them — the newer run's timestamp
|
||||
* next to the older run's execution id, so "View log" opens a run that is not the
|
||||
* one the row says it last happened at. The guard keeps both from the same run.
|
||||
*
|
||||
* Postgres evaluates every SET expression against the pre-update row, so
|
||||
* `secret_usage.last_used_at` here is the stored value, not the one being written.
|
||||
*/
|
||||
lastExecutionId: sql`case when excluded.last_used_at >= ${secretUsage.lastUsedAt} then excluded.last_execution_id else ${secretUsage.lastExecutionId} end`,
|
||||
lastTrigger: sql`case when excluded.last_used_at >= ${secretUsage.lastUsedAt} then excluded.last_trigger else ${secretUsage.lastTrigger} end`,
|
||||
updatedAt: now,
|
||||
},
|
||||
})
|
||||
}
|
||||
@@ -542,6 +542,7 @@ async function executeWorkflowCoreImpl(
|
||||
personalDecrypted,
|
||||
workspaceDecrypted,
|
||||
decryptionFailures,
|
||||
personalOwners,
|
||||
} = env
|
||||
|
||||
// Use encrypted values for logging (don't log decrypted secrets)
|
||||
@@ -564,6 +565,7 @@ async function executeWorkflowCoreImpl(
|
||||
personalDecrypted,
|
||||
workspaceDecrypted,
|
||||
decryptionFailures,
|
||||
personalOwners,
|
||||
restoredProvenance: restoreTrusted ? restoredState?.resolvedSecretTraceProvenance : undefined,
|
||||
restoredCheckpointVersion: restoredState?.resolvedSecretTraceCheckpointVersion,
|
||||
restoreTrusted,
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
CREATE TYPE "public"."secret_usage_scope" AS ENUM('workspace', 'personal');--> statement-breakpoint
|
||||
CREATE TYPE "public"."secret_usage_source" AS ENUM('workflow', 'copilot', 'mcp');--> statement-breakpoint
|
||||
CREATE TABLE "secret_usage" (
|
||||
"id" text PRIMARY KEY NOT NULL,
|
||||
"workspace_id" text NOT NULL,
|
||||
"secret_name" text NOT NULL,
|
||||
"secret_scope" "secret_usage_scope" NOT NULL,
|
||||
"secret_owner_user_id" text DEFAULT '' NOT NULL,
|
||||
"source" "secret_usage_source" NOT NULL,
|
||||
"workflow_id" text DEFAULT '' NOT NULL,
|
||||
"actor_user_id" text DEFAULT '' NOT NULL,
|
||||
"usage_date" date NOT NULL,
|
||||
"use_count" integer DEFAULT 0 NOT NULL,
|
||||
"last_used_at" timestamp NOT NULL,
|
||||
"last_execution_id" text,
|
||||
"last_trigger" text,
|
||||
"created_at" timestamp DEFAULT now() NOT NULL,
|
||||
"updated_at" timestamp DEFAULT now() NOT NULL
|
||||
);
|
||||
--> statement-breakpoint
|
||||
ALTER TABLE "secret_usage" ADD CONSTRAINT "secret_usage_workspace_id_workspace_id_fk" FOREIGN KEY ("workspace_id") REFERENCES "public"."workspace"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX "secret_usage_bucket_unique" ON "secret_usage" USING btree ("workspace_id","secret_name","secret_scope","secret_owner_user_id","source","workflow_id","actor_user_id","usage_date");--> statement-breakpoint
|
||||
CREATE INDEX "secret_usage_secret_recent_idx" ON "secret_usage" USING btree ("workspace_id","secret_name","secret_scope","secret_owner_user_id","last_used_at" DESC NULLS LAST);
|
||||
File diff suppressed because it is too large
Load Diff
@@ -2038,6 +2038,13 @@
|
||||
"when": 1786704849273,
|
||||
"tag": "0291_fuzzy_wong",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 292,
|
||||
"version": "7",
|
||||
"when": 1787098574507,
|
||||
"tag": "0292_yielding_tarantula",
|
||||
"breakpoints": true
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -5,6 +5,7 @@ import {
|
||||
boolean,
|
||||
check,
|
||||
customType,
|
||||
date,
|
||||
decimal,
|
||||
doublePrecision,
|
||||
index,
|
||||
@@ -686,6 +687,106 @@ export const workspaceEnvironment = pgTable(
|
||||
})
|
||||
)
|
||||
|
||||
/** Which principal a run resolved a secret under, and which surface asked for it. */
|
||||
export const secretUsageScopeEnum = pgEnum('secret_usage_scope', ['workspace', 'personal'])
|
||||
export const secretUsageSourceEnum = pgEnum('secret_usage_source', ['workflow', 'copilot', 'mcp'])
|
||||
|
||||
/**
|
||||
* Per-day rollup of which secrets a run actually resolved.
|
||||
*
|
||||
* Execution logs cannot answer this. They persist the whole *available* encrypted
|
||||
* environment rather than what a run referenced, they only evidence a secret where
|
||||
* value-matching redaction happened to fire, and they expire under
|
||||
* `DataRetentionSettings.logRetentionHours`. A secret's usage trail has to outlive its
|
||||
* runs' logs, so it is written here instead of derived from them.
|
||||
*
|
||||
* Rows are a rollup rather than one per run: a workflow on a one-minute schedule
|
||||
* touching three secrets would otherwise write thousands of rows a day, which is also
|
||||
* why this is not `audit_log` — that table is a human-scale compliance surface and
|
||||
* machine-scale rows would drown it.
|
||||
*
|
||||
* `secretScope` and `secretOwnerUserId` are part of the key because a workspace secret and a
|
||||
* personal secret can share a name — as can two people's personal secrets — and none of them
|
||||
* may merge. `lastTriggeredByUserId` is deliberately *not* in the key: a public endpoint
|
||||
* called by many people would otherwise fragment one bucket per caller.
|
||||
*/
|
||||
export const secretUsage = pgTable(
|
||||
'secret_usage',
|
||||
{
|
||||
id: text('id').primaryKey(),
|
||||
workspaceId: text('workspace_id')
|
||||
.notNull()
|
||||
.references(() => workspace.id, { onDelete: 'cascade' }),
|
||||
/** Matches `credential.envKey`; the trail is keyed by name, not by credential row. */
|
||||
secretName: text('secret_name').notNull(),
|
||||
secretScope: secretUsageScopeEnum('secret_scope').notNull(),
|
||||
/**
|
||||
* Whose personal secret this was; empty for a workspace one, which the workspace owns.
|
||||
*
|
||||
* Two people can hold personal secrets under the same name, and a personal secret shared
|
||||
* with the workspace resolves for callers who do not own it, so name and scope alone do
|
||||
* not identify a secret. Without this column one person's trail would show another's runs.
|
||||
* Not the same as `actorUserId`: a scheduled run resolves the workflow owner's personal
|
||||
* slice under the workspace's execution actor.
|
||||
*/
|
||||
secretOwnerUserId: text('secret_owner_user_id').notNull().default(''),
|
||||
source: secretUsageSourceEnum('source').notNull(),
|
||||
/**
|
||||
* Empty for a Copilot or MCP resolution, which has no workflow.
|
||||
*
|
||||
* Empty string rather than null because both this and `actorUserId` sit inside the unique
|
||||
* key below, and Postgres treats nulls as distinct — two Copilot rows would never collide,
|
||||
* so the upsert would insert forever instead of incrementing. `NULLS NOT DISTINCT` fixes
|
||||
* that but requires Postgres 15, and this is self-hosted software that must not raise its
|
||||
* database floor for one table. A sentinel keeps the key null-free on every version.
|
||||
*
|
||||
* Deliberately not a foreign key, and neither is `actorUserId`. An `onDelete: 'set null'`
|
||||
* would rewrite a key column, so two rows differing only by the deleted id would collide
|
||||
* and an ordinary workflow or account deletion would fail on this constraint. They are
|
||||
* historical facts in a usage ledger rather than live references, so they are stored as
|
||||
* plain ids and joined leniently; a row outliving its workflow is the point of a trail.
|
||||
*/
|
||||
workflowId: text('workflow_id').notNull().default(''),
|
||||
/** Whose access authorized the resolution — the run's actor; empty when there is none. */
|
||||
actorUserId: text('actor_user_id').notNull().default(''),
|
||||
/** UTC day bucket. */
|
||||
usageDate: date('usage_date').notNull(),
|
||||
useCount: integer('use_count').notNull().default(0),
|
||||
lastUsedAt: timestamp('last_used_at').notNull(),
|
||||
/** Deep-links the most recent run in Logs, where the block and its code are visible. */
|
||||
lastExecutionId: text('last_execution_id'),
|
||||
/**
|
||||
* The surface the most recent run came in through (`api`, `webhook`, `schedule`,
|
||||
* `manual`, `chat`, `copilot`). There is deliberately no separate "triggered by" column:
|
||||
* for every trigger kind the executor can name a caller, that caller *is* `actorUserId`,
|
||||
* and for the rest (schedule, webhook, workspace key) no human triggered the run at all.
|
||||
*/
|
||||
lastTrigger: text('last_trigger'),
|
||||
createdAt: timestamp('created_at').notNull().defaultNow(),
|
||||
updatedAt: timestamp('updated_at').notNull().defaultNow(),
|
||||
},
|
||||
(table) => ({
|
||||
/** Every column is non-null, so ordinary unique semantics make the upsert increment. */
|
||||
bucketUnique: uniqueIndex('secret_usage_bucket_unique').on(
|
||||
table.workspaceId,
|
||||
table.secretName,
|
||||
table.secretScope,
|
||||
table.secretOwnerUserId,
|
||||
table.source,
|
||||
table.workflowId,
|
||||
table.actorUserId,
|
||||
table.usageDate
|
||||
),
|
||||
secretRecentIdx: index('secret_usage_secret_recent_idx').on(
|
||||
table.workspaceId,
|
||||
table.secretName,
|
||||
table.secretScope,
|
||||
table.secretOwnerUserId,
|
||||
table.lastUsedAt.desc()
|
||||
),
|
||||
})
|
||||
)
|
||||
|
||||
export const workspaceBYOKKeys = pgTable(
|
||||
'workspace_byok_keys',
|
||||
{
|
||||
|
||||
@@ -159,6 +159,23 @@ export const schemaMock = {
|
||||
stateData: 'stateData',
|
||||
createdAt: 'createdAt',
|
||||
},
|
||||
secretUsage: {
|
||||
id: 'id',
|
||||
workspaceId: 'workspaceId',
|
||||
secretName: 'secretName',
|
||||
secretScope: 'secretScope',
|
||||
source: 'source',
|
||||
workflowId: 'workflowId',
|
||||
actorUserId: 'actorUserId',
|
||||
usageDate: 'usageDate',
|
||||
useCount: 'useCount',
|
||||
firstUsedAt: 'firstUsedAt',
|
||||
lastUsedAt: 'lastUsedAt',
|
||||
lastExecutionId: 'lastExecutionId',
|
||||
lastTrigger: 'lastTrigger',
|
||||
createdAt: 'createdAt',
|
||||
updatedAt: 'updatedAt',
|
||||
},
|
||||
workflowExecutionLogs: {
|
||||
id: 'id',
|
||||
workflowId: 'workflowId',
|
||||
|
||||
@@ -9,8 +9,8 @@ const QUERY_HOOKS_DIR = path.join(ROOT, 'apps/sim/hooks/queries')
|
||||
const SELECTOR_HOOKS_DIR = path.join(ROOT, 'apps/sim/hooks/selectors')
|
||||
|
||||
const BASELINE = {
|
||||
totalRoutes: 1122,
|
||||
zodRoutes: 1122,
|
||||
totalRoutes: 1123,
|
||||
zodRoutes: 1123,
|
||||
nonZodRoutes: 0,
|
||||
} as const
|
||||
|
||||
|
||||
Reference in New Issue
Block a user