mirror of
https://github.com/simstudioai/sim.git
synced 2026-10-30 19:32:46 +08:00
Waleed
6bbabbbac6
* chore: run the orphaned migration-safety test, and stop publishing real-looking ids One of the eight script tests was reachable from no entrypoint, so it had never run in CI — it passes, it was simply never invoked. This is the second time that hand-maintained list has drifted from the files beside it; the audit runner's own header records the first. The guard against a third is a `check:*` script rather than a test, because the runner derives its list from that namespace and so picks the guard up by name — a test would have had to be hand-added to the very list it guards. A published spec sat outside the generator's manifest and so outside its drift check, and carried six example ids with the texture of real generated ones rather than the pandigital placeholders the rest of the repo uses. The new check globs the directory instead of reading the manifest, since the manifest is what omitted the file. The one pre-existing borderline id is allowlisted with a reason: loosening the threshold to admit it would have hidden one of the six. * fix(cli): close the gaps black-box testing the shipped CLI found `sim profiles <anything>` still exited 0, so a probe reading the exit code to ask whether a command exists was told yes — the one group the earlier guard missed. The exemption was written for commands that are both a group and a leaf, but only `files restore` takes an operand; `profiles` takes none. Registering its listing as a default subcommand puts it back among the pure dispatchers the existing guard already covers, so the guard itself did not need widening. Three commands refuse a workspace API key and said nothing, while their menu siblings said so — reading as though they accept one. They are hand-written, so they never reached the code that appends the note. That note now comes from a helper taking the operation, so a command names the operation it invokes and the two cannot disagree, and a test fails if a hand-written command ever calls a restricted operation without it. A blank numeric value in a request body still became a real zero, the same coercion already fixed for query strings: the guard keyed off the slot when the distinction is the field's declared type. Twenty-one fields across fifteen operations were affected. An empty body string still clears a description. Blank values for the root endpoint, workspace and profile flags fell back to what was configured instead of being refused, and a whitespace workspace was accepted verbatim. A hand-written profile name carrying padding listed as reachable but resolved to defaults rather than erroring. Two schema descriptions named request fields that no flag spells, and a rejected value was echoed unredacted by four messages while their siblings redacted it. A write now re-emits a section header it was not asked to touch byte for byte. The blank-line normalisation around it is left alone: making the writer position-faithful is a change to its model, not a fix. * fix(scripts): match example uuids case-insensitively in the spec audit The pattern only recognised lowercase hex, so an uppercase id in a published spec was never examined and the audit reported success without having looked at it. Matching case-insensitively is not enough on its own: hex is case-insensitive, so a mixed-case id counts `A` and `a` as two digits and reports twenty distinct ones rather than sixteen. That inflated count clears the threshold the texture test uses to recognise a hand-authored placeholder, so a real id could have passed for one. The allowlist is an exact-string lookup and would likewise have missed an uppercase spelling of an entry. Both checks and the lookup now take a normalised id, while the finding still reports the spelling as it appears in the file. * fix(cli): stop a refusal being swallowed, and gate example ids by name A blank root flag was refused everywhere except `profiles`, where the catch that lets a broken profile still list absorbed it and the command exited 0 after printing the table. The refusal now carries its own error class, which is what the listing rethrows on — the two are distinguished by type rather than by matching message text, and a genuinely broken profile still lists. The unknown-profile message redacted the name the caller typed but not the suggestion or the list of configured names beside it, which come from the same file and are equally attacker-influenced once it has been hand-edited. Those are redacted now, as is every other message in these two files that quotes a name read out of the config, and the profile listing flattens the names it renders the way it already flattened the error column. The example-id audit judged a uuid by its digit texture, on the premise that a real one essentially never looks hand-authored. Measured against ten million generated ids, 0.81% of them do — one in 124, where this change alone replaced six. Requiring each digit exactly twice takes that to zero but rejects all fourteen placeholders now in the specs, so it is no cheaper than the alternative. The audit now holds the eighteen ids the specs actually use, which is one file rather than the twenty-seven a reserved format would touch, and a new id fails until someone lists it — which is the review the check exists to force. * fix(scripts): match the uuid sentinels exactly rather than by shape Accepting any id built from at most two distinct hex digits let something through that was never on the approved list. A generated id essentially never has that shape, so the practical risk was small — but this check had just stopped being a shape test and become a list, and a structural exception is the one thing that undoes that. The two ids it exists for are the nil and max sentinels, and both are matched by value now.
100 lines
5.1 KiB
TypeScript
100 lines
5.1 KiB
TypeScript
#!/usr/bin/env bun
|
|
/**
|
|
* Gates the example UUIDs published in the OpenAPI specs against an explicit approved set.
|
|
*
|
|
* The specs in `apps/docs/` ship in a public repository, so an example id copied from a real run
|
|
* publishes an opaque workspace, workflow, or execution identifier. This check exists because that
|
|
* has already happened: six such ids sat in `openapi-core.json` until they were replaced.
|
|
*
|
|
* The gate is enumeration, not inference. An earlier revision instead scored the *texture* of an
|
|
* id — the repo's hand-authored placeholders are pandigital, using every hex digit no more than
|
|
* three times, which a real v4 UUID was assumed never to be. Measured against 12M generated v4
|
|
* UUIDs, 0.81% of them have exactly that texture: roughly one copied id in 124 would have passed
|
|
* and shipped. Tightening the rule does not rescue it — requiring every digit exactly twice drops
|
|
* the random pass rate to 0 in 20M, but also rejects all fourteen placeholders in use today, so it
|
|
* is the same cost as changing every id. Enumeration reaches zero false negatives for the price of
|
|
* one line per id, and the ids are few: eighteen across eight specs, introduced by six commits in
|
|
* two and a half months, because the specs deliberately share a handful of placeholders.
|
|
*
|
|
* Adding an endpoint whose example uses a new id therefore fails this check until the id is listed
|
|
* below. That failure is the point — it is the moment a human confirms the id was authored rather
|
|
* than pasted, and the diff line records the answer for the next reader.
|
|
*
|
|
* The directory is globbed rather than read from `OPENAPI_SPEC_FILES`, because the gap this closes
|
|
* is precisely that `openapi-core.json` is absent from that manifest and from every other check.
|
|
*/
|
|
import { readdirSync } from 'node:fs'
|
|
import path from 'node:path'
|
|
|
|
const ROOT = path.resolve(import.meta.dir, '..')
|
|
const SPEC_DIR = path.join(ROOT, 'apps/docs')
|
|
const UUID_PATTERN = /[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/gi
|
|
|
|
/**
|
|
* Every example UUID the specs are allowed to publish, with what it stands for.
|
|
*
|
|
* The nil and max UUIDs are covered by {@link SENTINELS} and are not listed here.
|
|
*/
|
|
const APPROVED: Record<string, string> = {
|
|
'0f7c1a2e-9b3d-4c58-8a21-6d4e5f7a9b01': 'run id in the RUN_ID_CONFLICT error example',
|
|
'3b1f7c92-8d4e-4a6b-9c0d-5e2f8a714b36': 'the shared workflow id placeholder',
|
|
'4c1f9e77-2b3a-4f8d-9e10-6a2c8d4b1e05': 'KNOWLEDGE_CHUNK_ID',
|
|
'7c9e6679-7425-40de-944b-e07fc1f90ae7': "KNOWLEDGE_BASE_ID, Wikipedia's canonical example UUID",
|
|
'9d3b7f10-2c8e-4a56-b0f4-6e1a8c5d2b97': 'pausedExecutionId on the paused-execution example',
|
|
'9f4c2a10-3b7e-4d58-8f6a-2c1d0e5b7a94': 'second workspace id, where one example needs two',
|
|
'a3f1c0b2-7a44-4c1d-9d3a-2b8e5f0a1c77':
|
|
'minted block id, shared with the v2 workflow operations tests',
|
|
'a6f0c8d2-3e57-4b19-8d4a-1c9e2f6b0a35': 'block id in the selected-outputs selector example',
|
|
'a91c4b2e-6d3f-4e8a-b5c7-0d9e2f1a8c64': 'the shared workspace id placeholder',
|
|
'b2d4f8a0-1c3e-4a5b-9d7c-2e6f0a8b4c12': 'KNOWLEDGE_DOCUMENT_ID',
|
|
'b8c2e60f-1a47-4d35-9e8b-3f0d5a7c2e19': 'executionId on the failed-execution example',
|
|
'c7a92e15-3f4b-4d8c-a1e6-9b0d5f2c8e74': 'executionId returned by the execute example',
|
|
'd5e1a3c7-8f60-4b29-9c4d-2a6e0f8b3d17': 'executionId on the paused-execution example',
|
|
'e4f8d2b6-9a1c-4e3d-8b7f-5c0a2d9e6f13': 'the shared run id placeholder',
|
|
'f0b3d8c2-7e5a-4b9d-8c1f-6a4e2d0b9c58': 'executionId on the resume examples',
|
|
'f1c3a7d0-4b52-4a8e-9f61-2d7c8b3e5a04': 'run output file id',
|
|
}
|
|
|
|
/**
|
|
* The two UUIDs RFC 9562 reserves as sentinels: nil and max.
|
|
*
|
|
* Matched exactly rather than by shape. An earlier version accepted any id built from at most
|
|
* two distinct hex digits, which is astronomically unlikely for a generated id but is still a
|
|
* bypass of the approved list — and this check exists to be a list, not a shape test. Both
|
|
* values are used by the resources spec today.
|
|
*/
|
|
const SENTINELS = new Set([
|
|
'00000000-0000-0000-0000-000000000000',
|
|
'ffffffff-ffff-ffff-ffff-ffffffffffff',
|
|
])
|
|
|
|
const specFiles = readdirSync(SPEC_DIR)
|
|
.filter((file) => file.startsWith('openapi') && file.endsWith('.json'))
|
|
.sort()
|
|
|
|
const findings: string[] = []
|
|
|
|
for (const file of specFiles) {
|
|
const contents = await Bun.file(path.join(SPEC_DIR, file)).text()
|
|
const seen = new Set(contents.match(UUID_PATTERN) ?? [])
|
|
for (const uuid of [...seen].sort()) {
|
|
const normalized = uuid.toLowerCase()
|
|
if (normalized in APPROVED || SENTINELS.has(normalized)) continue
|
|
findings.push(` - ${file}: ${uuid}`)
|
|
}
|
|
}
|
|
|
|
if (findings.length > 0) {
|
|
console.error(
|
|
`Published OpenAPI specs contain example UUIDs that are not approved placeholders:\n${findings.join('\n')}\n` +
|
|
'Prefer reusing an id already in APPROVED in scripts/check-spec-example-ids.ts. If the example ' +
|
|
'genuinely needs its own, hand-author one in the house style — every hex digit, none more than ' +
|
|
'three times — then add it to APPROVED with what it stands for. Never paste an id from a real run.'
|
|
)
|
|
process.exit(1)
|
|
}
|
|
|
|
console.log(
|
|
`Spec example ids passed: ${specFiles.length} published specs publish only approved placeholders.`
|
|
)
|