fix(v2-api): standardization (#6542)

* fix(v2-api): stop leaking resolved secrets in logs and serving doc source

Two regressions shipped with the v2 API (#5273) where v2 diverged from the
v1 path it replaced, plus the hardening that fell out of auditing them.

**v2 logs bypassed secret redaction.** `getPublicLog` and `listPublicLogs`
called raw `materializeExecutionData`, while every other reader — v1 list and
detail, CSV export, `fetch-log-detail`, both data-drain sources — calls
`materializeExecutionDataForDisplay`, which applies the resolved-secret
provenance projection. Both v2 routes then serialize `traceSpans` and
`finalOutput` straight onto the wire, so unredacted secrets could reach the
public API. Swapped to the display projection and threaded the principal's
subject user into the read context.

**v2 file download served generation source.** `GET /api/v2/files/{fileId}`
streamed `file.key` raw. AI-generated docs store their generation source as
the primary file, so a raw download yields source text under a `.pdf` name —
a file the recipient cannot open. Generated docs now resolve to their compiled
artifact; ordinary uploads still stream and are never materialized, gated on
the recorded generation-source type rather than the extension. The resolve is
capped at MAX_RENDERED_DOCUMENT_BYTES, and a still-compiling artifact returns
a retryable 409 rather than a 500.

Also in this change:

- Reconcile the two v2 verbs that used PUT for PATCH semantics:
  `PUT /v2/knowledge/{id}` and `PUT /v2/tables/{tableId}/rows` are both
  all-optional partial updates. Breaking for API-key clients, but the surface
  is dark-launched behind the `v2-api` gate and no in-repo caller issues PUT.
- Close the OpenAPI coverage blind spot that hid two routes: contract
  discovery was a non-recursive read of the flat `contracts/v2/` directory,
  so a contract in a subdirectory — or beside its non-v2 siblings, which is
  where the uploads contracts live — escaped the gate. The sweep is now
  recursive over the whole contracts tree, and the two upload data-plane
  routes are named in an explicit allowlist with reasons and staleness guards.
- Extract `needsRenderedArtifact` so the "recorded type is authoritative,
  extension is fallback" rule has one home instead of being duplicated.
- Extract `DocCompileUserError` into a leaf module so recognizing it no longer
  drags `app/api/**` and `next/server` into application modules.
- Correct the stale pagination docstring in `contracts/v2/shared.ts` and pin
  the paged/full-set split in a test so it cannot drift again.

* fix(v2-api): absolute imports for the extracted doc-compile error

Review follow-up.

- Use the `@/lib/...` alias for `doc-compile-error` in the three modules that
  imported it relatively. The repo requires absolute imports, and having all
  four consumers share one specifier also removes any chance of two module
  instances resolving apart and breaking `instanceof`.
- Memoize the v2 list-pagination sweep. It re-imported the whole contracts
  tree once per test and timed out against the default 10s limit under load;
  it now sweeps once and declares an explicit timeout. Its failure message
  also still pointed at an enumeration in `v2/shared.ts` that this branch
  replaced with a pointer to the test itself.

* fix(v2-api): correct three inaccurate claims found in verification

None of these change behavior; each is a comment or test-config assertion that
was not true as written.

- The artifact resolver's TSDoc implied the byte cap prevents an oversized
  artifact being materialized. It does not: the artifact-store fetch is not
  streaming-bounded, so the bytes are resident before the ceiling rejects
  them. Say what it actually guarantees.
- `v2/shared.ts` pointed at per-contract documentation for the two lists that
  still filter in memory. Neither contract documents it, so name the two lists
  and what they do inline instead of pointing at a page that does not exist.
- The knowledge update contract said "every field of the body is optional";
  `workspaceId` is required. Narrow the claim to mutable fields.
- Scope the pagination sweep's extended timeout to the one test that pays for
  it, so a genuine hang in the other two surfaces in 10s rather than 60s.

scripts/check-openapi-specs.ts

@@ -12,10 +12,15 @@
  * 2. v2 conventions: every published operation is under `/api/v2/`, documents
  *    401, 429, and 503, and resolves every documented 4xx/5xx response to the
  *    canonical error envelope `{ error: { code, message } }`.
- * 3. Contract cross-check: every contract exported from
- *    `lib/api/contracts/v2/*` must be documented, every documented `/api/v2/`
- *    operation must have a contract, and for each pair the query params,
- *    body fields, and response fields are diffed via `z.toJSONSchema`.
+ * 3. Contract cross-check: every route contract anywhere under
+ *    `lib/api/contracts/**` whose path is under `/api/v2/` must be documented
+ *    (or listed in `UNDOCUMENTED_V2_ROUTES` with a reason), every documented
+ *    `/api/v2/` operation must have a contract, and for each pair the query
+ *    params, body fields, and response fields are diffed via
+ *    `z.toJSONSchema`. The sweep is recursive and rooted at the whole
+ *    contracts tree, not the flat `v2/` directory — a contract in a
+ *    subdirectory, or one that lives beside its non-v2 siblings, must never
+ *    be able to escape coverage by virtue of where its file sits.
  * 4. Examples: documented request/response examples are parsed with the
  *    matching contract's actual Zod schemas — a doc example that the runtime
  *    would reject fails the build.
@@ -29,10 +34,27 @@ import { OPENAPI_SPEC_FILES } from '../apps/docs/lib/openapi-specs'
 
 const ROOT = path.resolve(import.meta.dir, '..')
 const DOCS_DIR = path.join(ROOT, 'apps/docs')
-const V2_CONTRACTS_DIR = path.join(ROOT, 'apps/sim/lib/api/contracts/v2')
+const CONTRACTS_DIR = path.join(ROOT, 'apps/sim/lib/api/contracts')
 
 const SPEC_FILES = OPENAPI_SPEC_FILES
 
+/**
+ * `/api/v2/` routes that are deliberately absent from the public OpenAPI
+ * specs, each with the reason it is not public API surface. Anything not
+ * listed here fails the build, so an undocumented v2 route is always a
+ * conscious, reviewed decision rather than an accident of file layout.
+ *
+ * A stale entry — one whose contract no longer exists, or which has since
+ * been documented — also fails, so the list cannot rot into a blanket
+ * exemption.
+ */
+const UNDOCUMENTED_V2_ROUTES: Readonly<Record<string, string>> = {
+  'PUT /api/v2/uploads/{uploadId}':
+    'Local-storage data plane for a signed whole-object upload. Authenticated by the short-lived upload-token minted by the documented session-create operation, not by an API key; carries no v2 feature gate and returns bare error bodies rather than the canonical v2 envelope. The URL is handed to the client by the session response and is never constructed from docs.',
+  'PUT /api/v2/uploads/{uploadId}/parts/{partNumber}':
+    'Local-storage data plane for a signed multipart part upload. Authenticated by a per-part signed `token` query param minted by the documented part-URL operation, not by an API key; same non-canonical envelope and self-describing URL as the whole-object PUT above.',
+}
+
 /**
  * Every operation removed with the unversioned core specification has a
  * public v2 replacement. Keeping this mapping executable prevents a future
@@ -109,15 +131,31 @@ function isContract(value: unknown): value is ContractLike {
 const contractKey = (c: ContractLike) =>
   `${c.method.toUpperCase()} ${c.path.replace(/\[([^\]]+)\]/g, '{$1}')}`
 
+/** Every non-test `.ts` file under `dir`, recursively, in stable order. */
+function listContractFiles(dir: string): string[] {
+  const files: string[] = []
+  for (const entry of readdirSync(dir, { withFileTypes: true }).sort((a, b) =>
+    a.name.localeCompare(b.name)
+  )) {
+    const full = path.join(dir, entry.name)
+    if (entry.isDirectory()) {
+      if (entry.name === '__tests__') continue
+      files.push(...listContractFiles(full))
+    } else if (entry.name.endsWith('.ts') && !entry.name.endsWith('.test.ts')) {
+      files.push(full)
+    }
+  }
+  return files
+}
+
+/** Every `/api/v2/` route contract exported anywhere in the contracts tree. */
 async function loadContracts(): Promise<Map<string, { name: string; contract: ContractLike }>> {
   const registry = new Map<string, { name: string; contract: ContractLike }>()
-  const files = readdirSync(V2_CONTRACTS_DIR)
-    .filter((f) => f.endsWith('.ts') && f !== 'shared.ts')
-    .map((f) => path.join(V2_CONTRACTS_DIR, f))
-  for (const file of files) {
+  for (const file of listContractFiles(CONTRACTS_DIR)) {
     const mod = (await import(file)) as Record<string, unknown>
     for (const [name, value] of Object.entries(mod)) {
       if (!isContract(value)) continue
+      if (!value.path.startsWith('/api/v2/')) continue
       const key = contractKey(value)
       const existing = registry.get(key)
       if (existing) {
@@ -659,8 +697,20 @@ for (const specFile of SPEC_FILES) {
 }
 
 for (const [key, { name }] of registry) {
-  if (!documentedKeys.has(key)) {
-    errors.push(`registry: ${name} (${key}) is not documented in any OpenAPI spec`)
+  if (documentedKeys.has(key) || key in UNDOCUMENTED_V2_ROUTES) continue
+  errors.push(`registry: ${name} (${key}) is not documented in any OpenAPI spec`)
+}
+
+for (const [key, reason] of Object.entries(UNDOCUMENTED_V2_ROUTES)) {
+  if (!reason.trim()) {
+    errors.push(`undocumented v2 allowlist: ${key} needs a reason explaining why it is not public`)
+  }
+  if (!registry.has(key)) {
+    errors.push(`undocumented v2 allowlist: ${key} matches no contract — remove the stale entry`)
+  } else if (documentedKeys.has(key)) {
+    errors.push(
+      `undocumented v2 allowlist: ${key} is documented after all — remove it from the allowlist`
+    )
   }
 }
 
@@ -733,6 +783,7 @@ if (errors.length > 0) {
   for (const message of errors) console.error(`  - ${message}`)
   process.exit(1)
 }
+const exemptCount = Object.keys(UNDOCUMENTED_V2_ROUTES).length
 console.log(
-  `OpenAPI spec validation passed: ${SPEC_FILES.length} specs, ${documentedKeys.size} operations, ${registry.size} contracts cross-checked.`
+  `OpenAPI spec validation passed: ${SPEC_FILES.length} specs, ${documentedKeys.size} operations, ${registry.size} contracts cross-checked (${exemptCount} explicitly undocumented).`
 )