mirror of
https://github.com/saltbo/zpan.git
synced 2026-08-29 00:01:42 +08:00
feat(ids)!: normalize persistent identifiers to Base62
BREAKING CHANGE: historical entity IDs, public links, sessions, API keys, OAuth credentials, and related references require the documented one-time backfill before this runtime is deployed.
This commit is contained in:
saltbo
@@ -0,0 +1,112 @@
|
||||
# Identifier normalization
|
||||
|
||||
## Release boundary
|
||||
|
||||
ZPan-owned persistent entity identifiers and public opaque tokens use the fixed alphabet
|
||||
`0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz` and satisfy
|
||||
`^[A-Za-z0-9]+$`. This is a breaking, maintenance-window migration: historical values are
|
||||
backfilled before the new application starts. Runtime code contains no legacy lookup,
|
||||
prefix parser, or old-to-new alias table.
|
||||
|
||||
The central generator is `shared/ids.ts`. A repository lint rejects imports of Nano ID's
|
||||
default `nanoid()` generator from production and operational source and permits reviewed
|
||||
`customAlphabet()` use only for the central generator, organization slugs, and usernames.
|
||||
Set-based SQLite inserts use `randomblob()` encoded as Base16 (or deterministic hex of a
|
||||
canonical JSON tuple for rebuildable stats rows); these values are cryptographically random
|
||||
or injective, ASCII alphanumeric, and cannot call a TypeScript generator inside SQL. Better Auth remains on its upstream
|
||||
database-ID generator because Better Auth 1.7 already uses `a-zA-Z0-9`; replacing it would
|
||||
couple ZPan to authentication internals without changing the contract.
|
||||
|
||||
## Entropy budget
|
||||
|
||||
Base62 provides `log2(62) = 5.954` bits per character. Lengths were rounded up when replacing
|
||||
Nano ID's 64-character alphabet, so entropy never decreases.
|
||||
|
||||
| Use | Old entropy | New format | New entropy |
|
||||
| --- | ---: | --- | ---: |
|
||||
| Default entity ID | Nano ID 21, 126 bits | Base62 22 | 131.0 bits |
|
||||
| Share / matter alias | Nano ID 10, 60 bits | Base62 11 | 65.5 bits |
|
||||
| Image ID | Nano ID 12, 72 bits | Base62 13 | 77.4 bits |
|
||||
| Image public token | prefixed/default Nano ID, at most 60 bits | Base62 12 | 71.4 bits |
|
||||
| Short filename suffix | Nano ID 4, 24 bits | Base62 5 | 29.8 bits |
|
||||
| Invite code | upper alphanumeric 8, 41.4 bits | Base62 8 | 47.6 bits |
|
||||
| OAuth PAR suffix | Nano ID 32, 192 bits | Base62 33 | 196.5 bits |
|
||||
| Registration management token | 32 random bytes, 256 bits | Base62 43 | 256.0 bits |
|
||||
|
||||
The collision budget uses `p ≈ n(n-1)/(2 × 62^length)`:
|
||||
|
||||
| Namespace | Population assumption | Approximate collision probability | Guard |
|
||||
| --- | ---: | ---: | --- |
|
||||
| 8-char invite codes | 100,000 simultaneously retained codes | `2.3e-5` | unique index; conflict fails the create |
|
||||
| 11-char share/alias tokens | 10,000,000 retained tokens | `9.6e-7` | unique index; `/r` tokens reserve a shared registry row in the resource transaction and retry bounded collisions |
|
||||
| 12-char image tokens | 10,000,000 retained tokens | `1.5e-8` | unique index; same transactional `/r` registry and bounded retry |
|
||||
| 22-char entity IDs | 1,000,000,000 retained IDs | `1.8e-22` | primary/unique key |
|
||||
|
||||
The shortest token is a one-time invite code and its active population is expected to stay
|
||||
well below the stated 100,000-code budget. A collision never aliases an existing resource:
|
||||
the database rejects it. Entity IDs have a materially larger margin.
|
||||
|
||||
## Repository-wide inventory
|
||||
|
||||
The baseline audit found 57 direct default `nanoid()` calls in 31 production files. The
|
||||
following table classifies generators and persisted or embedded values by ownership. “Migrate”
|
||||
means the one-time backfill rewrites historical values. “Future only” means the value is not
|
||||
an opaque entity ID, but its random component now uses the central generator.
|
||||
|
||||
| Class | Values and locations | Action |
|
||||
| --- | --- | --- |
|
||||
| ZPan entity primary IDs | matters, storages, quotas/entitlements, invites, notifications, jobs, downloaders/tasks, upload sessions, usage reports, announcements, audit/stat/ledger rows, shares/recipients, image hosting, WebDAV state, license binding, webhook/x402/cloud report rows | Migrate PK plus references; all creation paths use `generateId()` |
|
||||
| Better Auth entity IDs | user, account, organization, member, invitation, API key, OAuth client/resource linking rows | Verify/migrate any historical exception; retain upstream Base62 generator and contract tests |
|
||||
| Public opaque tokens | matter alias, invite code, site/team invite, share, image-hosting, image-domain verification, downloader JTI, instance ID | Migrate; all new values are Base62 |
|
||||
| Direct-share `ds_` token | share token and `/r/:token` | Replace with 11-char Base62; reserve the shared redirect registry in the share transaction, then resolve database ownership and fail closed on any integrity mismatch |
|
||||
| Image public token | image-hosting public token | Replace with 12-char Base62 without a semantic prefix; reserve the same registry namespace transactionally and derive resource type from the database |
|
||||
| Event/source/idempotency keys | `traffic_<id>`, `admin_grant:<id>`, `mutation:<id>`, ledger event keys, webhook event IDs | Future random component uses Base62 where ZPan generates it; separators remain because these are structured business keys, not entity IDs |
|
||||
| Object/cache keys | R2/S3 `object`, `storage_key`, image paths, cache keys, file paths | Do not rename. The database stores the authoritative key, so remapping a row ID does not move the object |
|
||||
| JSON/polymorphic references | notification/job/audit/resource-change/stat/API-key metadata; `ref_id`, `target_id`, `scope_id`, `resource_id`, actor refs | Rewrite exact mapped values; malformed JSON fails the plan before mutation |
|
||||
| Cloud / Store / ZPan Cloud identifiers | binding/store/account IDs, cloud event IDs and raw webhook payloads | External system owns them; never rewrite. If the instance ID changes, disconnect the local binding and reconcile/rebind explicitly |
|
||||
| Better Auth credentials | sessions, verification, JWKS, device codes, OAuth grants/access/refresh tokens, consent, PAR, JWT revocations, registration management credentials, and provider access/refresh/ID tokens stored on accounts | Invalidate or clear during maintenance; require login/re-authorization/key rotation instead of synthesizing compatibility; password hashes and provider linkage remain |
|
||||
| Downloader credential claims | PASETO subject/JTI and bootstrap credentials | Disable downloader and delete bootstrap credentials; re-register to issue claims over the new ID |
|
||||
| HTTP request ID | `crypto.randomUUID()` | Protocol/observability identifier; preserve UUID |
|
||||
| WebDAV lock token | `opaquelocktoken:${crypto.randomUUID()}` | WebDAV URI token; preserve standard format |
|
||||
| Object challenge ID | UUID inside a signed upload challenge | Signed protocol field; preserve until challenge expiry |
|
||||
| OAuth request URI | `urn:ietf:params:oauth:request_uri:<Base62>` | Preserve registered URN prefix; only the opaque suffix changes |
|
||||
| Signed page cursor / share child ref | ZPan-owned transient public opaque references | Encode a versioned signed binary envelope as Base62; legacy Base64url/dotted forms expire at the release boundary with no dual decoder |
|
||||
| OAuth state, PKCE, JWT claims | provider-owned Base64url, URIs and signed claims | Protocol-owned; do not apply the entity-ID regex |
|
||||
| Cryptographic salt/hash/random bytes | password salts, secret hashes, signatures | Cryptographic encoding boundary; preserve |
|
||||
| UI-only IDs | transient React list/chip keys using UUID | Ephemeral and not persisted; preserve |
|
||||
|
||||
## Backfill invariants
|
||||
|
||||
The planner builds random old-to-new mappings; it never strips or substitutes punctuation.
|
||||
One invalid entity value maps to one stable replacement across PK/FK and embedded references.
|
||||
Table-scoped JSON handling rewrites scalar and array ID keys used by notifications, archive
|
||||
jobs, audit and quota attribution, plus notification share tokens from the share-token
|
||||
namespace; arbitrary strings and external IDs are left untouched.
|
||||
Token mappings remain table-scoped, while candidate generation and the durable
|
||||
`redirect_token_registry` enforce one shared namespace for direct-share and image `/r` tokens.
|
||||
Mappings live only in `_zpan_id_backfill_map`, an operational checkpoint table that runtime
|
||||
code never reads. Finalization drops it after verification.
|
||||
|
||||
Apply runs with foreign keys enabled and deferred inside one SQLite transaction. It checks
|
||||
database integrity, foreign keys, zero illegal values, redirect ambiguity, and unchanged row
|
||||
counts for every table except explicitly invalidated credential tables. The generated D1 plan
|
||||
keeps each statement under D1's 100 KB statement limit and is re-entrant through `INSERT OR
|
||||
IGNORE`. For a production D1 database, the application must be quiesced and the exact
|
||||
generated statement array must succeed in one rehearsed `D1Database.batch()` transaction;
|
||||
Wrangler SQL-file execution and ad hoc multi-batch splitting are not supported. No online
|
||||
mixed-format phase exists.
|
||||
|
||||
The planner coalesces reference assignments by table and JSON rewrites into size-bounded CASE
|
||||
updates. It refuses artifacts above 47 statements, leaving the executor's three preflight and
|
||||
checkpoint queries within D1 Free's 50-query invocation cap. Empty, representative cross-table,
|
||||
and 1,000-document JSON fixtures assert this ceiling; an oversized fixture proves fail-fast
|
||||
without mutation. Snapshots above the ceiling use the runbook's offline normalized-new-D1 import
|
||||
and application binding switch, with the untouched old D1 as the pre-write rollback point. They
|
||||
are never divided into in-place batches. After an exact artifact is applied, its
|
||||
digest is the cheap startup checkpoint; the full unindexed scans run in the maintenance
|
||||
finalizer rather than on every new Worker isolate.
|
||||
|
||||
Every public share/image/invite token is rotated, including already-alphanumeric values, so
|
||||
old links intentionally stop working. The optional
|
||||
0600 mapping export supports owner notification or a controlled link-export process, but must
|
||||
never be deployed as a runtime lookup table.
|
||||
@@ -26,7 +26,7 @@ image_hosting_configs
|
||||
image_hostings
|
||||
id PK // opaque internal id, also drives storageKey
|
||||
orgId
|
||||
token UNIQUE // "ih_" + nanoid(10); supports token URL access
|
||||
token UNIQUE // Base62 public token; supports token URL access
|
||||
path // user-visible virtual path, e.g. "blog/2026/04/screenshot.png"
|
||||
UNIQUE (orgId, path) // path must be unique within an org
|
||||
storageKey // "ih/<orgId>/<id>" — flat, ID-keyed, independent of path
|
||||
@@ -59,7 +59,7 @@ Two URL forms coexist, selected by domain:
|
||||
|
||||
| Domain | Accepted URL shape | Example |
|
||||
|---|---|---|
|
||||
| **Default app domain** (`zpan.io`) | Token URL only: `/r/:token` | `https://zpan.io/r/ih_aB3xK9.png` |
|
||||
| **Default app domain** (`zpan.io`) | Token URL only: `/r/:token` | `https://zpan.io/r/Ab3xK9Lm2PqR.png` |
|
||||
| **Custom domain** (`img.user.com`) | Path URL only: `/<virtualPath>` | `https://img.user.com/blog/2026/04/screenshot.png` |
|
||||
|
||||
Rationale for the strict split:
|
||||
@@ -70,14 +70,14 @@ Rationale for the strict split:
|
||||
|
||||
### Route: `/r/:token`
|
||||
|
||||
Shared with direct shares (old `/d/:token` is removed outright; no users in production, no alias kept). Token prefix disambiguates:
|
||||
Shared with direct shares (old `/d/:token` is removed outright; no users in production, no alias kept). The database record disambiguates resource type:
|
||||
|
||||
| Prefix | Kind | Cache-Control | Content-Disposition |
|
||||
| Database record | Kind | Cache-Control | Content-Disposition |
|
||||
|---|---|---|---|
|
||||
| `ds_` | Direct share (existing) | `no-store` | `attachment` |
|
||||
| `ih_` | Image hosting | `public, max-age=300` | `inline` |
|
||||
| active `shares` row with `kind=direct` | Direct share | `no-store` | `attachment` |
|
||||
| active `image_hostings` row | Image hosting | `public, max-age=300` | `inline` |
|
||||
|
||||
Optional file extension for Markdown / browser hinting: `/r/ih_aB3xK9.png`. Server ignores the extension when resolving — only the token is authoritative. Extension is derived from stored `mime` at upload.
|
||||
Optional file extension for Markdown / browser hinting: `/r/Ab3xK9Lm2PqR.png`. Server ignores the extension when resolving — only the token is authoritative. Extension is derived from stored `mime` at upload.
|
||||
|
||||
`max-age=300` is deliberately shorter than `PRESIGN_TTL_SECS` so cached 302s never point to expired presigned URLs.
|
||||
|
||||
@@ -126,7 +126,7 @@ app.route('/api', apiRouter)
|
||||
{
|
||||
"data": {
|
||||
"url": "https://img.myblog.com/blog/2026/04/screenshot.png",
|
||||
"urlAlt": "https://zpan.io/r/ih_aB3xK9.png",
|
||||
"urlAlt": "https://zpan.io/r/Ab3xK9Lm2PqR.png",
|
||||
"markdown": "",
|
||||
"html": "<img src=\"https://img.myblog.com/blog/2026/04/screenshot.png\" />",
|
||||
"bbcode": "[img]https://img.myblog.com/blog/2026/04/screenshot.png[/img]"
|
||||
|
||||
@@ -0,0 +1,190 @@
|
||||
# ID normalization release runbook
|
||||
|
||||
This runbook is destructive. It requires a separate, explicit production approval. The
|
||||
implementation and tests do not run `wrangler d1 execute --remote` or mutate production.
|
||||
|
||||
## Prepare and dry-run
|
||||
|
||||
1. Record the application commit and current D1 migration version. Announce a maintenance
|
||||
window that covers database rewrite, credential invalidation and post-release checks.
|
||||
2. Export a recoverable production backup with `wrangler d1 export <database> --remote
|
||||
--output pre-id-normalization.sql`. Store it according to the incident-recovery policy and
|
||||
test importing it into a new local D1 database.
|
||||
3. Import the export into an isolated SQLite/D1 rehearsal database. Apply migration
|
||||
`0092_base62_audit_event_key.sql`, `0093_redirect-token-registry.sql`, and
|
||||
`0094_redirect-token-kind-resource.sql` to the rehearsal
|
||||
copy first; the old application can tolerate the nullable audit column and unused registry,
|
||||
while the planner intentionally refuses a database without them. Then run:
|
||||
|
||||
```sh
|
||||
pnpm ids:backfill -- --sqlite rehearsal.sqlite \
|
||||
--plan-file id-normalization.sql \
|
||||
--batch-file id-normalization-batch.json \
|
||||
--mapping-file id-normalization-map.json
|
||||
```
|
||||
|
||||
All output files are sensitive plaintext, created with mode 0600, and refuse to overwrite
|
||||
an existing file. Encrypt them at rest under the incident-recovery policy and delete them
|
||||
after the observation window. The console prints counts only, never old/new values.
|
||||
4. Review counts, malformed-JSON failures, external binding/downloader impact, and the secure
|
||||
share/image/invite mapping export with the operations owner. Resolve any direct-share/image
|
||||
token collision before proceeding.
|
||||
|
||||
## Maintenance-window apply
|
||||
|
||||
1. Stop writes and background consumers. Confirm no upload, WebDAV, download, quota, webhook,
|
||||
or OAuth request can write during the rewrite. Take a second export and record row counts.
|
||||
2. Apply migrations `0092_base62_audit_event_key.sql`, `0093_redirect-token-registry.sql`, and
|
||||
`0094_redirect-token-kind-resource.sql`
|
||||
while the old application remains quiesced, export again, and regenerate the plan from that
|
||||
exact post-migration snapshot.
|
||||
Do not reuse a plan from an earlier snapshot.
|
||||
3. On a local SQLite deployment, apply with the explicit credential acknowledgement:
|
||||
|
||||
```sh
|
||||
pnpm ids:backfill -- --sqlite zpan.sqlite --apply \
|
||||
--confirm-credential-invalidation \
|
||||
--mapping-file id-normalization-map.json
|
||||
```
|
||||
|
||||
The same transaction writes `id_normalization_pending_artifact_digest` from the exact plan.
|
||||
The breaking application accepts that verified observation-window checkpoint. Local rollback
|
||||
removes it; local finalize moves the exact value to `id_normalization_applied_artifact_digest`
|
||||
while writing `id_normalization_version=1`.
|
||||
|
||||
4. On D1, do **not** pass the SQL file to `wrangler d1 execute`: file execution is not the
|
||||
atomicity boundary this migration requires. The approved maintenance executor must load
|
||||
the exact generated JSON statement array and submit all statements in one `env.DB.batch()`.
|
||||
D1 keeps foreign keys enabled; the first statement is `PRAGMA defer_foreign_keys = ON`.
|
||||
Copy `wrangler.id-backfill.example.toml` outside the repository, replace both D1 placeholders,
|
||||
and have a second operator verify that they identify the clone. The dedicated template has no
|
||||
application routes, assets, queues, cron triggers, or service bindings. Deploy it with
|
||||
`pnpm wrangler deploy --config /secure/path/wrangler.id-backfill.toml`, then set its secret with
|
||||
`pnpm wrangler secret put ID_BACKFILL_AUTH_TOKEN --config /secure/path/wrangler.id-backfill.toml`.
|
||||
Never use the application's default `wrangler.toml` for this executor. Then POST the generated artifact
|
||||
with `Authorization: Bearer …`, `X-ZPan-ID-Backfill-Confirm:
|
||||
invalidate-credentials-and-links`, and `X-ZPan-ID-Backfill-Digest: <artifact.digest>`.
|
||||
The executor verifies the SHA-256 digest, completion marker, a 47-statement artifact ceiling,
|
||||
and statement-size/first-statement contract before issuing exactly one `env.DB.batch()`.
|
||||
That same batch records the pending
|
||||
artifact digest; a retry of the identical artifact is a no-op and a different artifact is
|
||||
rejected. The ceiling leaves three queries for preflight and the pending-digest insert under
|
||||
D1 Free's 50-query invocation cap; JSON rewrites are coalesced into size-bounded CASE updates.
|
||||
The response contains only the digest and statement count. Delete
|
||||
the Worker after the rehearsal. Repeat the same executor and exact artifact in the approved
|
||||
production window; never expose it through the application Worker.
|
||||
|
||||
Rehearse that exact artifact through the one-shot Worker against an isolated D1 clone and
|
||||
prove both successful commit and injected-failure rollback. The checked-in CF test proves
|
||||
the executor's digest gate plus representative PK/FK, structured key, JSON, public-token,
|
||||
credential invalidation and rollback behavior; the clone rehearsal proves the exact
|
||||
database-sized artifact. If the exact artifact exceeds request, batch, CPU, statement,
|
||||
or database limits, **do not apply it to the serving database**. The planner fails before
|
||||
writing when it cannot fit the exact snapshot into the safe one-batch ceiling. Use the
|
||||
offline new-database procedure below instead; never split an in-place rewrite into batches.
|
||||
5. Run the planner again in dry-run mode. It must report zero invalid IDs/tokens, zero
|
||||
ambiguous redirects, zero credentials awaiting invalidation, and zero JSON rewrites.
|
||||
Run `PRAGMA foreign_key_check`, compare every pre/post table row count except documented
|
||||
credential tables, check all unique indexes, and reconcile quota/audit/job invariants.
|
||||
6. Deploy the breaking application build. Re-register downloaders, rotate JWKS through normal
|
||||
authentication startup, require users to sign in again, and require OAuth clients and
|
||||
social-login accounts to authorize again. Provider access/refresh/ID tokens are cleared,
|
||||
while credential password hashes and provider account linkage remain. Rebind ZPan Cloud if
|
||||
the instance ID changed; its local binding is deliberately marked disconnected.
|
||||
7. Notify link owners that old share, image and invite URLs are invalid. Use the encrypted-at-
|
||||
rest mapping export only for notification/export; do not serve redirects from it. Every
|
||||
public token is rotated, including values that were already alphanumeric.
|
||||
|
||||
D1 limits and behavior should be rechecked immediately before the production window:
|
||||
[limits](https://developers.cloudflare.com/d1/platform/limits/),
|
||||
[foreign keys](https://developers.cloudflare.com/d1/sql-api/foreign-keys/),
|
||||
[SQL statements](https://developers.cloudflare.com/d1/sql-api/sql-statements/), and
|
||||
[import/export](https://developers.cloudflare.com/d1/best-practices/import-export-data/).
|
||||
|
||||
## D1 snapshots above the atomic query ceiling
|
||||
|
||||
An exact artifact above 47 statements is a supported release shape, but only through an
|
||||
offline blue-green database replacement. This path does not mutate the old D1 database and
|
||||
therefore does not need runtime old/new compatibility or an unsafe partially normalized state.
|
||||
Every remote command and the final binding switch still require separate production approval.
|
||||
|
||||
1. Keep the application and every consumer quiesced from the final export until the binding
|
||||
switch is accepted. Record a D1 Time Travel bookmark and export the exact post-0094 database.
|
||||
The old D1 database is the immutable rollback point; do not apply the generated D1 artifact
|
||||
to it.
|
||||
2. Import that export into local SQLite, run `pnpm ids:backfill -- --sqlite ... --apply
|
||||
--confirm-credential-invalidation`, perform all pre/post invariants, and run `--finalize`.
|
||||
This uses one local SQLite transaction without D1's per-invocation query ceiling. Repeat from
|
||||
the untouched export after an injected interruption and prove that only the complete result
|
||||
survives. The finalized database must contain `id_normalization_version=1`, no pending digest
|
||||
and no mapping table.
|
||||
3. Convert the normalized SQLite database to D1-compatible SQL. Follow Cloudflare's documented
|
||||
SQLite `.dump` procedure exactly: remove the outer `BEGIN TRANSACTION`/`COMMIT` and the
|
||||
`_cf_KV` statements. Preserve the source export, normalized SQLite file, final SQL, SHA-256
|
||||
digests, table row counts and verification report as separate encrypted release artifacts.
|
||||
4. Create a **new, empty, unbound** D1 database and import the normalized SQL with Wrangler.
|
||||
Cloudflare currently accepts import files up to 5 GiB; split larger imports according to the
|
||||
current official procedure. Because this target serves no traffic, a failed or interrupted
|
||||
import is discarded and recreated instead of resumed in place. For a split import, record
|
||||
the expected and actual row count after every file, and do not proceed to the next file on a
|
||||
mismatch. Do not bind the target to an application until the final global checks pass.
|
||||
5. Re-export the new D1 database and compare every table row count with the normalized local
|
||||
source. Run `foreign_key_check`, zero-invalid ID/token scans, redirect-registry consistency,
|
||||
uniqueness checks and business invariants against the re-export. Rehearse the new application
|
||||
against this isolated target using a maintenance-only configuration.
|
||||
6. In one reviewed deployment, switch the application D1 binding to the new database and deploy
|
||||
the breaking application. Keep writes stopped while read-only routing, authentication
|
||||
invalidation and representative reads are checked. If any check fails, restore the previous
|
||||
application deployment and old database binding; the old D1 is unchanged. Open writes only
|
||||
after explicit release acceptance. Once writes open, reverting to the old database would lose
|
||||
accepted writes and requires a separately approved reconciliation; prefer a forward fix.
|
||||
7. Retain the old D1, exports and digests through the observation window. Delete the replacement
|
||||
attempt on any pre-switch failure. Do not run old and new applications concurrently, replicate
|
||||
writes, or add legacy token lookup during this process.
|
||||
|
||||
Cloudflare D1 export blocks other requests while it runs, import/export has format constraints,
|
||||
and Time Travel restore is destructive and in-place. Reconfirm those current platform semantics,
|
||||
limits and retention immediately before approval; do not treat this runbook as authority to run
|
||||
the remote commands.
|
||||
|
||||
## Rollback and reconciliation
|
||||
|
||||
Do not finalize until the release owner accepts post-deploy verification.
|
||||
|
||||
- Before credential invalidation or any uncertain D1 partial execution, the authoritative
|
||||
rollback is to restore the quiesced pre-change export and redeploy the previous application.
|
||||
- While `_zpan_id_backfill_map` remains, `pnpm ids:backfill -- --sqlite zpan.sqlite --rollback`
|
||||
reverses mapped IDs/tokens and embedded references for local recovery. Deliberately deleted
|
||||
credentials are not reconstructible; restore the backup if they are needed.
|
||||
- For D1, generate/review reversal from the applied snapshot or restore the backup into a new
|
||||
database and switch bindings according to the D1 recovery procedure. Never run old and new
|
||||
applications concurrently against a partially reversed database.
|
||||
- Reconcile S3/R2 objects by stored `object`/`storage_key`; those keys are intentionally not
|
||||
renamed. Reconcile Cloud binding state manually because Cloud-owned IDs are not rewritten.
|
||||
|
||||
After the observation window, verify again and drop the checkpoint:
|
||||
|
||||
```sh
|
||||
pnpm ids:backfill -- --sqlite zpan.sqlite --finalize
|
||||
```
|
||||
|
||||
For D1, keep the same temporary maintenance Worker bound to the quiesced database and POST
|
||||
`{"version":1,"digest":"<artifact.digest>"}` to `/finalize` with the same bearer secret and
|
||||
`X-ZPan-ID-Backfill-Confirm: finalize-id-normalization`. The executor rechecks all foreign keys,
|
||||
the Base62 ID/token scans and redirect-registry consistency; requires the
|
||||
mapping checkpoint and exact pending digest to exist; and atomically writes both the completion
|
||||
version and applied artifact digest before dropping the mapping table. Credentials created by
|
||||
the new application during the observation window are allowed; invalidation of pre-migration
|
||||
credentials is proven in the apply artifact rehearsal. Its CF test
|
||||
proves completion and repeat-finalize rejection. Remove the maintenance Worker immediately
|
||||
after the successful response with
|
||||
`pnpm wrangler delete --config /secure/path/wrangler.id-backfill.toml`.
|
||||
|
||||
D1 rejects `PRAGMA integrity_check` through its prepared API. Run that check on the exact
|
||||
exported SQLite rehearsal snapshot before finalization; the live D1 finalizer uses the
|
||||
supported foreign-key and semantic scans listed above.
|
||||
|
||||
Finalization removes the local mapping table and therefore removes mapping-based rollback;
|
||||
the retained database export remains the disaster-recovery point. It atomically writes
|
||||
`system_options.id_normalization_version=1`; later dry-run/apply/finalize attempts fail fast
|
||||
instead of rotating public tokens a second time.
|
||||
@@ -0,0 +1,2 @@
|
||||
ALTER TABLE `audit_events` ADD `event_key` text;--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX `audit_events_event_key_unique` ON `audit_events` (`event_key`);
|
||||
@@ -0,0 +1,7 @@
|
||||
CREATE TABLE `redirect_token_registry` (
|
||||
`token` text PRIMARY KEY NOT NULL,
|
||||
`kind` text NOT NULL,
|
||||
`resource_id` text NOT NULL
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX `redirect_token_registry_resource_id_unique` ON `redirect_token_registry` (`resource_id`);
|
||||
@@ -0,0 +1,2 @@
|
||||
DROP INDEX `redirect_token_registry_resource_id_unique`;--> statement-breakpoint
|
||||
CREATE UNIQUE INDEX `redirect_token_registry_kind_resource_id_unique` ON `redirect_token_registry` (`kind`,`resource_id`);
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -638,6 +638,27 @@
|
||||
"when": 1785721689950,
|
||||
"tag": "0091_oauth-client-registration-management",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 92,
|
||||
"version": "6",
|
||||
"when": 1785868980173,
|
||||
"tag": "0092_base62_audit_event_key",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 93,
|
||||
"version": "6",
|
||||
"when": 1785875392712,
|
||||
"tag": "0093_redirect-token-registry",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 94,
|
||||
"version": "6",
|
||||
"when": 1785876121935,
|
||||
"tag": "0094_redirect-token-kind-resource",
|
||||
"breakpoints": true
|
||||
}
|
||||
]
|
||||
}
|
||||
+2
-1
@@ -30,12 +30,13 @@
|
||||
"storage-status:backfill": "tsx scripts/backfill-storage-enabled-status.ts",
|
||||
"api-key-scopes:backfill": "tsx scripts/backfill-api-key-scopes.ts",
|
||||
"oauth-scopes:backfill": "tsx scripts/backfill-oauth-scopes.ts",
|
||||
"ids:backfill": "tsx scripts/backfill-ids.ts",
|
||||
"typecheck": "tsc --noEmit -p server/tsconfig.json && tsc --noEmit -p src/tsconfig.json",
|
||||
"test": "vitest run --project unit --project integration",
|
||||
"test:cf": "vitest run --project cloudflare",
|
||||
"test:libsql": "vitest run --project libsql",
|
||||
"test:watch": "vitest --project unit --project integration",
|
||||
"lint": "biome check .",
|
||||
"lint": "biome check . && node scripts/lint-id-generation.mjs",
|
||||
"lint:fix": "biome check --write .",
|
||||
"lint:arch": "depcruise server/ shared/ --config .dependency-cruiser.cjs",
|
||||
"lint:http": "tsx scripts/lint-http-boundary.ts",
|
||||
|
||||
@@ -19,13 +19,13 @@ import { ADMIN_STATS_METRICS as M } from '../server/domain/admin-stats-metrics'
|
||||
const MIN_VALID_TIMESTAMP_MS = Date.UTC(2000, 0, 1)
|
||||
const MAX_BACKFILL_HOURS = 100_000
|
||||
const STATISTICS_OPENING_SOURCE_ID = 'v3-authoritative-sources'
|
||||
const STATISTICS_OPENING_EVENT_ID = `audit:statistics_source_initialized:${STATISTICS_OPENING_SOURCE_ID}`
|
||||
const STATISTICS_OPENING_EVENT_KEY = `audit:statistics_source_initialized:${STATISTICS_OPENING_SOURCE_ID}`
|
||||
const STATISTICS_OPENING_OPTION_KEY = 'stats_integrity_exact_from_v3'
|
||||
const TRAFFIC_LEDGER_OPENING_EVENT_ID = 'traffic_ledger_opening_v1'
|
||||
|
||||
const statisticsExactFromMsSql = `COALESCE(
|
||||
(SELECT unixepoch(value) * 1000 FROM system_options WHERE key = '${STATISTICS_OPENING_OPTION_KEY}'),
|
||||
(SELECT created_at * 1000 FROM audit_events WHERE id = '${STATISTICS_OPENING_EVENT_ID}'),
|
||||
(SELECT created_at * 1000 FROM audit_events WHERE event_key = '${STATISTICS_OPENING_EVENT_KEY}'),
|
||||
(unixepoch() + 1) * 1000
|
||||
)`
|
||||
const statisticsFirstFullHourMsSql = `CAST((${statisticsExactFromMsSql} + 3599999) / 3600000 AS INTEGER) * 3600000`
|
||||
@@ -113,19 +113,19 @@ INSERT OR IGNORE INTO system_options (key, value)
|
||||
VALUES (
|
||||
'${STATISTICS_OPENING_OPTION_KEY}',
|
||||
COALESCE(
|
||||
(SELECT strftime('%Y-%m-%dT%H:%M:%fZ', created_at, 'unixepoch') FROM audit_events WHERE id = '${STATISTICS_OPENING_EVENT_ID}'),
|
||||
(SELECT strftime('%Y-%m-%dT%H:%M:%fZ', created_at, 'unixepoch') FROM audit_events WHERE event_key = '${STATISTICS_OPENING_EVENT_KEY}'),
|
||||
'${openingAt}'
|
||||
)
|
||||
);
|
||||
|
||||
DELETE FROM audit_events WHERE id = '${STATISTICS_OPENING_EVENT_ID}';
|
||||
DELETE FROM audit_events WHERE event_key = '${STATISTICS_OPENING_EVENT_KEY}';
|
||||
|
||||
INSERT OR IGNORE INTO cloud_traffic_reports (
|
||||
id, org_id, period, source, source_id, event_id, bytes, storage_id,
|
||||
unit_bytes, credits_per_unit, status, error, attempt_count, next_retry_at,
|
||||
issued_at, created_at, updated_at
|
||||
) VALUES (
|
||||
'traffic_ledger_opening_v1', '', '${trafficPeriod}', 'object_download',
|
||||
lower(hex('traffic_ledger_opening_v1')), '', '${trafficPeriod}', 'object_download',
|
||||
'traffic_ledger_opening_v1', 'traffic_ledger_opening_v1', 0, NULL,
|
||||
NULL, NULL, 'ledger_opening', NULL, 0, NULL, NULL, ${now.getTime()}, ${now.getTime()}
|
||||
);
|
||||
@@ -135,10 +135,11 @@ SET actor_type = CASE WHEN user_id IS NULL THEN 'anonymous' ELSE 'user' END
|
||||
WHERE actor_type IS NULL;
|
||||
|
||||
INSERT OR IGNORE INTO audit_events (
|
||||
id, org_id, user_id, actor_type, actor_ref, action, target_type,
|
||||
id, event_key, org_id, user_id, actor_type, actor_ref, action, target_type,
|
||||
target_id, target_name, metadata, created_at
|
||||
)
|
||||
SELECT
|
||||
lower(hex('event:user_register:' || registered_user.id)),
|
||||
'event:user_register:' || registered_user.id,
|
||||
'',
|
||||
registered_user.id,
|
||||
@@ -324,7 +325,7 @@ function userSignupHistoryStartSql(currentHour: number): string {
|
||||
WHERE action = 'user_register'
|
||||
AND target_id IS NOT NULL
|
||||
AND user_id = target_id
|
||||
AND id = 'event:user_register:' || target_id
|
||||
AND event_key = 'event:user_register:' || target_id
|
||||
AND json_valid(metadata) = 1
|
||||
AND json_type(metadata, '$.provider') = 'text'
|
||||
AND length(json_extract(metadata, '$.provider')) > 0
|
||||
@@ -477,7 +478,7 @@ INSERT INTO stats_rollups_hourly (
|
||||
count, bytes, unique_count, metadata, updated_at
|
||||
)
|
||||
SELECT
|
||||
CAST(bucket_start AS TEXT) || ':global:stats.rollup_run:metric_key:${metric}',
|
||||
lower(hex(json_array(bucket_start, '', 'stats.rollup_run', 'metric_key', '${metric}'))),
|
||||
bucket_start, '', 'stats.rollup_run', 'metric_key', '${metric}', 1, 0, 0,
|
||||
json_object('version', 3, 'scope', 'counters', 'quality', 'exact'),
|
||||
bucket_start + 3600000
|
||||
@@ -529,7 +530,7 @@ INSERT INTO stats_rollups_hourly (
|
||||
count, bytes, unique_count, metadata, updated_at
|
||||
)
|
||||
SELECT
|
||||
CAST(buckets.bucket_start AS TEXT) || ':global:stats.rollup_run:all:all',
|
||||
lower(hex(json_array(buckets.bucket_start, '', 'stats.rollup_run', '', ''))),
|
||||
buckets.bucket_start, '', 'stats.rollup_run', '', '', 1, 0, 0,
|
||||
CASE WHEN snapshot_markers.bucket_start IS NULL THEN
|
||||
json_object(
|
||||
@@ -686,7 +687,7 @@ export function buildValidationSql(now = new Date()): string {
|
||||
WHERE action = 'user_register'
|
||||
AND target_id IS NOT NULL
|
||||
AND user_id = target_id
|
||||
AND id = 'event:user_register:' || target_id
|
||||
AND event_key = 'event:user_register:' || target_id
|
||||
AND json_valid(metadata) = 1
|
||||
AND json_type(metadata, '$.provider') = 'text'
|
||||
AND length(json_extract(metadata, '$.provider')) > 0
|
||||
@@ -777,7 +778,7 @@ export function buildValidationSql(now = new Date()): string {
|
||||
json_valid(metadata) = 0
|
||||
OR target_id IS NULL
|
||||
OR user_id <> target_id
|
||||
OR id <> 'event:user_register:' || target_id
|
||||
OR event_key <> 'event:user_register:' || target_id
|
||||
OR COALESCE(json_type(metadata, '$.provider') = 'text', 0) = 0
|
||||
OR COALESCE(length(json_extract(metadata, '$.provider')), 0) = 0
|
||||
))
|
||||
@@ -807,7 +808,7 @@ export function buildValidationSql(now = new Date()): string {
|
||||
SELECT 1
|
||||
FROM audit_events registration_event
|
||||
WHERE registration_event.action = 'user_register'
|
||||
AND registration_event.id = 'event:user_register:' || registered_user.id
|
||||
AND registration_event.event_key = 'event:user_register:' || registered_user.id
|
||||
AND registration_event.user_id = registered_user.id
|
||||
AND registration_event.target_id = registered_user.id
|
||||
AND registration_event.created_at = CAST(registered_user.created_at / 1000 AS INTEGER)
|
||||
@@ -932,7 +933,7 @@ authoritative_registration_sources AS MATERIALIZED (
|
||||
WHERE action = 'user_register'
|
||||
AND target_id IS NOT NULL
|
||||
AND user_id = target_id
|
||||
AND id = 'event:user_register:' || target_id
|
||||
AND event_key = 'event:user_register:' || target_id
|
||||
AND json_valid(metadata) = 1
|
||||
AND json_type(metadata, '$.provider') = 'text'
|
||||
AND length(json_extract(metadata, '$.provider')) > 0
|
||||
@@ -1225,7 +1226,7 @@ SELECT json_object(
|
||||
AND (
|
||||
ae.target_id IS NULL
|
||||
OR ae.user_id <> ae.target_id
|
||||
OR ae.id <> 'event:user_register:' || ae.target_id
|
||||
OR ae.event_key <> 'event:user_register:' || ae.target_id
|
||||
OR
|
||||
COALESCE(json_type(ae.metadata, '$.provider') = 'text', 0) = 0
|
||||
OR COALESCE(length(json_extract(ae.metadata, '$.provider')), 0) = 0
|
||||
@@ -1238,7 +1239,7 @@ SELECT json_object(
|
||||
WHERE NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM audit_events registration_event
|
||||
WHERE registration_event.id = 'event:user_register:' || registered_user.id
|
||||
WHERE registration_event.event_key = 'event:user_register:' || registered_user.id
|
||||
)
|
||||
),
|
||||
'issuedTrafficReportsToRecover', (
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
import { writeFileSync } from 'node:fs'
|
||||
import Database from 'better-sqlite3'
|
||||
import {
|
||||
applyBackfill,
|
||||
backfillPlanDigest,
|
||||
createBackfillPlan,
|
||||
finalizeBackfill,
|
||||
inspectBackfill,
|
||||
pendingBackfillDigest,
|
||||
rollbackBackfill,
|
||||
} from './id-backfill-core'
|
||||
|
||||
interface Options {
|
||||
database: string
|
||||
mode: 'dry-run' | 'apply' | 'rollback' | 'finalize'
|
||||
planFile?: string
|
||||
batchFile?: string
|
||||
mappingFile?: string
|
||||
confirmInvalidation: boolean
|
||||
}
|
||||
|
||||
function valueAfter(argv: string[], flag: string): string | undefined {
|
||||
const index = argv.indexOf(flag)
|
||||
return index >= 0 ? argv[index + 1] : undefined
|
||||
}
|
||||
|
||||
function parseOptions(argv: string[]): Options {
|
||||
const database = valueAfter(argv, '--sqlite')
|
||||
if (!database) usage()
|
||||
const modes = ['--apply', '--rollback', '--finalize'].filter((flag) => argv.includes(flag))
|
||||
if (modes.length > 1) usage()
|
||||
return {
|
||||
database,
|
||||
mode: modes[0]?.slice(2) as Options['mode'] | undefined ?? 'dry-run',
|
||||
planFile: valueAfter(argv, '--plan-file'),
|
||||
batchFile: valueAfter(argv, '--batch-file'),
|
||||
mappingFile: valueAfter(argv, '--mapping-file'),
|
||||
confirmInvalidation: argv.includes('--confirm-credential-invalidation'),
|
||||
}
|
||||
}
|
||||
|
||||
function usage(): never {
|
||||
throw new Error(
|
||||
'Usage: pnpm ids:backfill -- --sqlite <export.sqlite> [--apply|--rollback|--finalize] [--confirm-credential-invalidation] [--plan-file <sql>] [--batch-file <json>] [--mapping-file <json>]',
|
||||
)
|
||||
}
|
||||
|
||||
function writePrivate(path: string, contents: string): void {
|
||||
writeFileSync(path, contents, { encoding: 'utf8', mode: 0o600, flag: 'wx' })
|
||||
}
|
||||
|
||||
function main(): void {
|
||||
const options = parseOptions(process.argv.slice(2))
|
||||
const db = new Database(options.database, options.mode === 'dry-run' ? { readonly: true } : undefined)
|
||||
try {
|
||||
db.pragma('foreign_keys = ON')
|
||||
if (options.mode === 'rollback') {
|
||||
const after = rollbackBackfill(db)
|
||||
console.log(JSON.stringify({ mode: 'rollback-complete', after }, null, 2))
|
||||
return
|
||||
}
|
||||
if (options.mode === 'finalize') {
|
||||
finalizeBackfill(db)
|
||||
console.log(JSON.stringify({ mode: 'finalize-complete' }, null, 2))
|
||||
return
|
||||
}
|
||||
if (options.mode === 'apply' && pendingBackfillDigest(db)) {
|
||||
const after = applyBackfill(db)
|
||||
console.log(JSON.stringify({ mode: 'apply-already-complete', after }, null, 2))
|
||||
return
|
||||
}
|
||||
|
||||
const plan = createBackfillPlan(db)
|
||||
console.log(JSON.stringify({ mode: options.mode, before: plan.before, statements: plan.sql.length }, null, 2))
|
||||
if (options.planFile) writePrivate(options.planFile, `${plan.sql.join('\n')}\n`)
|
||||
if (options.batchFile) {
|
||||
const digest = backfillPlanDigest(plan)
|
||||
writePrivate(options.batchFile, `${JSON.stringify({ version: 1, digest, statements: plan.sql }, null, 2)}\n`)
|
||||
}
|
||||
if (options.mappingFile) writePrivate(options.mappingFile, `${JSON.stringify(plan.mappings, null, 2)}\n`)
|
||||
if (options.mode === 'dry-run') return
|
||||
if (plan.before.credentialsToInvalidate > 0 && !options.confirmInvalidation) {
|
||||
throw new Error('credential_invalidation_confirmation_required')
|
||||
}
|
||||
const after = applyBackfill(db, plan)
|
||||
console.log(JSON.stringify({ mode: 'apply-complete', before: plan.before, after }, null, 2))
|
||||
} finally {
|
||||
db.close()
|
||||
}
|
||||
}
|
||||
|
||||
main()
|
||||
@@ -0,0 +1,413 @@
|
||||
import Database from 'better-sqlite3'
|
||||
import { drizzle } from 'drizzle-orm/better-sqlite3'
|
||||
import { migrate } from 'drizzle-orm/better-sqlite3/migrator'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { ID_NORMALIZATION_DATA_TABLES } from '../shared/id-normalization-inventory'
|
||||
import { assertIdIntegrity } from '../server/db/id-integrity'
|
||||
import { applyIdBackfillArtifact, type IdBackfillBatchArtifact } from '../workers/id-backfill'
|
||||
import {
|
||||
applyBackfill,
|
||||
createBackfillPlan,
|
||||
finalizeBackfill,
|
||||
inspectBackfill,
|
||||
idBackfillDataTables,
|
||||
rollbackBackfill,
|
||||
verifyBackfill,
|
||||
} from './id-backfill-core'
|
||||
|
||||
function sqliteD1(db: Database.Database): D1Database {
|
||||
const statement = (query: string, values: unknown[] = []) => ({
|
||||
bind: (...next: unknown[]) => statement(query, next),
|
||||
first: async <T>() => db.prepare(query).get(...values) as T | null,
|
||||
all: async <T>() => ({ results: db.prepare(query).all(...values) as T[] }),
|
||||
run: async () => db.prepare(query).run(...values),
|
||||
raw: async () => [],
|
||||
})
|
||||
return {
|
||||
prepare: (query: string) => statement(query),
|
||||
batch: async (statements: Array<{ run(): Promise<unknown> }>) =>
|
||||
db.transaction(() => statements.map((prepared) => prepared.run()))(),
|
||||
} as unknown as D1Database
|
||||
}
|
||||
|
||||
async function artifact(statements: string[]): Promise<IdBackfillBatchArtifact> {
|
||||
const bytes = new Uint8Array(
|
||||
await crypto.subtle.digest('SHA-256', new TextEncoder().encode(JSON.stringify(statements))),
|
||||
)
|
||||
return {
|
||||
version: 1,
|
||||
digest: Array.from(bytes, (byte) => byte.toString(16).padStart(2, '0')).join(''),
|
||||
statements,
|
||||
}
|
||||
}
|
||||
|
||||
function fixture(): Database.Database {
|
||||
const db = new Database(':memory:')
|
||||
db.pragma('foreign_keys = ON')
|
||||
db.exec(`
|
||||
CREATE TABLE organization (id TEXT PRIMARY KEY);
|
||||
CREATE TABLE user (id TEXT PRIMARY KEY);
|
||||
CREATE TABLE account (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL REFERENCES user(id),
|
||||
access_token TEXT,
|
||||
refresh_token TEXT,
|
||||
id_token TEXT,
|
||||
access_token_expires_at INTEGER,
|
||||
refresh_token_expires_at INTEGER,
|
||||
scope TEXT,
|
||||
password TEXT
|
||||
);
|
||||
CREATE TABLE storages (id TEXT PRIMARY KEY);
|
||||
CREATE TABLE matters (
|
||||
id TEXT PRIMARY KEY,
|
||||
org_id TEXT NOT NULL REFERENCES organization(id),
|
||||
alias TEXT NOT NULL UNIQUE,
|
||||
storage_id TEXT NOT NULL REFERENCES storages(id),
|
||||
object TEXT NOT NULL
|
||||
);
|
||||
CREATE TABLE shares (
|
||||
id TEXT PRIMARY KEY,
|
||||
token TEXT NOT NULL UNIQUE,
|
||||
kind TEXT NOT NULL,
|
||||
status TEXT NOT NULL,
|
||||
matter_id TEXT NOT NULL REFERENCES matters(id),
|
||||
org_id TEXT NOT NULL REFERENCES organization(id),
|
||||
creator_id TEXT NOT NULL REFERENCES user(id)
|
||||
);
|
||||
CREATE TABLE image_hostings (
|
||||
id TEXT PRIMARY KEY,
|
||||
token TEXT NOT NULL UNIQUE,
|
||||
status TEXT NOT NULL,
|
||||
org_id TEXT NOT NULL REFERENCES organization(id),
|
||||
storage_id TEXT NOT NULL REFERENCES storages(id),
|
||||
storage_key TEXT NOT NULL
|
||||
);
|
||||
CREATE TABLE redirect_token_registry (
|
||||
token TEXT PRIMARY KEY,
|
||||
kind TEXT NOT NULL,
|
||||
resource_id TEXT NOT NULL
|
||||
);
|
||||
CREATE UNIQUE INDEX redirect_token_registry_kind_resource_id_unique
|
||||
ON redirect_token_registry(kind, resource_id);
|
||||
CREATE TABLE image_hosting_configs (
|
||||
org_id TEXT PRIMARY KEY REFERENCES organization(id),
|
||||
verification_token TEXT
|
||||
);
|
||||
CREATE TABLE team_invite_links (id TEXT PRIMARY KEY, token TEXT NOT NULL UNIQUE);
|
||||
CREATE TABLE notifications (id TEXT PRIMARY KEY, user_id TEXT NOT NULL, ref_id TEXT, metadata TEXT);
|
||||
CREATE TABLE background_jobs (id TEXT PRIMARY KEY, org_id TEXT NOT NULL, user_id TEXT NOT NULL, metadata TEXT, result_metadata TEXT);
|
||||
CREATE TABLE audit_events (
|
||||
id TEXT PRIMARY KEY, event_key TEXT UNIQUE, org_id TEXT NOT NULL, user_id TEXT,
|
||||
actor_type TEXT, actor_ref TEXT, target_type TEXT NOT NULL, target_id TEXT, metadata TEXT
|
||||
);
|
||||
CREATE TABLE resource_changes (sequence INTEGER PRIMARY KEY AUTOINCREMENT, scope_id TEXT, resource_id TEXT, metadata TEXT);
|
||||
CREATE TABLE oauthResource (id TEXT PRIMARY KEY, identifier TEXT NOT NULL UNIQUE, signing_key_id TEXT);
|
||||
CREATE TABLE oauthClientResource (id TEXT PRIMARY KEY, client_id TEXT NOT NULL, resource_id TEXT NOT NULL);
|
||||
CREATE TABLE x402_capacity_purchase_intents (id TEXT PRIMARY KEY, org_id TEXT NOT NULL, resource_id TEXT NOT NULL);
|
||||
CREATE TABLE org_quota_entitlements (id TEXT PRIMARY KEY, org_id TEXT NOT NULL, metadata TEXT);
|
||||
CREATE TABLE cloud_traffic_reports (
|
||||
id TEXT PRIMARY KEY, org_id TEXT NOT NULL, storage_id TEXT, source TEXT NOT NULL, source_id TEXT NOT NULL,
|
||||
event_id TEXT NOT NULL UNIQUE
|
||||
);
|
||||
CREATE TABLE storage_usage_ledger (
|
||||
id TEXT PRIMARY KEY, event_key TEXT NOT NULL UNIQUE, org_id TEXT NOT NULL, storage_id TEXT NOT NULL,
|
||||
resource_type TEXT NOT NULL, resource_id TEXT NOT NULL
|
||||
);
|
||||
CREATE TABLE object_upload_sessions (
|
||||
id TEXT PRIMARY KEY, org_id TEXT NOT NULL, object_id TEXT NOT NULL, storage_id TEXT NOT NULL, created_by TEXT NOT NULL
|
||||
);
|
||||
CREATE TABLE apikey (id TEXT PRIMARY KEY, reference_id TEXT NOT NULL, metadata TEXT);
|
||||
CREATE TABLE session (id TEXT PRIMARY KEY, user_id TEXT, token TEXT);
|
||||
CREATE TABLE oauthAccessToken (id TEXT PRIMARY KEY, user_id TEXT, token TEXT);
|
||||
CREATE TABLE jwks (id TEXT PRIMARY KEY);
|
||||
CREATE TABLE downloaders (id TEXT PRIMARY KEY, token_jti TEXT NOT NULL UNIQUE, enabled INTEGER NOT NULL, status TEXT NOT NULL);
|
||||
CREATE TABLE license_bindings (
|
||||
id TEXT PRIMARY KEY, instance_id TEXT NOT NULL, status TEXT NOT NULL, refresh_token TEXT,
|
||||
cached_certificate TEXT, cached_certificate_expires_at INTEGER
|
||||
);
|
||||
CREATE TABLE system_options (key TEXT PRIMARY KEY, value TEXT NOT NULL);
|
||||
|
||||
INSERT INTO organization VALUES ('_org');
|
||||
INSERT INTO user VALUES ('user-');
|
||||
INSERT INTO account VALUES ('socialAccount', 'user-', 'access', 'refresh', 'identity', 1, 2, 'openid', NULL);
|
||||
INSERT INTO storages VALUES ('sto-rage');
|
||||
INSERT INTO matters VALUES ('matter-', '_org', 'alias_', 'sto-rage', 'objects/matter-/file.bin');
|
||||
`)
|
||||
db.exec(`
|
||||
INSERT INTO shares VALUES ('share_', 'ds_legacy', 'direct', 'active', 'matter-', '_org', 'user-');
|
||||
INSERT INTO image_hostings VALUES ('share_', '_org', 'active', '_org', 'sto-rage', 'ih/_org/_image.png');
|
||||
INSERT INTO image_hosting_configs VALUES ('_org', 'verify_token');
|
||||
INSERT INTO team_invite_links VALUES ('InviteBase62', 'AlreadyBase62');
|
||||
INSERT INTO notifications VALUES ('notice-', 'user-', 'share_', '{"shareId":"share_","jobId":"job_","token":"ds_legacy","nested":{"matterId":"matter-"}}');
|
||||
INSERT INTO background_jobs VALUES ('job_', '_org', 'user-', '{"matterIds":["matter-"],"jobId":"job_"}', '{"matterIds":["matter-"]}');
|
||||
INSERT INTO audit_events VALUES ('event:user_register:user-', NULL, '_org', 'user-', 'user', NULL, 'user', 'user-', '{"storageId":"sto-rage","sessionId":"session_upload_","sourceId":"share_","entitlementId":"entitlement_","externalId":"sto-rage"}');
|
||||
INSERT INTO resource_changes(scope_id, resource_id, metadata) VALUES ('_org', 'share_', '{"userId":"user-"}');
|
||||
INSERT INTO apikey VALUES ('api_key', 'user-', '{"workspaceId":"_org"}');
|
||||
INSERT INTO oauthResource VALUES ('oauthResource1', '_org', 'jwk_bad');
|
||||
INSERT INTO oauthClientResource VALUES ('client::resource', 'external-client', '_org');
|
||||
INSERT INTO x402_capacity_purchase_intents VALUES ('intent_', '_org', '_org');
|
||||
INSERT INTO org_quota_entitlements VALUES ('entitlement_', '_org', '{"grantedBy":"user-","updatedBy":"user-","revokedBy":"user-"}');
|
||||
INSERT INTO cloud_traffic_reports VALUES ('traffic-row_', '_org', 'sto-rage', 'direct_share', 'share_', 'external-event');
|
||||
INSERT INTO cloud_traffic_reports VALUES ('TrafficWebdav', '_org', 'sto-rage', 'webdav_download', 'matter-', 'external-webdav');
|
||||
INSERT INTO cloud_traffic_reports VALUES ('TrafficImage', '_org', 'sto-rage', 'custom_domain_image', 'share_', 'external-image');
|
||||
INSERT INTO storage_usage_ledger VALUES ('opening:_org:sto-rage', 'opening:_org:sto-rage', '_org', 'sto-rage', 'storage', 'sto-rage');
|
||||
INSERT INTO object_upload_sessions VALUES ('session_upload_', '_org', 'matter-', 'sto-rage', 'downloader:down_loader');
|
||||
INSERT INTO session VALUES ('session_bad', 'user-', 'session-token');
|
||||
INSERT INTO oauthAccessToken VALUES ('access_bad', 'user-', 'oauth-token');
|
||||
INSERT INTO jwks VALUES ('jwk_bad');
|
||||
INSERT INTO downloaders VALUES ('down_loader', 'jti_bad', 1, 'online');
|
||||
INSERT INTO license_bindings VALUES ('binding_', 'instance_bad', 'active', 'secret', 'certificate', 99);
|
||||
INSERT INTO license_bindings VALUES ('BindingBase62', 'InstanceBase62', 'disconnected', 'stale-secret', 'stale-certificate', 100);
|
||||
INSERT INTO system_options VALUES ('instance_id', 'instance_bad');
|
||||
`)
|
||||
return db
|
||||
}
|
||||
|
||||
describe('ID backfill', () => {
|
||||
it('shares the complete touched-table inventory with the fresh-database guard', () => {
|
||||
expect(idBackfillDataTables()).toEqual([...ID_NORMALIZATION_DATA_TABLES].sort())
|
||||
})
|
||||
|
||||
it('dry-runs without mutation and emits D1-sized, idempotent SQL', () => {
|
||||
const db = fixture()
|
||||
const plan = createBackfillPlan(db)
|
||||
expect(plan.before).toMatchObject({
|
||||
invalidIds: 18,
|
||||
invalidTokens: 6,
|
||||
credentialsToInvalidate: 7,
|
||||
ambiguousRedirectTokens: 0,
|
||||
})
|
||||
expect(plan.sql.every((statement) => Buffer.byteLength(statement) <= 100_000)).toBe(true)
|
||||
expect(plan.sql.length).toBeLessThanOrEqual(47)
|
||||
expect((db.prepare('SELECT id FROM organization').get() as { id: string }).id).toBe('_org')
|
||||
db.close()
|
||||
})
|
||||
|
||||
it('rewrites PK/FK, polymorphic and JSON references and invalidates credentials', () => {
|
||||
const db = fixture()
|
||||
const objectKeys = {
|
||||
object: (db.prepare('SELECT object FROM matters').get() as { object: string }).object,
|
||||
storageKey: (db.prepare('SELECT storage_key AS storageKey FROM image_hostings').get() as { storageKey: string })
|
||||
.storageKey,
|
||||
}
|
||||
const beforeCounts = {
|
||||
matters: (db.prepare('SELECT COUNT(*) count FROM matters').get() as { count: number }).count,
|
||||
shares: (db.prepare('SELECT COUNT(*) count FROM shares').get() as { count: number }).count,
|
||||
}
|
||||
const after = applyBackfill(db)
|
||||
expect(after).toMatchObject({ invalidIds: 0, invalidTokens: 0, credentialsToInvalidate: 0 })
|
||||
expect(db.pragma('foreign_key_check')).toEqual([])
|
||||
expect(db.prepare('SELECT COUNT(*) count FROM session').get()).toEqual({ count: 0 })
|
||||
expect(db.prepare('SELECT COUNT(*) count FROM oauthAccessToken').get()).toEqual({ count: 0 })
|
||||
expect(db.prepare('SELECT COUNT(*) count FROM jwks').get()).toEqual({ count: 0 })
|
||||
expect(db.prepare('SELECT signing_key_id FROM oauthResource').get()).toEqual({ signing_key_id: null })
|
||||
expect(db.prepare('SELECT access_token, refresh_token, id_token, scope FROM account').get()).toEqual({
|
||||
access_token: null,
|
||||
refresh_token: null,
|
||||
id_token: null,
|
||||
scope: null,
|
||||
})
|
||||
expect(db.prepare('SELECT enabled, status FROM downloaders').get()).toEqual({ enabled: 0, status: 'offline' })
|
||||
expect(
|
||||
db.prepare('SELECT COUNT(*) AS count FROM license_bindings WHERE status != \'disconnected\' OR refresh_token IS NOT NULL OR cached_certificate IS NOT NULL OR cached_certificate_expires_at IS NOT NULL').get(),
|
||||
).toEqual({ count: 0 })
|
||||
expect((db.prepare('SELECT instance_id FROM license_bindings').get() as { instance_id: string }).instance_id).toMatch(
|
||||
/^[A-Za-z0-9]+$/,
|
||||
)
|
||||
expect(db.prepare('SELECT COUNT(*) count FROM matters').get()).toEqual({ count: beforeCounts.matters })
|
||||
expect(db.prepare('SELECT COUNT(*) count FROM shares').get()).toEqual({ count: beforeCounts.shares })
|
||||
expect(db.prepare('SELECT object FROM matters').get()).toEqual({ object: objectKeys.object })
|
||||
expect(db.prepare('SELECT storage_key AS storageKey FROM image_hostings').get()).toEqual({
|
||||
storageKey: objectKeys.storageKey,
|
||||
})
|
||||
|
||||
const share = db.prepare('SELECT id, token, matter_id, org_id, creator_id FROM shares').get() as Record<string, string>
|
||||
expect(Object.values(share).every((value) => /^[A-Za-z0-9]+$/.test(value))).toBe(true)
|
||||
expect((db.prepare('SELECT token FROM image_hostings').get() as { token: string }).token).not.toBe(share.token)
|
||||
expect(db.prepare('SELECT kind, resource_id FROM redirect_token_registry ORDER BY kind').all()).toEqual([
|
||||
{ kind: 'direct_share', resource_id: share.id },
|
||||
{ kind: 'image_hosting', resource_id: (db.prepare('SELECT id FROM image_hostings').get() as { id: string }).id },
|
||||
])
|
||||
const rotatedValidToken = (db.prepare('SELECT token FROM team_invite_links').get() as { token: string }).token
|
||||
expect(rotatedValidToken).toMatch(/^[A-Za-z0-9]{32}$/)
|
||||
expect(rotatedValidToken).not.toBe('AlreadyBase62')
|
||||
const metadata = JSON.parse((db.prepare('SELECT metadata FROM notifications').get() as { metadata: string }).metadata)
|
||||
expect(metadata.shareId).toBe(share.id)
|
||||
expect(metadata.nested.matterId).toBe(share.matter_id)
|
||||
expect(metadata.token).toBe(share.token)
|
||||
const job = db.prepare('SELECT id, metadata, result_metadata FROM background_jobs').get() as Record<string, string>
|
||||
expect(JSON.parse(job.metadata)).toEqual({ matterIds: [share.matter_id], jobId: job.id })
|
||||
expect(JSON.parse(job.result_metadata)).toEqual({ matterIds: [share.matter_id] })
|
||||
expect((db.prepare('SELECT event_key FROM audit_events').get() as { event_key: string }).event_key).toBe(
|
||||
`event:user_register:${share.creator_id}`,
|
||||
)
|
||||
const auditMetadata = JSON.parse((db.prepare('SELECT metadata FROM audit_events').get() as { metadata: string }).metadata)
|
||||
expect(auditMetadata.storageId).toBe(
|
||||
(db.prepare('SELECT storage_id FROM matters').get() as { storage_id: string }).storage_id,
|
||||
)
|
||||
expect(auditMetadata.externalId).toBe('sto-rage')
|
||||
expect(auditMetadata.sessionId).toBe(
|
||||
(db.prepare('SELECT id FROM object_upload_sessions').get() as { id: string }).id,
|
||||
)
|
||||
expect(auditMetadata.sourceId).toBe(share.id)
|
||||
const entitlement = db.prepare('SELECT id, metadata FROM org_quota_entitlements').get() as { id: string; metadata: string }
|
||||
expect(auditMetadata.entitlementId).toBe(entitlement.id)
|
||||
expect(Object.values(JSON.parse(entitlement.metadata)).every((value) => value === share.creator_id)).toBe(true)
|
||||
expect((db.prepare('SELECT resource_id FROM oauthClientResource').get() as { resource_id: string }).resource_id).toBe('_org')
|
||||
expect((db.prepare('SELECT resource_id FROM x402_capacity_purchase_intents').get() as { resource_id: string }).resource_id).toBe('_org')
|
||||
expect((db.prepare("SELECT source_id FROM cloud_traffic_reports WHERE source = 'direct_share'").get() as { source_id: string }).source_id).toBe(share.id)
|
||||
expect((db.prepare("SELECT source_id FROM cloud_traffic_reports WHERE source = 'webdav_download'").get() as { source_id: string }).source_id).toBe(share.matter_id)
|
||||
expect((db.prepare("SELECT source_id FROM cloud_traffic_reports WHERE source = 'custom_domain_image'").get() as { source_id: string }).source_id).toBe(
|
||||
(db.prepare('SELECT id FROM image_hostings').get() as { id: string }).id,
|
||||
)
|
||||
const ledger = db.prepare('SELECT event_key, org_id, storage_id, resource_id FROM storage_usage_ledger').get() as Record<string, string>
|
||||
expect(Object.values(ledger).every((value) => /^[A-Za-z0-9:]+$/.test(value))).toBe(true)
|
||||
expect(ledger.event_key).toBe(`opening:${ledger.org_id}:${ledger.storage_id}`)
|
||||
expect((db.prepare('SELECT created_by FROM object_upload_sessions').get() as { created_by: string }).created_by).toMatch(/^downloader:[A-Za-z0-9]+$/)
|
||||
expect(() => verifyBackfill(db)).not.toThrow()
|
||||
expect(db.prepare("SELECT value FROM system_options WHERE key = 'id_normalization_pending_artifact_digest'").get()).toMatchObject({
|
||||
value: expect.stringMatching(/^[0-9a-f]{64}$/),
|
||||
})
|
||||
|
||||
const second = applyBackfill(db)
|
||||
expect(second).toMatchObject({ invalidIds: 0, invalidTokens: 0, credentialsToInvalidate: 0 })
|
||||
db.close()
|
||||
})
|
||||
|
||||
it('executes the complete representative planner artifact through the maintenance executor', async () => {
|
||||
const db = fixture()
|
||||
const plan = createBackfillPlan(db)
|
||||
const batch = await artifact(plan.sql)
|
||||
|
||||
await expect(applyIdBackfillArtifact(sqliteD1(db), batch, batch.digest)).resolves.toEqual({
|
||||
statements: plan.sql.length,
|
||||
digest: batch.digest,
|
||||
})
|
||||
expect(verifyBackfill(db)).toMatchObject({ invalidIds: 0, invalidTokens: 0, credentialsToInvalidate: 0 })
|
||||
expect(db.prepare('SELECT COUNT(*) AS count FROM redirect_token_registry').get()).toEqual({ count: 2 })
|
||||
db.close()
|
||||
})
|
||||
|
||||
it('rolls back mapped identifiers while keeping deliberately invalidated credentials absent', () => {
|
||||
const db = fixture()
|
||||
applyBackfill(db)
|
||||
const summary = rollbackBackfill(db)
|
||||
expect(summary.invalidIds).toBeGreaterThan(0)
|
||||
expect((db.prepare('SELECT id FROM organization').get() as { id: string }).id).toBe('_org')
|
||||
expect((db.prepare('SELECT token FROM shares').get() as { token: string }).token).toBe('ds_legacy')
|
||||
expect((db.prepare('SELECT token FROM team_invite_links').get() as { token: string }).token).toBe('AlreadyBase62')
|
||||
expect(db.prepare('SELECT token, kind FROM redirect_token_registry ORDER BY kind').all()).toEqual([
|
||||
{ token: 'ds_legacy', kind: 'direct_share' },
|
||||
{ token: '_org', kind: 'image_hosting' },
|
||||
])
|
||||
expect((db.prepare('SELECT event_key FROM audit_events').get() as { event_key: string }).event_key).toBe(
|
||||
'event:user_register:user-',
|
||||
)
|
||||
expect(JSON.parse((db.prepare('SELECT metadata FROM notifications').get() as { metadata: string }).metadata)).toEqual({
|
||||
shareId: 'share_',
|
||||
jobId: 'job_',
|
||||
token: 'ds_legacy',
|
||||
nested: { matterId: 'matter-' },
|
||||
})
|
||||
expect(db.prepare('SELECT COUNT(*) count FROM session').get()).toEqual({ count: 0 })
|
||||
expect(
|
||||
db.prepare("SELECT value FROM system_options WHERE key = 'id_normalization_pending_artifact_digest'").get(),
|
||||
).toBeUndefined()
|
||||
db.close()
|
||||
})
|
||||
|
||||
it('rolls back an interrupted transaction completely', () => {
|
||||
const db = fixture()
|
||||
const plan = createBackfillPlan(db)
|
||||
const run = db.transaction(() => {
|
||||
db.pragma('defer_foreign_keys = ON')
|
||||
for (const statement of plan.sql.slice(0, Math.ceil(plan.sql.length / 2))) db.exec(statement)
|
||||
throw new Error('simulated interruption')
|
||||
})
|
||||
expect(run).toThrow('simulated interruption')
|
||||
expect((db.prepare('SELECT id FROM organization').get() as { id: string }).id).toBe('_org')
|
||||
expect(inspectBackfill(db).invalidIds).toBeGreaterThan(0)
|
||||
db.close()
|
||||
})
|
||||
|
||||
it('handles an empty migrated database', () => {
|
||||
const db = new Database(':memory:')
|
||||
migrate(drizzle(db), { migrationsFolder: 'migrations' })
|
||||
const plan = createBackfillPlan(db)
|
||||
expect(plan.sql.length).toBeLessThanOrEqual(47)
|
||||
expect(applyBackfill(db, plan)).toMatchObject({ invalidIds: 0, invalidTokens: 0 })
|
||||
db.close()
|
||||
})
|
||||
|
||||
it('coalesces a large set of JSON rewrites within the D1 free-plan query budget', () => {
|
||||
const db = fixture()
|
||||
const baselineDocuments = inspectBackfill(db).jsonDocumentsToRewrite
|
||||
const insert = db.prepare('INSERT INTO notifications VALUES (?, ?, ?, ?)')
|
||||
db.transaction(() => {
|
||||
for (let index = 0; index < 1_000; index += 1) {
|
||||
insert.run(`Notice${index}`, 'user-', 'share_', '{"shareId":"share_","matterId":"matter-"}')
|
||||
}
|
||||
})()
|
||||
|
||||
expect(inspectBackfill(db).jsonDocumentsToRewrite).toBe(baselineDocuments + 1_000)
|
||||
const plan = createBackfillPlan(db)
|
||||
expect(plan.sql.length).toBeLessThanOrEqual(47)
|
||||
applyBackfill(db, plan)
|
||||
expect(
|
||||
db.prepare('SELECT COUNT(*) AS count FROM notifications WHERE instr(metadata, \'"share_"\') > 0').get(),
|
||||
).toEqual({ count: 0 })
|
||||
db.close()
|
||||
})
|
||||
|
||||
it('fails before mutation when a snapshot cannot fit the atomic D1 plan', () => {
|
||||
const db = fixture()
|
||||
const beforeCount = (
|
||||
db.prepare("SELECT COUNT(*) AS count FROM notifications WHERE id LIKE 'notice-%'").get() as { count: number }
|
||||
).count
|
||||
const insert = db.prepare('INSERT INTO notifications VALUES (?, ?, NULL, NULL)')
|
||||
db.transaction(() => {
|
||||
for (let index = 0; index < 24_000; index += 1) insert.run(`notice-${index}`, 'user-')
|
||||
})()
|
||||
|
||||
expect(() => createBackfillPlan(db)).toThrow(/d1_query_limit_exceeded:\d+:47/)
|
||||
expect(db.prepare("SELECT COUNT(*) AS count FROM notifications WHERE id LIKE 'notice-%'").get()).toEqual({
|
||||
count: beforeCount + 24_000,
|
||||
})
|
||||
expect(db.prepare("SELECT name FROM sqlite_master WHERE name = '_zpan_id_backfill_map'").get()).toBeUndefined()
|
||||
db.close()
|
||||
})
|
||||
|
||||
it('transitions the pending digest to completion and applied-digest markers', () => {
|
||||
const db = fixture()
|
||||
applyBackfill(db)
|
||||
const pending = db
|
||||
.prepare("SELECT value FROM system_options WHERE key = 'id_normalization_pending_artifact_digest'")
|
||||
.get() as { value: string }
|
||||
finalizeBackfill(db)
|
||||
expect(db.prepare("SELECT value FROM system_options WHERE key = 'id_normalization_version'").get()).toEqual({
|
||||
value: '1',
|
||||
})
|
||||
expect(db.prepare("SELECT value FROM system_options WHERE key = 'id_normalization_applied_artifact_digest'").get()).toEqual({
|
||||
value: pending.value,
|
||||
})
|
||||
expect(
|
||||
db.prepare("SELECT value FROM system_options WHERE key = 'id_normalization_pending_artifact_digest'").get(),
|
||||
).toBeUndefined()
|
||||
expect(() => createBackfillPlan(db)).toThrow('id_backfill_already_finalized:1')
|
||||
expect(() => finalizeBackfill(db)).toThrow('id_backfill_already_finalized:1')
|
||||
db.close()
|
||||
})
|
||||
|
||||
it('allows runtime bootstrap during the local observation window', async () => {
|
||||
const sqlite = new Database(':memory:')
|
||||
migrate(drizzle(sqlite), { migrationsFolder: 'migrations' })
|
||||
sqlite.exec(`
|
||||
INSERT INTO organization (id, name, slug, created_at) VALUES ('legacy_org', 'Legacy', 'legacy', 1);
|
||||
`)
|
||||
applyBackfill(sqlite)
|
||||
|
||||
await expect(assertIdIntegrity(drizzle(sqlite))).resolves.toBeUndefined()
|
||||
sqlite.close()
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,725 @@
|
||||
import { createHash } from 'node:crypto'
|
||||
import type Database from 'better-sqlite3'
|
||||
import {
|
||||
ID_NORMALIZATION_DATA_TABLES,
|
||||
INVALIDATED_CREDENTIAL_TABLES,
|
||||
OWNED_ID_TABLES,
|
||||
} from '../shared/id-normalization-inventory'
|
||||
import { DEFAULT_ID_LENGTH, generateToken, isBase62 } from '../shared/ids'
|
||||
|
||||
const MAP_TABLE = '_zpan_id_backfill_map'
|
||||
const ID_NAMESPACE = 'id'
|
||||
const COMPLETION_KEY = 'id_normalization_version'
|
||||
const COMPLETION_VERSION = '1'
|
||||
const PENDING_DIGEST_KEY = 'id_normalization_pending_artifact_digest'
|
||||
const APPLIED_DIGEST_KEY = 'id_normalization_applied_artifact_digest'
|
||||
const D1_MAX_ARTIFACT_STATEMENTS = 47
|
||||
const REDIRECT_TOKEN_NAMESPACES = new Set(['token:shares.token', 'token:image_hostings.token'])
|
||||
|
||||
const LOCAL_REFERENCE_COLUMNS = [
|
||||
['matters', 'org_id'], ['matters', 'storage_id'], ['org_quotas', 'org_id'],
|
||||
['account', 'user_id'], ['member', 'organization_id'], ['member', 'user_id'],
|
||||
['invitation', 'organization_id'], ['invitation', 'inviter_id'],
|
||||
['invite_codes', 'created_by'], ['invite_codes', 'used_by'],
|
||||
['team_invite_links', 'organization_id'], ['team_invite_links', 'inviter_id'],
|
||||
['shares', 'matter_id'], ['shares', 'org_id'], ['shares', 'creator_id'],
|
||||
['share_recipients', 'share_id'], ['share_recipients', 'recipient_user_id'],
|
||||
['notifications', 'user_id'], ['notifications', 'ref_id'],
|
||||
['image_hosting_configs', 'org_id'], ['image_hostings', 'org_id'], ['image_hostings', 'storage_id'],
|
||||
['apikey', 'reference_id'], ['site_invitations', 'invited_by'], ['site_invitations', 'accepted_by'],
|
||||
['site_invitations', 'revoked_by'], ['announcements', 'created_by'],
|
||||
['org_quota_entitlements', 'org_id'], ['cloud_traffic_reports', 'org_id'],
|
||||
['cloud_traffic_reports', 'storage_id'], ['background_jobs', 'org_id'], ['background_jobs', 'user_id'],
|
||||
['background_jobs', 'retried_from_job_id'], ['webdav_dead_properties', 'org_id'], ['webdav_locks', 'org_id'],
|
||||
['download_tasks', 'org_id'], ['download_tasks', 'created_by_user_id'],
|
||||
['download_tasks', 'assigned_downloader_id'], ['download_tasks', 'result_object_id'],
|
||||
['remote_download_usage_reports', 'org_id'], ['remote_download_usage_reports', 'downloader_id'],
|
||||
['remote_download_usage_reports', 'task_id'], ['object_upload_sessions', 'org_id'],
|
||||
['object_upload_sessions', 'object_id'], ['object_upload_sessions', 'storage_id'],
|
||||
['object_upload_sessions', 'created_by'],
|
||||
['downloaders', 'created_by'], ['stats_rollups_hourly', 'org_id'], ['audit_events', 'org_id'],
|
||||
['audit_events', 'user_id'], ['storage_usage_ledger', 'org_id'], ['storage_usage_ledger', 'storage_id'],
|
||||
['storage_usage_breakdowns', 'org_id'], ['resource_changes', 'scope_id'],
|
||||
['resource_changes', 'resource_id'], ['oauthClient', 'user_id'], ['x402_capacity_purchase_intents', 'org_id'],
|
||||
] as const
|
||||
|
||||
const GENERAL_LOCAL_JSON_ID_KEYS = [
|
||||
'shareId', 'matterId', 'storageId', 'userId', 'workspaceId', 'orgId', 'organizationId',
|
||||
'downloaderId', 'taskId', 'objectId', 'imageId', 'resultObjectId', 'recipientUserId', 'scopeId',
|
||||
'jobId', 'sessionId', 'entitlementId',
|
||||
] as const
|
||||
|
||||
export const TOKEN_COLUMNS = [
|
||||
{ table: 'matters', column: 'alias', length: 11 },
|
||||
{ table: 'invite_codes', column: 'code', length: 8 },
|
||||
{ table: 'site_invitations', column: 'token', length: 33 },
|
||||
{ table: 'team_invite_links', column: 'token', length: 32 },
|
||||
{ table: 'shares', column: 'token', length: 11 },
|
||||
{ table: 'image_hostings', column: 'token', length: 12 },
|
||||
{ table: 'image_hosting_configs', column: 'verification_token', length: 33 },
|
||||
{ table: 'downloaders', column: 'token_jti', length: DEFAULT_ID_LENGTH },
|
||||
] as const
|
||||
|
||||
const JSON_COLUMNS = [
|
||||
{ table: 'notifications', key: 'id', columns: ['metadata'], idKeys: ['shareId', 'matterId', 'jobId'], arrayIdKeys: [], shareTokenKeys: ['token'] },
|
||||
{ table: 'background_jobs', key: 'id', columns: ['metadata', 'result_metadata'], idKeys: [...GENERAL_LOCAL_JSON_ID_KEYS], arrayIdKeys: ['matterIds'], shareTokenKeys: [] },
|
||||
{ table: 'download_tasks', key: 'id', columns: ['events'], idKeys: [...GENERAL_LOCAL_JSON_ID_KEYS], arrayIdKeys: ['matterIds'], shareTokenKeys: [] },
|
||||
{ table: 'audit_events', key: 'id', columns: ['metadata'], idKeys: [...GENERAL_LOCAL_JSON_ID_KEYS, 'sourceId'], arrayIdKeys: ['matterIds'], shareTokenKeys: [] },
|
||||
{ table: 'resource_changes', key: 'sequence', columns: ['metadata'], idKeys: [...GENERAL_LOCAL_JSON_ID_KEYS], arrayIdKeys: ['matterIds'], shareTokenKeys: [] },
|
||||
{ table: 'stats_rollups_hourly', key: 'id', columns: ['metadata'], idKeys: [...GENERAL_LOCAL_JSON_ID_KEYS], arrayIdKeys: [], shareTokenKeys: [] },
|
||||
{ table: 'org_quota_entitlements', key: 'id', columns: ['metadata'], idKeys: ['grantedBy', 'updatedBy', 'revokedBy'], arrayIdKeys: [], shareTokenKeys: [] },
|
||||
{ table: 'apikey', key: 'id', columns: ['metadata'], idKeys: ['workspaceId', 'orgId', 'userId'], arrayIdKeys: [], shareTokenKeys: [] },
|
||||
] as const
|
||||
|
||||
export function idBackfillDataTables(): string[] {
|
||||
return [
|
||||
...new Set([
|
||||
...OWNED_ID_TABLES,
|
||||
...INVALIDATED_CREDENTIAL_TABLES,
|
||||
...TOKEN_COLUMNS.map(({ table }) => table),
|
||||
...LOCAL_REFERENCE_COLUMNS.map(([table]) => table),
|
||||
...JSON_COLUMNS.map(({ table }) => table),
|
||||
'redirect_token_registry',
|
||||
]),
|
||||
].sort()
|
||||
}
|
||||
|
||||
if (idBackfillDataTables().join('\0') !== [...ID_NORMALIZATION_DATA_TABLES].sort().join('\0')) {
|
||||
throw new Error('id_normalization_inventory_mismatch')
|
||||
}
|
||||
|
||||
export interface BackfillMapping {
|
||||
namespace: string
|
||||
oldValue: string
|
||||
newValue: string
|
||||
}
|
||||
|
||||
export interface BackfillSummary {
|
||||
invalidIds: number
|
||||
invalidTokens: number
|
||||
mappings: number
|
||||
tokenRotations: number
|
||||
credentialsToInvalidate: number
|
||||
jsonDocumentsToRewrite: number
|
||||
ambiguousRedirectTokens: number
|
||||
}
|
||||
|
||||
export interface BackfillPlan {
|
||||
sql: string[]
|
||||
mappings: BackfillMapping[]
|
||||
before: BackfillSummary
|
||||
}
|
||||
|
||||
function ident(value: string): string {
|
||||
return `"${value.replaceAll('"', '""')}"`
|
||||
}
|
||||
|
||||
function literal(value: string): string {
|
||||
return `'${value.replaceAll("'", "''")}'`
|
||||
}
|
||||
|
||||
function tableExists(db: Database.Database, table: string): boolean {
|
||||
return Boolean(db.prepare("SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = ?").get(table))
|
||||
}
|
||||
|
||||
function indexExists(db: Database.Database, index: string): boolean {
|
||||
return Boolean(db.prepare("SELECT 1 FROM sqlite_master WHERE type = 'index' AND name = ?").get(index))
|
||||
}
|
||||
|
||||
function columns(db: Database.Database, table: string): string[] {
|
||||
return (db.prepare(`PRAGMA table_info(${ident(table)})`).all() as Array<{ name: string }>).map((row) => row.name)
|
||||
}
|
||||
|
||||
function tableHasRows(db: Database.Database, table: string, where = ''): boolean {
|
||||
if (!tableExists(db, table)) return false
|
||||
return Boolean(db.prepare(`SELECT 1 FROM ${ident(table)}${where ? ` WHERE ${where}` : ''} LIMIT 1`).get())
|
||||
}
|
||||
|
||||
function columnHasMapping(
|
||||
db: Database.Database,
|
||||
table: string,
|
||||
column: string,
|
||||
mappings: readonly BackfillMapping[],
|
||||
namespace = ID_NAMESPACE,
|
||||
): boolean {
|
||||
if (!tableExists(db, table) || !columns(db, table).includes(column)) return false
|
||||
const oldValues = new Set(mappings.filter((entry) => entry.namespace === namespace).map((entry) => entry.oldValue))
|
||||
if (oldValues.size === 0) return false
|
||||
const values = db
|
||||
.prepare(`SELECT DISTINCT ${ident(column)} AS value FROM ${ident(table)} WHERE ${ident(column)} IS NOT NULL`)
|
||||
.all() as Array<{ value: unknown }>
|
||||
return values.some(({ value }) => typeof value === 'string' && oldValues.has(value))
|
||||
}
|
||||
|
||||
function existingMappings(db: Database.Database): BackfillMapping[] {
|
||||
if (!tableExists(db, MAP_TABLE)) return []
|
||||
return db
|
||||
.prepare(`SELECT namespace, old_value AS oldValue, new_value AS newValue FROM ${ident(MAP_TABLE)}`)
|
||||
.all() as BackfillMapping[]
|
||||
}
|
||||
|
||||
function assertNotFinalized(db: Database.Database): void {
|
||||
if (!tableExists(db, 'system_options')) return
|
||||
const marker = db.prepare('SELECT value FROM system_options WHERE key = ?').get(COMPLETION_KEY) as
|
||||
| { value: string }
|
||||
| undefined
|
||||
if (marker) throw new Error(`id_backfill_already_finalized:${marker.value}`)
|
||||
}
|
||||
|
||||
export function backfillPlanDigest(plan: Pick<BackfillPlan, 'sql'>): string {
|
||||
return createHash('sha256').update(JSON.stringify(plan.sql)).digest('hex')
|
||||
}
|
||||
|
||||
export function pendingBackfillDigest(db: Database.Database): string | undefined {
|
||||
if (!tableExists(db, 'system_options')) return undefined
|
||||
return (
|
||||
db.prepare('SELECT value FROM system_options WHERE key = ?').get(PENDING_DIGEST_KEY) as
|
||||
| { value: string }
|
||||
| undefined
|
||||
)?.value
|
||||
}
|
||||
|
||||
function invalidValues(db: Database.Database, table: string, column: string): string[] {
|
||||
if (!tableExists(db, table) || !columns(db, table).includes(column)) return []
|
||||
const rows = db.prepare(`SELECT DISTINCT ${ident(column)} AS value FROM ${ident(table)}`).all() as Array<{
|
||||
value: unknown
|
||||
}>
|
||||
return rows.flatMap(({ value }) => (typeof value === 'string' && !isBase62(value) ? [value] : []))
|
||||
}
|
||||
|
||||
function collectMappings(db: Database.Database): BackfillMapping[] {
|
||||
const persisted = existingMappings(db)
|
||||
const byKey = new Map(persisted.map((entry) => [`${entry.namespace}\0${entry.oldValue}`, entry]))
|
||||
const used = new Map<string, Set<string>>()
|
||||
const reserve = (namespace: string, value: string): void => {
|
||||
const values = used.get(namespace) ?? new Set<string>()
|
||||
values.add(value)
|
||||
used.set(namespace, values)
|
||||
if (REDIRECT_TOKEN_NAMESPACES.has(namespace)) {
|
||||
const redirectValues = used.get('redirect-token') ?? new Set<string>()
|
||||
redirectValues.add(value)
|
||||
used.set('redirect-token', redirectValues)
|
||||
}
|
||||
}
|
||||
for (const entry of persisted) reserve(entry.namespace, entry.newValue)
|
||||
for (const table of OWNED_ID_TABLES) {
|
||||
if (!tableExists(db, table) || !columns(db, table).includes('id')) continue
|
||||
const rows = db.prepare(`SELECT id FROM ${ident(table)}`).all() as Array<{ id: string }>
|
||||
for (const row of rows) if (isBase62(row.id)) reserve(ID_NAMESPACE, row.id)
|
||||
}
|
||||
const uniqueValue = (namespace: string, length: number): string => {
|
||||
let candidate = generateToken(length)
|
||||
while (used.get(namespace)?.has(candidate) || (REDIRECT_TOKEN_NAMESPACES.has(namespace) && used.get('redirect-token')?.has(candidate))) {
|
||||
candidate = generateToken(length)
|
||||
}
|
||||
reserve(namespace, candidate)
|
||||
if (REDIRECT_TOKEN_NAMESPACES.has(namespace)) reserve('redirect-token', candidate)
|
||||
return candidate
|
||||
}
|
||||
for (const table of OWNED_ID_TABLES) {
|
||||
for (const oldValue of invalidValues(db, table, 'id')) {
|
||||
const key = `${ID_NAMESPACE}\0${oldValue}`
|
||||
if (!byKey.has(key)) {
|
||||
byKey.set(key, { namespace: ID_NAMESPACE, oldValue, newValue: uniqueValue(ID_NAMESPACE, DEFAULT_ID_LENGTH) })
|
||||
}
|
||||
}
|
||||
}
|
||||
for (const token of TOKEN_COLUMNS) {
|
||||
const namespace = `token:${token.table}.${token.column}`
|
||||
const settledValues = new Set(persisted.filter((entry) => entry.namespace === namespace).map((entry) => entry.newValue))
|
||||
if (tableExists(db, token.table)) {
|
||||
const rows = db.prepare(`SELECT ${ident(token.column)} AS value FROM ${ident(token.table)}`).all() as Array<{
|
||||
value: unknown
|
||||
}>
|
||||
const changed: Array<{ rowKey: string | number; document: string }> = []
|
||||
for (const row of rows) {
|
||||
if (typeof row.value !== 'string') continue
|
||||
reserve(namespace, row.value)
|
||||
if (settledValues.has(row.value)) continue
|
||||
const key = `${namespace}\0${row.value}`
|
||||
if (!byKey.has(key)) {
|
||||
byKey.set(key, { namespace, oldValue: row.value, newValue: uniqueValue(namespace, token.length) })
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if (tableExists(db, 'system_options')) {
|
||||
const row = db.prepare("SELECT value FROM system_options WHERE key = 'instance_id'").get() as
|
||||
| { value: string }
|
||||
| undefined
|
||||
if (row && !isBase62(row.value)) {
|
||||
const namespace = 'token:system_options.instance_id'
|
||||
const key = `${namespace}\0${row.value}`
|
||||
if (!byKey.has(key)) {
|
||||
byKey.set(key, { namespace, oldValue: row.value, newValue: uniqueValue(namespace, DEFAULT_ID_LENGTH) })
|
||||
}
|
||||
}
|
||||
}
|
||||
return [...byKey.values()]
|
||||
}
|
||||
|
||||
function rewriteJson(
|
||||
value: unknown,
|
||||
replacement: ReadonlyMap<string, string>,
|
||||
shareTokenReplacement: ReadonlyMap<string, string>,
|
||||
idKeys: ReadonlySet<string>,
|
||||
arrayIdKeys: ReadonlySet<string>,
|
||||
shareTokenKeys: ReadonlySet<string>,
|
||||
): unknown {
|
||||
if (Array.isArray(value)) {
|
||||
return value.map((item) =>
|
||||
rewriteJson(item, replacement, shareTokenReplacement, idKeys, arrayIdKeys, shareTokenKeys),
|
||||
)
|
||||
}
|
||||
if (!value || typeof value !== 'object') return value
|
||||
return Object.fromEntries(
|
||||
Object.entries(value).map(([key, item]) => [
|
||||
key,
|
||||
idKeys.has(key) && typeof item === 'string'
|
||||
? replacement.get(item) ?? item
|
||||
: arrayIdKeys.has(key) && Array.isArray(item)
|
||||
? item.map((entry) => (typeof entry === 'string' ? replacement.get(entry) ?? entry : entry))
|
||||
: shareTokenKeys.has(key) && typeof item === 'string'
|
||||
? shareTokenReplacement.get(item) ?? item
|
||||
: rewriteJson(item, replacement, shareTokenReplacement, idKeys, arrayIdKeys, shareTokenKeys),
|
||||
]),
|
||||
)
|
||||
}
|
||||
|
||||
function jsonUpdates(
|
||||
db: Database.Database,
|
||||
mappings: BackfillMapping[],
|
||||
): { statements: string[]; documentCount: number } {
|
||||
const replacement = new Map<string, string>()
|
||||
for (const { oldValue, newValue } of mappings.filter(({ namespace }) => namespace === ID_NAMESPACE)) {
|
||||
const existing = replacement.get(oldValue)
|
||||
if (existing && existing !== newValue) throw new Error('ambiguous_embedded_mapping')
|
||||
replacement.set(oldValue, newValue)
|
||||
}
|
||||
const shareTokenReplacement = new Map(
|
||||
mappings
|
||||
.filter(({ namespace }) => namespace === 'token:shares.token')
|
||||
.map(({ oldValue, newValue }) => [oldValue, newValue]),
|
||||
)
|
||||
const updates: string[] = []
|
||||
let documentCount = 0
|
||||
for (const config of JSON_COLUMNS) {
|
||||
const idKeys = new Set<string>(config.idKeys)
|
||||
const arrayIdKeys = new Set<string>(config.arrayIdKeys)
|
||||
const shareTokenKeys = new Set<string>(config.shareTokenKeys)
|
||||
if (!tableExists(db, config.table)) continue
|
||||
const available = new Set(columns(db, config.table))
|
||||
if (!available.has(config.key)) continue
|
||||
for (const column of config.columns) {
|
||||
if (!available.has(column)) continue
|
||||
const rows = db
|
||||
.prepare(
|
||||
`SELECT ${ident(config.key)} AS rowKey, ${ident(column)} AS document FROM ${ident(config.table)} WHERE ${ident(column)} IS NOT NULL`,
|
||||
)
|
||||
.all() as Array<{ rowKey: string | number; document: string }>
|
||||
const changed: Array<{ rowKey: string | number; document: string }> = []
|
||||
for (const row of rows) {
|
||||
let parsed: unknown
|
||||
try {
|
||||
parsed = JSON.parse(row.document)
|
||||
} catch {
|
||||
throw new Error(`invalid_json:${config.table}.${column}`)
|
||||
}
|
||||
const rewritten = JSON.stringify(
|
||||
rewriteJson(parsed, replacement, shareTokenReplacement, idKeys, arrayIdKeys, shareTokenKeys),
|
||||
)
|
||||
if (rewritten === row.document) continue
|
||||
changed.push({ rowKey: row.rowKey, document: rewritten })
|
||||
documentCount += 1
|
||||
}
|
||||
let batch: typeof changed = []
|
||||
const build = (entries: typeof changed): string => {
|
||||
const cases = entries
|
||||
.map(({ rowKey, document }) => {
|
||||
const key = typeof rowKey === 'number' ? String(rowKey) : literal(rowKey)
|
||||
return `WHEN ${key} THEN ${literal(document)}`
|
||||
})
|
||||
.join(' ')
|
||||
const keys = entries
|
||||
.map(({ rowKey }) => (typeof rowKey === 'number' ? String(rowKey) : literal(rowKey)))
|
||||
.join(', ')
|
||||
return `UPDATE ${ident(config.table)} SET ${ident(column)} = CASE ${ident(config.key)} ${cases} END WHERE ${ident(config.key)} IN (${keys});`
|
||||
}
|
||||
for (const entry of changed) {
|
||||
const candidate = [...batch, entry]
|
||||
if (batch.length > 0 && Buffer.byteLength(build(candidate), 'utf8') > 95_000) {
|
||||
updates.push(build(batch))
|
||||
batch = [entry]
|
||||
} else {
|
||||
batch = candidate
|
||||
}
|
||||
}
|
||||
if (batch.length > 0) updates.push(build(batch))
|
||||
}
|
||||
}
|
||||
return { statements: updates, documentCount }
|
||||
}
|
||||
|
||||
function mappingTableSql(): string {
|
||||
return `CREATE TABLE IF NOT EXISTS ${ident(MAP_TABLE)} (namespace TEXT NOT NULL, old_value TEXT NOT NULL, new_value TEXT NOT NULL, PRIMARY KEY (namespace, old_value), UNIQUE (namespace, new_value));`
|
||||
}
|
||||
|
||||
function mapExpression(column: string, namespace = ID_NAMESPACE): string {
|
||||
return `(SELECT new_value FROM ${ident(MAP_TABLE)} WHERE namespace = ${literal(namespace)} AND old_value = ${ident(column)})`
|
||||
}
|
||||
|
||||
function mapPredicate(column: string, namespace = ID_NAMESPACE): string {
|
||||
return `EXISTS (SELECT 1 FROM ${ident(MAP_TABLE)} WHERE namespace = ${literal(namespace)} AND old_value = ${ident(column)})`
|
||||
}
|
||||
|
||||
function updateSql(db: Database.Database, mappings: BackfillMapping[]): string[] {
|
||||
const sql: string[] = ['PRAGMA defer_foreign_keys = ON;', mappingTableSql()]
|
||||
for (let offset = 0; offset < mappings.length; offset += 500) {
|
||||
const values = mappings
|
||||
.slice(offset, offset + 500)
|
||||
.map((entry) => `(${literal(entry.namespace)}, ${literal(entry.oldValue)}, ${literal(entry.newValue)})`)
|
||||
.join(', ')
|
||||
sql.push(
|
||||
`INSERT OR IGNORE INTO ${ident(MAP_TABLE)} (namespace, old_value, new_value) VALUES ${values};`,
|
||||
)
|
||||
}
|
||||
sql.push(...jsonUpdates(db, mappings).statements)
|
||||
|
||||
if (!tableExists(db, 'audit_events') || !columns(db, 'audit_events').includes('event_key')) {
|
||||
throw new Error('required_migration_missing:audit_events.event_key')
|
||||
}
|
||||
if (!tableExists(db, 'redirect_token_registry')) {
|
||||
throw new Error('required_migration_missing:redirect_token_registry')
|
||||
}
|
||||
if (!indexExists(db, 'redirect_token_registry_kind_resource_id_unique')) {
|
||||
throw new Error('required_migration_missing:redirect_token_registry_kind_resource_id_unique')
|
||||
}
|
||||
if (tableHasRows(db, 'audit_events', "event_key IS NULL AND (id LIKE 'event:%' OR id LIKE 'audit:%')")) {
|
||||
sql.push("UPDATE audit_events SET event_key = id WHERE event_key IS NULL AND (id LIKE 'event:%' OR id LIKE 'audit:%');")
|
||||
}
|
||||
|
||||
const tableUpdates = new Map<string, { assignments: string[]; predicates: string[] }>()
|
||||
const addUpdate = (table: string, column: string, expression: string, predicate: string): void => {
|
||||
const update = tableUpdates.get(table) ?? { assignments: [], predicates: [] }
|
||||
update.assignments.push(`${ident(column)} = CASE WHEN ${predicate} THEN ${expression} ELSE ${ident(column)} END`)
|
||||
update.predicates.push(predicate)
|
||||
tableUpdates.set(table, update)
|
||||
}
|
||||
const addMappedUpdate = (table: string, column: string, namespace = ID_NAMESPACE, extra = ''): void => {
|
||||
if (!columnHasMapping(db, table, column, mappings, namespace)) return
|
||||
const predicate = `${extra ? `${extra} AND ` : ''}${mapPredicate(column, namespace)}`
|
||||
addUpdate(table, column, mapExpression(column, namespace), predicate)
|
||||
}
|
||||
|
||||
for (const [table, column] of LOCAL_REFERENCE_COLUMNS) addMappedUpdate(table, column)
|
||||
addMappedUpdate(
|
||||
'audit_events',
|
||||
'target_id',
|
||||
ID_NAMESPACE,
|
||||
"target_type IN ('team','user','file','folder','share','image','remote_download','quota')",
|
||||
)
|
||||
addMappedUpdate(
|
||||
'audit_events',
|
||||
'actor_ref',
|
||||
ID_NAMESPACE,
|
||||
"actor_type IN ('api_key','downloader','task-upload')",
|
||||
)
|
||||
addMappedUpdate(
|
||||
'storage_usage_ledger',
|
||||
'resource_id',
|
||||
ID_NAMESPACE,
|
||||
"resource_type IN ('matter','image_hosting','storage')",
|
||||
)
|
||||
addMappedUpdate(
|
||||
'cloud_traffic_reports',
|
||||
'source_id',
|
||||
ID_NAMESPACE,
|
||||
"source IN ('object_download','webdav_download','direct_share','landing_share','image_hosting','custom_domain_image')",
|
||||
)
|
||||
const idOldValues = mappings.filter(({ namespace }) => namespace === ID_NAMESPACE).map(({ oldValue }) => oldValue)
|
||||
for (const table of ['audit_events', 'storage_usage_ledger'].filter((table) => {
|
||||
if (!tableHasRows(db, table)) return false
|
||||
const eventKeys = db
|
||||
.prepare(`SELECT COALESCE(event_key, id) AS value FROM ${ident(table)}`)
|
||||
.all() as Array<{ value: string }>
|
||||
return eventKeys.some(({ value }) => idOldValues.some((oldValue) => value.includes(oldValue)))
|
||||
})) {
|
||||
sql.push(rewriteStructuredReferencesSql(table, 'event_key'))
|
||||
}
|
||||
if (tableHasRows(db, 'object_upload_sessions', "created_by LIKE 'downloader:%'")) {
|
||||
const predicate = `created_by LIKE 'downloader:%' AND EXISTS (
|
||||
SELECT 1 FROM ${ident(MAP_TABLE)}
|
||||
WHERE namespace = ${literal(ID_NAMESPACE)} AND old_value = substr(object_upload_sessions.created_by, 12)
|
||||
)`
|
||||
addUpdate(
|
||||
'object_upload_sessions',
|
||||
'created_by',
|
||||
`'downloader:' || (
|
||||
SELECT new_value FROM ${ident(MAP_TABLE)}
|
||||
WHERE namespace = ${literal(ID_NAMESPACE)} AND old_value = substr(object_upload_sessions.created_by, 12)
|
||||
)
|
||||
`,
|
||||
predicate,
|
||||
)
|
||||
}
|
||||
|
||||
for (const table of OWNED_ID_TABLES) {
|
||||
addMappedUpdate(table, 'id')
|
||||
}
|
||||
for (const token of TOKEN_COLUMNS) {
|
||||
const namespace = `token:${token.table}.${token.column}`
|
||||
addMappedUpdate(token.table, token.column, namespace)
|
||||
}
|
||||
addMappedUpdate('system_options', 'value', 'token:system_options.instance_id', "key = 'instance_id'")
|
||||
addMappedUpdate('license_bindings', 'instance_id', 'token:system_options.instance_id')
|
||||
if (tableHasRows(db, 'downloaders', 'enabled != 0')) {
|
||||
addUpdate('downloaders', 'enabled', '0', 'enabled != 0')
|
||||
addUpdate('downloaders', 'status', "'offline'", 'enabled != 0')
|
||||
}
|
||||
if (tableHasRows(db, 'license_bindings')) {
|
||||
const available = new Set(columns(db, 'license_bindings'))
|
||||
addUpdate('license_bindings', 'status', "'disconnected'", '1')
|
||||
for (const column of ['refresh_token', 'cached_certificate', 'cached_certificate_expires_at']) {
|
||||
if (available.has(column)) addUpdate('license_bindings', column, 'NULL', '1')
|
||||
}
|
||||
}
|
||||
if (tableHasRows(db, 'account')) {
|
||||
const available = new Set(columns(db, 'account'))
|
||||
const credentialColumns = [
|
||||
'access_token',
|
||||
'refresh_token',
|
||||
'id_token',
|
||||
'access_token_expires_at',
|
||||
'refresh_token_expires_at',
|
||||
'scope',
|
||||
].filter((column) => available.has(column))
|
||||
for (const column of credentialColumns) addUpdate('account', column, 'NULL', '1')
|
||||
}
|
||||
if (tableHasRows(db, 'oauthResource', 'signing_key_id IS NOT NULL')) {
|
||||
addUpdate('oauthResource', 'signing_key_id', 'NULL', 'signing_key_id IS NOT NULL')
|
||||
}
|
||||
for (const [table, update] of tableUpdates) {
|
||||
sql.push(
|
||||
`UPDATE ${ident(table)} SET ${update.assignments.join(', ')} WHERE ${update.predicates.map((predicate) => `(${predicate})`).join(' OR ')};`,
|
||||
)
|
||||
}
|
||||
if (tableExists(db, 'redirect_token_registry')) {
|
||||
if (tableHasRows(db, 'redirect_token_registry')) sql.push('DELETE FROM redirect_token_registry;')
|
||||
if (tableHasRows(db, 'shares', "kind = 'direct'")) {
|
||||
sql.push("INSERT INTO redirect_token_registry (token, kind, resource_id) SELECT token, 'direct_share', id FROM shares WHERE kind = 'direct';")
|
||||
}
|
||||
if (tableHasRows(db, 'image_hostings')) {
|
||||
sql.push("INSERT INTO redirect_token_registry (token, kind, resource_id) SELECT token, 'image_hosting', id FROM image_hostings;")
|
||||
}
|
||||
}
|
||||
for (const table of INVALIDATED_CREDENTIAL_TABLES) {
|
||||
if (tableHasRows(db, table)) sql.push(`DELETE FROM ${ident(table)};`)
|
||||
}
|
||||
return sql
|
||||
}
|
||||
|
||||
function rewriteStructuredReferencesSql(table: string, column: string): string {
|
||||
const delimited = `namespace = ${literal(ID_NAMESPACE)} AND instr(rewritten.value, ':' || old_value || ':') > 0`
|
||||
const suffixed = `namespace = ${literal(ID_NAMESPACE)} AND substr(rewritten.value, -length(old_value) - 1) = ':' || old_value`
|
||||
return `WITH RECURSIVE rewritten(row_key, value, step) AS (
|
||||
SELECT rowid, ${ident(column)}, 0 FROM ${ident(table)} WHERE ${ident(column)} IS NOT NULL
|
||||
UNION ALL
|
||||
SELECT row_key,
|
||||
CASE
|
||||
WHEN EXISTS (SELECT 1 FROM ${ident(MAP_TABLE)} WHERE ${delimited}) THEN replace(
|
||||
value,
|
||||
':' || (SELECT old_value FROM ${ident(MAP_TABLE)} WHERE ${delimited} LIMIT 1) || ':',
|
||||
':' || (SELECT new_value FROM ${ident(MAP_TABLE)} WHERE ${delimited} LIMIT 1) || ':'
|
||||
)
|
||||
WHEN EXISTS (SELECT 1 FROM ${ident(MAP_TABLE)} WHERE ${suffixed}) THEN
|
||||
substr(value, 1, length(value) - length((SELECT old_value FROM ${ident(MAP_TABLE)} WHERE ${suffixed} LIMIT 1))) ||
|
||||
(SELECT new_value FROM ${ident(MAP_TABLE)} WHERE ${suffixed} LIMIT 1)
|
||||
ELSE value
|
||||
END,
|
||||
step + 1
|
||||
FROM rewritten WHERE step < 8
|
||||
)
|
||||
UPDATE ${ident(table)} SET ${ident(column)} = (
|
||||
SELECT value FROM rewritten WHERE row_key = ${ident(table)}.rowid ORDER BY step DESC LIMIT 1
|
||||
) WHERE ${ident(column)} IS NOT NULL;`
|
||||
}
|
||||
|
||||
function countRows(db: Database.Database, table: string): number {
|
||||
if (!tableExists(db, table)) return 0
|
||||
return (db.prepare(`SELECT COUNT(*) AS count FROM ${ident(table)}`).get() as { count: number }).count
|
||||
}
|
||||
|
||||
function countInvalid(db: Database.Database, table: string, column: string): number {
|
||||
return invalidValues(db, table, column).length
|
||||
}
|
||||
|
||||
function preservedRowCounts(db: Database.Database): Map<string, number> {
|
||||
const invalidated = new Set<string>(INVALIDATED_CREDENTIAL_TABLES)
|
||||
const derived = new Set(['redirect_token_registry'])
|
||||
const rows = db
|
||||
.prepare("SELECT name FROM sqlite_master WHERE type = 'table' AND name NOT LIKE 'sqlite_%'")
|
||||
.all() as Array<{ name: string }>
|
||||
return new Map(
|
||||
rows
|
||||
.filter(({ name }) => name !== MAP_TABLE && !invalidated.has(name) && !derived.has(name))
|
||||
.map(({ name }) => [name, countRows(db, name)]),
|
||||
)
|
||||
}
|
||||
|
||||
function assertPreservedRowCounts(db: Database.Database, before: ReadonlyMap<string, number>): void {
|
||||
for (const [table, count] of before) {
|
||||
const after = countRows(db, table)
|
||||
if (after !== count) throw new Error(`row_count_changed:${table}:${count}:${after}`)
|
||||
}
|
||||
}
|
||||
|
||||
export function inspectBackfill(db: Database.Database): BackfillSummary {
|
||||
const invalidIds = OWNED_ID_TABLES.reduce((count, table) => count + countInvalid(db, table, 'id'), 0)
|
||||
let invalidTokens = TOKEN_COLUMNS.reduce((count, token) => count + countInvalid(db, token.table, token.column), 0)
|
||||
if (tableExists(db, 'system_options')) {
|
||||
const instance = db.prepare("SELECT value FROM system_options WHERE key = 'instance_id'").get() as
|
||||
| { value: string }
|
||||
| undefined
|
||||
if (instance && !isBase62(instance.value)) invalidTokens += 1
|
||||
}
|
||||
let credentialsToInvalidate = INVALIDATED_CREDENTIAL_TABLES.reduce(
|
||||
(count, table) => count + countRows(db, table),
|
||||
0,
|
||||
)
|
||||
if (tableExists(db, 'account')) {
|
||||
const available = new Set(columns(db, 'account'))
|
||||
const credentialColumns = ['access_token', 'refresh_token', 'id_token'].filter((column) => available.has(column))
|
||||
if (credentialColumns.length > 0) {
|
||||
credentialsToInvalidate += (
|
||||
db
|
||||
.prepare(`SELECT COUNT(*) AS count FROM account WHERE ${credentialColumns.map((column) => `${ident(column)} IS NOT NULL`).join(' OR ')}`)
|
||||
.get() as { count: number }
|
||||
).count
|
||||
}
|
||||
}
|
||||
if (tableExists(db, 'downloaders') && columns(db, 'downloaders').includes('enabled')) {
|
||||
credentialsToInvalidate += (
|
||||
db.prepare('SELECT COUNT(*) AS count FROM downloaders WHERE enabled != 0').get() as { count: number }
|
||||
).count
|
||||
}
|
||||
if (tableExists(db, 'license_bindings') && columns(db, 'license_bindings').includes('status')) {
|
||||
const available = new Set(columns(db, 'license_bindings'))
|
||||
const credentialColumns = ['refresh_token', 'cached_certificate', 'cached_certificate_expires_at'].filter((column) =>
|
||||
available.has(column),
|
||||
)
|
||||
const conditions = ["status != 'disconnected'", ...credentialColumns.map((column) => `${ident(column)} IS NOT NULL`)]
|
||||
credentialsToInvalidate += (
|
||||
db.prepare(`SELECT COUNT(*) AS count FROM license_bindings WHERE ${conditions.join(' OR ')}`).get() as { count: number }
|
||||
).count
|
||||
}
|
||||
const mappings = existingMappings(db).length
|
||||
const jsonDocumentsToRewrite = jsonUpdates(db, collectMappings(db)).documentCount
|
||||
let ambiguousRedirectTokens = 0
|
||||
if (tableExists(db, 'shares') && tableExists(db, 'image_hostings')) {
|
||||
ambiguousRedirectTokens = (
|
||||
db
|
||||
.prepare(
|
||||
"SELECT COUNT(*) AS count FROM shares s INNER JOIN image_hostings i ON i.token = s.token WHERE s.kind = 'direct' AND s.status = 'active' AND i.status = 'active'",
|
||||
)
|
||||
.get() as { count: number }
|
||||
).count
|
||||
}
|
||||
const tokenRotations = existingMappings(db).filter(({ namespace }) => namespace.startsWith('token:')).length
|
||||
return { invalidIds, invalidTokens, mappings, tokenRotations, credentialsToInvalidate, jsonDocumentsToRewrite, ambiguousRedirectTokens }
|
||||
}
|
||||
|
||||
export function createBackfillPlan(db: Database.Database): BackfillPlan {
|
||||
assertNotFinalized(db)
|
||||
const mappings = collectMappings(db)
|
||||
const before = {
|
||||
...inspectBackfill(db),
|
||||
mappings: mappings.length,
|
||||
tokenRotations: mappings.filter(({ namespace }) => namespace.startsWith('token:')).length,
|
||||
}
|
||||
const sql = updateSql(db, mappings)
|
||||
if (sql.length > D1_MAX_ARTIFACT_STATEMENTS) {
|
||||
throw new Error(`d1_query_limit_exceeded:${sql.length}:${D1_MAX_ARTIFACT_STATEMENTS}`)
|
||||
}
|
||||
const oversized = sql.find((statement) => Buffer.byteLength(statement, 'utf8') > 100_000)
|
||||
if (oversized) throw new Error('d1_statement_limit_exceeded')
|
||||
return { sql, mappings, before }
|
||||
}
|
||||
|
||||
export function verifyBackfill(db: Database.Database): BackfillSummary {
|
||||
const summary = inspectBackfill(db)
|
||||
const integrity = db.pragma('integrity_check') as Array<{ integrity_check: string }>
|
||||
if (integrity.length !== 1 || integrity[0]?.integrity_check !== 'ok') {
|
||||
throw new Error(`integrity_check_failed:${integrity.length}`)
|
||||
}
|
||||
const foreignKeys = db.pragma('foreign_key_check') as unknown[]
|
||||
if (foreignKeys.length > 0) throw new Error(`foreign_key_check_failed:${foreignKeys.length}`)
|
||||
if (summary.invalidIds > 0) throw new Error(`invalid_ids_remaining:${summary.invalidIds}`)
|
||||
if (summary.invalidTokens > 0) throw new Error(`invalid_tokens_remaining:${summary.invalidTokens}`)
|
||||
if (summary.ambiguousRedirectTokens > 0) {
|
||||
throw new Error(`ambiguous_redirect_tokens:${summary.ambiguousRedirectTokens}`)
|
||||
}
|
||||
return summary
|
||||
}
|
||||
|
||||
export function applyBackfill(db: Database.Database, requestedPlan?: BackfillPlan): BackfillSummary {
|
||||
assertNotFinalized(db)
|
||||
const pendingDigest = pendingBackfillDigest(db)
|
||||
if (pendingDigest) {
|
||||
if (requestedPlan && backfillPlanDigest(requestedPlan) !== pendingDigest) {
|
||||
throw new Error(`id_backfill_different_artifact_pending:${pendingDigest}`)
|
||||
}
|
||||
return verifyBackfill(db)
|
||||
}
|
||||
const plan = requestedPlan ?? createBackfillPlan(db)
|
||||
const digest = backfillPlanDigest(plan)
|
||||
const rowCounts = preservedRowCounts(db)
|
||||
const run = db.transaction(() => {
|
||||
db.pragma('defer_foreign_keys = ON')
|
||||
for (const statement of plan.sql) db.exec(statement)
|
||||
const summary = verifyBackfill(db)
|
||||
assertPreservedRowCounts(db, rowCounts)
|
||||
db.prepare('INSERT INTO system_options (key, value) VALUES (?, ?)').run(PENDING_DIGEST_KEY, digest)
|
||||
return summary
|
||||
})
|
||||
return run()
|
||||
}
|
||||
|
||||
export function rollbackBackfill(db: Database.Database): BackfillSummary {
|
||||
assertNotFinalized(db)
|
||||
if (!tableExists(db, MAP_TABLE)) throw new Error('backfill_mapping_missing')
|
||||
const rowCounts = preservedRowCounts(db)
|
||||
const mappings = existingMappings(db).map((entry) => ({
|
||||
namespace: entry.namespace,
|
||||
oldValue: entry.newValue,
|
||||
newValue: entry.oldValue,
|
||||
}))
|
||||
const plan = { sql: updateSql(db, mappings), mappings, before: inspectBackfill(db) }
|
||||
plan.sql.splice(1, 0, `DROP TABLE ${ident(MAP_TABLE)};`)
|
||||
const run = db.transaction(() => {
|
||||
db.pragma('defer_foreign_keys = ON')
|
||||
for (const statement of plan.sql) db.exec(statement)
|
||||
db.exec(`DROP TABLE ${ident(MAP_TABLE)}`)
|
||||
const foreignKeys = db.pragma('foreign_key_check') as unknown[]
|
||||
if (foreignKeys.length > 0) throw new Error(`foreign_key_check_failed:${foreignKeys.length}`)
|
||||
assertPreservedRowCounts(db, rowCounts)
|
||||
db.prepare('DELETE FROM system_options WHERE key = ?').run(PENDING_DIGEST_KEY)
|
||||
return inspectBackfill(db)
|
||||
})
|
||||
return run()
|
||||
}
|
||||
|
||||
export function finalizeBackfill(db: Database.Database): void {
|
||||
assertNotFinalized(db)
|
||||
if (!tableExists(db, 'system_options')) throw new Error('system_options_missing')
|
||||
const pendingDigest = pendingBackfillDigest(db)
|
||||
if (!pendingDigest || !/^[0-9a-f]{64}$/.test(pendingDigest)) throw new Error('id_backfill_pending_digest_missing')
|
||||
const run = db.transaction(() => {
|
||||
verifyBackfill(db)
|
||||
if (!tableExists(db, MAP_TABLE)) throw new Error('backfill_mapping_missing')
|
||||
db.prepare('INSERT INTO system_options (key, value) VALUES (?, ?)').run(COMPLETION_KEY, COMPLETION_VERSION)
|
||||
db.prepare('INSERT INTO system_options (key, value) VALUES (?, ?)').run(APPLIED_DIGEST_KEY, pendingDigest)
|
||||
db.prepare('DELETE FROM system_options WHERE key = ?').run(PENDING_DIGEST_KEY)
|
||||
db.exec(`DROP TABLE ${ident(MAP_TABLE)}`)
|
||||
})
|
||||
run()
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
import { readFileSync } from 'node:fs'
|
||||
import { execFileSync } from 'node:child_process'
|
||||
|
||||
const roots = ['server', 'shared', 'src', 'workers', 'scripts']
|
||||
const files = execFileSync('git', ['ls-files', ...roots], { encoding: 'utf8' })
|
||||
.trim()
|
||||
.split('\n')
|
||||
.filter((file) => file && !/\.(?:test|cf-test|libsql-test)\.[jt]sx?$/.test(file))
|
||||
|
||||
const violations = []
|
||||
const customAlphabetAllowlist = new Set(['shared/ids.ts', 'shared/org-slugs.ts', 'server/auth.ts'])
|
||||
for (const file of files) {
|
||||
const source = readFileSync(file, 'utf8')
|
||||
if (/import\s*\{[^}]*\bnanoid\b[^}]*\}\s*from\s*['"]nanoid['"]/.test(source)) {
|
||||
violations.push(`${file}: imports the default nanoid generator`)
|
||||
}
|
||||
if (
|
||||
/import\s*\{[^}]*\bcustomAlphabet\b[^}]*\}\s*from\s*['"]nanoid['"]/.test(source) &&
|
||||
!customAlphabetAllowlist.has(file)
|
||||
) {
|
||||
violations.push(`${file}: imports customAlphabet outside the reviewed generator allowlist`)
|
||||
}
|
||||
}
|
||||
|
||||
if (violations.length > 0) {
|
||||
throw new Error(`Uncontrolled ID generation:\n${violations.join('\n')}`)
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { execFileSync } from 'node:child_process'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
|
||||
describe('ID generation lint', () => {
|
||||
it('rejects default nanoid imports in production sources', () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'zpan-id-lint-'))
|
||||
try {
|
||||
execFileSync('git', ['init', '-q'], { cwd: dir })
|
||||
mkdirSync(join(dir, 'server'))
|
||||
writeFileSync(join(dir, 'server/bad.ts'), "import { nanoid } from 'nanoid'\nexport const id = nanoid()\n")
|
||||
execFileSync('git', ['add', 'server/bad.ts'], { cwd: dir })
|
||||
const script = join(process.cwd(), 'scripts/lint-id-generation.mjs')
|
||||
expect(() => execFileSync('node', [script], { cwd: dir, stdio: 'pipe' })).toThrow(/default nanoid generator/)
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true })
|
||||
}
|
||||
})
|
||||
})
|
||||
@@ -3,7 +3,7 @@ import { mkdtempSync, rmSync, writeFileSync } from 'node:fs'
|
||||
import { tmpdir } from 'node:os'
|
||||
import path from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { generateId } from '../shared/ids'
|
||||
|
||||
export const PREVIEW_ADMIN_EMAIL = 'admin@zpan.space'
|
||||
export const CREDENTIAL_ACCOUNT_ISSUER = 'local:credential'
|
||||
@@ -100,8 +100,8 @@ credential source. Do not commit or print it.`)
|
||||
const sql = buildPreviewAdminSeedSql({
|
||||
email: PREVIEW_ADMIN_EMAIL,
|
||||
passwordHash: hashPassword(password),
|
||||
userId: `preview-admin-${nanoid(12)}`,
|
||||
accountId: `preview-admin-account-${nanoid(12)}`,
|
||||
userId: generateId(),
|
||||
accountId: generateId(),
|
||||
now: Date.now(),
|
||||
})
|
||||
|
||||
|
||||
@@ -177,7 +177,7 @@ const HOURLY_SOURCES: readonly HourlySource[] = [
|
||||
where: `registered_user.action = 'user_register'
|
||||
AND registered_user.target_id IS NOT NULL
|
||||
AND registered_user.user_id = registered_user.target_id
|
||||
AND registered_user.id = 'event:user_register:' || registered_user.target_id
|
||||
AND registered_user.event_key = 'event:user_register:' || registered_user.target_id
|
||||
AND json_valid(registered_user.metadata) = 1
|
||||
AND json_type(registered_user.metadata, '$.provider') = 'text'
|
||||
AND length(json_extract(registered_user.metadata, '$.provider')) > 0`,
|
||||
@@ -216,9 +216,7 @@ export function buildAdminStatsCounterRollupInsertSqlStatements(range: AdminStat
|
||||
count, bytes, unique_count, metadata, updated_at
|
||||
)
|
||||
SELECT
|
||||
CAST(bucket_start AS TEXT) || ':' || COALESCE(NULLIF(org_id, ''), 'global') || ':' ||
|
||||
metric_key || ':' || COALESCE(NULLIF(dimension_key, ''), 'all') || ':' ||
|
||||
COALESCE(NULLIF(hex(dimension_value), ''), 'all'),
|
||||
lower(hex(json_array(bucket_start, org_id, metric_key, dimension_key, dimension_value))),
|
||||
bucket_start, org_id, metric_key, dimension_key, dimension_value,
|
||||
count, bytes, unique_count,
|
||||
json_object('version', ${ROLLUP_VERSION}, 'scope', 'counters', 'quality', 'exact'),
|
||||
|
||||
@@ -83,7 +83,7 @@ describe('admin stats source integrity', () => {
|
||||
expect(() => assertAdminStatsSourceIntegrity(integrity)).not.toThrow()
|
||||
const rows = await db.all<{ events: number; issuedAt: number }>(sql`
|
||||
SELECT
|
||||
(SELECT COUNT(*) FROM audit_events WHERE id = 'event:download_issued:traffic-integrity-ok') AS events,
|
||||
(SELECT COUNT(*) FROM audit_events WHERE event_key = 'event:download_issued:traffic-integrity-ok') AS events,
|
||||
(SELECT issued_at FROM cloud_traffic_reports WHERE event_id = 'traffic-integrity-ok') AS issuedAt
|
||||
`)
|
||||
expect(rows).toEqual([{ events: 0, issuedAt: issuedAt.getTime() }])
|
||||
@@ -130,7 +130,7 @@ describe('admin stats source integrity', () => {
|
||||
await expect(reports.markIssued('traffic-does-not-exist', now)).rejects.toThrow('traffic_report_not_found')
|
||||
const rows = await db.all<{ issuedAt: number | null; events: number }>(sql`
|
||||
SELECT issued_at AS issuedAt,
|
||||
(SELECT COUNT(*) FROM audit_events WHERE id LIKE 'event:download_issued:traffic-%') AS events
|
||||
(SELECT COUNT(*) FROM audit_events WHERE event_key LIKE 'event:download_issued:traffic-%') AS events
|
||||
FROM cloud_traffic_reports
|
||||
WHERE event_id = 'traffic-activity-mismatch'
|
||||
`)
|
||||
|
||||
@@ -5,7 +5,7 @@ import type { Database } from '../../platform/interface'
|
||||
import { createCloudTrafficReportRepo, trafficLedgerExactFrom } from './cloud-traffic-report'
|
||||
|
||||
const OPENING_SOURCE_ID = 'v3-authoritative-sources'
|
||||
const OPENING_EVENT_ID = `audit:statistics_source_initialized:${OPENING_SOURCE_ID}`
|
||||
const OPENING_EVENT_KEY = `audit:statistics_source_initialized:${OPENING_SOURCE_ID}`
|
||||
const OPENING_OPTION_KEY = 'stats_integrity_exact_from_v3'
|
||||
|
||||
export interface AdminStatsSourceIntegrity {
|
||||
@@ -31,14 +31,14 @@ export async function ensureAdminStatsIntegrityOpening(db: Database, now = new D
|
||||
const legacy = await db
|
||||
.select({ createdAt: auditEvents.createdAt })
|
||||
.from(auditEvents)
|
||||
.where(eq(auditEvents.id, OPENING_EVENT_ID))
|
||||
.where(eq(auditEvents.eventKey, OPENING_EVENT_KEY))
|
||||
.limit(1)
|
||||
const exactFrom = legacy[0]?.createdAt ?? new Date((Math.floor(now.getTime() / 1000) + 1) * 1000)
|
||||
await db
|
||||
.insert(systemOptions)
|
||||
.values({ key: OPENING_OPTION_KEY, value: exactFrom.toISOString() })
|
||||
.onConflictDoNothing({ target: systemOptions.key })
|
||||
await db.delete(auditEvents).where(eq(auditEvents.id, OPENING_EVENT_ID))
|
||||
await db.delete(auditEvents).where(eq(auditEvents.eventKey, OPENING_EVENT_KEY))
|
||||
|
||||
const rows = await db
|
||||
.select({ value: systemOptions.value })
|
||||
@@ -115,7 +115,7 @@ export async function inspectAdminStatsSourceIntegrity(
|
||||
json_valid(metadata) = 0
|
||||
OR target_id IS NULL
|
||||
OR user_id <> target_id
|
||||
OR id <> 'event:user_register:' || target_id
|
||||
OR event_key <> 'event:user_register:' || target_id
|
||||
OR COALESCE(json_type(metadata, '$.provider') = 'text', 0) = 0
|
||||
OR COALESCE(length(json_extract(metadata, '$.provider')), 0) = 0
|
||||
))
|
||||
@@ -166,7 +166,7 @@ export async function inspectAdminStatsSourceIntegrity(
|
||||
SELECT 1
|
||||
FROM audit_events registration_event
|
||||
WHERE registration_event.action = 'user_register'
|
||||
AND registration_event.id = 'event:user_register:' || registered_user.id
|
||||
AND registration_event.event_key = 'event:user_register:' || registered_user.id
|
||||
AND registration_event.user_id = registered_user.id
|
||||
AND registration_event.target_id = registered_user.id
|
||||
AND registration_event.created_at = CAST(registered_user.created_at / 1000 AS INTEGER)
|
||||
|
||||
@@ -67,7 +67,7 @@ describe('admin hourly stats rollup', () => {
|
||||
await db.run(sql`
|
||||
UPDATE audit_events
|
||||
SET created_at = ${atSec}, metadata = '{"provider":"credential"}'
|
||||
WHERE id = ${`event:user_register:${userId}`}
|
||||
WHERE event_key = ${`event:user_register:${userId}`}
|
||||
`)
|
||||
await db.run(sql`UPDATE session SET created_at = ${atMs}, updated_at = ${atMs} WHERE user_id = ${userId}`)
|
||||
await db.run(sql`UPDATE organization SET created_at = ${atMs}, updated_at = ${atMs} WHERE id = ${orgId}`)
|
||||
@@ -166,9 +166,14 @@ describe('admin hourly stats rollup', () => {
|
||||
('rollup-share-created-limited', ${orgId}, ${userId}, 'user', 'share_create', 'share', 'rollup-share-limited', 'rollup-share-limited',
|
||||
'{"kind":"direct"}', ${atSec}),
|
||||
('rollup-job-finished-fact', ${orgId}, NULL, 'system', 'background_job_failed', 'background_job', 'rollup-job-finished', 'rollup-job-finished',
|
||||
'{"jobType":"archive","outcome":"failed"}', ${atSec}),
|
||||
('event:user_register:rollup-direct-user', '', 'rollup-direct-user', 'user', 'user_register', 'user',
|
||||
'rollup-direct-user', 'rollup-direct-user', '{"provider":"unknown"}', ${atSec})
|
||||
'{"jobType":"archive","outcome":"failed"}', ${atSec})
|
||||
`)
|
||||
await db.run(sql`
|
||||
INSERT INTO audit_events
|
||||
(id, event_key, org_id, user_id, actor_type, action, target_type, target_id, target_name, metadata, created_at)
|
||||
VALUES
|
||||
('RollupDirectRegistration', 'event:user_register:rollup-direct-user', '', 'rollup-direct-user', 'user',
|
||||
'user_register', 'user', 'rollup-direct-user', 'rollup-direct-user', '{"provider":"unknown"}', ${atSec})
|
||||
`)
|
||||
await db.run(sql`
|
||||
INSERT INTO object_upload_sessions
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { generateId } from '@shared/ids'
|
||||
import { and, eq, inArray, isNull, ne, or, sql } from 'drizzle-orm'
|
||||
import { organization } from '../../db/auth-schema'
|
||||
import {
|
||||
@@ -133,7 +134,7 @@ export async function rebuildAdminStatsHour(
|
||||
|
||||
const updatedAt = generatedAt
|
||||
const rows = rollups.values().map((row) => ({
|
||||
id: rollupId(bucketStart, row),
|
||||
id: generateId(),
|
||||
bucketStart,
|
||||
orgId: row.orgId,
|
||||
metricKey: row.metric,
|
||||
@@ -198,7 +199,7 @@ export async function captureAdminStatsSnapshot(
|
||||
await addSnapshotMetrics(db, rollups, observedAt)
|
||||
rollups.add(M.statsRollupRun, '', 1, 0, { outcome: 'success' })
|
||||
const rows = rollups.values().map((row) => ({
|
||||
id: rollupId(bucketStart, row),
|
||||
id: generateId(),
|
||||
bucketStart,
|
||||
orgId: row.orgId,
|
||||
metricKey: row.metric,
|
||||
@@ -897,10 +898,6 @@ function key(metric: AdminStatsMetric, orgId: string, dimensionKey: string, dime
|
||||
return `${metric}\u0000${orgId}\u0000${dimensionKey}\u0000${dimensionValue}`
|
||||
}
|
||||
|
||||
function rollupId(bucketStart: Date, row: RollupValue): string {
|
||||
return `${bucketStart.getTime()}:${row.orgId || 'global'}:${row.metric}:${row.dimensionKey || 'all'}:${row.dimensionValue || 'all'}`
|
||||
}
|
||||
|
||||
async function queryStage<T>(name: string, query: PromiseLike<T>): Promise<T> {
|
||||
try {
|
||||
return await query
|
||||
|
||||
@@ -151,7 +151,7 @@ async function getLiveUserSummary(db: Database, now: Date): Promise<{ total: num
|
||||
eq(auditEvents.action, 'user_register'),
|
||||
sql`${auditEvents.targetId} IS NOT NULL`,
|
||||
sql`${auditEvents.userId} = ${auditEvents.targetId}`,
|
||||
sql`${auditEvents.id} = 'event:user_register:' || ${auditEvents.targetId}`,
|
||||
sql`${auditEvents.eventKey} = 'event:user_register:' || ${auditEvents.targetId}`,
|
||||
gte(auditEvents.createdAt, new Date(fromMs)),
|
||||
lte(auditEvents.createdAt, new Date(nowMs)),
|
||||
sql`json_valid(${auditEvents.metadata}) = 1`,
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { generateId } from '@shared/ids'
|
||||
import type { AnnouncementInput } from '@shared/schemas'
|
||||
import { count, desc, eq, getTableColumns, ne, sql } from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { announcements } from '../../db/schema'
|
||||
import type { Database } from '../../platform/interface'
|
||||
import type { AnnouncementRecord, AnnouncementRepo } from '../../usecases/ports'
|
||||
@@ -31,7 +31,7 @@ export function createAnnouncementRepo(db: Database): AnnouncementRepo {
|
||||
async create(input, createdBy) {
|
||||
const now = new Date()
|
||||
const row: AnnouncementRow = {
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
title: input.title,
|
||||
body: input.body,
|
||||
status: input.status,
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { generateId } from '@shared/ids'
|
||||
import { and, count, desc, eq, getTableColumns, gte, lte, sql } from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { organization, user } from '../../db/auth-schema'
|
||||
import { auditEvents } from '../../db/schema'
|
||||
import { assertAuditEvent } from '../../domain/audit-events'
|
||||
@@ -9,7 +9,7 @@ import type { AuditActorType, AuditRepo, RecordAuditEventInput } from '../../use
|
||||
export function auditEventValues(event: RecordAuditEventInput): typeof auditEvents.$inferInsert {
|
||||
assertAuditEvent(event)
|
||||
return {
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
orgId: event.orgId,
|
||||
userId: event.userId ?? null,
|
||||
actorType: event.actorType ?? (event.userId ? 'user' : 'anonymous'),
|
||||
@@ -52,7 +52,7 @@ export function idempotentSystemEventValues(input: {
|
||||
targetName: input.targetName ?? targetId,
|
||||
metadata: input.metadata,
|
||||
}),
|
||||
id: `event:${input.action}:${input.sourceId}`,
|
||||
eventKey: `event:${input.action}:${input.sourceId}`,
|
||||
createdAt: input.occurredAt,
|
||||
}
|
||||
}
|
||||
@@ -96,10 +96,10 @@ export function createAuditRepo(db: Database): AuditRepo {
|
||||
.insert(auditEvents)
|
||||
.values({
|
||||
...auditEventValues(event),
|
||||
id: `event:${event.action}:${idempotencyKey}`,
|
||||
eventKey: `event:${event.action}:${idempotencyKey}`,
|
||||
createdAt: occurredAt,
|
||||
})
|
||||
.onConflictDoNothing({ target: auditEvents.id })
|
||||
.onConflictDoNothing({ target: auditEvents.eventKey })
|
||||
},
|
||||
|
||||
async list(orgId, opts) {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { generateId } from '@shared/ids'
|
||||
import type { BackgroundJob, BackgroundJobStatus } from '@shared/types'
|
||||
import { and, count, desc, eq, inArray, lt, or, type SQL, sql } from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { backgroundJobs } from '../../db/schema'
|
||||
import { executeWriteTransaction } from '../../db/transaction'
|
||||
import type { Database } from '../../platform/interface'
|
||||
@@ -91,7 +91,7 @@ export function createBackgroundJobRepo(db: Database): BackgroundJobRepo {
|
||||
async create(input) {
|
||||
const now = new Date()
|
||||
const row: typeof backgroundJobs.$inferInsert = {
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
orgId: input.orgId,
|
||||
userId: input.userId,
|
||||
type: input.type,
|
||||
@@ -239,7 +239,7 @@ export function createBackgroundJobRepo(db: Database): BackgroundJobRepo {
|
||||
|
||||
const now = new Date()
|
||||
const retry: typeof backgroundJobs.$inferInsert = {
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
orgId: row.orgId,
|
||||
userId: row.userId,
|
||||
type: row.type,
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
import { generateId } from '@shared/ids'
|
||||
import { isPersonalOrgLike } from '@shared/org-slugs'
|
||||
import type { CloudOrderQuotaChange } from '@shared/schemas'
|
||||
import type { CloudStoreTarget } from '@shared/types'
|
||||
import { and, eq, sql } from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { member, organization, user } from '../../db/auth-schema'
|
||||
import { orgQuotaEntitlements, orgQuotas, webhookEvents } from '../../db/schema'
|
||||
import { type AtomicQuery, executeRows, executeWriteTransaction } from '../../db/transaction'
|
||||
@@ -205,7 +205,7 @@ function quotaEntitlementValue(
|
||||
): typeof orgQuotaEntitlements.$inferInsert | null {
|
||||
if (bytes === 0) return null
|
||||
return {
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
orgId: event.targetOrgId,
|
||||
resourceType,
|
||||
entitlementType: entitlementType(event),
|
||||
@@ -272,7 +272,7 @@ async function beginWebhookEvent(
|
||||
rawPayload: string,
|
||||
payloadHash: string,
|
||||
): Promise<{ id: string; duplicate: boolean }> {
|
||||
const id = nanoid()
|
||||
const id = generateId()
|
||||
const inserted = await executeRows(
|
||||
db
|
||||
.insert(webhookEvents)
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { generateId } from '@shared/ids'
|
||||
import { and, asc, eq, isNull, lte, ne, or, sql } from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { cloudTrafficReports } from '../../db/schema'
|
||||
import { executeWriteTransaction } from '../../db/transaction'
|
||||
import { currentTrafficPeriod } from '../../domain/quota'
|
||||
@@ -64,7 +64,7 @@ export function createCloudTrafficReportRepo(db: Database): CloudTrafficReportRe
|
||||
await executeWriteTransaction(db, [
|
||||
ledgerOpeningInsert(db, input.now),
|
||||
db.insert(cloudTrafficReports).values({
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
orgId: input.orgId,
|
||||
period: input.period,
|
||||
source: input.source,
|
||||
@@ -143,7 +143,7 @@ function ledgerOpeningInsert(db: Database, now: Date) {
|
||||
return db
|
||||
.insert(cloudTrafficReports)
|
||||
.values({
|
||||
id: TRAFFIC_LEDGER_OPENING_EVENT_ID,
|
||||
id: generateId(),
|
||||
orgId: '',
|
||||
period: currentTrafficPeriod(now),
|
||||
source: 'object_download',
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { generateId } from '@shared/ids'
|
||||
import { and, eq, gt, isNull, sql } from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { downloaderBootstrapCredential, session } from '../../db/auth-schema'
|
||||
import { downloaders } from '../../db/schema'
|
||||
import { executeRows, executeWriteTransactionWithResults } from '../../db/transaction'
|
||||
@@ -15,7 +15,7 @@ export function createDownloaderBootstrapCredentialRepo(
|
||||
return {
|
||||
async issue(input) {
|
||||
await db.insert(downloaderBootstrapCredential).values({
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
tokenHash: await tokens.hashDownloadToken(input.platform, input.token),
|
||||
userId: input.userId,
|
||||
deviceCode: input.deviceCode,
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { generateId, generateToken } from '@shared/ids'
|
||||
import { and, asc, eq, gt, isNotNull, isNull, like, or, sql } from 'drizzle-orm'
|
||||
import { customAlphabet, nanoid } from 'nanoid'
|
||||
import { imageHostingConfigs, imageHostings } from '../../db/schema'
|
||||
import { imageHostingConfigs, imageHostings, redirectTokenRegistry } from '../../db/schema'
|
||||
import { type AtomicQuery, executeWriteTransaction, executeWriteTransactionWithResults } from '../../db/transaction'
|
||||
import { mimeToExt } from '../../lib/mime-utils'
|
||||
import type { Database } from '../../platform/interface'
|
||||
@@ -10,6 +10,7 @@ import type {
|
||||
ImageHostingRepo,
|
||||
ImageResolution,
|
||||
} from '../../usecases/ports'
|
||||
import { withRedirectToken } from './redirect-token'
|
||||
import { resourceChangeQuery } from './resource-change'
|
||||
import {
|
||||
imageActivationLedgerQuery,
|
||||
@@ -21,8 +22,6 @@ import { imageAddedProjectionQueries, imageRemovedProjectionQueries } from './st
|
||||
type ImageHostingRow = typeof imageHostings.$inferSelect
|
||||
|
||||
const MAX_COLLISION_RETRIES = 5
|
||||
const imageTokenSuffix = customAlphabet('0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz', 10)
|
||||
|
||||
function toRecord(row: ImageHostingRow): ImageHostingRecord {
|
||||
return row as unknown as ImageHostingRecord
|
||||
}
|
||||
@@ -66,8 +65,7 @@ export function createImageHostingRepo(db: Database): ImageHostingRepo {
|
||||
if (conflict.length === 0) return candidate
|
||||
}
|
||||
|
||||
// Exhausted retries — use nanoid suffix as fallback
|
||||
return `${prefix}${stem}-${nanoid(4)}${ext}`
|
||||
return `${prefix}${stem}-${generateToken(5)}${ext}`
|
||||
}
|
||||
|
||||
return {
|
||||
@@ -136,49 +134,72 @@ export function createImageHostingRepo(db: Database): ImageHostingRepo {
|
||||
},
|
||||
|
||||
async create(input: CreateImageHostingInput) {
|
||||
const id = nanoid(12)
|
||||
const token = `ih${imageTokenSuffix()}`
|
||||
const id = generateId(13)
|
||||
const ext = mimeToExt(input.mime)
|
||||
const storageKey = `ih/${input.orgId}/${id}.${ext}`
|
||||
const now = new Date()
|
||||
|
||||
const resolvedPath = await resolveUniquePath(input.orgId, input.path)
|
||||
|
||||
const row: ImageHostingRow = {
|
||||
id,
|
||||
orgId: input.orgId,
|
||||
token,
|
||||
path: resolvedPath,
|
||||
storageId: input.storageId,
|
||||
storageKey,
|
||||
size: input.size,
|
||||
mime: input.mime,
|
||||
width: null,
|
||||
height: null,
|
||||
status: input.status,
|
||||
purgedAt: null,
|
||||
accessCount: 0,
|
||||
lastAccessedAt: null,
|
||||
createdAt: now,
|
||||
}
|
||||
return withRedirectToken(
|
||||
12,
|
||||
async (token) => {
|
||||
const row: ImageHostingRow = {
|
||||
id,
|
||||
orgId: input.orgId,
|
||||
token,
|
||||
path: resolvedPath,
|
||||
storageId: input.storageId,
|
||||
storageKey,
|
||||
size: input.size,
|
||||
mime: input.mime,
|
||||
width: null,
|
||||
height: null,
|
||||
status: input.status,
|
||||
purgedAt: null,
|
||||
accessCount: 0,
|
||||
lastAccessedAt: null,
|
||||
createdAt: now,
|
||||
}
|
||||
|
||||
await executeWriteTransaction(db, [
|
||||
db.insert(imageHostings).values(row),
|
||||
...(row.status === 'active'
|
||||
? [
|
||||
resourceChangeQuery(db, {
|
||||
scopeType: 'organization',
|
||||
scopeId: row.orgId,
|
||||
resourceType: 'image_hosting',
|
||||
resourceId: row.id,
|
||||
changeType: 'upsert',
|
||||
action: 'created',
|
||||
occurredAt: now,
|
||||
}),
|
||||
]
|
||||
: []),
|
||||
])
|
||||
return toRecord(row)
|
||||
await executeWriteTransaction(db, [
|
||||
db.insert(imageHostings).values(row),
|
||||
db.insert(redirectTokenRegistry).values({
|
||||
token: row.token,
|
||||
kind: 'image_hosting',
|
||||
resourceId: row.id,
|
||||
}),
|
||||
...(row.status === 'active'
|
||||
? [
|
||||
resourceChangeQuery(db, {
|
||||
scopeType: 'organization',
|
||||
scopeId: row.orgId,
|
||||
resourceType: 'image_hosting',
|
||||
resourceId: row.id,
|
||||
changeType: 'upsert',
|
||||
action: 'created',
|
||||
occurredAt: now,
|
||||
}),
|
||||
]
|
||||
: []),
|
||||
])
|
||||
return toRecord(row)
|
||||
},
|
||||
async (token) => {
|
||||
const existingImage = await db
|
||||
.select({ id: imageHostings.id })
|
||||
.from(imageHostings)
|
||||
.where(eq(imageHostings.token, token))
|
||||
.limit(1)
|
||||
if (existingImage.length > 0) return true
|
||||
const reservation = await db
|
||||
.select({ token: redirectTokenRegistry.token })
|
||||
.from(redirectTokenRegistry)
|
||||
.where(eq(redirectTokenRegistry.token, token))
|
||||
.limit(1)
|
||||
return reservation.length > 0
|
||||
},
|
||||
)
|
||||
},
|
||||
|
||||
async get(id, orgId) {
|
||||
@@ -283,16 +304,27 @@ export function createImageHostingRepo(db: Database): ImageHostingRepo {
|
||||
const row = existing[0]
|
||||
if (!row) return
|
||||
if (row.status === 'draft') {
|
||||
await db
|
||||
.delete(imageHostings)
|
||||
.where(
|
||||
and(
|
||||
eq(imageHostings.id, id),
|
||||
eq(imageHostings.orgId, orgId),
|
||||
eq(imageHostings.status, 'draft'),
|
||||
isNull(imageHostings.purgedAt),
|
||||
await executeWriteTransaction(db, [
|
||||
db
|
||||
.delete(imageHostings)
|
||||
.where(
|
||||
and(
|
||||
eq(imageHostings.id, id),
|
||||
eq(imageHostings.orgId, orgId),
|
||||
eq(imageHostings.status, 'draft'),
|
||||
isNull(imageHostings.purgedAt),
|
||||
),
|
||||
),
|
||||
)
|
||||
db
|
||||
.delete(redirectTokenRegistry)
|
||||
.where(
|
||||
and(
|
||||
eq(redirectTokenRegistry.resourceId, id),
|
||||
eq(redirectTokenRegistry.kind, 'image_hosting'),
|
||||
sql`NOT EXISTS (SELECT 1 FROM image_hostings WHERE image_hostings.id = ${id})`,
|
||||
),
|
||||
),
|
||||
])
|
||||
return
|
||||
}
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { generateId } from '@shared/ids'
|
||||
import { eq } from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { systemOptions } from '../../db/schema'
|
||||
import type { Database } from '../../platform/interface'
|
||||
import type { InstanceRepo } from '../../usecases/ports'
|
||||
@@ -16,7 +16,7 @@ export function createInstanceRepo(db: Database): InstanceRepo {
|
||||
.limit(1)
|
||||
if (rows[0]?.value) return rows[0].value
|
||||
|
||||
const id = nanoid(21)
|
||||
const id = generateId()
|
||||
await db
|
||||
.insert(systemOptions)
|
||||
.values({ key: INSTANCE_ID_KEY, value: id })
|
||||
|
||||
@@ -22,11 +22,11 @@ describe('generateInviteCodes', () => {
|
||||
expect(uniqueCodes.size).toBe(10)
|
||||
})
|
||||
|
||||
it('each code has an 8-character uppercase alphanumeric code field', async () => {
|
||||
it('each code has an 8-character Base62 code field', async () => {
|
||||
const { db } = await createTestApp()
|
||||
const codes = await createInviteRepo(db).generate('admin-1', 3)
|
||||
for (const code of codes) {
|
||||
expect(code.code).toMatch(/^[0-9A-Z]{8}$/)
|
||||
expect(code.code).toMatch(/^[A-Za-z0-9]{8}$/)
|
||||
}
|
||||
})
|
||||
|
||||
|
||||
@@ -1,18 +1,16 @@
|
||||
import { generateId, generateToken } from '@shared/ids'
|
||||
import { and, count, desc, eq, getTableColumns, gt, isNull, or, sql } from 'drizzle-orm'
|
||||
import { customAlphabet, nanoid } from 'nanoid'
|
||||
import { inviteCodes } from '../../db/schema'
|
||||
import type { Database } from '../../platform/interface'
|
||||
import type { InviteCodeRecord, InviteRepo } from '../../usecases/ports'
|
||||
|
||||
const generateCode = customAlphabet('0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ', 8)
|
||||
|
||||
export function createInviteRepo(db: Database): InviteRepo {
|
||||
return {
|
||||
async generate(adminUserId, quantity, expiresAt) {
|
||||
const now = new Date()
|
||||
const rows: InviteCodeRecord[] = Array.from({ length: quantity }, () => ({
|
||||
id: nanoid(),
|
||||
code: generateCode(),
|
||||
id: generateId(),
|
||||
code: generateToken(8),
|
||||
createdBy: adminUserId,
|
||||
usedBy: null,
|
||||
usedAt: null,
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { generateId } from '@shared/ids'
|
||||
import { eq } from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { licenseBindings } from '../../db/schema'
|
||||
import { executeWriteTransaction } from '../../db/transaction'
|
||||
import type { Database } from '../../platform/interface'
|
||||
@@ -54,7 +54,7 @@ async function createLicenseBinding(db: Database, input: CreateLicenseBindingInp
|
||||
})
|
||||
.where(eq(licenseBindings.status, 'active')),
|
||||
db.insert(licenseBindings).values({
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
cloudBindingId: input.cloudBindingId,
|
||||
cloudStoreId: input.cloudStoreId ?? null,
|
||||
instanceId: input.instanceId,
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { DirType, ObjectStatus } from '@shared/constants'
|
||||
import { generateId, generateToken } from '@shared/ids'
|
||||
import type { SQL } from 'drizzle-orm'
|
||||
import {
|
||||
aliasedTable,
|
||||
@@ -18,7 +19,6 @@ import {
|
||||
or,
|
||||
sql,
|
||||
} from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { matters } from '../../db/schema'
|
||||
import { type AtomicQuery, executeWriteTransaction, executeWriteTransactionWithResults } from '../../db/transaction'
|
||||
import { suggestRenamed } from '../../domain/matter-name-conflict'
|
||||
@@ -297,9 +297,9 @@ export function createMatterRepo(db: Database): MatterRepo {
|
||||
const finalName = plan.finalName
|
||||
|
||||
const row: MatterRow = {
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
orgId: input.orgId,
|
||||
alias: nanoid(10),
|
||||
alias: generateToken(11),
|
||||
name: finalName,
|
||||
type: input.type,
|
||||
size: input.size ?? 0,
|
||||
@@ -508,9 +508,9 @@ export function createMatterRepo(db: Database): MatterRepo {
|
||||
)
|
||||
|
||||
const row: MatterRow = {
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
orgId: source.orgId,
|
||||
alias: nanoid(10),
|
||||
alias: generateToken(11),
|
||||
name: finalName,
|
||||
type: source.type,
|
||||
size: source.size,
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { generateId } from '@shared/ids'
|
||||
import { and, count, desc, eq, isNull, lt, or } from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { notifications } from '../../db/schema'
|
||||
import { executeWriteTransaction } from '../../db/transaction'
|
||||
import type { Database } from '../../platform/interface'
|
||||
@@ -16,7 +16,7 @@ export function createNotificationRepo(db: Database): NotificationRepo {
|
||||
return {
|
||||
async create(input) {
|
||||
const row: NotificationRow = {
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
userId: input.userId,
|
||||
type: input.type,
|
||||
title: input.title,
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { generateId } from '@shared/ids'
|
||||
import { and, eq } from 'drizzle-orm'
|
||||
import { oauthClient, oauthClientRegistration, oauthClientResource, oauthResource } from '../../db/auth-schema'
|
||||
import { executeWriteTransaction } from '../../db/transaction'
|
||||
@@ -50,7 +51,7 @@ export async function replaceManagedOAuthClient(
|
||||
): Promise<void> {
|
||||
const resourceQueries = resourceIds.map((resourceId) =>
|
||||
db.insert(oauthClientResource).values({
|
||||
id: `${clientId}::${resourceId}`,
|
||||
id: generateId(),
|
||||
clientId,
|
||||
resourceId,
|
||||
}),
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { generateId } from '@shared/ids'
|
||||
import { and, eq } from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { objectUploadSessions } from '../../db/schema'
|
||||
import type { Database } from '../../platform/interface'
|
||||
import type { ObjectUploadSessionRecord, ObjectUploadSessionRepo } from '../../usecases/ports'
|
||||
@@ -29,7 +29,7 @@ export function createObjectUploadSessionRepo(db: Database): ObjectUploadSession
|
||||
async create(input) {
|
||||
const now = new Date()
|
||||
const row: typeof objectUploadSessions.$inferInsert = {
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
orgId: input.orgId,
|
||||
objectId: input.objectId,
|
||||
storageId: input.storageId,
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
import { env } from 'cloudflare:workers'
|
||||
import { beforeEach, describe, expect, it } from 'vitest'
|
||||
import { withRedirectToken } from './redirect-token'
|
||||
|
||||
const RESOURCE_TABLE = 'test_redirect_token_resource'
|
||||
const REGISTRY_TABLE = 'test_redirect_token_registry'
|
||||
|
||||
describe('[CF] redirect token collision recovery', () => {
|
||||
beforeEach(async () => {
|
||||
await env.DB.exec(`
|
||||
DROP TABLE IF EXISTS ${RESOURCE_TABLE};
|
||||
DROP TABLE IF EXISTS ${REGISTRY_TABLE};
|
||||
CREATE TABLE ${RESOURCE_TABLE} (id TEXT PRIMARY KEY, token TEXT NOT NULL UNIQUE);
|
||||
CREATE TABLE ${REGISTRY_TABLE} (token TEXT PRIMARY KEY, resource_id TEXT NOT NULL UNIQUE);
|
||||
INSERT INTO ${REGISTRY_TABLE} VALUES ('TakenToken1', 'ExistingImage1');
|
||||
`)
|
||||
})
|
||||
|
||||
it('retries a real D1 uniqueness conflict without inspecting driver error text', async () => {
|
||||
const tokens = ['TakenToken1', 'FreshToken2']
|
||||
let forcedRace = true
|
||||
const isTaken = async (token: string) => {
|
||||
if (token === 'TakenToken1' && forcedRace) {
|
||||
forcedRace = false
|
||||
return false
|
||||
}
|
||||
return (
|
||||
(await env.DB.prepare(
|
||||
`SELECT 1 FROM ${RESOURCE_TABLE} WHERE token = ?1 UNION ALL SELECT 1 FROM ${REGISTRY_TABLE} WHERE token = ?1 LIMIT 1`,
|
||||
)
|
||||
.bind(token)
|
||||
.first()) !== null
|
||||
)
|
||||
}
|
||||
|
||||
const result = await withRedirectToken(
|
||||
11,
|
||||
async (token) => {
|
||||
await env.DB.batch([
|
||||
env.DB.prepare(`INSERT INTO ${RESOURCE_TABLE} VALUES (?1, ?2)`).bind(`Resource${token}`, token),
|
||||
env.DB.prepare(`INSERT INTO ${REGISTRY_TABLE} VALUES (?1, ?2)`).bind(token, `Resource${token}`),
|
||||
])
|
||||
return token
|
||||
},
|
||||
isTaken,
|
||||
() => tokens.shift()!,
|
||||
)
|
||||
|
||||
expect(result).toBe('FreshToken2')
|
||||
expect(await env.DB.prepare(`SELECT token FROM ${RESOURCE_TABLE}`).all()).toMatchObject({
|
||||
results: [{ token: 'FreshToken2' }],
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,81 @@
|
||||
import Database from 'better-sqlite3'
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { withRedirectToken } from './redirect-token'
|
||||
|
||||
describe('withRedirectToken', () => {
|
||||
it('skips candidates already reserved in the token namespace', async () => {
|
||||
const tokens = ['FirstToken1', 'SecondToken2', 'ThirdToken3']
|
||||
const taken = new Set(['FirstToken1', 'SecondToken2'])
|
||||
const operation = vi.fn(async (token: string) => token)
|
||||
|
||||
await expect(
|
||||
withRedirectToken(
|
||||
11,
|
||||
operation,
|
||||
async (token) => taken.has(token),
|
||||
() => tokens.shift()!,
|
||||
),
|
||||
).resolves.toBe('ThirdToken3')
|
||||
expect(operation).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('fails after the bounded collision budget is exhausted', async () => {
|
||||
const operation = vi.fn(async () => {
|
||||
throw new Error('driver-specific unique failure')
|
||||
})
|
||||
|
||||
await expect(
|
||||
withRedirectToken(
|
||||
11,
|
||||
operation,
|
||||
async () => true,
|
||||
() => 'SameToken11',
|
||||
),
|
||||
).rejects.toThrow('redirect_token_collision_budget_exhausted')
|
||||
expect(operation).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('does not hide unrelated database failures', async () => {
|
||||
const failure = new Error('database is locked')
|
||||
const operation = vi.fn(async () => {
|
||||
throw failure
|
||||
})
|
||||
|
||||
await expect(withRedirectToken(11, operation, async () => false)).rejects.toBe(failure)
|
||||
expect(operation).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('retries a real unique conflict after the failed resource transaction rolls back', async () => {
|
||||
const db = new Database(':memory:')
|
||||
db.exec(`
|
||||
CREATE TABLE resources (id TEXT PRIMARY KEY, token TEXT NOT NULL);
|
||||
CREATE TABLE redirect_token_registry (token TEXT PRIMARY KEY, kind TEXT NOT NULL, resource_id TEXT NOT NULL);
|
||||
INSERT INTO redirect_token_registry VALUES ('TakenToken1', 'image_hosting', 'ExistingImage1');
|
||||
`)
|
||||
const tokens = ['TakenToken1', 'FreshToken2']
|
||||
let forcedRace = true
|
||||
|
||||
const result = await withRedirectToken(
|
||||
11,
|
||||
async (token) => {
|
||||
db.transaction(() => {
|
||||
db.prepare('INSERT INTO resources VALUES (?, ?)').run(`Resource${token}`, token)
|
||||
db.prepare("INSERT INTO redirect_token_registry VALUES (?, 'direct_share', ?)").run(token, `Resource${token}`)
|
||||
})()
|
||||
return token
|
||||
},
|
||||
async (token) => {
|
||||
if (token === 'TakenToken1' && forcedRace) {
|
||||
forcedRace = false
|
||||
return false
|
||||
}
|
||||
return db.prepare('SELECT 1 FROM redirect_token_registry WHERE token = ? LIMIT 1').get(token) !== undefined
|
||||
},
|
||||
() => tokens.shift()!,
|
||||
)
|
||||
|
||||
expect(result).toBe('FreshToken2')
|
||||
expect(db.prepare('SELECT token FROM resources').all()).toEqual([{ token: 'FreshToken2' }])
|
||||
db.close()
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,24 @@
|
||||
import { generateToken } from '@shared/ids'
|
||||
|
||||
const MAX_TOKEN_COLLISION_RETRIES = 5
|
||||
|
||||
export async function withRedirectToken<T>(
|
||||
length: number,
|
||||
operation: (token: string) => Promise<T>,
|
||||
isTaken: (token: string) => Promise<boolean>,
|
||||
tokenGenerator: (length: number) => string = generateToken,
|
||||
): Promise<T> {
|
||||
for (let attempt = 0; attempt < MAX_TOKEN_COLLISION_RETRIES; attempt += 1) {
|
||||
const token = tokenGenerator(length)
|
||||
if (await isTaken(token)) continue
|
||||
try {
|
||||
return await operation(token)
|
||||
} catch (error) {
|
||||
// Database drivers do not expose a portable SQLite constraint code. Query
|
||||
// the shared namespace after a failed transaction and retry only when the
|
||||
// candidate is demonstrably occupied; unrelated failures retain identity.
|
||||
if (!(await isTaken(token))) throw error
|
||||
}
|
||||
}
|
||||
throw new Error('redirect_token_collision_budget_exhausted')
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
import { generateId } from '@shared/ids'
|
||||
import { asc, eq, inArray } from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { remoteDownloadUsageReports } from '../../db/schema'
|
||||
import type { Database } from '../../platform/interface'
|
||||
import type {
|
||||
@@ -39,7 +39,7 @@ export function createRemoteDownloadUsageRepo(db: Database): RemoteDownloadUsage
|
||||
|
||||
async insert(input: InsertRemoteDownloadUsageReportInput) {
|
||||
await db.insert(remoteDownloadUsageReports).values({
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
orgId: input.orgId,
|
||||
downloaderId: input.downloaderId,
|
||||
taskId: input.taskId,
|
||||
|
||||
@@ -3,7 +3,7 @@ import { nanoid } from 'nanoid'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { DirType } from '../../../shared/constants'
|
||||
import type { CreateShareInput } from '../../../shared/schemas/share'
|
||||
import { matters } from '../../db/schema'
|
||||
import { matters, redirectTokenRegistry } from '../../db/schema'
|
||||
import { isAccessibleByUser } from '../../domain/share'
|
||||
import { verifyPassword as verifyPasswordHash } from '../../lib/password'
|
||||
import type { Database } from '../../platform/interface'
|
||||
@@ -80,7 +80,8 @@ describe('createShare', () => {
|
||||
})
|
||||
|
||||
expect(share.id).toBeTruthy()
|
||||
expect(share.token).toHaveLength(10)
|
||||
expect(share.token).toHaveLength(11)
|
||||
expect(share.token).toMatch(/^[A-Za-z0-9]+$/)
|
||||
expect(share.kind).toBe('landing')
|
||||
expect(share.status).toBe('active')
|
||||
expect(share.passwordHash).toBeTruthy()
|
||||
@@ -101,6 +102,9 @@ describe('createShare', () => {
|
||||
|
||||
expect(share.kind).toBe('direct')
|
||||
expect(share.passwordHash).toBeNull()
|
||||
await expect(
|
||||
db.select().from(redirectTokenRegistry).where(eq(redirectTokenRegistry.token, share.token)),
|
||||
).resolves.toEqual([{ token: share.token, kind: 'direct_share', resourceId: share.id }])
|
||||
})
|
||||
|
||||
it('throws DIRECT_NO_PASSWORD when direct share has a password', async () => {
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
import { DirType } from '@shared/constants'
|
||||
import { generateId } from '@shared/ids'
|
||||
import type { CreateShareInput } from '@shared/schemas/share'
|
||||
import { and, count, desc, eq, isNotNull, isNull, like, lt, or, sql } from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { user } from '../../db/auth-schema'
|
||||
import { matters, shareRecipients, shares } from '../../db/schema'
|
||||
import { matters, redirectTokenRegistry, shareRecipients, shares } from '../../db/schema'
|
||||
import { type AtomicQuery, executeWriteTransaction } from '../../db/transaction'
|
||||
import { hashPassword } from '../../lib/password'
|
||||
import type { Database } from '../../platform/interface'
|
||||
@@ -16,6 +16,7 @@ import {
|
||||
type ShareResolution,
|
||||
} from '../../usecases/ports'
|
||||
import { createQuotaRepo } from './quota'
|
||||
import { withRedirectToken } from './redirect-token'
|
||||
import { resourceChangeQuery } from './resource-change'
|
||||
|
||||
function buildPath(parent: string, name: string): string {
|
||||
@@ -45,49 +46,73 @@ export function createShareRepo(db: Database): ShareRepo {
|
||||
if (input.kind === 'direct' && matter.dirtype !== DirType.FILE) throw new CreateShareError('DIRECT_NO_FOLDER')
|
||||
|
||||
const now = new Date()
|
||||
const token = input.kind === 'direct' ? `ds_${nanoid(10)}` : nanoid(10)
|
||||
const share: ShareRecord = {
|
||||
id: nanoid(),
|
||||
token,
|
||||
kind: input.kind,
|
||||
matterId: input.matterId,
|
||||
orgId: input.orgId,
|
||||
creatorId: input.creatorId,
|
||||
passwordHash: input.password ? hashPassword(input.password) : null,
|
||||
expiresAt: input.expiresAt ?? null,
|
||||
downloadLimit: input.downloadLimit ?? null,
|
||||
views: 0,
|
||||
downloads: 0,
|
||||
status: 'active',
|
||||
private: input.private ?? false,
|
||||
createdAt: now,
|
||||
}
|
||||
const id = generateId()
|
||||
return withRedirectToken(
|
||||
11,
|
||||
async (token) => {
|
||||
const share: ShareRecord = {
|
||||
id,
|
||||
token,
|
||||
kind: input.kind,
|
||||
matterId: input.matterId,
|
||||
orgId: input.orgId,
|
||||
creatorId: input.creatorId,
|
||||
passwordHash: input.password ? hashPassword(input.password) : null,
|
||||
expiresAt: input.expiresAt ?? null,
|
||||
downloadLimit: input.downloadLimit ?? null,
|
||||
views: 0,
|
||||
downloads: 0,
|
||||
status: 'active',
|
||||
private: input.private ?? false,
|
||||
createdAt: now,
|
||||
}
|
||||
|
||||
const queries: AtomicQuery[] = [
|
||||
db.insert(shares).values(share),
|
||||
resourceChangeQuery(db, {
|
||||
scopeType: 'user',
|
||||
scopeId: share.creatorId,
|
||||
resourceType: 'share',
|
||||
resourceId: share.id,
|
||||
changeType: 'upsert',
|
||||
action: 'created',
|
||||
occurredAt: now,
|
||||
}),
|
||||
]
|
||||
if (input.recipients && input.recipients.length > 0) {
|
||||
const recipientRows = input.recipients.map((r) => ({
|
||||
id: nanoid(),
|
||||
shareId: share.id,
|
||||
recipientUserId: r.recipientUserId ?? null,
|
||||
recipientEmail: r.recipientEmail ?? null,
|
||||
createdAt: now,
|
||||
}))
|
||||
queries.push(db.insert(shareRecipients).values(recipientRows))
|
||||
}
|
||||
const queries: AtomicQuery[] = [
|
||||
db.insert(shares).values(share),
|
||||
...(share.kind === 'direct'
|
||||
? [
|
||||
db.insert(redirectTokenRegistry).values({
|
||||
token: share.token,
|
||||
kind: 'direct_share',
|
||||
resourceId: share.id,
|
||||
}),
|
||||
]
|
||||
: []),
|
||||
resourceChangeQuery(db, {
|
||||
scopeType: 'user',
|
||||
scopeId: share.creatorId,
|
||||
resourceType: 'share',
|
||||
resourceId: share.id,
|
||||
changeType: 'upsert',
|
||||
action: 'created',
|
||||
occurredAt: now,
|
||||
}),
|
||||
]
|
||||
if (input.recipients && input.recipients.length > 0) {
|
||||
const recipientRows = input.recipients.map((r) => ({
|
||||
id: generateId(),
|
||||
shareId: share.id,
|
||||
recipientUserId: r.recipientUserId ?? null,
|
||||
recipientEmail: r.recipientEmail ?? null,
|
||||
createdAt: now,
|
||||
}))
|
||||
queries.push(db.insert(shareRecipients).values(recipientRows))
|
||||
}
|
||||
|
||||
await executeWriteTransaction(db, queries)
|
||||
return share
|
||||
await executeWriteTransaction(db, queries)
|
||||
return share
|
||||
},
|
||||
async (token) => {
|
||||
const existingShare = await db.select({ id: shares.id }).from(shares).where(eq(shares.token, token)).limit(1)
|
||||
if (existingShare.length > 0 || input.kind !== 'direct') return existingShare.length > 0
|
||||
const reservation = await db
|
||||
.select({ token: redirectTokenRegistry.token })
|
||||
.from(redirectTokenRegistry)
|
||||
.where(eq(redirectTokenRegistry.token, token))
|
||||
.limit(1)
|
||||
return reservation.length > 0
|
||||
},
|
||||
)
|
||||
},
|
||||
|
||||
async resolveByToken(token: string): Promise<ShareResolution> {
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { DEFAULT_SITE_NAME } from '@shared/constants'
|
||||
import { generateId, generateToken } from '@shared/ids'
|
||||
import type { SiteInvitation } from '@shared/types'
|
||||
import { and, count, desc, eq, gt, isNull, sql } from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import * as authSchema from '../../db/auth-schema'
|
||||
import { siteInvitations, systemOptions } from '../../db/schema'
|
||||
import type { Database } from '../../platform/interface'
|
||||
@@ -108,9 +108,9 @@ async function createSiteInvitation(db: Database, adminUserId: string, rawEmail:
|
||||
if (pendingInvite) throw new Error('A pending invitation already exists for this email')
|
||||
|
||||
const row: typeof siteInvitations.$inferInsert = {
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
email,
|
||||
token: nanoid(32),
|
||||
token: generateToken(33),
|
||||
invitedBy: adminUserId,
|
||||
acceptedBy: null,
|
||||
acceptedAt: null,
|
||||
@@ -148,7 +148,7 @@ async function resendSiteInvitation(
|
||||
if (row.revokedAt) return 'already_revoked'
|
||||
|
||||
const now = new Date()
|
||||
const nextToken = nanoid(32)
|
||||
const nextToken = generateToken(33)
|
||||
await db
|
||||
.update(siteInvitations)
|
||||
.set({
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { DirType } from '@shared/constants'
|
||||
import { generateId, generateToken } from '@shared/ids'
|
||||
import { and, asc, eq, isNull, sql } from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { imageHostings, matters, storageUsageLedger } from '../../db/schema'
|
||||
import { type AtomicQuery, executeWriteTransaction } from '../../db/transaction'
|
||||
import type { Database } from '../../platform/interface'
|
||||
@@ -65,7 +65,7 @@ export function storageUsageOpeningBalanceQuery(
|
||||
return db
|
||||
.insert(storageUsageLedger)
|
||||
.values({
|
||||
id: eventKey,
|
||||
id: generateId(),
|
||||
eventKey,
|
||||
orgId,
|
||||
storageId,
|
||||
@@ -95,11 +95,11 @@ export function storageUsageOpeningBalanceQuery(
|
||||
}
|
||||
|
||||
export function storageUsageMutationQuery(db: Database, mutation: StorageUsageLedgerMutation): AtomicQuery {
|
||||
const eventKey = mutation.eventKey ?? `mutation:${nanoid()}`
|
||||
const eventKey = mutation.eventKey ?? `mutation:${generateToken(22)}`
|
||||
return db
|
||||
.insert(storageUsageLedger)
|
||||
.values({
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
eventKey,
|
||||
orgId: mutation.orgId,
|
||||
storageId: mutation.storageId,
|
||||
@@ -127,7 +127,7 @@ export function matterActivationLedgerQuery(
|
||||
return conditionalMutationQuery(
|
||||
db,
|
||||
sql`SELECT
|
||||
${nanoid()}, ${eventKey}, ${matters.orgId}, ${matters.storageId}, 'matter', ${matters.id},
|
||||
${generateId()}, ${eventKey}, ${matters.orgId}, ${matters.storageId}, 'matter', ${matters.id},
|
||||
${matters.size}, 'matter_activated', ${occurredAt.getTime()}, ${occurredAt.getTime()}
|
||||
FROM ${matters}
|
||||
WHERE ${matters.id} = ${matterId}
|
||||
@@ -149,7 +149,7 @@ export function matterResizeLedgerQuery(
|
||||
return conditionalMutationQuery(
|
||||
db,
|
||||
sql`SELECT
|
||||
${nanoid()}, ${`matter:${matterId}:resized:${nanoid()}`}, ${matters.orgId}, ${matters.storageId}, 'matter',
|
||||
${generateId()}, ${`matter:${matterId}:resized:${generateToken(22)}`}, ${matters.orgId}, ${matters.storageId}, 'matter',
|
||||
${matters.id}, ${nextSize} - COALESCE(${matters.size}, 0), 'matter_resized',
|
||||
${occurredAt.getTime()}, ${occurredAt.getTime()}
|
||||
FROM ${matters}
|
||||
@@ -167,7 +167,7 @@ export function matterPurgeLedgerQuery(db: Database, orgId: string, matterId: st
|
||||
return conditionalMutationQuery(
|
||||
db,
|
||||
sql`SELECT
|
||||
${nanoid()}, ${eventKey}, ${matters.orgId}, ${matters.storageId}, 'matter', ${matters.id},
|
||||
${generateId()}, ${eventKey}, ${matters.orgId}, ${matters.storageId}, 'matter', ${matters.id},
|
||||
-COALESCE(${matters.size}, 0), 'matter_purged', ${occurredAt.getTime()}, ${occurredAt.getTime()}
|
||||
FROM ${matters}
|
||||
WHERE ${matters.id} = ${matterId}
|
||||
@@ -189,7 +189,7 @@ export function imageActivationLedgerQuery(
|
||||
return conditionalMutationQuery(
|
||||
db,
|
||||
sql`SELECT
|
||||
${nanoid()}, ${eventKey}, ${imageHostings.orgId}, ${imageHostings.storageId}, 'image_hosting',
|
||||
${generateId()}, ${eventKey}, ${imageHostings.orgId}, ${imageHostings.storageId}, 'image_hosting',
|
||||
${imageHostings.id}, ${imageHostings.size}, 'image_activated', ${occurredAt.getTime()}, ${occurredAt.getTime()}
|
||||
FROM ${imageHostings}
|
||||
WHERE ${imageHostings.id} = ${imageId}
|
||||
@@ -205,7 +205,7 @@ export function imagePurgeLedgerQuery(db: Database, orgId: string, imageId: stri
|
||||
return conditionalMutationQuery(
|
||||
db,
|
||||
sql`SELECT
|
||||
${nanoid()}, ${eventKey}, ${imageHostings.orgId}, ${imageHostings.storageId}, 'image_hosting',
|
||||
${generateId()}, ${eventKey}, ${imageHostings.orgId}, ${imageHostings.storageId}, 'image_hosting',
|
||||
${imageHostings.id}, -${imageHostings.size}, 'image_purged', ${occurredAt.getTime()}, ${occurredAt.getTime()}
|
||||
FROM ${imageHostings}
|
||||
WHERE ${imageHostings.id} = ${imageId}
|
||||
@@ -278,7 +278,7 @@ export async function ensureStorageUsageIntegrityOpeningBalances(db: Database, o
|
||||
return db
|
||||
.insert(storageUsageLedger)
|
||||
.values({
|
||||
id: eventKey,
|
||||
id: generateId(),
|
||||
eventKey,
|
||||
orgId,
|
||||
storageId,
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { generateId } from '@shared/ids'
|
||||
import { and, asc, count, eq, isNull, lt, or } from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { matters, storages } from '../../db/schema'
|
||||
import type { Database } from '../../platform/interface'
|
||||
import type { CachePolicy, CacheService, StorageRecord, StorageRepo } from '../../usecases/ports'
|
||||
@@ -70,7 +70,7 @@ export function createStorageRepo(db: Database, cache?: CacheService): StorageRe
|
||||
async create(input) {
|
||||
const now = new Date()
|
||||
const row: StorageRow = {
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
provider: input.provider ?? '',
|
||||
bucket: input.bucket,
|
||||
endpoint: input.endpoint,
|
||||
|
||||
@@ -1,20 +1,19 @@
|
||||
import { generateId, generateToken } from '@shared/ids'
|
||||
import { and, desc, eq, gt, isNull, or } from 'drizzle-orm'
|
||||
import { customAlphabet, nanoid } from 'nanoid'
|
||||
import { invitation, member, organization } from '../../db/auth-schema'
|
||||
import { teamInviteLinks } from '../../db/schema'
|
||||
import type { Database } from '../../platform/interface'
|
||||
import type { TeamInviteLinkRecord, TeamInviteRepo } from '../../usecases/ports'
|
||||
|
||||
const generateToken = customAlphabet('0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz', 32)
|
||||
const DEFAULT_EXPIRES_IN_MS = 7 * 24 * 60 * 60 * 1000 // 7 days
|
||||
|
||||
export function createTeamInviteRepo(db: Database): TeamInviteRepo {
|
||||
return {
|
||||
async createInviteLink(organizationId, inviterId, role, expiresIn) {
|
||||
const token = generateToken()
|
||||
const token = generateToken(32)
|
||||
const now = new Date()
|
||||
const row: TeamInviteLinkRecord = {
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
token,
|
||||
organizationId,
|
||||
role,
|
||||
@@ -68,7 +67,7 @@ export function createTeamInviteRepo(db: Database): TeamInviteRepo {
|
||||
if (existing[0]) return 'already_member'
|
||||
|
||||
await db.insert(member).values({
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
organizationId: link.organizationId,
|
||||
userId,
|
||||
role: link.role,
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { generateId, generateToken } from '@shared/ids'
|
||||
import { isPersonalOrgLike } from '@shared/org-slugs'
|
||||
import { and, desc, eq, inArray, isNull, or } from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { member, organization, user } from '../../db/auth-schema'
|
||||
import { orgQuotaEntitlements } from '../../db/schema'
|
||||
import type { Database } from '../../platform/interface'
|
||||
@@ -142,12 +142,12 @@ async function grantOrgEntitlement(
|
||||
if ('error' in org) return org
|
||||
const now = new Date()
|
||||
const entitlement = {
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
orgId: input.orgId,
|
||||
resourceType: input.resourceType,
|
||||
entitlementType: 'grant',
|
||||
source: 'admin_grant',
|
||||
sourceId: `admin_grant:${nanoid()}`,
|
||||
sourceId: `admin_grant:${generateToken(22)}`,
|
||||
bytes: input.bytes,
|
||||
startsAt: now,
|
||||
expiresAt: input.expiresAt ?? null,
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { generateId } from '@shared/ids'
|
||||
import { and, eq, inArray, or, type SQL, sql } from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { webdavDeadProperties, webdavLocks } from '../../db/schema'
|
||||
import { type AtomicQuery, executeWriteTransaction } from '../../db/transaction'
|
||||
import type { Database } from '../../platform/interface'
|
||||
@@ -53,7 +53,7 @@ export function createWebDavStateRepo(db: Database): WebDavStateRepo {
|
||||
db
|
||||
.insert(webdavDeadProperties)
|
||||
.values({
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
orgId,
|
||||
resourcePath,
|
||||
namespace: property.namespace,
|
||||
@@ -89,7 +89,7 @@ export function createWebDavStateRepo(db: Database): WebDavStateRepo {
|
||||
db
|
||||
.insert(webdavDeadProperties)
|
||||
.values({
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
orgId,
|
||||
resourcePath: targetPath,
|
||||
namespace: row.namespace,
|
||||
@@ -280,7 +280,7 @@ function newLock(input: {
|
||||
}): DavLock {
|
||||
const now = new Date()
|
||||
return {
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
token: `opaquelocktoken:${crypto.randomUUID()}`,
|
||||
orgId: input.orgId,
|
||||
resourcePath: input.resourcePath,
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { generateId } from '@shared/ids'
|
||||
import { and, eq, isNull, lt, or, sql } from 'drizzle-orm'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { x402CapacityPurchaseIntents } from '../../db/schema'
|
||||
import { executeWriteTransactionWithResults } from '../../db/transaction'
|
||||
import type { Database } from '../../platform/interface'
|
||||
@@ -30,7 +30,7 @@ export function createX402CapacityPurchaseRepo(db: Database): X402CapacityPurcha
|
||||
async create(input) {
|
||||
const now = new Date()
|
||||
const nowMs = now.getTime()
|
||||
const id = nanoid()
|
||||
const id = generateId()
|
||||
const abandonedBefore = new Date(nowMs - PENDING_INTENT_TTL_MS)
|
||||
const creationWindowStart = new Date(nowMs - INTENT_CREATION_WINDOW_MS)
|
||||
const retentionBefore = new Date(nowMs - ABANDONED_INTENT_RETENTION_MS)
|
||||
|
||||
@@ -81,6 +81,27 @@ describe('registration gate — first user always allowed', () => {
|
||||
expect(res.status).toBe(200)
|
||||
})
|
||||
|
||||
it('keeps Better Auth and ZPan-created authentication records on the Base62 contract', async () => {
|
||||
const ctx = await createTestApp()
|
||||
const res = await signUp(ctx, 'base62-contract@example.com')
|
||||
const body = (await res.json()) as { user: { id: string } }
|
||||
const rows = await ctx.db.all<{ id: string }>(sql`
|
||||
SELECT id FROM user
|
||||
UNION ALL SELECT id FROM account
|
||||
UNION ALL SELECT id FROM session
|
||||
UNION ALL SELECT id FROM organization
|
||||
UNION ALL SELECT id FROM member
|
||||
UNION ALL SELECT id FROM org_quotas
|
||||
UNION ALL SELECT id FROM org_quota_entitlements
|
||||
UNION ALL SELECT id FROM audit_events
|
||||
`)
|
||||
|
||||
expect(res.status).toBe(200)
|
||||
expect(body.user.id).toMatch(/^[A-Za-z0-9]+$/)
|
||||
expect(rows.length).toBeGreaterThanOrEqual(8)
|
||||
expect(rows.every(({ id }) => /^[A-Za-z0-9]+$/.test(id))).toBe(true)
|
||||
})
|
||||
|
||||
it('first user can register when auth_signup_mode is invite_only without a code', async () => {
|
||||
const ctx = await createTestApp()
|
||||
await ctx.db.insert(schema.systemOptions).values({ key: 'auth_signup_mode', value: 'invite_only' })
|
||||
@@ -875,7 +896,7 @@ describe('OAuth consent guards', () => {
|
||||
expect(body).toMatchObject({
|
||||
client_id: expect.any(String),
|
||||
client_secret: expect.any(String),
|
||||
registration_access_token: expect.stringMatching(/^zpr_/),
|
||||
registration_access_token: expect.stringMatching(/^[A-Za-z0-9]{43}$/),
|
||||
registration_client_uri: expect.stringMatching(/^http:\/\/localhost:3000\/api\/auth\/oauth2\/register\//),
|
||||
token_endpoint_auth_method: 'client_secret_basic',
|
||||
authorization_details_types: [WORKSPACE_AUTHORIZATION_DETAIL_TYPE],
|
||||
@@ -1564,7 +1585,7 @@ describe('OAuth consent guards', () => {
|
||||
const pushedBody = (await pushed.json()) as { request_uri: string; expires_in: number }
|
||||
expect(pushed.status, JSON.stringify(pushedBody)).toBe(201)
|
||||
expect(pushedBody).toMatchObject({
|
||||
request_uri: expect.stringMatching(/^urn:ietf:params:oauth:request_uri:/),
|
||||
request_uri: expect.stringMatching(/^urn:ietf:params:oauth:request_uri:[A-Za-z0-9]{33}$/),
|
||||
expires_in: 90,
|
||||
})
|
||||
|
||||
|
||||
+6
-5
@@ -18,7 +18,7 @@ import {
|
||||
import { genericOAuth } from 'better-auth/plugins/generic-oauth'
|
||||
import { adminAc, memberAc, ownerAc } from 'better-auth/plugins/organization/access'
|
||||
import { count, eq, like } from 'drizzle-orm'
|
||||
import { customAlphabet, nanoid } from 'nanoid'
|
||||
import { customAlphabet } from 'nanoid'
|
||||
import {
|
||||
API_KEY_TEMPLATES,
|
||||
ApiKeyTemplate,
|
||||
@@ -31,6 +31,7 @@ import {
|
||||
WEBDAV_RATE_LIMITER_BINDING,
|
||||
} from '../shared/api-key-templates'
|
||||
import { DEFAULT_ORG_QUOTA, DEFAULT_ORG_TRAFFIC_QUOTA, SignupMode } from '../shared/constants'
|
||||
import { generateId } from '../shared/ids'
|
||||
import { JWT_BEARER_GRANT_TYPE, OAUTH_SCOPES, TOKEN_EXCHANGE_GRANT_TYPE } from '../shared/oauth'
|
||||
import {
|
||||
BUILTIN_PROVIDER_IDS,
|
||||
@@ -1021,7 +1022,7 @@ async function createPersonalOrg(
|
||||
db: Database,
|
||||
user: { id: string; name: string; username?: string | null },
|
||||
): Promise<string> {
|
||||
const orgId = nanoid()
|
||||
const orgId = generateId()
|
||||
const now = new Date()
|
||||
const displayName = user.name || user.username
|
||||
const orgName = displayName ? `${displayName}'s Space` : 'Personal Space'
|
||||
@@ -1037,7 +1038,7 @@ async function createPersonalOrg(
|
||||
createdAt: now,
|
||||
}),
|
||||
db.insert(authSchema.member).values({
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
organizationId: orgId,
|
||||
userId: user.id,
|
||||
role: 'owner',
|
||||
@@ -1067,7 +1068,7 @@ async function findPersonalOrgFromExistingSession(db: Database, userId: string):
|
||||
|
||||
async function createOrgQuotaValues(_db: Database, orgId: string, now: Date): Promise<typeof orgQuotas.$inferInsert> {
|
||||
return {
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
orgId,
|
||||
quota: 0,
|
||||
used: 0,
|
||||
@@ -1125,7 +1126,7 @@ function freePlanEntitlementValue(
|
||||
settingKey: string,
|
||||
): typeof orgQuotaEntitlements.$inferInsert {
|
||||
return {
|
||||
id: nanoid(),
|
||||
id: generateId(),
|
||||
orgId,
|
||||
resourceType,
|
||||
entitlementType: 'plan',
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { z } from 'zod'
|
||||
import { generateToken } from '../../shared/ids'
|
||||
import {
|
||||
JWT_BEARER_GRANT_TYPE,
|
||||
OAUTH_SCOPES,
|
||||
@@ -111,7 +112,7 @@ export function addOAuthClientRegistrationManagementOpenApi(document: { paths: R
|
||||
registrationSchema.properties = {
|
||||
...properties,
|
||||
registration_client_uri: { type: 'string', format: 'uri' },
|
||||
registration_access_token: { type: 'string' },
|
||||
registration_access_token: { type: 'string', pattern: '^[A-Za-z0-9]{43}$' },
|
||||
}
|
||||
registrationSchema.required = [
|
||||
...new Set([
|
||||
@@ -128,7 +129,7 @@ export function addOAuthClientRegistrationManagementOpenApi(document: { paths: R
|
||||
properties: {
|
||||
client_id: { type: 'string' },
|
||||
registration_client_uri: { type: 'string', format: 'uri' },
|
||||
registration_access_token: { type: 'string' },
|
||||
registration_access_token: { type: 'string', pattern: '^[A-Za-z0-9]{43}$' },
|
||||
scope: { type: 'string' },
|
||||
},
|
||||
required: ['client_id', 'registration_client_uri', 'registration_access_token'],
|
||||
@@ -441,8 +442,7 @@ function mergedHeaders(...inputs: Array<HeadersInit | undefined>): Headers {
|
||||
}
|
||||
|
||||
function registrationToken(): string {
|
||||
const bytes = crypto.getRandomValues(new Uint8Array(32))
|
||||
return `zpr_${base64Url(bytes)}`
|
||||
return generateToken(43)
|
||||
}
|
||||
|
||||
function configurationUrl(clientId: string, baseUrl: string): URL {
|
||||
|
||||
@@ -2,8 +2,8 @@ import { getOAuthProviderApi, type oauthProvider } from '@better-auth/oauth-prov
|
||||
import { parseWorkspaceAuthorizationDetails } from '@shared/schemas'
|
||||
import { APIError, type BetterAuthPlugin } from 'better-auth'
|
||||
import { createAuthEndpoint } from 'better-auth/api'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { z } from 'zod'
|
||||
import { generateToken } from '../../shared/ids'
|
||||
|
||||
type OAuthProviderOptions = Parameters<typeof oauthProvider>[0]
|
||||
|
||||
@@ -44,7 +44,7 @@ export function oauthPushedAuthorizationRequests(options: OAuthProviderOptions):
|
||||
await validatePushedAuthorizationRequest(ctx.body, client, options)
|
||||
|
||||
const parameters = stripClientCredentials(ctx.body)
|
||||
const requestUri = `${PAR_REQUEST_URI_PREFIX}${nanoid(32)}`
|
||||
const requestUri = `${PAR_REQUEST_URI_PREFIX}${generateToken(33)}`
|
||||
const now = new Date()
|
||||
await ctx.context.adapter.create({
|
||||
model: 'oauthPushedAuthorizationRequest',
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { createApp } from './app'
|
||||
import { createAuth } from './auth'
|
||||
import { createDeps } from './composition'
|
||||
import { assertIdIntegrity } from './db/id-integrity'
|
||||
import type { Platform } from './platform/interface'
|
||||
import type { Deps } from './usecases/deps'
|
||||
|
||||
@@ -17,6 +18,7 @@ export async function createBootstrap(platform: Platform, deps: Deps = createDep
|
||||
.map((o) => o.trim())
|
||||
.filter(Boolean) || ['http://localhost:5185']
|
||||
|
||||
await assertIdIntegrity(platform.db)
|
||||
const auth = await createAuth(platform, secret, baseURL, trustedOrigins)
|
||||
return createApp(platform, auth, deps)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,177 @@
|
||||
import { env } from 'cloudflare:workers'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import idBackfillWorker, {
|
||||
applyIdBackfillArtifact,
|
||||
finalizeIdBackfill,
|
||||
type IdBackfillBatchArtifact,
|
||||
} from '../../workers/id-backfill'
|
||||
|
||||
const SUBJECT = '_zpan_id_backfill_cf_subject'
|
||||
const CHILD = '_zpan_id_backfill_cf_child'
|
||||
const MAP = '_zpan_id_backfill_map'
|
||||
const PUBLIC = '_zpan_id_backfill_cf_public'
|
||||
const EVENT = '_zpan_id_backfill_cf_event'
|
||||
const CREDENTIAL = '_zpan_id_backfill_cf_credential'
|
||||
|
||||
async function artifact(statements: string[]): Promise<IdBackfillBatchArtifact> {
|
||||
const bytes = new Uint8Array(
|
||||
await crypto.subtle.digest('SHA-256', new TextEncoder().encode(JSON.stringify(statements))),
|
||||
)
|
||||
const digest = Array.from(bytes, (byte) => byte.toString(16).padStart(2, '0')).join('')
|
||||
return { version: 1, digest, statements }
|
||||
}
|
||||
|
||||
async function resetFixture(): Promise<void> {
|
||||
await env.DB.exec(`
|
||||
DROP TABLE IF EXISTS ${CHILD};
|
||||
DROP TABLE IF EXISTS ${SUBJECT};
|
||||
DROP TABLE IF EXISTS ${MAP};
|
||||
DROP TABLE IF EXISTS ${PUBLIC};
|
||||
DROP TABLE IF EXISTS ${EVENT};
|
||||
DROP TABLE IF EXISTS ${CREDENTIAL};
|
||||
CREATE TABLE ${SUBJECT} (id TEXT PRIMARY KEY);
|
||||
CREATE TABLE ${CHILD} (id TEXT PRIMARY KEY, subject_id TEXT NOT NULL REFERENCES ${SUBJECT}(id));
|
||||
CREATE TABLE ${MAP} (old_value TEXT PRIMARY KEY, new_value TEXT NOT NULL UNIQUE);
|
||||
CREATE TABLE ${PUBLIC} (id TEXT PRIMARY KEY, token TEXT NOT NULL UNIQUE);
|
||||
CREATE TABLE ${EVENT} (id TEXT PRIMARY KEY, event_key TEXT, metadata TEXT);
|
||||
CREATE TABLE ${CREDENTIAL} (id TEXT PRIMARY KEY, secret TEXT);
|
||||
CREATE TABLE IF NOT EXISTS system_options (key TEXT PRIMARY KEY, value TEXT NOT NULL);
|
||||
DELETE FROM system_options WHERE key IN ('id_normalization_version', 'id_normalization_artifact_digest', 'id_normalization_pending_artifact_digest');
|
||||
INSERT INTO ${SUBJECT} VALUES ('legacy_id');
|
||||
INSERT INTO ${CHILD} VALUES ('child', 'legacy_id');
|
||||
INSERT INTO ${PUBLIC} VALUES ('public', 'ds_legacy');
|
||||
INSERT INTO ${EVENT} VALUES ('event:user:legacy_id', NULL, '{"matterId":"legacy_id","matterIds":["legacy_id"]}');
|
||||
INSERT INTO ${CREDENTIAL} VALUES ('credential', 'secret');
|
||||
`)
|
||||
}
|
||||
|
||||
function representativeStatements(): string[] {
|
||||
return [
|
||||
'PRAGMA defer_foreign_keys = ON;',
|
||||
`INSERT INTO ${MAP} VALUES ('legacy_id', 'Base62Replacement')`,
|
||||
`UPDATE ${CHILD} SET subject_id = (SELECT new_value FROM ${MAP} WHERE old_value = subject_id)`,
|
||||
`UPDATE ${EVENT} SET event_key = replace(id, 'legacy_id', 'Base62Replacement'), metadata = json_set(metadata, '$.matterId', 'Base62Replacement', '$.matterIds[0]', 'Base62Replacement')`,
|
||||
`UPDATE ${SUBJECT} SET id = (SELECT new_value FROM ${MAP} WHERE old_value = id)`,
|
||||
`UPDATE ${PUBLIC} SET token = 'PublicBase62Token'`,
|
||||
`DELETE FROM ${CREDENTIAL}`,
|
||||
]
|
||||
}
|
||||
|
||||
describe('[CF] ID backfill D1 transaction rehearsal', () => {
|
||||
it('defers FKs while atomically rewriting a PK and its reference', async () => {
|
||||
await resetFixture()
|
||||
const plan = await artifact(representativeStatements())
|
||||
const response = await idBackfillWorker.fetch(
|
||||
new Request('https://maintenance.invalid/', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
Authorization: 'Bearer rehearsal-secret',
|
||||
'Content-Type': 'application/json',
|
||||
'X-ZPan-ID-Backfill-Confirm': 'invalidate-credentials-and-links',
|
||||
'X-ZPan-ID-Backfill-Digest': plan.digest,
|
||||
},
|
||||
body: JSON.stringify(plan),
|
||||
}),
|
||||
{ DB: env.DB, ID_BACKFILL_AUTH_TOKEN: 'rehearsal-secret' },
|
||||
)
|
||||
expect(response.status).toBe(200)
|
||||
await expect(response.json()).resolves.toEqual({
|
||||
ok: true,
|
||||
statements: plan.statements.length,
|
||||
digest: plan.digest,
|
||||
})
|
||||
|
||||
const row = await env.DB.prepare(
|
||||
`SELECT s.id, c.subject_id AS subjectId FROM ${SUBJECT} s JOIN ${CHILD} c ON c.subject_id = s.id`,
|
||||
).first<{ id: string; subjectId: string }>()
|
||||
expect(row).toEqual({ id: 'Base62Replacement', subjectId: 'Base62Replacement' })
|
||||
const foreignKeys = await env.DB.prepare(`PRAGMA foreign_key_check(${CHILD})`).all()
|
||||
expect(foreignKeys.results).toEqual([])
|
||||
expect(await env.DB.prepare(`SELECT token FROM ${PUBLIC}`).first()).toEqual({ token: 'PublicBase62Token' })
|
||||
expect(await env.DB.prepare(`SELECT event_key AS eventKey, metadata FROM ${EVENT}`).first()).toEqual({
|
||||
eventKey: 'event:user:Base62Replacement',
|
||||
metadata: '{"matterId":"Base62Replacement","matterIds":["Base62Replacement"]}',
|
||||
})
|
||||
expect(
|
||||
(await env.DB.prepare(`SELECT COUNT(*) AS count FROM ${CREDENTIAL}`).first<{ count: number }>())?.count,
|
||||
).toBe(0)
|
||||
})
|
||||
|
||||
it('rolls the whole D1 batch back when a later statement fails', async () => {
|
||||
await resetFixture()
|
||||
const plan = await artifact([...representativeStatements(), `INSERT INTO ${SUBJECT} VALUES ('Base62Replacement')`])
|
||||
await expect(applyIdBackfillArtifact(env.DB, plan, plan.digest)).rejects.toThrow()
|
||||
|
||||
const subject = await env.DB.prepare(`SELECT id FROM ${SUBJECT}`).first<{ id: string }>()
|
||||
const child = await env.DB.prepare(`SELECT subject_id AS subjectId FROM ${CHILD}`).first<{ subjectId: string }>()
|
||||
expect(subject?.id).toBe('legacy_id')
|
||||
expect(child?.subjectId).toBe('legacy_id')
|
||||
expect(await env.DB.prepare(`SELECT token FROM ${PUBLIC}`).first()).toEqual({ token: 'ds_legacy' })
|
||||
expect(
|
||||
(await env.DB.prepare(`SELECT COUNT(*) AS count FROM ${CREDENTIAL}`).first<{ count: number }>())?.count,
|
||||
).toBe(1)
|
||||
})
|
||||
|
||||
it('rejects a modified or mismatched artifact before sending a D1 batch', async () => {
|
||||
await resetFixture()
|
||||
const plan = await artifact(representativeStatements())
|
||||
plan.statements[1] = `DELETE FROM ${SUBJECT}`
|
||||
await expect(applyIdBackfillArtifact(env.DB, plan, plan.digest)).rejects.toThrow(
|
||||
'backfill_artifact_digest_mismatch',
|
||||
)
|
||||
expect(await env.DB.prepare(`SELECT id FROM ${SUBJECT}`).first()).toEqual({ id: 'legacy_id' })
|
||||
})
|
||||
|
||||
it('rejects an artifact that would exceed the D1 free-plan invocation budget', async () => {
|
||||
await resetFixture()
|
||||
const plan = await artifact(['PRAGMA defer_foreign_keys = ON;', ...Array.from({ length: 47 }, () => 'SELECT 1')])
|
||||
|
||||
await expect(applyIdBackfillArtifact(env.DB, plan, plan.digest)).rejects.toThrow('d1_query_limit_exceeded')
|
||||
})
|
||||
|
||||
it('rejects an artifact after the backfill has been finalized', async () => {
|
||||
await resetFixture()
|
||||
await env.DB.prepare("INSERT INTO system_options VALUES ('id_normalization_version', '1')").run()
|
||||
const plan = await artifact(representativeStatements())
|
||||
|
||||
await expect(applyIdBackfillArtifact(env.DB, plan, plan.digest)).rejects.toThrow('id_backfill_already_finalized:1')
|
||||
expect(await env.DB.prepare(`SELECT id FROM ${SUBJECT}`).first()).toEqual({ id: 'legacy_id' })
|
||||
})
|
||||
|
||||
it('finalizes atomically through the authenticated maintenance endpoint', async () => {
|
||||
await resetFixture()
|
||||
const plan = await artifact(representativeStatements())
|
||||
await applyIdBackfillArtifact(env.DB, plan, plan.digest)
|
||||
await expect(finalizeIdBackfill(env.DB, { version: 1, digest: plan.digest })).resolves.toEqual({
|
||||
digest: plan.digest,
|
||||
})
|
||||
|
||||
await resetFixture()
|
||||
await applyIdBackfillArtifact(env.DB, plan, plan.digest)
|
||||
const response = await idBackfillWorker.fetch(
|
||||
new Request('https://maintenance.invalid/finalize', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
Authorization: 'Bearer rehearsal-secret',
|
||||
'Content-Type': 'application/json',
|
||||
'X-ZPan-ID-Backfill-Confirm': 'finalize-id-normalization',
|
||||
},
|
||||
body: JSON.stringify({ version: 1, digest: plan.digest }),
|
||||
}),
|
||||
{ DB: env.DB, ID_BACKFILL_AUTH_TOKEN: 'rehearsal-secret' },
|
||||
)
|
||||
|
||||
expect(response.status).toBe(200)
|
||||
expect(
|
||||
await env.DB.prepare("SELECT value FROM system_options WHERE key = 'id_normalization_version'").first(),
|
||||
).toEqual({
|
||||
value: '1',
|
||||
})
|
||||
expect(
|
||||
await env.DB.prepare("SELECT name FROM sqlite_master WHERE name = '_zpan_id_backfill_map'").first(),
|
||||
).toBeNull()
|
||||
await expect(finalizeIdBackfill(env.DB, { version: 1, digest: plan.digest })).rejects.toThrow(
|
||||
'id_backfill_already_finalized:1',
|
||||
)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,88 @@
|
||||
import { sql } from 'drizzle-orm'
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { createTestApp } from '../test/setup'
|
||||
import { assertIdIntegrity } from './id-integrity'
|
||||
|
||||
describe('ID integrity release guard', () => {
|
||||
it('marks a fresh empty database once and later starts through one checkpoint query', async () => {
|
||||
const { app, db } = await createTestApp()
|
||||
// createTestApp initializes Better Auth first; production bootstrap runs
|
||||
// the integrity guard before auth creates its initial signing key.
|
||||
await db.run(sql`DELETE FROM jwks`)
|
||||
await db.run(sql`DELETE FROM oauthResource`)
|
||||
await expect(assertIdIntegrity(db)).resolves.toBeUndefined()
|
||||
expect(
|
||||
await db.all<{ value: string }>(sql`
|
||||
SELECT value FROM system_options WHERE key = 'id_normalization_version'
|
||||
`),
|
||||
).toEqual([{ value: '1' }])
|
||||
|
||||
await app.request('/api/auth/sign-up/email', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ name: 'ID Guard', email: 'id-guard@example.com', password: 'password123456' }),
|
||||
})
|
||||
const all = vi.spyOn(db, 'all')
|
||||
await expect(assertIdIntegrity(db)).resolves.toBeUndefined()
|
||||
expect(all).toHaveBeenCalledTimes(1)
|
||||
all.mockRestore()
|
||||
})
|
||||
|
||||
it('rejects malformed completion and pending checkpoints', async () => {
|
||||
const { db } = await createTestApp()
|
||||
await db.run(sql`INSERT INTO system_options (key, value) VALUES ('id_normalization_version', '2')`)
|
||||
await expect(assertIdIntegrity(db)).rejects.toThrow('id_integrity_checkpoint_invalid:id_normalization_version')
|
||||
await db.run(sql`DELETE FROM system_options WHERE key = 'id_normalization_version'`)
|
||||
await db.run(sql`
|
||||
INSERT INTO system_options (key, value)
|
||||
VALUES ('id_normalization_pending_artifact_digest', 'not-a-digest')
|
||||
`)
|
||||
await expect(assertIdIntegrity(db)).rejects.toThrow(
|
||||
'id_integrity_checkpoint_invalid:id_normalization_pending_artifact_digest',
|
||||
)
|
||||
})
|
||||
|
||||
it('does not mark an upgrade database with only a retained structured reference', async () => {
|
||||
const { db } = await createTestApp()
|
||||
await db.run(sql`DELETE FROM jwks`)
|
||||
await db.run(sql`DELETE FROM oauthResource`)
|
||||
await db.run(sql`
|
||||
INSERT INTO resource_changes (
|
||||
scope_type, scope_id, resource_type, resource_id, change_type, metadata, occurred_at
|
||||
) VALUES ('organization', 'legacy_org', 'share', 'legacy-share', 'upsert', '{"shareId":"legacy-share"}', 1)
|
||||
`)
|
||||
|
||||
await expect(assertIdIntegrity(db)).rejects.toThrow('id_normalization_checkpoint_required')
|
||||
expect(await db.all(sql`SELECT value FROM system_options WHERE key = 'id_normalization_version'`)).toEqual([])
|
||||
})
|
||||
|
||||
it('does not mark an upgrade database with only an invalidated credential row', async () => {
|
||||
const { db } = await createTestApp()
|
||||
await db.run(sql`DELETE FROM jwks`)
|
||||
await db.run(sql`DELETE FROM oauthResource`)
|
||||
await db.run(sql`INSERT INTO oauthClientAssertion (id, expires_at) VALUES ('legacy_assertion', 1)`)
|
||||
|
||||
await expect(assertIdIntegrity(db)).rejects.toThrow('id_normalization_checkpoint_required')
|
||||
expect(await db.all(sql`SELECT value FROM system_options WHERE key = 'id_normalization_version'`)).toEqual([])
|
||||
})
|
||||
|
||||
it('fails fast with counts but does not print identifier values', async () => {
|
||||
const { db } = await createTestApp()
|
||||
await db.run(sql`
|
||||
INSERT INTO storages (id, bucket, endpoint, region, access_key, secret_key, file_path, custom_host, capacity, used, status, created_at, updated_at)
|
||||
VALUES ('secret_bad-id', 'bucket', 'https://s3.example.com', 'auto', 'key', 'secret', '', '', 0, 0, 'active', 0, 0)
|
||||
`)
|
||||
await expect(assertIdIntegrity(db)).rejects.toThrow('id_integrity_failed:storages.id=1')
|
||||
await expect(assertIdIntegrity(db)).rejects.not.toThrow('secret_bad-id')
|
||||
})
|
||||
|
||||
it('fails when a redirect resource is missing its transactional reservation', async () => {
|
||||
const { db } = await createTestApp()
|
||||
await db.run(sql`
|
||||
INSERT INTO shares (id, token, kind, matter_id, org_id, creator_id, status, created_at)
|
||||
VALUES ('ShareId1', 'SharedToken1', 'direct', 'MatterId1', 'OrgId1', 'UserId1', 'active', 0)
|
||||
`)
|
||||
|
||||
await expect(assertIdIntegrity(db)).rejects.toThrow('redirect_tokens.registry=1')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,176 @@
|
||||
import { ID_NORMALIZATION_DATA_TABLES } from '@shared/id-normalization-inventory'
|
||||
import { sql } from 'drizzle-orm'
|
||||
import type { Database } from '../platform/interface'
|
||||
|
||||
const ID_TABLES = [
|
||||
'user',
|
||||
'session',
|
||||
'account',
|
||||
'verification',
|
||||
'jwks',
|
||||
'organization',
|
||||
'member',
|
||||
'invitation',
|
||||
'apikey',
|
||||
'deviceCode',
|
||||
'oauthClient',
|
||||
'oauthResource',
|
||||
'oauthClientResource',
|
||||
'oauthRefreshToken',
|
||||
'oauthAccessToken',
|
||||
'oauthConsent',
|
||||
'oauthPushedAuthorizationRequest',
|
||||
'downloader_bootstrap_credentials',
|
||||
'matters',
|
||||
'webdav_dead_properties',
|
||||
'webdav_locks',
|
||||
'storages',
|
||||
'org_quotas',
|
||||
'cloud_traffic_reports',
|
||||
'org_quota_entitlements',
|
||||
'webhook_events',
|
||||
'x402_capacity_purchase_intents',
|
||||
'invite_codes',
|
||||
'site_invitations',
|
||||
'license_bindings',
|
||||
'team_invite_links',
|
||||
'notifications',
|
||||
'background_jobs',
|
||||
'downloaders',
|
||||
'download_tasks',
|
||||
'object_upload_sessions',
|
||||
'remote_download_usage_reports',
|
||||
'announcements',
|
||||
'audit_events',
|
||||
'stats_rollups_hourly',
|
||||
'storage_usage_ledger',
|
||||
'shares',
|
||||
'share_recipients',
|
||||
'image_hostings',
|
||||
] as const
|
||||
|
||||
const TOKEN_COLUMNS = [
|
||||
['matters', 'alias'],
|
||||
['invite_codes', 'code'],
|
||||
['site_invitations', 'token'],
|
||||
['team_invite_links', 'token'],
|
||||
['shares', 'token'],
|
||||
['image_hostings', 'token'],
|
||||
['image_hosting_configs', 'verification_token'],
|
||||
['downloaders', 'token_jti'],
|
||||
] as const
|
||||
|
||||
const quote = (value: string) => `"${value.replaceAll('"', '""')}"`
|
||||
const literal = (value: string) => `'${value.replaceAll("'", "''")}'`
|
||||
const invalid = (column: string) => `${quote(column)} = '' OR ${quote(column)} GLOB '*[^A-Za-z0-9]*'`
|
||||
const COMPLETION_VERSION = '1'
|
||||
const DIGEST_PATTERN = /^[0-9a-f]{64}$/
|
||||
|
||||
type ReleaseCheckpoint = { key: string; value: string }
|
||||
|
||||
function validateReleaseCheckpoint(rows: ReleaseCheckpoint[]): boolean {
|
||||
const completion = rows.find(({ key }) => key === 'id_normalization_version')
|
||||
const pending = rows.find(({ key }) => key === 'id_normalization_pending_artifact_digest')
|
||||
if (completion && completion.value !== COMPLETION_VERSION) {
|
||||
throw new Error(`id_integrity_checkpoint_invalid:id_normalization_version`)
|
||||
}
|
||||
if (pending && !DIGEST_PATTERN.test(pending.value)) {
|
||||
throw new Error(`id_integrity_checkpoint_invalid:id_normalization_pending_artifact_digest`)
|
||||
}
|
||||
return completion !== undefined || pending !== undefined
|
||||
}
|
||||
|
||||
function markFreshEmptyDatabaseSql(): string {
|
||||
const emptyPredicates = ID_NORMALIZATION_DATA_TABLES.map(
|
||||
(table) => `NOT EXISTS (SELECT 1 FROM ${quote(table)} LIMIT 1)`,
|
||||
)
|
||||
emptyPredicates.push("NOT EXISTS (SELECT 1 FROM system_options WHERE key = 'instance_id' LIMIT 1)")
|
||||
return `INSERT INTO system_options (key, value)
|
||||
SELECT 'id_normalization_version', '${COMPLETION_VERSION}'
|
||||
WHERE ${emptyPredicates.join(' AND ')}
|
||||
ON CONFLICT(key) DO NOTHING
|
||||
RETURNING key, value`
|
||||
}
|
||||
|
||||
export function idIntegrityScanSql(): string[] {
|
||||
return [
|
||||
...ID_TABLES.map(
|
||||
(table) =>
|
||||
`SELECT ${literal(`${table}.id`)} AS field, COUNT(*) AS count FROM ${quote(table)} WHERE ${invalid('id')}`,
|
||||
),
|
||||
...TOKEN_COLUMNS.map(
|
||||
([table, column]) =>
|
||||
`SELECT ${literal(`${table}.${column}`)} AS field, COUNT(*) AS count FROM ${quote(table)} WHERE ${quote(column)} IS NOT NULL AND (${invalid(column)})`,
|
||||
),
|
||||
"SELECT 'system_options.instance_id' AS field, COUNT(*) AS count FROM system_options WHERE key = 'instance_id' AND (value = '' OR value GLOB '*[^A-Za-z0-9]*')",
|
||||
"SELECT 'redirect_token_registry.token' AS field, COUNT(*) AS count FROM redirect_token_registry WHERE token = '' OR token GLOB '*[^A-Za-z0-9]*'",
|
||||
"SELECT 'redirect_token_registry.resource_id' AS field, COUNT(*) AS count FROM redirect_token_registry WHERE resource_id = '' OR resource_id GLOB '*[^A-Za-z0-9]*'",
|
||||
]
|
||||
}
|
||||
|
||||
export const AMBIGUOUS_REDIRECT_SQL =
|
||||
"SELECT COUNT(*) AS count FROM shares s INNER JOIN image_hostings i ON i.token = s.token WHERE s.kind = 'direct' AND s.status = 'active' AND i.status = 'active'"
|
||||
|
||||
export const REDIRECT_REGISTRY_INTEGRITY_SQL = `SELECT COUNT(*) AS count FROM redirect_token_registry r
|
||||
WHERE (r.kind = 'direct_share' AND NOT EXISTS (
|
||||
SELECT 1 FROM shares s WHERE s.kind = 'direct' AND s.id = r.resource_id AND s.token = r.token
|
||||
)) OR (r.kind = 'image_hosting' AND NOT EXISTS (
|
||||
SELECT 1 FROM image_hostings i WHERE i.id = r.resource_id AND i.token = r.token
|
||||
)) OR r.kind NOT IN ('direct_share', 'image_hosting')
|
||||
UNION ALL
|
||||
SELECT COUNT(*) AS count FROM shares s
|
||||
WHERE s.kind = 'direct' AND NOT EXISTS (
|
||||
SELECT 1 FROM redirect_token_registry r WHERE r.kind = 'direct_share' AND r.resource_id = s.id AND r.token = s.token
|
||||
)
|
||||
UNION ALL
|
||||
SELECT COUNT(*) AS count FROM image_hostings i
|
||||
WHERE NOT EXISTS (
|
||||
SELECT 1 FROM redirect_token_registry r WHERE r.kind = 'image_hosting' AND r.resource_id = i.id AND r.token = i.token
|
||||
)`
|
||||
|
||||
export async function assertIdIntegrity(db: Database): Promise<void> {
|
||||
let releaseCheckpoint = (await db.all(
|
||||
sql.raw(
|
||||
"SELECT key, value FROM system_options WHERE key IN ('id_normalization_version', 'id_normalization_pending_artifact_digest')",
|
||||
),
|
||||
)) as ReleaseCheckpoint[]
|
||||
// The D1 maintenance executor writes the pending digest atomically with the
|
||||
// backfill and finalize replaces it with the completion marker after full
|
||||
// semantic scans. Avoid repeating unindexed release scans on every isolate.
|
||||
if (validateReleaseCheckpoint(releaseCheckpoint)) return
|
||||
|
||||
const initialized = (await db.all(sql.raw(markFreshEmptyDatabaseSql()))) as ReleaseCheckpoint[]
|
||||
if (initialized.length > 0) return
|
||||
// A concurrent isolate may have installed the same empty-database marker.
|
||||
releaseCheckpoint = (await db.all(
|
||||
sql.raw(
|
||||
"SELECT key, value FROM system_options WHERE key IN ('id_normalization_version', 'id_normalization_pending_artifact_digest')",
|
||||
),
|
||||
)) as ReleaseCheckpoint[]
|
||||
if (validateReleaseCheckpoint(releaseCheckpoint)) return
|
||||
const scans = idIntegrityScanSql()
|
||||
const rows: Array<{ field: string; count: number }> = []
|
||||
// D1 caps compound SELECT terms below SQLite's upstream default, so keep each
|
||||
// release-boundary scan comfortably below that limit.
|
||||
for (let offset = 0; offset < scans.length; offset += 5) {
|
||||
rows.push(
|
||||
...((await db.all(sql.raw(scans.slice(offset, offset + 5).join(' UNION ALL ')))) as Array<{
|
||||
field: string
|
||||
count: number
|
||||
}>),
|
||||
)
|
||||
}
|
||||
const failures = rows.filter(({ count }) => count > 0)
|
||||
const ambiguous = (await db.all(sql.raw(AMBIGUOUS_REDIRECT_SQL))) as Array<{ count: number }>
|
||||
if ((ambiguous[0]?.count ?? 0) > 0) failures.push({ field: 'redirect_tokens.ambiguous', count: ambiguous[0]!.count })
|
||||
const registryFailures = (await db.all(sql.raw(REDIRECT_REGISTRY_INTEGRITY_SQL))) as Array<{ count: number }>
|
||||
const registryFailureCount = registryFailures.reduce((total, row) => total + row.count, 0)
|
||||
if (registryFailureCount > 0) failures.push({ field: 'redirect_tokens.registry', count: registryFailureCount })
|
||||
if (failures.length > 0) {
|
||||
throw new Error(`id_integrity_failed:${failures.map(({ field, count }) => `${field}=${count}`).join(',')}`)
|
||||
}
|
||||
// A non-empty database cannot prove that already-Base62 public tokens were
|
||||
// rotated or credentials invalidated through value scans alone. Only the
|
||||
// rehearsal-backed apply/finalize flow may establish that release boundary.
|
||||
throw new Error('id_normalization_checkpoint_required')
|
||||
}
|
||||
@@ -82,6 +82,70 @@ describe('migration 0022_kind_storm.sql', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('migration 0092_base62_audit_event_key.sql', () => {
|
||||
const migrationPath = join(process.cwd(), 'migrations/0092_base62_audit_event_key.sql')
|
||||
const migration = readFileSync(migrationPath, 'utf-8')
|
||||
|
||||
it('adds a nullable unique idempotency key without rewriting audit primary IDs', () => {
|
||||
const db = new Database(':memory:')
|
||||
try {
|
||||
db.exec('CREATE TABLE audit_events (id TEXT PRIMARY KEY)')
|
||||
db.exec("INSERT INTO audit_events (id) VALUES ('legacy-event-id')")
|
||||
for (const statement of migration.split('--> statement-breakpoint')) db.exec(statement)
|
||||
|
||||
expect(db.prepare('SELECT id, event_key FROM audit_events').get()).toEqual({
|
||||
id: 'legacy-event-id',
|
||||
event_key: null,
|
||||
})
|
||||
db.exec("INSERT INTO audit_events (id, event_key) VALUES ('new-id', 'event:key')")
|
||||
expect(() => db.exec("INSERT INTO audit_events (id, event_key) VALUES ('other-id', 'event:key')")).toThrow()
|
||||
} finally {
|
||||
db.close()
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('migration 0093_redirect-token-registry.sql', () => {
|
||||
const migrationPath = join(process.cwd(), 'migrations/0093_redirect-token-registry.sql')
|
||||
const migration = readFileSync(migrationPath, 'utf-8')
|
||||
|
||||
it('creates a shared unique namespace for redirect token reservations', () => {
|
||||
const db = new Database(':memory:')
|
||||
try {
|
||||
for (const statement of migration.split('--> statement-breakpoint')) db.exec(statement)
|
||||
db.exec("INSERT INTO redirect_token_registry VALUES ('SharedToken1', 'direct_share', 'ShareId1')")
|
||||
expect(() =>
|
||||
db.exec("INSERT INTO redirect_token_registry VALUES ('SharedToken1', 'image_hosting', 'ImageId1')"),
|
||||
).toThrow()
|
||||
expect(() =>
|
||||
db.exec("INSERT INTO redirect_token_registry VALUES ('OtherToken2', 'image_hosting', 'ShareId1')"),
|
||||
).toThrow()
|
||||
} finally {
|
||||
db.close()
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('migration 0094_redirect-token-kind-resource.sql', () => {
|
||||
const createMigration = readFileSync(join(process.cwd(), 'migrations/0093_redirect-token-registry.sql'), 'utf-8')
|
||||
const migration = readFileSync(join(process.cwd(), 'migrations/0094_redirect-token-kind-resource.sql'), 'utf-8')
|
||||
|
||||
it('allows independent resource ID namespaces while retaining per-kind uniqueness', () => {
|
||||
const db = new Database(':memory:')
|
||||
try {
|
||||
for (const statement of createMigration.split('--> statement-breakpoint')) db.exec(statement)
|
||||
for (const statement of migration.split('--> statement-breakpoint')) db.exec(statement)
|
||||
db.exec("INSERT INTO redirect_token_registry VALUES ('ShareToken1', 'direct_share', 'SameId1')")
|
||||
db.exec("INSERT INTO redirect_token_registry VALUES ('ImageToken1', 'image_hosting', 'SameId1')")
|
||||
expect(() =>
|
||||
db.exec("INSERT INTO redirect_token_registry VALUES ('ShareToken2', 'direct_share', 'SameId1')"),
|
||||
).toThrow()
|
||||
} finally {
|
||||
db.close()
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('migration 0067_storage-health-status-default.sql', () => {
|
||||
const migrationPath = join(process.cwd(), 'migrations/0067_storage-health-status-default.sql')
|
||||
const migration = readFileSync(migrationPath, 'utf-8')
|
||||
|
||||
+16
-2
@@ -529,6 +529,7 @@ export const auditEvents = sqliteTable(
|
||||
'audit_events',
|
||||
{
|
||||
id: text('id').primaryKey(),
|
||||
eventKey: text('event_key').unique(),
|
||||
orgId: text('org_id').notNull(),
|
||||
userId: text('user_id'),
|
||||
action: text('action').notNull(), // 'upload', 'create', 'delete', 'rename', 'move', 'restore'
|
||||
@@ -644,6 +645,19 @@ export const shares = sqliteTable(
|
||||
],
|
||||
)
|
||||
|
||||
// Shared namespace for resources resolved by /r/:token. Keeping the reservation
|
||||
// in the same transaction as the resource insert makes cross-table uniqueness
|
||||
// an invariant instead of a best-effort preflight check.
|
||||
export const redirectTokenRegistry = sqliteTable(
|
||||
'redirect_token_registry',
|
||||
{
|
||||
token: text('token').primaryKey(),
|
||||
kind: text('kind').notNull(), // 'direct_share' | 'image_hosting'
|
||||
resourceId: text('resource_id').notNull(),
|
||||
},
|
||||
(t) => [uniqueIndex('redirect_token_registry_kind_resource_id_unique').on(t.kind, t.resourceId)],
|
||||
)
|
||||
|
||||
export const shareRecipients = sqliteTable(
|
||||
'share_recipients',
|
||||
{
|
||||
@@ -682,11 +696,11 @@ export const imageHostingConfigs = sqliteTable('image_hosting_configs', {
|
||||
export const imageHostings = sqliteTable(
|
||||
'image_hostings',
|
||||
{
|
||||
id: text('id').primaryKey(), // nanoid(12)
|
||||
id: text('id').primaryKey(), // 13-character Base62 ID
|
||||
orgId: text('org_id')
|
||||
.notNull()
|
||||
.references(() => organization.id, { onDelete: 'cascade' }),
|
||||
token: text('token').notNull().unique(), // "ih" + 10 alphanumeric characters
|
||||
token: text('token').notNull().unique(), // 12-character Base62 public token
|
||||
path: text('path').notNull(), // virtual path e.g. "blog/2026/04/shot.png"
|
||||
storageId: text('storage_id')
|
||||
.notNull()
|
||||
|
||||
@@ -35,10 +35,6 @@ export interface ImageUrlConfig {
|
||||
domainVerifiedAt: Date | null
|
||||
}
|
||||
|
||||
export function isImageHostingToken(token: string): boolean {
|
||||
return token.startsWith('ih')
|
||||
}
|
||||
|
||||
export function buildImageUrl(config: ImageUrlConfig | null, path: string, tokenUrl: string): string {
|
||||
if (config?.customDomain && config.domainVerifiedAt) {
|
||||
return `https://${config.customDomain}/${path}`
|
||||
|
||||
@@ -1200,7 +1200,7 @@ async function seedStatsFixture(db: Awaited<ReturnType<typeof createTestApp>>['d
|
||||
await db.run(sql`
|
||||
UPDATE audit_events
|
||||
SET created_at = ${nowSec}
|
||||
WHERE id = ${`event:user_register:${userId}`}
|
||||
WHERE event_key = ${`event:user_register:${userId}`}
|
||||
`)
|
||||
|
||||
await db.run(sql`
|
||||
|
||||
@@ -122,7 +122,7 @@ describe('better-auth admin user endpoints (migration target)', () => {
|
||||
const registrationBefore = await db.all<{ provider: string }>(sql`
|
||||
SELECT json_extract(metadata, '$.provider') AS provider
|
||||
FROM audit_events
|
||||
WHERE id = ${`event:user_register:${userId}`}
|
||||
WHERE event_key = ${`event:user_register:${userId}`}
|
||||
`)
|
||||
expect(registrationBefore).toEqual([{ provider: 'credential' }])
|
||||
|
||||
@@ -146,7 +146,7 @@ describe('better-auth admin user endpoints (migration target)', () => {
|
||||
const registrationAfter = await db.all<{ provider: string }>(sql`
|
||||
SELECT json_extract(metadata, '$.provider') AS provider
|
||||
FROM audit_events
|
||||
WHERE id = ${`event:user_register:${userId}`}
|
||||
WHERE event_key = ${`event:user_register:${userId}`}
|
||||
`)
|
||||
expect(registrationAfter).toEqual(registrationBefore)
|
||||
})
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { OpenAPIHono, z } from '@hono/zod-openapi'
|
||||
import { AuthorizationScope } from '@shared/authorization'
|
||||
import { createBackgroundJobRequestSchema, cursorPageSchema, listBackgroundJobsQuerySchema } from '../../shared/schemas'
|
||||
import { opaqueIdSchema } from '../../shared/schemas/identifiers'
|
||||
import type { Env } from '../middleware/platform'
|
||||
import {
|
||||
cancelBackgroundJob,
|
||||
@@ -30,9 +31,9 @@ const backgroundJobProgressSchema = z.object({
|
||||
|
||||
const backgroundJobSchema = z
|
||||
.object({
|
||||
id: z.string(),
|
||||
orgId: z.string(),
|
||||
userId: z.string(),
|
||||
id: opaqueIdSchema,
|
||||
orgId: opaqueIdSchema,
|
||||
userId: opaqueIdSchema,
|
||||
type: z.string(),
|
||||
status: z.string(),
|
||||
targetFolder: z.string().nullable(),
|
||||
@@ -119,7 +120,7 @@ const getJobRoute = authRoute(
|
||||
tags: ['Background Jobs'],
|
||||
method: 'get',
|
||||
path: '/{id}',
|
||||
request: { params: z.object({ id: z.string() }) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }) },
|
||||
responses: {
|
||||
200: jsonContent(backgroundJobSchema, 'Background job'),
|
||||
404: errorResponse('Not found'),
|
||||
@@ -135,7 +136,7 @@ const cancelJobRoute = authRoute(
|
||||
tags: ['Background Jobs'],
|
||||
method: 'put',
|
||||
path: '/{id}/status',
|
||||
request: { params: z.object({ id: z.string() }), ...jsonBody(cancelJobSchema) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }), ...jsonBody(cancelJobSchema) },
|
||||
responses: {
|
||||
200: jsonContent(backgroundJobSchema, 'Canceled background job'),
|
||||
404: errorResponse('Not found'),
|
||||
@@ -152,7 +153,7 @@ const retryJobRoute = authRoute(
|
||||
tags: ['Background Jobs'],
|
||||
method: 'post',
|
||||
path: '/{id}/retries',
|
||||
request: { params: z.object({ id: z.string() }) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }) },
|
||||
responses: {
|
||||
201: jsonContent(backgroundJobSchema, 'Retried background job'),
|
||||
404: errorResponse('Not found'),
|
||||
|
||||
@@ -11,6 +11,7 @@ import {
|
||||
listDownloadTasksQuerySchema,
|
||||
updateDownloadTaskSchema,
|
||||
} from '@shared/schemas'
|
||||
import { opaqueIdSchema } from '@shared/schemas/identifiers'
|
||||
import type { Env } from '../../middleware/platform'
|
||||
import {
|
||||
createDownloadTask,
|
||||
@@ -162,7 +163,7 @@ const getRoute = authRoute(
|
||||
tags: ['Download Tasks'],
|
||||
method: 'get',
|
||||
path: '/{id}',
|
||||
request: { params: z.object({ id: z.string() }) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }) },
|
||||
responses: {
|
||||
200: jsonContent(downloadTaskSchema, 'Download task'),
|
||||
...taskErrorResponses,
|
||||
@@ -178,7 +179,7 @@ const eventsRoute = authRoute(
|
||||
tags: ['Download Tasks'],
|
||||
method: 'get',
|
||||
path: '/{id}/events',
|
||||
request: { params: z.object({ id: z.string() }) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }) },
|
||||
responses: {
|
||||
200: jsonContent(downloadTaskTimelineSchema, 'Download task timeline'),
|
||||
...taskErrorResponses,
|
||||
@@ -194,7 +195,7 @@ const updateRoute = authRoute(
|
||||
tags: ['Download Tasks'],
|
||||
method: 'patch',
|
||||
path: '/{id}',
|
||||
request: { params: z.object({ id: z.string() }), ...jsonBody(updateDownloadTaskSchema) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }), ...jsonBody(updateDownloadTaskSchema) },
|
||||
responses: {
|
||||
200: jsonContent(downloadTaskSchema, 'Updated download task'),
|
||||
...taskErrorResponses,
|
||||
@@ -210,7 +211,7 @@ const statusRoute = authRoute(
|
||||
tags: ['Download Tasks'],
|
||||
method: 'put',
|
||||
path: '/{id}/status',
|
||||
request: { params: z.object({ id: z.string() }), ...jsonBody(downloadTaskStatusUpdateSchema) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }), ...jsonBody(downloadTaskStatusUpdateSchema) },
|
||||
responses: {
|
||||
200: jsonContent(downloadTaskSchema, 'Updated download task'),
|
||||
...taskErrorResponses,
|
||||
@@ -226,7 +227,7 @@ const attemptRoute = authRoute(
|
||||
tags: ['Download Tasks'],
|
||||
method: 'post',
|
||||
path: '/{id}/attempts',
|
||||
request: { params: z.object({ id: z.string() }), ...jsonBody(downloadTaskAttemptSchema) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }), ...jsonBody(downloadTaskAttemptSchema) },
|
||||
responses: {
|
||||
201: jsonContent(downloadTaskSchema, 'New download attempt'),
|
||||
...taskErrorResponses,
|
||||
@@ -242,7 +243,7 @@ const deleteRoute = authRoute(
|
||||
tags: ['Download Tasks'],
|
||||
method: 'delete',
|
||||
path: '/{id}',
|
||||
request: { params: z.object({ id: z.string() }) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }) },
|
||||
responses: {
|
||||
204: { description: 'Deleted download task' },
|
||||
...taskErrorResponses,
|
||||
|
||||
@@ -10,6 +10,7 @@ import {
|
||||
updateDownloaderCreditBillingSchema,
|
||||
updateDownloaderSchema,
|
||||
} from '@shared/schemas'
|
||||
import { opaqueIdSchema } from '@shared/schemas/identifiers'
|
||||
import { FREE_DOWNLOADER_LIMIT } from '../../../shared/constants'
|
||||
import { hasFeature } from '../../domain/licensing'
|
||||
import type { Env } from '../../middleware/platform'
|
||||
@@ -71,7 +72,7 @@ const updateRoute = authRoute(
|
||||
tags: ['Downloaders'],
|
||||
method: 'patch',
|
||||
path: '/{id}',
|
||||
request: { params: z.object({ id: z.string() }), ...jsonBody(updateDownloaderSchema) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }), ...jsonBody(updateDownloaderSchema) },
|
||||
responses: {
|
||||
200: jsonContent(downloaderSchema, 'Updated downloader'),
|
||||
402: errorResponse('Feature not available'),
|
||||
@@ -88,7 +89,7 @@ const updateCreditBillingRoute = authRoute(
|
||||
tags: ['Downloaders'],
|
||||
method: 'put',
|
||||
path: '/{id}/credit-billing',
|
||||
request: { params: z.object({ id: z.string() }), ...jsonBody(updateDownloaderCreditBillingSchema) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }), ...jsonBody(updateDownloaderCreditBillingSchema) },
|
||||
responses: {
|
||||
200: jsonContent(downloaderSchema, 'Updated downloader'),
|
||||
402: errorResponse('Feature not available'),
|
||||
@@ -105,7 +106,7 @@ const deleteRoute = authRoute(
|
||||
tags: ['Downloaders'],
|
||||
method: 'delete',
|
||||
path: '/{id}',
|
||||
request: { params: z.object({ id: z.string() }) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }) },
|
||||
responses: {
|
||||
204: { description: 'Deleted downloader' },
|
||||
404: errorResponse('Not found'),
|
||||
|
||||
@@ -1,13 +1,14 @@
|
||||
import { z } from '@hono/zod-openapi'
|
||||
import { pageSchema } from '@shared/schemas'
|
||||
import { opaqueIdSchema } from '@shared/schemas/identifiers'
|
||||
import type { EntitlementResult, QuotaEntitlementItem } from '../usecases/ports'
|
||||
|
||||
// Quota entitlement DTO shared by the team- and user-scoped admin endpoints. The
|
||||
// domain record carries Date timestamps; toQuotaEntitlementDTO serializes them.
|
||||
export const quotaEntitlementSchema = z
|
||||
.object({
|
||||
id: z.string(),
|
||||
orgId: z.string(),
|
||||
id: opaqueIdSchema,
|
||||
orgId: opaqueIdSchema,
|
||||
resourceType: z.string(),
|
||||
entitlementType: z.string(),
|
||||
source: z.string(),
|
||||
@@ -35,7 +36,7 @@ export function toQuotaEntitlementDTO(e: QuotaEntitlementItem): QuotaEntitlement
|
||||
}
|
||||
|
||||
export const entitlementResultSchema = z
|
||||
.object({ orgId: z.string(), entitlement: quotaEntitlementSchema })
|
||||
.object({ orgId: opaqueIdSchema, entitlement: quotaEntitlementSchema })
|
||||
.openapi('EntitlementResult')
|
||||
|
||||
export function toEntitlementResultDTO(r: EntitlementResult): z.infer<typeof entitlementResultSchema> {
|
||||
|
||||
@@ -309,9 +309,14 @@ describe('POST /api/image-hosting/images/presign (JSON two-stage)', () => {
|
||||
const body = (await res.json()) as Record<string, unknown>
|
||||
expect(body.uploadUrl).toBe('https://presigned-upload.example.com')
|
||||
expect(body.id).toBeTruthy()
|
||||
expect(String(body.token)).toMatch(/^ih[A-Za-z0-9]{10}$/)
|
||||
expect(String(body.token)).toMatch(/^[A-Za-z0-9]{12}$/)
|
||||
expect(body.path).toBe('blog/2026/shot.png')
|
||||
expect(String(body.storageKey)).toMatch(/^ih\//)
|
||||
expect(
|
||||
await db.all(
|
||||
sql`SELECT token, kind, resource_id AS resourceId FROM redirect_token_registry WHERE token = ${String(body.token)}`,
|
||||
),
|
||||
).toEqual([{ token: body.token, kind: 'image_hosting', resourceId: body.id }])
|
||||
})
|
||||
|
||||
it('returns 400 for path with .. [spec: image-hosting/path-traversal]', async () => {
|
||||
@@ -589,7 +594,7 @@ describe('POST /api/image-hosting/images (multipart)', () => {
|
||||
const body = (await res.json()) as { data: Record<string, unknown> }
|
||||
expect(body.data).toBeDefined()
|
||||
expect(body.data.url).toBeTruthy()
|
||||
expect(String(body.data.urlAlt)).toMatch(/\/r\/ih[A-Za-z0-9]{10}$/)
|
||||
expect(String(body.data.urlAlt)).toMatch(/\/r\/[A-Za-z0-9]{12}$/)
|
||||
expect(String(body.data.markdown)).toContain('
|
||||
expect(String(body.data.html)).toContain('<img src=')
|
||||
expect(String(body.data.bbcode)).toContain('[img]')
|
||||
@@ -724,7 +729,7 @@ describe('POST /api/image-hosting/images (multipart)', () => {
|
||||
const body = (await res.json()) as { data: Record<string, unknown> }
|
||||
// url and urlAlt should both be the token URL when no custom domain
|
||||
expect(body.data.url).toBe(body.data.urlAlt)
|
||||
expect(String(body.data.url)).toMatch(/\/r\/ih[A-Za-z0-9]{10}$/)
|
||||
expect(String(body.data.url)).toMatch(/\/r\/[A-Za-z0-9]{12}$/)
|
||||
})
|
||||
|
||||
it('cleans up DB row and refunds quota when S3 put fails', async () => {
|
||||
@@ -1209,6 +1214,23 @@ describe('DELETE /api/image-hosting/images/:id', () => {
|
||||
expect(res.status).toBe(404)
|
||||
})
|
||||
|
||||
it('removes the redirect token reservation when a draft is hard-deleted', async () => {
|
||||
const { app, db } = await createTestApp()
|
||||
await insertStorage(db)
|
||||
const headers = await authedHeaders(app)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHostingConfig(db, orgId)
|
||||
const createRes = await app.request('/api/image-hosting/images/presign', {
|
||||
method: 'POST',
|
||||
headers: { ...headers, 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ path: 'discard-draft.png', mime: 'image/png', size: 2048 }),
|
||||
})
|
||||
const image = (await createRes.json()) as { id: string; token: string }
|
||||
|
||||
expect((await app.request(`/api/image-hosting/images/${image.id}`, { method: 'DELETE', headers })).status).toBe(204)
|
||||
expect(await db.all(sql`SELECT token FROM redirect_token_registry WHERE resource_id = ${image.id}`)).toEqual([])
|
||||
})
|
||||
|
||||
it('removes the S3 object and retains a hidden DB tombstone', async () => {
|
||||
const { app, db } = await createTestApp()
|
||||
await insertStorage(db)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { OpenAPIHono, z } from '@hono/zod-openapi'
|
||||
import { AuthorizationScope } from '@shared/authorization'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { generateToken } from '@shared/ids'
|
||||
import { opaqueIdSchema, opaqueTokenSchema } from '@shared/schemas/identifiers'
|
||||
import {
|
||||
ALLOWED_IMAGE_MIMES,
|
||||
createIhostImageSchema,
|
||||
@@ -42,12 +43,12 @@ import {
|
||||
// them as Date).
|
||||
const imageHostingSchema = z
|
||||
.object({
|
||||
id: z.string(),
|
||||
orgId: z.string(),
|
||||
token: z.string(),
|
||||
id: opaqueIdSchema,
|
||||
orgId: opaqueIdSchema,
|
||||
token: opaqueTokenSchema,
|
||||
path: z.string(),
|
||||
url: z.string(),
|
||||
storageId: z.string(),
|
||||
storageId: opaqueIdSchema,
|
||||
storageKey: z.string(),
|
||||
size: z.number().int(),
|
||||
mime: z.string(),
|
||||
@@ -78,8 +79,8 @@ function toImageHostingDTO(
|
||||
|
||||
const imageDraftSchema = z
|
||||
.object({
|
||||
id: z.string(),
|
||||
token: z.string(),
|
||||
id: opaqueIdSchema,
|
||||
token: opaqueTokenSchema,
|
||||
path: z.string(),
|
||||
uploadUrl: z.string(),
|
||||
storageKey: z.string(),
|
||||
@@ -87,12 +88,12 @@ const imageDraftSchema = z
|
||||
.openapi('ImageHostingDraft')
|
||||
|
||||
const imageListSchema = z
|
||||
.object({ items: z.array(imageHostingSchema), nextPageToken: z.string().nullable() })
|
||||
.object({ items: z.array(imageHostingSchema), nextPageToken: opaqueTokenSchema.nullable() })
|
||||
.openapi('ImageHostingList')
|
||||
|
||||
// Derive a storage path from the upload's filename, falling back to a random name.
|
||||
function deriveDefaultPath(filename: string, mime: string): string {
|
||||
if (!filename || filename === 'blob') return `image-${nanoid(8)}.${mimeToExt(mime)}`
|
||||
if (!filename || filename === 'blob') return `image-${generateToken(9)}.${mimeToExt(mime)}`
|
||||
return filename.replace(/[/\\]/g, '_')
|
||||
}
|
||||
|
||||
@@ -161,7 +162,7 @@ const getRoute = authRoute(
|
||||
tags: ['Image Hosting'],
|
||||
method: 'get',
|
||||
path: '/images/{id}',
|
||||
request: { params: z.object({ id: z.string() }) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }) },
|
||||
responses: {
|
||||
200: jsonContent(imageHostingSchema, 'Hosted image'),
|
||||
400: errorResponse('No active organization'),
|
||||
@@ -179,7 +180,7 @@ const confirmRoute = authRoute(
|
||||
tags: ['Image Hosting'],
|
||||
method: 'put',
|
||||
path: '/images/{id}/status',
|
||||
request: { params: z.object({ id: z.string() }) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }) },
|
||||
responses: {
|
||||
200: jsonContent(imageHostingSchema, 'Confirmed image'),
|
||||
400: errorResponse('No active organization'),
|
||||
@@ -198,7 +199,7 @@ const deleteRoute = authRoute(
|
||||
tags: ['Image Hosting'],
|
||||
method: 'delete',
|
||||
path: '/images/{id}',
|
||||
request: { params: z.object({ id: z.string() }) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }) },
|
||||
responses: {
|
||||
204: { description: 'Deleted' },
|
||||
400: errorResponse('No active organization'),
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { OpenAPIHono, z } from '@hono/zod-openapi'
|
||||
import { AuthorizationScope } from '@shared/authorization'
|
||||
import { cursorPageSchema, listNotificationsQuerySchema } from '@shared/schemas'
|
||||
import { opaqueIdSchema } from '@shared/schemas/identifiers'
|
||||
import type { Env } from '../middleware/platform'
|
||||
import {
|
||||
getUnreadCount,
|
||||
@@ -19,8 +20,8 @@ import {
|
||||
|
||||
const notificationSchema = z
|
||||
.object({
|
||||
id: z.string(),
|
||||
userId: z.string(),
|
||||
id: opaqueIdSchema,
|
||||
userId: opaqueIdSchema,
|
||||
type: z.string(),
|
||||
title: z.string(),
|
||||
body: z.string(),
|
||||
@@ -88,7 +89,7 @@ const markReadRoute = authRoute(
|
||||
tags: ['Notifications'],
|
||||
method: 'patch',
|
||||
path: '/{id}',
|
||||
request: { params: z.object({ id: z.string() }) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }) },
|
||||
responses: {
|
||||
204: { description: 'Marked read' },
|
||||
404: errorResponse('Not found'),
|
||||
|
||||
@@ -14,6 +14,7 @@ import {
|
||||
} from '@shared/schemas'
|
||||
import type { Context } from 'hono'
|
||||
import { ZPAN_CLOUD_URL_DEFAULT } from '../../shared/constants'
|
||||
import { opaqueIdSchema, opaqueTokenSchema } from '../../shared/schemas/identifiers'
|
||||
import { transferAuditActor } from '../middleware/audit-transfers'
|
||||
import { boundWorkspaceOrgId, type Env } from '../middleware/platform'
|
||||
import {
|
||||
@@ -45,9 +46,9 @@ import { decodeOptionalPageToken, directoryCursorCodec, encodeNextPageToken, pag
|
||||
// OpenAPI document and SDKs derive the `Matter` model from.
|
||||
const matterSchema = z
|
||||
.object({
|
||||
id: z.string(),
|
||||
orgId: z.string(),
|
||||
alias: z.string(),
|
||||
id: opaqueIdSchema,
|
||||
orgId: opaqueIdSchema,
|
||||
alias: opaqueTokenSchema,
|
||||
name: z.string(),
|
||||
type: z.string(),
|
||||
size: z.number().int().nullable(),
|
||||
@@ -56,7 +57,7 @@ const matterSchema = z
|
||||
.string()
|
||||
.describe('Slash-delimited parent folder path relative to the workspace root; empty for root objects.'),
|
||||
object: z.string(),
|
||||
storageId: z.string(),
|
||||
storageId: opaqueIdSchema,
|
||||
status: z.string(),
|
||||
trashedAt: z.number().int().nullable(),
|
||||
createdAt: z.string(),
|
||||
@@ -156,11 +157,11 @@ const listObjectsQuerySchema = cursorPageQuerySchema.extend({
|
||||
.optional(),
|
||||
type: z.string().optional(),
|
||||
search: z.string().optional(),
|
||||
orgId: z.string().optional(),
|
||||
orgId: opaqueIdSchema.optional(),
|
||||
})
|
||||
|
||||
const idParam = z.object({ id: z.string() })
|
||||
const sessionParams = z.object({ id: z.string(), uploadSessionId: z.string() })
|
||||
const idParam = z.object({ id: opaqueIdSchema })
|
||||
const sessionParams = z.object({ id: opaqueIdSchema, uploadSessionId: opaqueIdSchema })
|
||||
const abortUploadQuerySchema = z.object({
|
||||
strictStorageCleanup: z.enum(['1', 'true']).optional(),
|
||||
})
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { encodeBase62Bytes } from '@shared/ids'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import type { Platform } from '../platform/interface'
|
||||
import {
|
||||
@@ -20,7 +21,11 @@ function platform(secret = 'test-secret'): Platform {
|
||||
}
|
||||
|
||||
async function signRawBody(body: string, secret: string): Promise<string> {
|
||||
const encodedBody = Buffer.from(body).toString('base64url')
|
||||
const payload = new TextEncoder().encode(body)
|
||||
const signed = new Uint8Array(5 + payload.length)
|
||||
signed[0] = 1
|
||||
new DataView(signed.buffer).setUint32(1, payload.length)
|
||||
signed.set(payload, 5)
|
||||
const key = await crypto.subtle.importKey(
|
||||
'raw',
|
||||
new TextEncoder().encode(`zpan:page-token:v1:${secret}`),
|
||||
@@ -28,8 +33,11 @@ async function signRawBody(body: string, secret: string): Promise<string> {
|
||||
false,
|
||||
['sign'],
|
||||
)
|
||||
const signature = await crypto.subtle.sign('HMAC', key, new TextEncoder().encode(encodedBody))
|
||||
return `${encodedBody}.${Buffer.from(signature).toString('base64url')}`
|
||||
const signature = new Uint8Array(await crypto.subtle.sign('HMAC', key, signed))
|
||||
const envelope = new Uint8Array(signed.length + signature.length)
|
||||
envelope.set(signed)
|
||||
envelope.set(signature, signed.length)
|
||||
return encodeBase62Bytes(envelope)
|
||||
}
|
||||
|
||||
describe('page tokens', () => {
|
||||
@@ -41,6 +49,8 @@ describe('page tokens', () => {
|
||||
now: 1_000,
|
||||
})
|
||||
|
||||
expect(query).toMatch(/^[A-Za-z0-9]+$/)
|
||||
expect(token).toMatch(/^[A-Za-z0-9]+$/)
|
||||
await expect(decodePageToken(platform(), token, { query, now: 2_000 })).resolves.toEqual({
|
||||
createdAt: 123,
|
||||
id: 'item-1',
|
||||
@@ -131,4 +141,11 @@ describe('page tokens', () => {
|
||||
meta: { reason: 'INVALID_PAGE_TOKEN' },
|
||||
})
|
||||
})
|
||||
|
||||
it('rejects the pre-release Base64url dotted format without a legacy decoder', async () => {
|
||||
await expect(decodePageToken(platform(), 'eyJ2IjoxfQ.signature', { query: 'query' })).rejects.toMatchObject({
|
||||
httpStatus: 400,
|
||||
meta: { reason: 'INVALID_PAGE_TOKEN' },
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
+29
-25
@@ -1,9 +1,12 @@
|
||||
import { decodeBase62Bytes, encodeBase62Bytes, isBase62 } from '@shared/ids'
|
||||
import type { Platform } from '../platform/interface'
|
||||
import { badRequest } from '../usecases/ports'
|
||||
|
||||
const TOKEN_VERSION = 1
|
||||
const TOKEN_TTL_MS = 72 * 60 * 60 * 1000
|
||||
const TOKEN_PURPOSE = 'zpan:page-token:v1'
|
||||
const TOKEN_HEADER_BYTES = 5
|
||||
const TOKEN_SIGNATURE_BYTES = 32
|
||||
|
||||
export type PageBoundary = Record<string, string | number | null>
|
||||
|
||||
@@ -23,14 +26,6 @@ function invalidPageToken(): never {
|
||||
throw badRequest('Invalid page token', 'INVALID_PAGE_TOKEN')
|
||||
}
|
||||
|
||||
function encodeBase64Url(value: Uint8Array): string {
|
||||
return Buffer.from(value).toString('base64url')
|
||||
}
|
||||
|
||||
function decodeBase64Url(value: string): Uint8Array {
|
||||
return new Uint8Array(Buffer.from(value, 'base64url'))
|
||||
}
|
||||
|
||||
function secret(platform: Platform): string {
|
||||
const value = platform.getEnv('BETTER_AUTH_SECRET')
|
||||
if (!value) throw new Error('BETTER_AUTH_SECRET is required')
|
||||
@@ -49,7 +44,7 @@ async function signingKey(platform: Platform): Promise<CryptoKey> {
|
||||
|
||||
export async function pageQueryFingerprint(value: unknown): Promise<string> {
|
||||
const digest = await crypto.subtle.digest('SHA-256', new TextEncoder().encode(JSON.stringify(value)))
|
||||
return encodeBase64Url(new Uint8Array(digest))
|
||||
return encodeBase62Bytes(new Uint8Array(digest))
|
||||
}
|
||||
|
||||
export async function encodePageToken(
|
||||
@@ -62,9 +57,16 @@ export async function encodePageToken(
|
||||
query: input.query,
|
||||
expiresAt: (input.now ?? Date.now()) + TOKEN_TTL_MS,
|
||||
}
|
||||
const body = encodeBase64Url(new TextEncoder().encode(JSON.stringify(payload)))
|
||||
const signature = await crypto.subtle.sign('HMAC', await signingKey(platform), new TextEncoder().encode(body))
|
||||
return `${body}.${encodeBase64Url(new Uint8Array(signature))}`
|
||||
const payloadBytes = new TextEncoder().encode(JSON.stringify(payload))
|
||||
const signed = new Uint8Array(TOKEN_HEADER_BYTES + payloadBytes.length)
|
||||
signed[0] = TOKEN_VERSION
|
||||
new DataView(signed.buffer).setUint32(1, payloadBytes.length)
|
||||
signed.set(payloadBytes, TOKEN_HEADER_BYTES)
|
||||
const signature = new Uint8Array(await crypto.subtle.sign('HMAC', await signingKey(platform), signed))
|
||||
const envelope = new Uint8Array(signed.length + signature.length)
|
||||
envelope.set(signed)
|
||||
envelope.set(signature, signed.length)
|
||||
return encodeBase62Bytes(envelope)
|
||||
}
|
||||
|
||||
export async function decodePageToken(
|
||||
@@ -72,23 +74,25 @@ export async function decodePageToken(
|
||||
token: string,
|
||||
input: { query: string; now?: number },
|
||||
): Promise<PageBoundary> {
|
||||
const [body, signature, extra] = token.split('.')
|
||||
if (!body || !signature || extra) invalidPageToken()
|
||||
|
||||
const signatureBytes = decodeBase64Url(signature)
|
||||
const verificationSignature = new Uint8Array(signatureBytes.byteLength)
|
||||
verificationSignature.set(signatureBytes)
|
||||
const valid = await crypto.subtle.verify(
|
||||
'HMAC',
|
||||
await signingKey(platform),
|
||||
verificationSignature,
|
||||
new TextEncoder().encode(body),
|
||||
)
|
||||
if (!isBase62(token)) invalidPageToken()
|
||||
let envelope: Uint8Array
|
||||
try {
|
||||
envelope = decodeBase62Bytes(token)
|
||||
} catch {
|
||||
invalidPageToken()
|
||||
}
|
||||
if (envelope.length < TOKEN_HEADER_BYTES + TOKEN_SIGNATURE_BYTES || envelope[0] !== TOKEN_VERSION) invalidPageToken()
|
||||
const payloadLength = new DataView(envelope.buffer, envelope.byteOffset, envelope.byteLength).getUint32(1)
|
||||
const signedLength = TOKEN_HEADER_BYTES + payloadLength
|
||||
if (envelope.length !== signedLength + TOKEN_SIGNATURE_BYTES) invalidPageToken()
|
||||
const signed = envelope.slice(0, signedLength)
|
||||
const verificationSignature = envelope.slice(signedLength)
|
||||
const valid = await crypto.subtle.verify('HMAC', await signingKey(platform), verificationSignature, signed)
|
||||
if (!valid) invalidPageToken()
|
||||
|
||||
let payload: PageTokenPayload
|
||||
try {
|
||||
payload = JSON.parse(new TextDecoder().decode(decodeBase64Url(body))) as PageTokenPayload
|
||||
payload = JSON.parse(new TextDecoder().decode(signed.slice(TOKEN_HEADER_BYTES))) as PageTokenPayload
|
||||
} catch {
|
||||
invalidPageToken()
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { OpenAPIHono, z } from '@hono/zod-openapi'
|
||||
import { AuthorizationScope } from '@shared/authorization'
|
||||
import { pageSchema } from '@shared/schemas'
|
||||
import { opaqueIdSchema } from '@shared/schemas/identifiers'
|
||||
import type { Env } from '../middleware/platform'
|
||||
import { notFound } from '../usecases/ports'
|
||||
import { getUserQuota, listQuotaOverview } from '../usecases/quota'
|
||||
@@ -21,7 +22,7 @@ const currentStoragePlanSchema = z.object({
|
||||
|
||||
const effectiveQuotaSchema = z
|
||||
.object({
|
||||
orgId: z.string(),
|
||||
orgId: opaqueIdSchema,
|
||||
baseQuota: z.number().int(),
|
||||
entitlementQuota: z.number().int(),
|
||||
quota: z.number().int(),
|
||||
@@ -40,7 +41,7 @@ const effectiveQuotaSchema = z
|
||||
.openapi('EffectiveQuota')
|
||||
|
||||
const quotaOverviewItemSchema = effectiveQuotaSchema
|
||||
.extend({ id: z.string(), orgName: z.string(), orgType: z.string() })
|
||||
.extend({ id: opaqueIdSchema, orgName: z.string(), orgType: z.string() })
|
||||
.openapi('QuotaOverviewItem')
|
||||
|
||||
const quotaOverviewSchema = pageSchema(quotaOverviewItemSchema, 'QuotaOverview')
|
||||
|
||||
@@ -129,7 +129,7 @@ describe('[CF] /r/:token image hosting', () => {
|
||||
const { app, db } = await buildApp()
|
||||
const { orgId } = await signUpAndGetIds(app, db)
|
||||
await insertStorage(db)
|
||||
const token = `ih_cfext${Date.now()}`
|
||||
const token = `ihCfext${Date.now()}`
|
||||
await insertImageHosting(db, orgId, { id: `cf-ihext-${Date.now()}`, token })
|
||||
|
||||
const res = await app.request(`/r/${token}.jpg`, { redirect: 'manual' })
|
||||
|
||||
@@ -102,9 +102,9 @@ async function getAccessCount(db: Awaited<ReturnType<typeof createTestApp>>['db'
|
||||
return rows[0]?.access_count ?? 0
|
||||
}
|
||||
|
||||
// ─── ds_ direct share tests ───────────────────────────────────────────────────
|
||||
// ─── direct share tests ───────────────────────────────────────────────────────
|
||||
|
||||
describe('GET /r/:token (ds_ direct shares)', () => {
|
||||
describe('GET /r/:token (direct shares)', () => {
|
||||
it('returns 302 with attachment disposition and no-store cache for valid direct share [spec: redirect/direct-share]', async () => {
|
||||
const { app, db } = await createTestApp()
|
||||
await authedHeaders(app)
|
||||
@@ -132,12 +132,34 @@ describe('GET /r/:token (ds_ direct shares)', () => {
|
||||
])
|
||||
})
|
||||
|
||||
it('returns 404 for unknown ds_ token [spec: redirect/unknown-ds-token]', async () => {
|
||||
it('returns 404 for a legacy punctuated token [spec: redirect/unknown-ds-token]', async () => {
|
||||
const { app } = await createTestApp()
|
||||
const res = await app.request('/r/ds_unknowntoken', { redirect: 'manual' })
|
||||
expect(res.status).toBe(404)
|
||||
})
|
||||
|
||||
it('fails closed when a token is owned by both redirect resource tables', async () => {
|
||||
const { app, db } = await createTestApp()
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
const creatorId = await getUserId(db)
|
||||
await insertFile(db, orgId, { id: 'collision-file', name: 'collision.bin' })
|
||||
const share = await createShareRepo(db).create({
|
||||
matterId: 'collision-file',
|
||||
orgId,
|
||||
creatorId,
|
||||
kind: 'direct',
|
||||
})
|
||||
await db.run(sql`UPDATE shares SET token = 'CollisionToken' WHERE id = ${share.id}`)
|
||||
await insertImageHosting(db, orgId, { id: 'collision-image', token: 'CollisionToken' })
|
||||
|
||||
const res = await app.request('/r/CollisionToken', { redirect: 'manual' })
|
||||
expect(res.status).toBe(404)
|
||||
expect(S3Service.prototype.presignDownload).not.toHaveBeenCalled()
|
||||
expect(S3Service.prototype.presignInline).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('returns 404 for landing share token at /r/ [spec: redirect/landing-token-rejected]', async () => {
|
||||
const { app, db } = await createTestApp()
|
||||
await authedHeaders(app)
|
||||
@@ -293,9 +315,9 @@ describe('GET /r/:token (image hosting)', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, { id: 'ih-img1', token: 'ih_testtoken1' })
|
||||
await insertImageHosting(db, orgId, { id: 'ih-img1', token: 'ihTesttoken1' })
|
||||
|
||||
const res = await app.request('/r/ih_testtoken1', { redirect: 'manual' })
|
||||
const res = await app.request('/r/ihTesttoken1', { redirect: 'manual' })
|
||||
expect(res.status).toBe(302)
|
||||
expect(res.headers.get('location')).toBe(MOCK_INLINE_URL)
|
||||
const cc = res.headers.get('cache-control') ?? ''
|
||||
@@ -332,9 +354,9 @@ describe('GET /r/:token (image hosting)', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, { id: 'ih-img2', token: 'ih_exttest1' })
|
||||
await insertImageHosting(db, orgId, { id: 'ih-img2', token: 'ihExttest1' })
|
||||
|
||||
const res = await app.request('/r/ih_exttest1.png', { redirect: 'manual' })
|
||||
const res = await app.request('/r/ihExttest1.png', { redirect: 'manual' })
|
||||
expect(res.status).toBe(302)
|
||||
expect(res.headers.get('location')).toBe(MOCK_INLINE_URL)
|
||||
})
|
||||
@@ -344,16 +366,16 @@ describe('GET /r/:token (image hosting)', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, { id: 'ih-img3', token: 'ih_exttest2' })
|
||||
await insertImageHosting(db, orgId, { id: 'ih-img3', token: 'ihExttest2' })
|
||||
|
||||
const res = await app.request('/r/ih_exttest2.webp', { redirect: 'manual' })
|
||||
const res = await app.request('/r/ihExttest2.webp', { redirect: 'manual' })
|
||||
expect(res.status).toBe(302)
|
||||
expect(res.headers.get('location')).toBe(MOCK_INLINE_URL)
|
||||
})
|
||||
|
||||
it('returns 404 for non-existent ih_ token [spec: redirect/unknown-ih-token]', async () => {
|
||||
const { app } = await createTestApp()
|
||||
const res = await app.request('/r/ih_doesnotexist', { redirect: 'manual' })
|
||||
const res = await app.request('/r/ihDoesnotexist', { redirect: 'manual' })
|
||||
expect(res.status).toBe(404)
|
||||
})
|
||||
|
||||
@@ -362,9 +384,9 @@ describe('GET /r/:token (image hosting)', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, { id: 'ih-draft1', token: 'ih_drafttoken', status: 'draft' })
|
||||
await insertImageHosting(db, orgId, { id: 'ih-draft1', token: 'ihDrafttoken', status: 'draft' })
|
||||
|
||||
const res = await app.request('/r/ih_drafttoken', { redirect: 'manual' })
|
||||
const res = await app.request('/r/ihDrafttoken', { redirect: 'manual' })
|
||||
expect(res.status).toBe(404)
|
||||
})
|
||||
|
||||
@@ -375,11 +397,11 @@ describe('GET /r/:token (image hosting)', () => {
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, {
|
||||
id: 'ih-no-storage',
|
||||
token: 'ih_nostorage',
|
||||
token: 'ihNostorage',
|
||||
storageId: 'st-missing-storage',
|
||||
})
|
||||
|
||||
const res = await app.request('/r/ih_nostorage', { redirect: 'manual' })
|
||||
const res = await app.request('/r/ihNostorage', { redirect: 'manual' })
|
||||
expect(res.status).toBe(404)
|
||||
const body = (await res.json()) as { error: { message: string; status: string } }
|
||||
expect(body.error.message).toBe('Storage not found')
|
||||
@@ -393,7 +415,7 @@ describe('GET /r/:token (image hosting)', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, { id: 'ih-credits', token: 'ih_credits' })
|
||||
await insertImageHosting(db, orgId, { id: 'ih-credits', token: 'ihCredits' })
|
||||
|
||||
const redirectUsecase = await import('../usecases/redirect.js')
|
||||
vi.spyOn(redirectUsecase, 'resolveImageHostingDownload').mockResolvedValueOnce({
|
||||
@@ -401,7 +423,7 @@ describe('GET /r/:token (image hosting)', () => {
|
||||
error: insufficientCredits('Insufficient credits', { metadata: { resource: 'storage_egress' } }),
|
||||
})
|
||||
|
||||
const res = await app.request('/r/ih_credits', { redirect: 'manual' })
|
||||
const res = await app.request('/r/ihCredits', { redirect: 'manual' })
|
||||
expect(res.status).toBe(402)
|
||||
const body = (await res.json()) as {
|
||||
error: {
|
||||
@@ -422,10 +444,10 @@ describe('GET /r/:token (image hosting)', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, { id: 'ih-cnt1', token: 'ih_counttest1' })
|
||||
await insertImageHosting(db, orgId, { id: 'ih-cnt1', token: 'ihCounttest1' })
|
||||
|
||||
expect(await getAccessCount(db, 'ih-cnt1')).toBe(0)
|
||||
await app.request('/r/ih_counttest1', { redirect: 'manual' })
|
||||
await app.request('/r/ihCounttest1', { redirect: 'manual' })
|
||||
expect(await getAccessCount(db, 'ih-cnt1')).toBe(1)
|
||||
})
|
||||
|
||||
@@ -434,7 +456,7 @@ describe('GET /r/:token (image hosting)', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, { id: 'ih-quota-ok', token: 'ih_quotaok' })
|
||||
await insertImageHosting(db, orgId, { id: 'ih-quota-ok', token: 'ihQuotaok' })
|
||||
const trafficPeriod = currentTrafficPeriod()
|
||||
await db.run(sql`
|
||||
UPDATE org_quotas
|
||||
@@ -442,7 +464,7 @@ describe('GET /r/:token (image hosting)', () => {
|
||||
WHERE org_id = ${orgId}
|
||||
`)
|
||||
|
||||
const res = await app.request('/r/ih_quotaok', { redirect: 'manual' })
|
||||
const res = await app.request('/r/ihQuotaok', { redirect: 'manual' })
|
||||
expect(res.status).toBe(302)
|
||||
|
||||
const rows = await db.all<{ trafficUsed: number }>(
|
||||
@@ -456,7 +478,7 @@ describe('GET /r/:token (image hosting)', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, { id: 'ih-sign-fail', token: 'ih_signfail' })
|
||||
await insertImageHosting(db, orgId, { id: 'ih-sign-fail', token: 'ihSignfail' })
|
||||
const trafficPeriod = currentTrafficPeriod()
|
||||
await db.run(sql`
|
||||
UPDATE org_quotas
|
||||
@@ -465,7 +487,7 @@ describe('GET /r/:token (image hosting)', () => {
|
||||
`)
|
||||
vi.mocked(S3Service.prototype.presignInline).mockRejectedValueOnce(new Error('sign failed'))
|
||||
|
||||
const res = await app.request('/r/ih_signfail', { redirect: 'manual' })
|
||||
const res = await app.request('/r/ihSignfail', { redirect: 'manual' })
|
||||
expect(res.status).toBe(500)
|
||||
|
||||
const rows = await db.all<{ trafficUsed: number }>(
|
||||
@@ -480,7 +502,7 @@ describe('GET /r/:token (image hosting)', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, { id: 'ih-quota-repeat', token: 'ih_quotarepeat' })
|
||||
await insertImageHosting(db, orgId, { id: 'ih-quota-repeat', token: 'ihQuotarepeat' })
|
||||
const trafficPeriod = currentTrafficPeriod()
|
||||
await db.run(sql`
|
||||
UPDATE org_quotas
|
||||
@@ -489,11 +511,11 @@ describe('GET /r/:token (image hosting)', () => {
|
||||
`)
|
||||
await setTrafficPlanEntitlement(db, orgId, 1024)
|
||||
|
||||
const first = await app.request('/r/ih_quotarepeat', { redirect: 'manual' })
|
||||
const first = await app.request('/r/ihQuotarepeat', { redirect: 'manual' })
|
||||
expect(first.status).toBe(302)
|
||||
expect(first.headers.get('cache-control')).toBe('no-store')
|
||||
|
||||
const second = await app.request('/r/ih_quotarepeat', { redirect: 'manual' })
|
||||
const second = await app.request('/r/ihQuotarepeat', { redirect: 'manual' })
|
||||
expect(second.status).toBe(422)
|
||||
const secondBody = (await second.json()) as { error: { message: string; details: Array<{ reason: string }> } }
|
||||
expect(secondBody.error.message).toBe('Traffic quota exceeded')
|
||||
@@ -507,9 +529,9 @@ describe('GET /r/:token (image hosting)', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, { id: 'ih-cnt2', token: 'ih_counttest2', status: 'draft' })
|
||||
await insertImageHosting(db, orgId, { id: 'ih-cnt2', token: 'ihCounttest2', status: 'draft' })
|
||||
|
||||
await app.request('/r/ih_counttest2', { redirect: 'manual' })
|
||||
await app.request('/r/ihCounttest2', { redirect: 'manual' })
|
||||
expect(await getAccessCount(db, 'ih-cnt2')).toBe(0)
|
||||
})
|
||||
})
|
||||
@@ -522,10 +544,10 @@ describe('GET /r/:token — referer allowlist enforcement', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, { id: 'ih-ref1', token: 'ih_reftest1' })
|
||||
await insertImageHosting(db, orgId, { id: 'ih-ref1', token: 'ihReftest1' })
|
||||
// No config inserted — no allowlist
|
||||
|
||||
const res = await app.request('/r/ih_reftest1', {
|
||||
const res = await app.request('/r/ihReftest1', {
|
||||
redirect: 'manual',
|
||||
headers: { Referer: 'https://anydomain.com/page' },
|
||||
})
|
||||
@@ -537,10 +559,10 @@ describe('GET /r/:token — referer allowlist enforcement', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, { id: 'ih-ref2', token: 'ih_reftest2' })
|
||||
await insertImageHosting(db, orgId, { id: 'ih-ref2', token: 'ihReftest2' })
|
||||
await insertImageHostingConfig(db, orgId, { refererAllowlist: ['https://myblog.com'] })
|
||||
|
||||
const res = await app.request('/r/ih_reftest2', {
|
||||
const res = await app.request('/r/ihReftest2', {
|
||||
redirect: 'manual',
|
||||
headers: { Referer: 'https://myblog.com/post/1' },
|
||||
})
|
||||
@@ -552,10 +574,10 @@ describe('GET /r/:token — referer allowlist enforcement', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, { id: 'ih-ref3', token: 'ih_reftest3' })
|
||||
await insertImageHosting(db, orgId, { id: 'ih-ref3', token: 'ihReftest3' })
|
||||
await insertImageHostingConfig(db, orgId, { refererAllowlist: ['https://myblog.com'] })
|
||||
|
||||
const res = await app.request('/r/ih_reftest3', { redirect: 'manual' })
|
||||
const res = await app.request('/r/ihReftest3', { redirect: 'manual' })
|
||||
expect(res.status).toBe(302)
|
||||
})
|
||||
|
||||
@@ -564,10 +586,10 @@ describe('GET /r/:token — referer allowlist enforcement', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, { id: 'ih-ref4', token: 'ih_reftest4' })
|
||||
await insertImageHosting(db, orgId, { id: 'ih-ref4', token: 'ihReftest4' })
|
||||
await insertImageHostingConfig(db, orgId, { refererAllowlist: ['https://myblog.com'] })
|
||||
|
||||
const res = await app.request('/r/ih_reftest4', {
|
||||
const res = await app.request('/r/ihReftest4', {
|
||||
redirect: 'manual',
|
||||
headers: { Referer: 'https://otherdomain.com/page' },
|
||||
})
|
||||
@@ -579,10 +601,10 @@ describe('GET /r/:token — referer allowlist enforcement', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, { id: 'ih-ref5', token: 'ih_reftest5' })
|
||||
await insertImageHosting(db, orgId, { id: 'ih-ref5', token: 'ihReftest5' })
|
||||
await insertImageHostingConfig(db, orgId, { refererAllowlist: ['https://myblog.com'] })
|
||||
|
||||
const res = await app.request('/r/ih_reftest5', {
|
||||
const res = await app.request('/r/ihReftest5', {
|
||||
redirect: 'manual',
|
||||
headers: { Referer: 'https://sub.myblog.com/page' },
|
||||
})
|
||||
@@ -594,10 +616,10 @@ describe('GET /r/:token — referer allowlist enforcement', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, { id: 'ih-ref6', token: 'ih_reftest6' })
|
||||
await insertImageHosting(db, orgId, { id: 'ih-ref6', token: 'ihReftest6' })
|
||||
await insertImageHostingConfig(db, orgId, { refererAllowlist: ['https://myblog.com'] })
|
||||
|
||||
await app.request('/r/ih_reftest6', {
|
||||
await app.request('/r/ihReftest6', {
|
||||
redirect: 'manual',
|
||||
headers: { Referer: 'https://evil.com/page' },
|
||||
})
|
||||
@@ -647,9 +669,9 @@ describe('GET /r/:token — two-org isolation', () => {
|
||||
INSERT OR IGNORE INTO storages (id, bucket, endpoint, region, access_key, secret_key, file_path, custom_host, capacity, used, status, created_at, updated_at)
|
||||
VALUES (${STORAGE_ID}, 'test-bucket', 'https://s3.amazonaws.com', 'us-east-1', 'AK', 'SK', '', '', 0, 0, 'active', ${now}, ${now})
|
||||
`)
|
||||
await insertImageHosting(db, orgId, { id: 'ih-iso1', token: 'ih_isolationtest' })
|
||||
await insertImageHosting(db, orgId, { id: 'ih-iso1', token: 'ihIsolationtest' })
|
||||
|
||||
const res = await app.request('/r/ih_isolationtest', { redirect: 'manual' })
|
||||
const res = await app.request('/r/ihIsolationtest', { redirect: 'manual' })
|
||||
expect(res.status).toBe(302)
|
||||
expect(res.headers.get('location')).toBe(MOCK_INLINE_URL)
|
||||
})
|
||||
@@ -659,7 +681,7 @@ describe('GET /r/:token — two-org isolation', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, { id: 'ih-quota', token: 'ih_quotatest' })
|
||||
await insertImageHosting(db, orgId, { id: 'ih-quota', token: 'ihQuotatest' })
|
||||
const trafficPeriod = currentTrafficPeriod()
|
||||
await db.run(sql`
|
||||
UPDATE org_quotas
|
||||
@@ -668,7 +690,7 @@ describe('GET /r/:token — two-org isolation', () => {
|
||||
`)
|
||||
await setTrafficPlanEntitlement(db, orgId, 512)
|
||||
|
||||
const res = await app.request('/r/ih_quotatest', { redirect: 'manual' })
|
||||
const res = await app.request('/r/ihQuotatest', { redirect: 'manual' })
|
||||
expect(res.status).toBe(422)
|
||||
const body = (await res.json()) as { error: { message: string; details: Array<{ reason: string }> } }
|
||||
expect(body.error.message).toBe('Traffic quota exceeded')
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import type { Context } from 'hono'
|
||||
import { Hono } from 'hono'
|
||||
import { ZPAN_CLOUD_URL_DEFAULT } from '../../shared/constants'
|
||||
import { isImageHostingToken } from '../domain/image-hosting'
|
||||
import { isDownloadFailureStatus, transferAuditActor, transferFailureReason } from '../middleware/audit-transfers'
|
||||
import type { Env } from '../middleware/platform'
|
||||
import { notFound } from '../usecases/ports'
|
||||
@@ -11,6 +10,7 @@ import {
|
||||
resolveDirectShareDownload,
|
||||
resolveImageHostingDownload,
|
||||
resolveRedirectDownloadAuditTarget,
|
||||
resolveRedirectTokenKind,
|
||||
} from '../usecases/redirect'
|
||||
import { recordDownloadFailure, recordDownloadIssued } from '../usecases/transfer-activity'
|
||||
|
||||
@@ -99,8 +99,9 @@ app.get('/:token', async (c) => {
|
||||
const raw = c.req.param('token')
|
||||
const token = stripExtension(raw)
|
||||
|
||||
if (token.startsWith('ds_')) return handleDirectShare(c, token)
|
||||
if (isImageHostingToken(token)) return handleImageHosting(c, token)
|
||||
const kind = await resolveRedirectTokenKind(c.get('deps'), token)
|
||||
if (kind === 'direct_share') return handleDirectShare(c, token)
|
||||
if (kind === 'image_hosting') return handleImageHosting(c, token)
|
||||
|
||||
throw notFound()
|
||||
})
|
||||
|
||||
@@ -6,6 +6,7 @@ import { createShareRepo } from '../adapters/repos/share'
|
||||
import { auditEvents, shareRecipients, shares } from '../db/schema.js'
|
||||
import { currentTrafficPeriod } from '../domain/quota.js'
|
||||
import { authedHeaders, createTestApp, seedProLicense } from '../test/setup.js'
|
||||
import { encodeChildRef } from './share-utils'
|
||||
|
||||
type TestApp = Awaited<ReturnType<typeof createTestApp>>['app']
|
||||
type TestDb = Awaited<ReturnType<typeof createTestApp>>['db']
|
||||
@@ -212,7 +213,7 @@ describe('POST /api/shares', () => {
|
||||
|
||||
const body = (await res.json()) as Record<string, unknown>
|
||||
expect(body.kind).toBe('direct')
|
||||
expect((body.urls as Record<string, string>).direct).toMatch(/^\/r\/ds_/)
|
||||
expect((body.urls as Record<string, string>).direct).toMatch(/^\/r\/[A-Za-z0-9]+$/)
|
||||
expect((body.urls as Record<string, string>).landing).toBeUndefined()
|
||||
})
|
||||
|
||||
@@ -2276,18 +2277,16 @@ describe('Public share routes', () => {
|
||||
await insertFile(db, orgId, { id: 'out1', name: 'outside.txt' })
|
||||
const share = await createShareRepo(db).create({ matterId: 'fld3', orgId, creatorId, kind: 'landing' })
|
||||
|
||||
const { createHmac } = await import('node:crypto')
|
||||
const sig = createHmac('sha256', share.token).update('out1').digest('hex').slice(0, 16)
|
||||
const fakeRef = Buffer.from(`out1.${sig}`).toString('base64url')
|
||||
const fakeRef = encodeChildRef(share.token, 'out1')
|
||||
|
||||
const res = await app.request(`/api/shares/${share.token}/objects/${fakeRef}`, { redirect: 'manual' })
|
||||
expect(res.status).toBe(404)
|
||||
})
|
||||
})
|
||||
|
||||
// ─── GET /r/:token — unified redirect for direct shares (ds_) ────────────────
|
||||
// ─── GET /r/:token — unified redirect for direct shares ─────────────────────
|
||||
|
||||
describe('GET /r/:token (ds_ direct shares)', () => {
|
||||
describe('GET /r/:token (direct shares)', () => {
|
||||
it('returns 302 redirect for valid direct share', async () => {
|
||||
const { app, db } = await createTestApp()
|
||||
await authedHeaders(app)
|
||||
@@ -2332,10 +2331,10 @@ describe('Public share routes', () => {
|
||||
const now = Date.now()
|
||||
await db.run(sql`
|
||||
INSERT INTO shares (id, token, kind, matter_id, org_id, creator_id, views, downloads, status, created_at)
|
||||
VALUES ('sh-dltrash', 'ds_token-dltrash', 'direct', 'dlx3', ${orgId}, ${creatorId}, 0, 0, 'active', ${now})
|
||||
VALUES ('sh-dltrash', 'DirectDltrash', 'direct', 'dlx3', ${orgId}, ${creatorId}, 0, 0, 'active', ${now})
|
||||
`)
|
||||
|
||||
const res = await app.request('/r/ds_token-dltrash', { redirect: 'manual' })
|
||||
const res = await app.request('/r/DirectDltrash', { redirect: 'manual' })
|
||||
expect(res.status).toBe(410)
|
||||
})
|
||||
|
||||
|
||||
+22
-21
@@ -4,6 +4,7 @@ import type { Context } from 'hono'
|
||||
import { getCookie, setCookie } from 'hono/cookie'
|
||||
import { ZPAN_CLOUD_URL_DEFAULT } from '../../shared/constants'
|
||||
import { cursorPageSchema } from '../../shared/schemas'
|
||||
import { opaqueIdSchema, opaqueTokenSchema } from '../../shared/schemas/identifiers'
|
||||
import {
|
||||
createShareRequestSchema,
|
||||
listSharesQuerySchema,
|
||||
@@ -50,7 +51,7 @@ const cloudBaseUrl = (c: Context<Env>) => c.get('platform').getEnv('ZPAN_CLOUD_U
|
||||
// ─── Schemas ─────────────────────────────────────────────────────────────────
|
||||
const shareViewSchema = z
|
||||
.object({
|
||||
token: z.string(),
|
||||
token: opaqueTokenSchema,
|
||||
kind: z.string(),
|
||||
status: z.string(),
|
||||
expiresAt: z.string().nullable(),
|
||||
@@ -71,9 +72,9 @@ const shareViewSchema = z
|
||||
views: z.number().int(),
|
||||
rootRef: z.string(),
|
||||
// creator-only fields
|
||||
id: z.string().optional(),
|
||||
matterId: z.string().optional(),
|
||||
orgId: z.string().optional(),
|
||||
id: opaqueIdSchema.optional(),
|
||||
matterId: opaqueIdSchema.optional(),
|
||||
orgId: opaqueIdSchema.optional(),
|
||||
creatorId: z.string().optional(),
|
||||
createdAt: z.string().optional(),
|
||||
recipients: z.array(shareRecipientViewSchema).optional(),
|
||||
@@ -117,11 +118,11 @@ function toShareViewDTO(dto: ShareViewerDto | ShareCreatorDto): z.infer<typeof s
|
||||
|
||||
const shareListItemSchema = z
|
||||
.object({
|
||||
id: z.string(),
|
||||
token: z.string(),
|
||||
id: opaqueIdSchema,
|
||||
token: opaqueTokenSchema,
|
||||
kind: z.string(),
|
||||
matterId: z.string(),
|
||||
orgId: z.string(),
|
||||
matterId: opaqueIdSchema,
|
||||
orgId: opaqueIdSchema,
|
||||
creatorId: z.string(),
|
||||
expiresAt: z.string().nullable(),
|
||||
downloadLimit: z.number().int().nullable(),
|
||||
@@ -150,7 +151,7 @@ const shareObjectsSchema = shareObjectsResponseSchema.openapi('ShareObjects')
|
||||
|
||||
const createdShareSchema = z
|
||||
.object({
|
||||
token: z.string(),
|
||||
token: opaqueTokenSchema,
|
||||
kind: z.string(),
|
||||
urls: z.object({ landing: z.string().optional(), direct: z.string().optional() }),
|
||||
expiresAt: z.string().nullable(),
|
||||
@@ -162,16 +163,16 @@ const createdShareSchema = z
|
||||
// `saved` carries full Matter records; serialized inline (the named `Matter`
|
||||
// component is owned by the objects router).
|
||||
const savedMatterSchema = z.object({
|
||||
id: z.string(),
|
||||
orgId: z.string(),
|
||||
alias: z.string(),
|
||||
id: opaqueIdSchema,
|
||||
orgId: opaqueIdSchema,
|
||||
alias: opaqueTokenSchema,
|
||||
name: z.string(),
|
||||
type: z.string(),
|
||||
size: z.number().int().nullable(),
|
||||
dirtype: z.number().int().nullable(),
|
||||
parent: z.string(),
|
||||
object: z.string(),
|
||||
storageId: z.string(),
|
||||
storageId: opaqueIdSchema,
|
||||
status: z.string(),
|
||||
trashedAt: z.number().int().nullable(),
|
||||
createdAt: z.string(),
|
||||
@@ -191,7 +192,7 @@ const saveShareResultSchema = z
|
||||
|
||||
const listObjectsQuerySchema = z.object({
|
||||
parent: z.string().optional(),
|
||||
pageToken: z.string().min(1).optional(),
|
||||
pageToken: opaqueTokenSchema.optional(),
|
||||
pageSize: z.coerce.number().int().min(1).max(100).default(50),
|
||||
})
|
||||
|
||||
@@ -206,7 +207,7 @@ const viewShareRoute = authRoute(
|
||||
tags: ['Shares'],
|
||||
method: 'get',
|
||||
path: '/{token}',
|
||||
request: { params: z.object({ token: z.string() }) },
|
||||
request: { params: z.object({ token: opaqueTokenSchema }) },
|
||||
responses: {
|
||||
200: jsonContent(shareViewSchema, 'Share'),
|
||||
404: errorResponse('Share not found or revoked'),
|
||||
@@ -223,7 +224,7 @@ const verifyShareRoute = authRoute(
|
||||
tags: ['Shares'],
|
||||
method: 'post',
|
||||
path: '/{token}/sessions',
|
||||
request: { params: z.object({ token: z.string() }), ...jsonBody(verifyPasswordSchema) },
|
||||
request: { params: z.object({ token: opaqueTokenSchema }), ...jsonBody(verifyPasswordSchema) },
|
||||
responses: {
|
||||
200: jsonContent(z.object({ ok: z.literal(true) }), 'Verified'),
|
||||
403: errorResponse('Invalid password'),
|
||||
@@ -240,7 +241,7 @@ const listShareObjectsRoute = authRoute(
|
||||
tags: ['Shares'],
|
||||
method: 'get',
|
||||
path: '/{token}/objects',
|
||||
request: { params: z.object({ token: z.string() }), query: listObjectsQuerySchema },
|
||||
request: { params: z.object({ token: opaqueTokenSchema }), query: listObjectsQuerySchema },
|
||||
responses: {
|
||||
200: jsonContent(shareObjectsSchema, 'Share objects'),
|
||||
400: errorResponse('Bad request'),
|
||||
@@ -259,7 +260,7 @@ const readShareReadmeRoute = authRoute(
|
||||
tags: ['Shares'],
|
||||
method: 'get',
|
||||
path: '/{token}/readme',
|
||||
request: { params: z.object({ token: z.string() }) },
|
||||
request: { params: z.object({ token: opaqueTokenSchema }) },
|
||||
responses: {
|
||||
200: jsonContent(shareReadmeResponseSchema, 'README.md content'),
|
||||
400: errorResponse('README.md is not valid UTF-8'),
|
||||
@@ -447,7 +448,7 @@ const revokeShareRoute = authRoute(
|
||||
method: 'put',
|
||||
path: '/{token}/status',
|
||||
request: {
|
||||
params: z.object({ token: z.string() }),
|
||||
params: z.object({ token: opaqueTokenSchema }),
|
||||
...jsonBody(z.object({ status: z.literal('revoked') })),
|
||||
},
|
||||
responses: {
|
||||
@@ -469,7 +470,7 @@ const putSharePrivacyRoute = authRoute(
|
||||
method: 'put',
|
||||
path: '/{token}/privacy',
|
||||
request: {
|
||||
params: z.object({ token: z.string() }),
|
||||
params: z.object({ token: opaqueTokenSchema }),
|
||||
...jsonBody(sharePrivacySchema),
|
||||
},
|
||||
responses: {
|
||||
@@ -489,7 +490,7 @@ const saveShareRoute = authRoute(
|
||||
tags: ['Shares'],
|
||||
method: 'post',
|
||||
path: '/{token}/objects',
|
||||
request: { params: z.object({ token: z.string() }), ...jsonBody(saveShareRequestSchema) },
|
||||
request: { params: z.object({ token: opaqueTokenSchema }), ...jsonBody(saveShareRequestSchema) },
|
||||
responses: {
|
||||
201: jsonContent(saveShareResultSchema, 'Saved'),
|
||||
400: errorResponse('Bad request'),
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { OpenAPIHono, z } from '@hono/zod-openapi'
|
||||
import { AuthorizationScope } from '@shared/authorization'
|
||||
import { announcementInputSchema, announcementStatusSchema, pageQuerySchema, pageSchema } from '@shared/schemas'
|
||||
import { opaqueIdSchema } from '@shared/schemas/identifiers'
|
||||
import type { Env } from '../../middleware/platform'
|
||||
import { requireFeature } from '../../middleware/require-feature'
|
||||
import { type AnnouncementRecord, forbidden, notFound } from '../../usecases/ports'
|
||||
@@ -16,14 +17,14 @@ import { authRoute, errorResponse, jsonBody, jsonContent } from '../openapi'
|
||||
|
||||
const announcementSchema = z
|
||||
.object({
|
||||
id: z.string(),
|
||||
id: opaqueIdSchema,
|
||||
title: z.string(),
|
||||
body: z.string(),
|
||||
status: z.string(),
|
||||
priority: z.number().int(),
|
||||
publishedAt: z.string().nullable(),
|
||||
expiresAt: z.string().nullable(),
|
||||
createdBy: z.string(),
|
||||
createdBy: opaqueIdSchema,
|
||||
createdAt: z.string(),
|
||||
updatedAt: z.string(),
|
||||
})
|
||||
@@ -90,7 +91,7 @@ const getAnnouncementRoute = authRoute(
|
||||
method: 'get',
|
||||
path: '/{id}',
|
||||
middleware: [requireFeature('site_announcements')] as const,
|
||||
request: { params: z.object({ id: z.string() }) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }) },
|
||||
responses: {
|
||||
200: jsonContent(announcementSchema, 'Announcement'),
|
||||
404: errorResponse('Announcement not found'),
|
||||
@@ -107,7 +108,7 @@ const updateAnnouncementRoute = authRoute(
|
||||
method: 'put',
|
||||
path: '/{id}',
|
||||
middleware: [requireFeature('site_announcements')] as const,
|
||||
request: { params: z.object({ id: z.string() }), ...jsonBody(announcementInputSchema) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }), ...jsonBody(announcementInputSchema) },
|
||||
responses: {
|
||||
200: jsonContent(announcementSchema, 'Updated announcement'),
|
||||
404: errorResponse('Announcement not found'),
|
||||
@@ -124,7 +125,7 @@ const deleteAnnouncementRoute = authRoute(
|
||||
method: 'delete',
|
||||
path: '/{id}',
|
||||
middleware: [requireFeature('site_announcements')] as const,
|
||||
request: { params: z.object({ id: z.string() }) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }) },
|
||||
responses: {
|
||||
204: { description: 'Deleted announcement' },
|
||||
404: errorResponse('Announcement not found'),
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { OpenAPIHono, z } from '@hono/zod-openapi'
|
||||
import { AuthorizationScope } from '@shared/authorization'
|
||||
import { pageQuerySchema, pageSchema } from '@shared/schemas'
|
||||
import { opaqueIdSchema } from '@shared/schemas/identifiers'
|
||||
import type { Env } from '../../middleware/platform'
|
||||
import { requireFeature } from '../../middleware/require-feature'
|
||||
import type { AdminAuditEventWithOrg } from '../../usecases/ports'
|
||||
@@ -10,9 +11,9 @@ import { authRoute, errorResponse, jsonContent } from '../openapi'
|
||||
|
||||
const auditEventSchema = z
|
||||
.object({
|
||||
id: z.string(),
|
||||
orgId: z.string(),
|
||||
userId: z.string().nullable(),
|
||||
id: opaqueIdSchema,
|
||||
orgId: opaqueIdSchema,
|
||||
userId: opaqueIdSchema.nullable(),
|
||||
actorType: z.enum(['user', 'api_key', 'oauth', 'agent', 'anonymous', 'system', 'downloader', 'task-upload']),
|
||||
actorRef: z.string().nullable(),
|
||||
actorIssuer: z.string().nullable(),
|
||||
@@ -22,7 +23,7 @@ const auditEventSchema = z
|
||||
targetName: z.string(),
|
||||
metadata: z.string().nullable(),
|
||||
createdAt: z.string(),
|
||||
user: z.object({ id: z.string().nullable(), name: z.string(), image: z.string().nullable() }),
|
||||
user: z.object({ id: opaqueIdSchema.nullable(), name: z.string(), image: z.string().nullable() }),
|
||||
orgName: z.string().nullable(),
|
||||
})
|
||||
.openapi('AuditEvent')
|
||||
@@ -36,8 +37,8 @@ function toAuditEventDTO(e: AdminAuditEventWithOrg): AuditEventDTO {
|
||||
const auditPageSchema = pageSchema(auditEventSchema, 'AuditEventPage')
|
||||
|
||||
const listAuditQuerySchema = pageQuerySchema.extend({
|
||||
orgId: z.string().optional(),
|
||||
userId: z.string().optional(),
|
||||
orgId: opaqueIdSchema.optional(),
|
||||
userId: opaqueIdSchema.optional(),
|
||||
action: z.string().optional(),
|
||||
targetType: z.string().optional(),
|
||||
createdFrom: z.string().datetime().optional(),
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { OpenAPIHono, z } from '@hono/zod-openapi'
|
||||
import { AuthorizationScope } from '@shared/authorization'
|
||||
import { pageQuerySchema, pageSchema } from '@shared/schemas'
|
||||
import { opaqueIdSchema, opaqueTokenSchema } from '@shared/schemas/identifiers'
|
||||
import type { Env } from '../../middleware/platform'
|
||||
import { notFound, unauthorized } from '../../usecases/ports'
|
||||
import {
|
||||
@@ -15,14 +16,14 @@ import { authRoute, errorResponse, jsonBody, jsonContent } from '../openapi'
|
||||
// SiteInvitation is already wire-shaped (ISO string timestamps) — no DTO mapper.
|
||||
const siteInvitationSchema = z
|
||||
.object({
|
||||
id: z.string(),
|
||||
id: opaqueIdSchema,
|
||||
email: z.string(),
|
||||
token: z.string(),
|
||||
invitedBy: z.string(),
|
||||
token: opaqueTokenSchema,
|
||||
invitedBy: opaqueIdSchema,
|
||||
invitedByName: z.string(),
|
||||
acceptedBy: z.string().nullable(),
|
||||
acceptedBy: opaqueIdSchema.nullable(),
|
||||
acceptedAt: z.string().nullable(),
|
||||
revokedBy: z.string().nullable(),
|
||||
revokedBy: opaqueIdSchema.nullable(),
|
||||
revokedAt: z.string().nullable(),
|
||||
expiresAt: z.string(),
|
||||
createdAt: z.string(),
|
||||
@@ -73,7 +74,7 @@ const resendRoute = authRoute(
|
||||
tags: ['Invitations'],
|
||||
method: 'post',
|
||||
path: '/{id}/deliveries',
|
||||
request: { params: z.object({ id: z.string() }) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }) },
|
||||
responses: {
|
||||
200: jsonContent(siteInvitationSchema, 'Resent invitation'),
|
||||
400: errorResponse('Invitation is no longer pending'),
|
||||
@@ -90,7 +91,7 @@ const revokeRoute = authRoute(
|
||||
tags: ['Invitations'],
|
||||
method: 'delete',
|
||||
path: '/{id}',
|
||||
request: { params: z.object({ id: z.string() }) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }) },
|
||||
responses: {
|
||||
204: { description: 'Revoked invitation' },
|
||||
400: errorResponse('Invitation is no longer pending'),
|
||||
@@ -108,7 +109,7 @@ const getByTokenRoute = authRoute(
|
||||
tags: ['Invitations'],
|
||||
method: 'get',
|
||||
path: '/{token}',
|
||||
request: { params: z.object({ token: z.string() }) },
|
||||
request: { params: z.object({ token: opaqueTokenSchema }) },
|
||||
responses: {
|
||||
200: jsonContent(siteInvitationSchema, 'Invitation'),
|
||||
404: errorResponse('Invitation not found'),
|
||||
|
||||
@@ -259,14 +259,14 @@ describe('Public Invite Codes API — POST /validate', () => {
|
||||
expect(res.status).toBe(400)
|
||||
})
|
||||
|
||||
it('returns 400 when code contains lowercase letters', async () => {
|
||||
it('accepts lowercase letters in the Base62 alphabet', async () => {
|
||||
const { app } = await createTestApp()
|
||||
const res = await app.request('/api/site/invite-codes/validations', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ code: 'abcd1234' }),
|
||||
})
|
||||
expect(res.status).toBe(400)
|
||||
expect(res.status).toBe(200)
|
||||
})
|
||||
|
||||
it('returns 400 when code is fewer than 8 characters', async () => {
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { OpenAPIHono, z } from '@hono/zod-openapi'
|
||||
import { AuthorizationScope } from '@shared/authorization'
|
||||
import { pageQuerySchema, pageSchema } from '@shared/schemas'
|
||||
import { opaqueIdSchema, opaqueTokenSchema } from '@shared/schemas/identifiers'
|
||||
import type { Env } from '../../middleware/platform'
|
||||
import { type InviteCodeRecord, unauthorized } from '../../usecases/ports'
|
||||
import {
|
||||
@@ -13,10 +14,10 @@ import { authRoute, errorResponse, jsonBody, jsonContent } from '../openapi'
|
||||
|
||||
const inviteCodeSchema = z
|
||||
.object({
|
||||
id: z.string(),
|
||||
code: z.string(),
|
||||
createdBy: z.string(),
|
||||
usedBy: z.string().nullable(),
|
||||
id: opaqueIdSchema,
|
||||
code: opaqueTokenSchema,
|
||||
createdBy: opaqueIdSchema,
|
||||
usedBy: opaqueIdSchema.nullable(),
|
||||
usedAt: z.string().nullable(),
|
||||
expiresAt: z.string().nullable(),
|
||||
createdAt: z.string(),
|
||||
@@ -45,7 +46,7 @@ const validateSchema = z.object({
|
||||
code: z
|
||||
.string()
|
||||
.length(8)
|
||||
.regex(/^[0-9A-Z]{8}$/),
|
||||
.regex(/^[A-Za-z0-9]{8}$/),
|
||||
})
|
||||
|
||||
const listRoute = authRoute(
|
||||
@@ -85,7 +86,7 @@ const deleteRoute = authRoute(
|
||||
tags: ['Invite Codes'],
|
||||
method: 'delete',
|
||||
path: '/{id}',
|
||||
request: { params: z.object({ id: z.string() }) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }) },
|
||||
responses: {
|
||||
204: { description: 'Deleted invite code' },
|
||||
409: errorResponse('Cannot delete a used invite code'),
|
||||
|
||||
@@ -7,6 +7,7 @@ import {
|
||||
replaceStorageSchema,
|
||||
updateStorageEgressBillingSchema,
|
||||
} from '@shared/schemas'
|
||||
import { opaqueIdSchema } from '@shared/schemas/identifiers'
|
||||
import type { Env } from '../../middleware/platform'
|
||||
import { type StorageRecord, storageNotFound } from '../../usecases/ports'
|
||||
import {
|
||||
@@ -25,7 +26,7 @@ import { authRoute, errorResponse, jsonBody, jsonContent } from '../openapi'
|
||||
// Timestamps are the only Date fields; toStorageDTO serializes them.
|
||||
const storageSchema = z
|
||||
.object({
|
||||
id: z.string(),
|
||||
id: opaqueIdSchema,
|
||||
provider: z.string(),
|
||||
bucket: z.string(),
|
||||
endpoint: z.string(),
|
||||
@@ -99,7 +100,7 @@ const getStorageRoute = authRoute(
|
||||
tags: ['Storages'],
|
||||
method: 'get',
|
||||
path: '/{id}',
|
||||
request: { params: z.object({ id: z.string() }) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }) },
|
||||
responses: {
|
||||
200: jsonContent(storageSchema, 'Storage'),
|
||||
404: errorResponse('Storage not found'),
|
||||
@@ -115,7 +116,7 @@ const replaceStorageRoute = authRoute(
|
||||
tags: ['Storages'],
|
||||
method: 'put',
|
||||
path: '/{id}',
|
||||
request: { params: z.object({ id: z.string() }), ...jsonBody(replaceStorageSchema) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }), ...jsonBody(replaceStorageSchema) },
|
||||
responses: {
|
||||
200: jsonContent(storageSchema, 'Replaced storage'),
|
||||
402: errorResponse('Feature not available'),
|
||||
@@ -132,7 +133,7 @@ const patchStorageRoute = authRoute(
|
||||
tags: ['Storages'],
|
||||
method: 'patch',
|
||||
path: '/{id}',
|
||||
request: { params: z.object({ id: z.string() }), ...jsonBody(patchStorageSchema) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }), ...jsonBody(patchStorageSchema) },
|
||||
responses: {
|
||||
200: jsonContent(storageSchema, 'Updated storage'),
|
||||
402: errorResponse('Feature not available'),
|
||||
@@ -149,7 +150,7 @@ const updateStorageEgressBillingRoute = authRoute(
|
||||
tags: ['Storages'],
|
||||
method: 'put',
|
||||
path: '/{id}/egress-billing',
|
||||
request: { params: z.object({ id: z.string() }), ...jsonBody(updateStorageEgressBillingSchema) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }), ...jsonBody(updateStorageEgressBillingSchema) },
|
||||
responses: {
|
||||
200: jsonContent(storageSchema, 'Updated storage'),
|
||||
402: errorResponse('Feature not available'),
|
||||
@@ -166,7 +167,7 @@ const deleteStorageRoute = authRoute(
|
||||
tags: ['Storages'],
|
||||
method: 'delete',
|
||||
path: '/{id}',
|
||||
request: { params: z.object({ id: z.string() }) },
|
||||
request: { params: z.object({ id: opaqueIdSchema }) },
|
||||
responses: {
|
||||
204: { description: 'Deleted storage' },
|
||||
404: errorResponse('Storage not found'),
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { OpenAPIHono, z } from '@hono/zod-openapi'
|
||||
import { AuthorizationScope } from '@shared/authorization'
|
||||
import { opaqueIdSchema } from '@shared/schemas/identifiers'
|
||||
import type { Env } from '../../middleware/platform'
|
||||
import { runtimeInfo } from '../../usecases/site/instance-info'
|
||||
import { getChangelog, resolveInstanceInfo } from '../../usecases/site/system'
|
||||
@@ -7,7 +8,7 @@ import { authRoute, jsonContent } from '../openapi'
|
||||
|
||||
const instanceInfoSchema = z
|
||||
.object({
|
||||
id: z.string(),
|
||||
id: opaqueIdSchema,
|
||||
name: z.string(),
|
||||
url: z.string(),
|
||||
version: z.string(),
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { OpenAPIHono, z } from '@hono/zod-openapi'
|
||||
import { AuthorizationScope } from '@shared/authorization'
|
||||
import { opaqueIdSchema } from '@shared/schemas/identifiers'
|
||||
import { STORAGE_USAGE_CATEGORIES, STORAGE_USAGE_SORT_FIELDS } from '@shared/storage-usage'
|
||||
import type { Env } from '../middleware/platform'
|
||||
import { notFound } from '../usecases/ports'
|
||||
@@ -24,7 +25,7 @@ const usageSchema = z
|
||||
.openapi('StorageUsage')
|
||||
|
||||
const itemSchema = z.object({
|
||||
id: z.string(),
|
||||
id: opaqueIdSchema,
|
||||
name: z.string(),
|
||||
path: z.string(),
|
||||
parentPath: z.string(),
|
||||
|
||||
@@ -429,10 +429,10 @@ describe('public redirect cloud traffic reporting', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImage(db, orgId, 'ih-cloud-token', 'ih_cloudtoken')
|
||||
await insertImage(db, orgId, 'ih-cloud-token', 'ihCloudtoken')
|
||||
await insertImageConfig(db, orgId)
|
||||
|
||||
const res = await app.request('/r/ih_cloudtoken', { redirect: 'manual' })
|
||||
const res = await app.request('/r/ihCloudtoken', { redirect: 'manual' })
|
||||
|
||||
expect(res.status).toBe(302)
|
||||
await expect(trafficReports(db)).resolves.toMatchObject([
|
||||
@@ -448,11 +448,11 @@ describe('public redirect cloud traffic reporting', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImage(db, orgId, 'ih-cloud-log-fail', 'ih_cloudlogfail')
|
||||
await insertImage(db, orgId, 'ih-cloud-log-fail', 'ihCloudlogfail')
|
||||
await insertImageConfig(db, orgId)
|
||||
vi.spyOn(db, 'run').mockRejectedValue(new Error('access failed'))
|
||||
|
||||
const res = await app.request('/r/ih_cloudlogfail', { redirect: 'manual' })
|
||||
const res = await app.request('/r/ihCloudlogfail', { redirect: 'manual' })
|
||||
|
||||
expect(res.status).toBe(302)
|
||||
expect(consoleError).toHaveBeenCalled()
|
||||
@@ -466,10 +466,10 @@ describe('public redirect cloud traffic reporting', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImage(db, orgId, 'ih-cloud-presign-fail', 'ih_cloudpresignfail')
|
||||
await insertImage(db, orgId, 'ih-cloud-presign-fail', 'ihCloudpresignfail')
|
||||
await insertImageConfig(db, orgId)
|
||||
|
||||
const res = await app.request('/r/ih_cloudpresignfail', { redirect: 'manual' })
|
||||
const res = await app.request('/r/ihCloudpresignfail', { redirect: 'manual' })
|
||||
|
||||
expect(res.status).toBe(500)
|
||||
expect(fetch).not.toHaveBeenCalled()
|
||||
@@ -483,7 +483,7 @@ describe('public redirect cloud traffic reporting', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImage(db, orgId, 'ih-cloud-domain', 'ih_clouddomain', 'blog/domain.png')
|
||||
await insertImage(db, orgId, 'ih-cloud-domain', 'ihClouddomain', 'blog/domain.png')
|
||||
await insertImageConfig(db, orgId, 'img.example.com')
|
||||
|
||||
const res = await app.request('https://img.example.com/blog/domain.png', {
|
||||
@@ -507,7 +507,7 @@ describe('public redirect cloud traffic reporting', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImage(db, orgId, 'ih-cloud-domain-blocked', 'ih_clouddomainblocked', 'blog/domain-blocked.png')
|
||||
await insertImage(db, orgId, 'ih-cloud-domain-blocked', 'ihClouddomainblocked', 'blog/domain-blocked.png')
|
||||
await insertImageConfig(db, orgId, 'img-blocked.example.com')
|
||||
await setTrafficQuota(db, orgId)
|
||||
|
||||
@@ -542,7 +542,7 @@ describe('public redirect cloud traffic reporting', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImage(db, orgId, 'ih-cloud-domain-confirm-fail', 'ih_clouddomainconfirmfail', 'blog/confirm-fail.png')
|
||||
await insertImage(db, orgId, 'ih-cloud-domain-confirm-fail', 'ihClouddomainconfirmfail', 'blog/confirm-fail.png')
|
||||
await insertImageConfig(db, orgId, 'img-confirm-fail.example.com')
|
||||
await setTrafficQuota(db, orgId)
|
||||
vi.spyOn(deps.cloudTrafficReports, 'markIssued').mockRejectedValueOnce(new Error('confirm failed'))
|
||||
@@ -574,7 +574,7 @@ describe('public redirect cloud traffic reporting', () => {
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImage(db, orgId, 'ih-cloud-domain-log-fail', 'ih_clouddomainlogfail', 'blog/domain-log-fail.png')
|
||||
await insertImage(db, orgId, 'ih-cloud-domain-log-fail', 'ihClouddomainlogfail', 'blog/domain-log-fail.png')
|
||||
await insertImageConfig(db, orgId, 'img.example.com')
|
||||
vi.spyOn(db, 'run').mockRejectedValue(new Error('access failed'))
|
||||
|
||||
|
||||
+27
-21
@@ -1,6 +1,7 @@
|
||||
import { OpenAPIHono, z } from '@hono/zod-openapi'
|
||||
import { AuthorizationScope } from '@shared/authorization'
|
||||
import { pageQuerySchema, pageSchema } from '@shared/schemas'
|
||||
import { opaqueIdSchema, opaqueTokenSchema } from '@shared/schemas/identifiers'
|
||||
import type { Env } from '../middleware/platform'
|
||||
import {
|
||||
type AuditEventWithUser,
|
||||
@@ -41,7 +42,7 @@ import { authRoute, errorResponse, jsonBody, jsonContent } from './openapi'
|
||||
|
||||
const inviteLinkInfoSchema = z
|
||||
.object({
|
||||
organizationId: z.string(),
|
||||
organizationId: opaqueIdSchema,
|
||||
organizationName: z.string(),
|
||||
role: z.string(),
|
||||
expiresAt: z.string().nullable(),
|
||||
@@ -52,11 +53,13 @@ function toInviteLinkInfoDTO(i: InviteLinkInfo): z.infer<typeof inviteLinkInfoSc
|
||||
return { ...i, expiresAt: i.expiresAt ? i.expiresAt.toISOString() : null }
|
||||
}
|
||||
|
||||
const inviteLinkCreatedSchema = z.object({ token: z.string(), expiresAt: z.string() }).openapi('TeamInviteLinkCreated')
|
||||
const inviteLinkCreatedSchema = z
|
||||
.object({ token: opaqueTokenSchema, expiresAt: z.string() })
|
||||
.openapi('TeamInviteLinkCreated')
|
||||
|
||||
const pendingInvitationSchema = z
|
||||
.object({
|
||||
id: z.string(),
|
||||
id: opaqueIdSchema,
|
||||
email: z.string(),
|
||||
role: z.string(),
|
||||
expiresAt: z.string().nullable(),
|
||||
@@ -72,9 +75,9 @@ const pendingInvitationListSchema = pageSchema(pendingInvitationSchema, 'TeamInv
|
||||
|
||||
const activityEventSchema = z
|
||||
.object({
|
||||
id: z.string(),
|
||||
orgId: z.string(),
|
||||
userId: z.string().nullable(),
|
||||
id: opaqueIdSchema,
|
||||
orgId: opaqueIdSchema,
|
||||
userId: opaqueIdSchema.nullable(),
|
||||
actorType: z.enum(['user', 'api_key', 'oauth', 'agent', 'anonymous', 'system', 'downloader', 'task-upload']),
|
||||
actorRef: z.string().nullable(),
|
||||
actorIssuer: z.string().nullable(),
|
||||
@@ -84,7 +87,7 @@ const activityEventSchema = z
|
||||
targetName: z.string(),
|
||||
metadata: z.string().nullable(),
|
||||
createdAt: z.string(),
|
||||
user: z.object({ id: z.string().nullable(), name: z.string(), image: z.string().nullable() }),
|
||||
user: z.object({ id: opaqueIdSchema.nullable(), name: z.string(), image: z.string().nullable() }),
|
||||
})
|
||||
.openapi('AuditEvent')
|
||||
|
||||
@@ -96,7 +99,7 @@ const activityPageSchema = pageSchema(activityEventSchema, 'ActivityPage')
|
||||
|
||||
const teamSummarySchema = z
|
||||
.object({
|
||||
id: z.string(),
|
||||
id: opaqueIdSchema,
|
||||
name: z.string(),
|
||||
slug: z.string(),
|
||||
logo: z.string().nullable(),
|
||||
@@ -115,7 +118,7 @@ const createLinkSchema = z.object({
|
||||
expiresIn: z.number().int().min(1).optional(),
|
||||
})
|
||||
|
||||
const joinSchema = z.object({ token: z.string().min(1) })
|
||||
const joinSchema = z.object({ token: opaqueTokenSchema })
|
||||
|
||||
const grantEntitlementSchema = z.object({
|
||||
resourceType: z.literal('storage'),
|
||||
@@ -155,7 +158,7 @@ const inviteLinkInfoRoute = authRoute(
|
||||
tags: ['Teams'],
|
||||
method: 'get',
|
||||
path: '/invite-links/{token}',
|
||||
request: { params: z.object({ token: z.string() }) },
|
||||
request: { params: z.object({ token: opaqueTokenSchema }) },
|
||||
responses: {
|
||||
200: jsonContent(inviteLinkInfoSchema, 'Invite link info'),
|
||||
404: errorResponse('Invalid or expired invite link'),
|
||||
@@ -178,7 +181,7 @@ const createInviteLinkRoute = authRoute(
|
||||
tags: ['Teams'],
|
||||
method: 'post',
|
||||
path: '/{teamId}/invite-links',
|
||||
request: { params: z.object({ teamId: z.string() }), ...jsonBody(createLinkSchema) },
|
||||
request: { params: z.object({ teamId: opaqueIdSchema }), ...jsonBody(createLinkSchema) },
|
||||
responses: {
|
||||
201: jsonContent(inviteLinkCreatedSchema, 'Created invite link'),
|
||||
403: errorResponse('Forbidden'),
|
||||
@@ -194,7 +197,7 @@ const listInvitationsRoute = authRoute(
|
||||
tags: ['Teams'],
|
||||
method: 'get',
|
||||
path: '/{teamId}/invitations',
|
||||
request: { params: z.object({ teamId: z.string() }) },
|
||||
request: { params: z.object({ teamId: opaqueIdSchema }) },
|
||||
responses: {
|
||||
200: jsonContent(pendingInvitationListSchema, 'Pending invitations'),
|
||||
403: errorResponse('Forbidden'),
|
||||
@@ -210,7 +213,7 @@ const joinTeamRoute = authRoute(
|
||||
tags: ['Teams'],
|
||||
method: 'post',
|
||||
path: '/{teamId}/members',
|
||||
request: { params: z.object({ teamId: z.string() }), ...jsonBody(joinSchema) },
|
||||
request: { params: z.object({ teamId: opaqueIdSchema }), ...jsonBody(joinSchema) },
|
||||
responses: {
|
||||
200: jsonContent(z.object({ ok: z.literal(true) }), 'Joined'),
|
||||
404: errorResponse('Invalid invite link'),
|
||||
@@ -229,7 +232,7 @@ const activityRoute = authRoute(
|
||||
method: 'get',
|
||||
path: '/{teamId}/activity',
|
||||
request: {
|
||||
params: z.object({ teamId: z.string() }),
|
||||
params: z.object({ teamId: opaqueIdSchema }),
|
||||
query: pageQuerySchema,
|
||||
},
|
||||
responses: {
|
||||
@@ -249,7 +252,7 @@ const setLogoRoute = authRoute(
|
||||
path: '/{teamId}/logo',
|
||||
// Body is multipart/form-data (a `file` field); parsed directly in the handler
|
||||
// rather than via a request schema (the form validator conflicts with formData()).
|
||||
request: { params: z.object({ teamId: z.string() }) },
|
||||
request: { params: z.object({ teamId: opaqueIdSchema }) },
|
||||
responses: {
|
||||
200: jsonContent(z.object({ url: z.string() }), 'Logo URL'),
|
||||
400: errorResponse('Bad request'),
|
||||
@@ -269,7 +272,7 @@ const deleteLogoRoute = authRoute(
|
||||
tags: ['Teams'],
|
||||
method: 'delete',
|
||||
path: '/{teamId}/logo',
|
||||
request: { params: z.object({ teamId: z.string() }) },
|
||||
request: { params: z.object({ teamId: opaqueIdSchema }) },
|
||||
responses: {
|
||||
204: { description: 'Deleted' },
|
||||
403: errorResponse('Forbidden'),
|
||||
@@ -370,7 +373,7 @@ const getTeamRoute = authRoute(
|
||||
tags: ['Teams'],
|
||||
method: 'get',
|
||||
path: '/{teamId}',
|
||||
request: { params: z.object({ teamId: z.string() }) },
|
||||
request: { params: z.object({ teamId: opaqueIdSchema }) },
|
||||
responses: {
|
||||
200: jsonContent(teamSummarySchema, 'Team'),
|
||||
404: errorResponse('Team not found'),
|
||||
@@ -386,7 +389,7 @@ const listEntitlementsRoute = authRoute(
|
||||
tags: ['Teams'],
|
||||
method: 'get',
|
||||
path: '/{teamId}/entitlements',
|
||||
request: { params: z.object({ teamId: z.string() }) },
|
||||
request: { params: z.object({ teamId: opaqueIdSchema }) },
|
||||
responses: {
|
||||
200: jsonContent(entitlementListSchema, 'Entitlements'),
|
||||
400: errorResponse('Bad request'),
|
||||
@@ -403,7 +406,7 @@ const grantEntitlementRoute = authRoute(
|
||||
tags: ['Teams'],
|
||||
method: 'post',
|
||||
path: '/{teamId}/entitlements',
|
||||
request: { params: z.object({ teamId: z.string() }), ...jsonBody(grantEntitlementSchema) },
|
||||
request: { params: z.object({ teamId: opaqueIdSchema }), ...jsonBody(grantEntitlementSchema) },
|
||||
responses: {
|
||||
201: jsonContent(entitlementResultSchema, 'Granted entitlement'),
|
||||
400: errorResponse('Bad request'),
|
||||
@@ -420,7 +423,10 @@ const updateEntitlementRoute = authRoute(
|
||||
tags: ['Teams'],
|
||||
method: 'patch',
|
||||
path: '/{teamId}/entitlements/{eid}',
|
||||
request: { params: z.object({ teamId: z.string(), eid: z.string() }), ...jsonBody(updateEntitlementSchema) },
|
||||
request: {
|
||||
params: z.object({ teamId: opaqueIdSchema, eid: opaqueIdSchema }),
|
||||
...jsonBody(updateEntitlementSchema),
|
||||
},
|
||||
responses: {
|
||||
200: jsonContent(entitlementResultSchema, 'Updated entitlement'),
|
||||
400: errorResponse('Bad request'),
|
||||
@@ -437,7 +443,7 @@ const revokeEntitlementRoute = authRoute(
|
||||
tags: ['Teams'],
|
||||
method: 'delete',
|
||||
path: '/{teamId}/entitlements/{eid}',
|
||||
request: { params: z.object({ teamId: z.string(), eid: z.string() }) },
|
||||
request: { params: z.object({ teamId: opaqueIdSchema, eid: opaqueIdSchema }) },
|
||||
responses: {
|
||||
204: { description: 'Revoked entitlement' },
|
||||
400: errorResponse('Bad request'),
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { OpenAPIHono, z } from '@hono/zod-openapi'
|
||||
import { AuthorizationScope } from '@shared/authorization'
|
||||
import { cursorPageQuerySchema, cursorPageSchema, restoreObjectSchema } from '@shared/schemas'
|
||||
import { opaqueIdSchema, opaqueTokenSchema } from '@shared/schemas/identifiers'
|
||||
import type { Env } from '../middleware/platform'
|
||||
import { deleteObject, getTrashObject, listTrashedObjects, restoreObject } from '../usecases/object'
|
||||
import { badRequest, type Matter, notFound } from '../usecases/ports'
|
||||
@@ -11,16 +12,16 @@ import { decodeOptionalPageToken, encodeNextPageToken, pageQueryFingerprint, tra
|
||||
// grouping/view of `objects`, not a separate resource.
|
||||
const matterSchema = z
|
||||
.object({
|
||||
id: z.string(),
|
||||
orgId: z.string(),
|
||||
alias: z.string(),
|
||||
id: opaqueIdSchema,
|
||||
orgId: opaqueIdSchema,
|
||||
alias: opaqueTokenSchema,
|
||||
name: z.string(),
|
||||
type: z.string(),
|
||||
size: z.number().int().nullable(),
|
||||
dirtype: z.number().int().nullable(),
|
||||
parent: z.string(),
|
||||
object: z.string(),
|
||||
storageId: z.string(),
|
||||
storageId: opaqueIdSchema,
|
||||
status: z.string(),
|
||||
trashedAt: z.number().int().nullable(),
|
||||
createdAt: z.string(),
|
||||
@@ -50,7 +51,7 @@ function toMatterDTO(m: Matter): MatterDTO {
|
||||
}
|
||||
|
||||
const trashPageSchema = cursorPageSchema(matterSchema, 'TrashObjectPage')
|
||||
const idParam = z.object({ id: z.string() })
|
||||
const idParam = z.object({ id: opaqueIdSchema })
|
||||
|
||||
const listTrashRoute = authRoute(
|
||||
{ scopes: [AuthorizationScope.OBJECTS_READ], minTeamRole: 'viewer' },
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { OpenAPIHono, z } from '@hono/zod-openapi'
|
||||
import { AuthorizationScope } from '@shared/authorization'
|
||||
import { opaqueIdSchema } from '@shared/schemas/identifiers'
|
||||
import { publicProfileSchema } from '@shared/schemas/profile'
|
||||
import type { Env } from '../middleware/platform'
|
||||
import {
|
||||
@@ -130,7 +131,7 @@ const getUserQuotaRoute = authRoute(
|
||||
tags: ['Users'],
|
||||
method: 'get',
|
||||
path: '/{userId}/quota',
|
||||
request: { params: z.object({ userId: z.string() }) },
|
||||
request: { params: z.object({ userId: opaqueIdSchema }) },
|
||||
responses: { 200: jsonContent(userQuotaSchema, 'User quota') },
|
||||
},
|
||||
)
|
||||
@@ -143,7 +144,7 @@ const listUserEntitlementsRoute = authRoute(
|
||||
tags: ['Users'],
|
||||
method: 'get',
|
||||
path: '/{userId}/entitlements',
|
||||
request: { params: z.object({ userId: z.string() }) },
|
||||
request: { params: z.object({ userId: opaqueIdSchema }) },
|
||||
responses: {
|
||||
200: jsonContent(entitlementListSchema, 'Entitlements'),
|
||||
400: errorResponse('Bad request'),
|
||||
@@ -160,7 +161,7 @@ const grantUserEntitlementRoute = authRoute(
|
||||
tags: ['Users'],
|
||||
method: 'post',
|
||||
path: '/{userId}/entitlements',
|
||||
request: { params: z.object({ userId: z.string() }), ...jsonBody(grantEntitlementSchema) },
|
||||
request: { params: z.object({ userId: opaqueIdSchema }), ...jsonBody(grantEntitlementSchema) },
|
||||
responses: {
|
||||
201: jsonContent(entitlementResultSchema, 'Granted'),
|
||||
400: errorResponse('Bad request'),
|
||||
@@ -177,7 +178,10 @@ const updateUserEntitlementRoute = authRoute(
|
||||
tags: ['Users'],
|
||||
method: 'patch',
|
||||
path: '/{userId}/entitlements/{eid}',
|
||||
request: { params: z.object({ userId: z.string(), eid: z.string() }), ...jsonBody(updateEntitlementSchema) },
|
||||
request: {
|
||||
params: z.object({ userId: opaqueIdSchema, eid: opaqueIdSchema }),
|
||||
...jsonBody(updateEntitlementSchema),
|
||||
},
|
||||
responses: {
|
||||
200: jsonContent(entitlementResultSchema, 'Updated'),
|
||||
400: errorResponse('Bad request'),
|
||||
@@ -194,7 +198,7 @@ const revokeUserEntitlementRoute = authRoute(
|
||||
tags: ['Users'],
|
||||
method: 'delete',
|
||||
path: '/{userId}/entitlements/{eid}',
|
||||
request: { params: z.object({ userId: z.string(), eid: z.string() }) },
|
||||
request: { params: z.object({ userId: opaqueIdSchema, eid: opaqueIdSchema }) },
|
||||
responses: {
|
||||
204: { description: 'Revoked' },
|
||||
400: errorResponse('Bad request'),
|
||||
|
||||
@@ -13,20 +13,20 @@ describe('buildObjectKey', () => {
|
||||
vi.setSystemTime(new Date('2026-03-15T12:00:00Z'))
|
||||
|
||||
const result = buildObjectKey(baseVars)
|
||||
// Template: $ORG_ID/$UID/$NOW_DATE/$RAND_16KEY$RAW_EXT
|
||||
expect(result).toMatch(/^org456\/user123\/20260315\/.{16}\.jpg$/)
|
||||
// 17 Base62 characters preserve the entropy of the previous 16-character Nano ID.
|
||||
expect(result).toMatch(/^org456\/user123\/20260315\/[A-Za-z0-9]{17}\.jpg$/)
|
||||
|
||||
vi.useRealTimers()
|
||||
})
|
||||
|
||||
it('includes a 16-char random key', () => {
|
||||
it('includes a 17-char Base62 random key', () => {
|
||||
vi.useFakeTimers()
|
||||
vi.setSystemTime(new Date('2026-03-15T12:00:00Z'))
|
||||
|
||||
const result = buildObjectKey(baseVars)
|
||||
const parts = result.split('/')
|
||||
const filename = parts[3] // RAND_16KEY + ext
|
||||
expect(filename.replace('.jpg', '')).toHaveLength(16)
|
||||
expect(filename.replace('.jpg', '')).toMatch(/^[A-Za-z0-9]{17}$/)
|
||||
|
||||
vi.useRealTimers()
|
||||
})
|
||||
@@ -36,7 +36,7 @@ describe('buildObjectKey', () => {
|
||||
vi.setSystemTime(new Date('2026-03-15T12:00:00Z'))
|
||||
|
||||
const result = buildObjectKey({ ...baseVars, rawExt: '' })
|
||||
expect(result).toMatch(/^org456\/user123\/20260315\/.{16}$/)
|
||||
expect(result).toMatch(/^org456\/user123\/20260315\/[A-Za-z0-9]{17}$/)
|
||||
|
||||
vi.useRealTimers()
|
||||
})
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { nanoid } from 'nanoid'
|
||||
import { generateToken } from '@shared/ids'
|
||||
|
||||
export interface TemplateVars {
|
||||
uid: string
|
||||
@@ -18,5 +18,5 @@ export function buildObjectKey(vars: TemplateVars): string {
|
||||
const month = String(now.getMonth() + 1).padStart(2, '0')
|
||||
const day = String(now.getDate()).padStart(2, '0')
|
||||
|
||||
return `${vars.orgId}/${vars.uid}/${year}${month}${day}/${nanoid(16)}${vars.rawExt}`
|
||||
return `${vars.orgId}/${vars.uid}/${year}${month}${day}/${generateToken(17)}${vars.rawExt}`
|
||||
}
|
||||
|
||||
+32
-1
@@ -60,7 +60,7 @@ describe('global OpenAPI document', () => {
|
||||
schema: {
|
||||
properties: {
|
||||
registration_client_uri: { type: 'string', format: 'uri' },
|
||||
registration_access_token: { type: 'string' },
|
||||
registration_access_token: { type: 'string', pattern: '^[A-Za-z0-9]{43}$' },
|
||||
},
|
||||
required: expect.arrayContaining(['registration_client_uri', 'registration_access_token']),
|
||||
},
|
||||
@@ -71,6 +71,37 @@ describe('global OpenAPI document', () => {
|
||||
})
|
||||
})
|
||||
|
||||
it('publishes Base62 contracts for ZPan-owned IDs and public tokens', async () => {
|
||||
const { app } = await createTestApp({ DOWNLOAD_TOKEN_SECRET: 'test-download-token-secret' })
|
||||
const res = await app.request('/api/openapi.json')
|
||||
const doc = (await res.json()) as {
|
||||
paths: Record<string, Record<string, { parameters?: Array<{ name?: string; schema?: { pattern?: string } }> }>>
|
||||
components?: { schemas?: Record<string, { properties?: Record<string, { pattern?: string }> }> }
|
||||
}
|
||||
|
||||
const parameterPattern = (path: string, method: string, name: string) =>
|
||||
doc.paths[path]?.[method]?.parameters?.find((parameter) => parameter.name === name)?.schema?.pattern
|
||||
const objectId = doc.paths['/api/objects/{id}']?.get?.parameters?.find(({ name }) => name === 'id')
|
||||
const shareToken = doc.paths['/api/shares/{token}']?.get?.parameters?.find(({ name }) => name === 'token')
|
||||
expect(objectId?.schema?.pattern).toBe('^[A-Za-z0-9]+$')
|
||||
expect(shareToken?.schema?.pattern).toBe('^[A-Za-z0-9]+$')
|
||||
expect(parameterPattern('/api/teams/{teamId}', 'get', 'teamId')).toBe('^[A-Za-z0-9]+$')
|
||||
expect(parameterPattern('/api/users/{userId}/entitlements/{eid}', 'delete', 'userId')).toBe('^[A-Za-z0-9]+$')
|
||||
expect(parameterPattern('/api/users/{userId}/entitlements/{eid}', 'delete', 'eid')).toBe('^[A-Za-z0-9]+$')
|
||||
expect(parameterPattern('/api/site/announcements/{id}', 'get', 'id')).toBe('^[A-Za-z0-9]+$')
|
||||
expect(parameterPattern('/api/downloads/tasks', 'get', 'pageToken')).toBe('^[A-Za-z0-9]+$')
|
||||
expect(parameterPattern('/api/image-hosting/images', 'get', 'pageToken')).toBe('^[A-Za-z0-9]+$')
|
||||
expect(doc.components?.schemas?.Matter?.properties?.id?.pattern).toBe('^[A-Za-z0-9]+$')
|
||||
expect(doc.components?.schemas?.TeamSummary?.properties?.id?.pattern).toBe('^[A-Za-z0-9]+$')
|
||||
expect(doc.components?.schemas?.Announcement?.properties?.createdBy?.pattern).toBe('^[A-Za-z0-9]+$')
|
||||
expect(doc.components?.schemas?.QuotaEntitlement?.properties?.orgId?.pattern).toBe('^[A-Za-z0-9]+$')
|
||||
expect(doc.components?.schemas?.DownloadTask?.properties?.id?.pattern).toBe('^[A-Za-z0-9]+$')
|
||||
expect(doc.components?.schemas?.DownloadTaskPage?.properties?.nextPageToken?.pattern).toBe('^[A-Za-z0-9]+$')
|
||||
expect(doc.components?.schemas?.DownloadTaskListPage?.properties?.nextPageToken?.pattern).toBe('^[A-Za-z0-9]+$')
|
||||
expect(doc.components?.schemas?.ImageHostingList?.properties?.nextPageToken?.pattern).toBe('^[A-Za-z0-9]+$')
|
||||
expect(doc.components?.schemas?.ShareObjects?.properties?.nextPageToken?.pattern).toBe('^[A-Za-z0-9]+$')
|
||||
})
|
||||
|
||||
it('serves the Scalar reference UI at /api/docs pointing at the spec', async () => {
|
||||
const { app } = await createTestApp({ DOWNLOAD_TOKEN_SECRET: 'test-download-token-secret' })
|
||||
const res = await app.request('/api/docs')
|
||||
|
||||
@@ -93,7 +93,7 @@ describe('admin stats backfill', () => {
|
||||
created_at INTEGER NOT NULL, creator_id TEXT
|
||||
);
|
||||
CREATE TABLE audit_events (
|
||||
id TEXT PRIMARY KEY, org_id TEXT NOT NULL, user_id TEXT, actor_type TEXT, actor_ref TEXT,
|
||||
id TEXT PRIMARY KEY, event_key TEXT UNIQUE, org_id TEXT NOT NULL, user_id TEXT, actor_type TEXT, actor_ref TEXT,
|
||||
action TEXT NOT NULL, target_type TEXT NOT NULL, target_id TEXT, target_name TEXT NOT NULL,
|
||||
metadata TEXT, created_at INTEGER NOT NULL
|
||||
);
|
||||
@@ -162,7 +162,9 @@ describe('admin stats backfill', () => {
|
||||
('m2', 'o2', 'u2', ${historyStartMs + 1000});
|
||||
INSERT INTO matters VALUES ('f1', 512, 0);
|
||||
INSERT INTO shares VALUES ('s1', 'landing', 'f1', 'o1', 'active', NULL, 10, 0, 1, ${eventSec}, 'u1');
|
||||
INSERT INTO audit_events VALUES
|
||||
INSERT INTO audit_events (
|
||||
id, org_id, user_id, actor_type, actor_ref, action, target_type, target_id, target_name, metadata, created_at
|
||||
) VALUES
|
||||
('audit:statistics_source_initialized:v3-authoritative-sources', '', NULL, 'system', 'statistics-integrity', 'statistics_source_initialized', 'statistics', 'v3-authoritative-sources', 'statistics source', '{"schemaVersion":3}', ${Math.floor(historyStartMs / 1000)}),
|
||||
('upload-1', 'o1', 'u1', NULL, NULL, 'upload_confirm', 'file', 'f1', 'file.bin',
|
||||
'{"bytes":512,"source":"upload","status":"success"}', ${eventSec}),
|
||||
@@ -182,6 +184,7 @@ describe('admin stats backfill', () => {
|
||||
('legacy-task-completed', 'o1', NULL, 'system', 'legacy-download-task-worker', 'download_task_completed', 'remote_download', 't1', 'task',
|
||||
'{"category":"video","outcome":"completed","bytes":512}', ${eventSec}),
|
||||
('legacy-cloud-customer', 'o1', 'cloud-customer-1', 'user', NULL, 'quota_order_increase', 'quota', 'o1', 'o1', NULL, ${eventSec});
|
||||
UPDATE audit_events SET event_key = id WHERE id = 'audit:statistics_source_initialized:v3-authoritative-sources';
|
||||
INSERT INTO cloud_traffic_reports (
|
||||
id, org_id, period, source, source_id, event_id, bytes, storage_id, unit_bytes, credits_per_unit,
|
||||
status, error, attempt_count, next_retry_at, issued_at, created_at, updated_at
|
||||
|
||||
@@ -500,6 +500,7 @@ const APP_SCHEMA_SQL = `
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS team_invite_links_token_unique ON team_invite_links(token);
|
||||
CREATE TABLE IF NOT EXISTS audit_events (
|
||||
id TEXT PRIMARY KEY,
|
||||
event_key TEXT UNIQUE,
|
||||
org_id TEXT NOT NULL,
|
||||
user_id TEXT,
|
||||
action TEXT NOT NULL,
|
||||
@@ -574,6 +575,12 @@ const APP_SCHEMA_SQL = `
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS shares_creator_status_created_idx ON shares(creator_id, status, created_at);
|
||||
CREATE INDEX IF NOT EXISTS shares_creator_private_created_idx ON shares(creator_id, private, created_at);
|
||||
CREATE TABLE IF NOT EXISTS redirect_token_registry (
|
||||
token TEXT PRIMARY KEY,
|
||||
kind TEXT NOT NULL,
|
||||
resource_id TEXT NOT NULL,
|
||||
UNIQUE (kind, resource_id)
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS share_recipients (
|
||||
id TEXT PRIMARY KEY,
|
||||
share_id TEXT NOT NULL,
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { generateId, generateToken } from '@shared/ids'
|
||||
import type {
|
||||
CreateDownloaderInput,
|
||||
CreateDownloadTaskInput,
|
||||
@@ -17,7 +18,6 @@ import type {
|
||||
DownloadTaskRuntime,
|
||||
DownloadTaskTimelineItem,
|
||||
} from '@shared/types'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { ZPAN_CLOUD_URL_DEFAULT } from '../../../shared/constants'
|
||||
import { parseDownloadTaskEvents } from '../../domain/download-task-events'
|
||||
import { hasFeature } from '../../domain/licensing'
|
||||
@@ -159,8 +159,8 @@ async function prepareDownloaderRegistration(
|
||||
userId: string,
|
||||
now: Date,
|
||||
) {
|
||||
const id = nanoid()
|
||||
const jti = nanoid()
|
||||
const id = generateId()
|
||||
const jti = generateToken(22)
|
||||
const token = await deps.downloadTokens.signDownloadToken(platform, {
|
||||
v: 1,
|
||||
typ: 'downloader',
|
||||
@@ -317,7 +317,7 @@ export async function createDownloadTask(
|
||||
folderPath: input.targetFolder,
|
||||
})
|
||||
const now = new Date()
|
||||
const id = nanoid()
|
||||
const id = generateId()
|
||||
await deps.downloadTasks.insert({
|
||||
id,
|
||||
orgId,
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { generateToken } from '@shared/ids'
|
||||
import type { PutIhostConfigInput } from '@shared/schemas'
|
||||
import { nanoid } from 'nanoid'
|
||||
import { hasFeature } from '../../domain/licensing'
|
||||
import {
|
||||
AppError,
|
||||
@@ -168,8 +168,8 @@ export async function putImageHostingConfig(
|
||||
verificationToken:
|
||||
newDomain && provider?.settings.provider === 'manual'
|
||||
? newDomain === oldDomain
|
||||
? (existing?.verificationToken ?? nanoid(32))
|
||||
: nanoid(32)
|
||||
? (existing?.verificationToken ?? generateToken(33))
|
||||
: generateToken(33)
|
||||
: null,
|
||||
} as const
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user