mirror of
https://github.com/saltbo/zpan.git
synced 2026-08-28 15:51:29 +08:00
fix(ids): make normalization optional
Keep historical identifiers readable and addressable while generating all new entity IDs with Base62. Reject unsafe external ID references and preserve API keys and device codes during optional normalization.
This commit is contained in:
saltbo
@@ -1,12 +1,26 @@
|
||||
# Base62 ID normalization release runbook
|
||||
# Optional Base62 ID normalization runbook
|
||||
|
||||
This release is a breaking, maintenance-window migration. ZPan-owned opaque IDs and public tokens use the fixed alphabet
|
||||
`0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz` and must match `^[A-Za-z0-9]+$`. The runtime dispatches
|
||||
public redirects only by the new `s`/`i` namespace prefix and has no legacy lookup, alias, or old/new format fallback.
|
||||
New ZPan-owned opaque IDs and public tokens use the fixed alphabet
|
||||
`0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz`. Existing entity IDs containing `_` or `-` remain
|
||||
supported, and historical `ds_`/`ih` public links remain readable. Administrators may leave historical values unchanged
|
||||
or run this maintenance-window tool to normalize them in bulk; normal application startup never requires it.
|
||||
|
||||
Production execution is intentionally outside this change. Do not run any `wrangler ... --remote` command until a
|
||||
maintainer has reviewed the dry-run statistics, secured backup, reconciliation report, and D1 SQL plan and has given a
|
||||
new, explicit production approval.
|
||||
## Runtime ID policy
|
||||
|
||||
- Every new ZPan entity receives its own ID from the central Base62 generator. Create/update APIs do not accept a
|
||||
caller-selected primary ID, and new share/image public tokens use their current Base62 namespace.
|
||||
- Historical IDs may be returned and may be used to locate or reference the already-existing row. This includes write
|
||||
operations against an existing legacy object and creating a new entity that refers to an existing legacy organization,
|
||||
user, matter, or storage. The referenced row must still exist and pass the normal ownership/authorization checks.
|
||||
- Accepting a historical reference never creates or rewrites an entity with that legacy ID. ID-shaped external inputs
|
||||
reject everything outside the compatibility alphabet (`A-Z`, `a-z`, `0-9`, `_`, `-`); path separators, punctuation,
|
||||
whitespace, Unicode, URNs, and structured event/provider identifiers are not accepted as local entity IDs.
|
||||
- Provider-owned, protocol, business/event, natural, and physical-storage identifiers follow their own contracts and are
|
||||
not treated as ZPan entity IDs.
|
||||
|
||||
Production normalization is intentionally an explicit operator choice. Do not run any `wrangler ... --remote` command
|
||||
until a maintainer has reviewed the dry-run statistics, secured backup, reconciliation report, and D1 SQL plan and has
|
||||
given a new, explicit production approval.
|
||||
|
||||
## Observable release baseline
|
||||
|
||||
@@ -28,8 +42,8 @@ direct default `nanoid()` call in production roots.
|
||||
|
||||
| Classification | Values | Migration action |
|
||||
| --- | --- | --- |
|
||||
| Must migrate: ZPan entity PKs | user, account, organization, member, invitation, OAuth client/resource/link row, matter, storage, share/recipient, audit event, notification, announcement, quota/entitlement, background job, downloader, download task/usage report, upload session, image, invite, WebDAV state row, traffic report, license binding, webhook row, x402 intent | Random old-to-new mapping for every non-Base62 value; update known references and embedded JSON. Audit event entity IDs are separated from `event_key`. |
|
||||
| Must migrate: public/unique tokens | share and direct-download token, image token, site invite token, team invite token, invite code, image-domain verification token | Rotate every value, including already-alphanumeric legacy values. Shares become `s` + 11 random Base62 characters and images become `i` + 11 random Base62 characters. Old public links intentionally stop working. |
|
||||
| Optional normalization: ZPan entity PKs | user, account, organization, member, invitation, OAuth client/resource/link row, matter, storage, share/recipient, audit event, notification, announcement, quota/entitlement, background job, downloader, download task/usage report, upload session, image, invite, WebDAV state row, traffic report, license binding, webhook row, x402 intent | When the tool is selected, create a random old-to-new mapping for every non-Base62 value and update known references and embedded JSON. Audit event entity IDs are separated from `event_key`. |
|
||||
| Optional normalization: public/unique tokens | share and direct-download token, image token, site invite token, team invite token, invite code, image-domain verification token | When the tool is selected, rotate every value, including already-alphanumeric legacy values. Shares become `s` + 11 random Base62 characters and images become `i` + 11 random Base62 characters. Old public links intentionally stop working only for an administrator who executes the tool. |
|
||||
| Future generator plus historical scan | matter alias; dynamic OAuth registration management token | Migrate invalid aliases. Dynamic registration token hashes cannot be rewritten without plaintext, so registrations are invalidated and clients must re-register. |
|
||||
| Structured business/event keys | `traffic_<id>`, `mutation:<id>`, `admin_grant:<id>`, storage ledger `event_key`, downloader event IDs, idempotency keys, webhook provider event IDs, stats rollup deterministic IDs | Not entity IDs. Preserve their structure. Update only typed local references: audit target/event keys, storage opening/matter/image keys, Free-plan source keys, and initial download-task event keys. Cloud/Webhook/downloader event IDs stay byte-for-byte unchanged because they are external idempotency identities. |
|
||||
| Protocol identifiers | HTTP request UUID, WebDAV `opaquelocktoken:` URI, OAuth/PAR `urn:ietf:params:oauth:request_uri:` value, OAuth state/PKCE values, JWT `jti`/claims, JWK `kid` | Preserve protocol format. The Base62 PAR suffix is new, but the standards-defined URN remains a URI and is not validated as an opaque ID. |
|
||||
@@ -37,7 +51,7 @@ direct default `nanoid()` call in production roots.
|
||||
| External-system-owned values | S3 multipart `upload_id`, Cloud/Store order/resource/account/binding/attempt IDs, OAuth `client_id` and resource URI, provider account ID, external access/refresh/ID tokens | Do not rewrite. Inbound schemas remain protocol/provider appropriate. |
|
||||
| Physical object references | `matters.object`, image `storage_key`, storage bucket/key/path | Do not rename objects. These persisted physical references continue to point to the existing object even when organization/matter/image IDs change. Reconciliation proves every referenced object remains accessible. |
|
||||
| ID-derived avatar/logo objects | `user.image`, `organization.logo`, local `PUBLIC_IMAGES` keys (`user/<id>`, `team/<id>`), Cloud avatar owner IDs | If an owner ID needs mapping and an image/logo exists, fail before mutation. Export the affected owners, delete/clear the old hosted image while the old ID is authoritative, then re-upload after migration. The database-only tool never copies, deletes, or rebinds external R2/Cloud objects. Unrelated provider avatar URLs must still be reviewed and explicitly cleared/restored because the tool cannot prove their ownership from the URL alone. |
|
||||
| Session and authorization artifacts | API keys, sessions, verification rows, device codes, downloader bootstrap credentials, downloader bearer credentials, task-upload grants, OAuth access/refresh tokens, consents, PAR rows, client assertions, server signing keys, dynamic registration management rows and their dynamic clients | Delete the credential rows listed by the tool. Downloader rows remain for history, but every stored token hash/JTI is rotated and the row is disabled, including IDs that were already Base62. Active task-upload states must be drained before mapping, so old task-upload tokens cannot remain usable. Users must sign in again and recreate API keys; downloaders must re-pair; OAuth clients must authorize again; dynamic clients must re-register. No synthetic compatibility tokens are created. |
|
||||
| Session and authorization artifacts | API keys, sessions, verification rows, device codes, downloader bootstrap credentials, downloader bearer credentials, task-upload grants, OAuth access/refresh tokens, consents, PAR rows, client assertions, server signing keys, dynamic registration management rows and their dynamic clients | Preserve API keys and device-code rows while remapping their row IDs, user references, and workspace-scope metadata; their stored key/code material, permissions, limits, status, and expiry remain unchanged. Delete the other credential rows listed by the tool. Downloader rows remain for history, but every stored token hash/JTI is rotated and the row is disabled, including IDs that were already Base62. Active task-upload states must be drained before mapping, so old task-upload tokens cannot remain usable. Users must sign in again; downloaders must re-pair; OAuth clients must authorize again; dynamic clients must re-register. No synthetic compatibility tokens are created. |
|
||||
| Cross-system instance identity | `system_options.instance_id` and active Cloud license binding | A locally rewritten value would orphan the Cloud-side binding. If it is non-Base62, stop and obtain a Cloud reconciliation decision: disconnect/rebind under a new Base62 instance ID or coordinate an external mapping. The local migration must not silently rewrite it. |
|
||||
| Pending cross-system usage/purchase state | pending/failed/blocked/skipped/dead-letter Cloud usage, non-reported remote-download usage, unfinished x402 purchases, Cloud-order entitlements | If a referenced local ID needs mapping, fail before creating the mapping table. Drain, cancel, or reconcile the external operation first. Historical provider event/idempotency IDs are never rewritten locally. |
|
||||
| Active Cloud customer identity | organization IDs used as Cloud/Store customer and order targets while a license binding is active | If any organization ID would change while an active binding exists, fail before mutation. Production requires a separately approved Cloud-side mapping/rebind; this tool never changes Cloud data. |
|
||||
@@ -46,10 +60,10 @@ direct default `nanoid()` call in production roots.
|
||||
|
||||
Existing physical object references stay unchanged during migration, but every future key allocation uses a guarded builder.
|
||||
Ordinary object, WebDAV, archive, copy, transfer, and downloader uploads use
|
||||
`<Base62 org ID>/<Base62 owner user ID>/<YYYYMMDD>/<17 Base62 random characters><extension>`. Image-hosting uploads use
|
||||
`ih/<Base62 org ID>/<13-character Base62 image ID>.<MIME-derived extension>`. The `/` separators and filename extension
|
||||
are S3 key structure, not opaque IDs. Key construction fails before an upload is presigned or written if an ID component
|
||||
is not Base62.
|
||||
`<compatible org ID>/<compatible owner user ID>/<YYYYMMDD>/<17 Base62 random characters><extension>`. Image-hosting
|
||||
uploads use `ih/<compatible org ID>/<new Base62 image ID>.<MIME-derived extension>`. Compatible historical components
|
||||
may contain `_` or `-`; `/` and other path syntax remain rejected. The `/` separators and filename extension are S3 key
|
||||
structure, not opaque IDs.
|
||||
|
||||
The inventory included `nanoid`, `customAlphabet`, `randomUUID`, and `randomBytes` calls; schema PKs/FKs/unique tokens;
|
||||
OpenAPI path parameters; clients/scripts/tests/docs; audit/notification/job JSON; R2/S3 physical keys; cache and idempotency
|
||||
@@ -58,8 +72,9 @@ organization plugin owns separate organization, member, and invitation create ho
|
||||
21-character Nano IDs with `_`/`-`. ZPan configures Better Auth's supported `advanced.database.generateId` hook and all
|
||||
three organization hooks with the same central generator. A database contract test covers user, account, organization,
|
||||
member, invitation, and session IDs plus session tokens.
|
||||
Better Auth API keys use a Base62 custom generator; caller prefixes containing punctuation are rejected. Because stored
|
||||
API-key hashes cannot be rewritten, all historical keys are intentionally revoked by the maintenance migration.
|
||||
Better Auth API keys use a Base62 custom generator; caller prefixes containing punctuation are rejected. Historical
|
||||
API-key hashes remain byte-for-byte unchanged. The migration remaps the API-key row ID, owner reference, audit actor
|
||||
references, and workspace organization ID in metadata so the same presented key keeps its permissions and limits.
|
||||
|
||||
## Generator entropy and collision budget
|
||||
|
||||
@@ -99,14 +114,15 @@ the public redirect namespaces disjoint without a cross-resource lookup or redir
|
||||
describe. Invalid historical references are assigned a stable pseudonym in the same mapping kind; already-Base62
|
||||
historical pseudonyms are retained. They are format-checked but are not misrepresented as live foreign keys. Empty
|
||||
ledger sentinels remain empty and are never added to the mapping table.
|
||||
6. Credential/session tables listed above are invalidated explicitly. Dynamic OAuth client rows with registration
|
||||
6. API keys and device codes are preserved with normalized internal IDs and references. The other credential/session
|
||||
tables listed above are invalidated explicitly. Dynamic OAuth client rows with registration
|
||||
management credentials are removed; statically configured clients remain. Every downloader token hash/JTI is
|
||||
replaced and the downloader is disabled even when its historical ID was already Base62.
|
||||
7. Validation checks row counts, uniqueness through constraints, `PRAGMA foreign_key_check`, zero illegal governed
|
||||
values, exact `s`/`i` public-token prefixes and lengths, live direct and typed-polymorphic target existence, typed JSON
|
||||
reference formats/targets according to their historical or live semantics, and structured upload creators. Any
|
||||
failure aborts the transaction. Legacy download tasks whose creator was stored as `api-key:<id>` are resolved to the
|
||||
API-key owner before the API-key rows are intentionally invalidated.
|
||||
API-key owner before the API-key row ID and owner reference are normalized.
|
||||
8. `--emit-d1-sql` writes the exact reviewed mapping and rewritten JSON to a mode-`0600` SQL plan. The plan has persistent
|
||||
`CHECK (violations = 0)` assertions for expected row counts, foreign keys, formats, exact rewritten values, logical
|
||||
references, and public-token namespace formats. The versioned completion marker is the last state change; a missed update or
|
||||
@@ -118,10 +134,9 @@ the public redirect namespaces disjoint without a cross-resource lookup or redir
|
||||
exceeds 1,000 statements. A database with an individually oversized rewritten JSON value or a plan that still needs
|
||||
more statements requires a separately engineered phased migration; the tool never emits a known-unexecutable plan.
|
||||
|
||||
The runtime startup gate is intentionally O(1) after migration: it requires both versioned proof markers instead of
|
||||
rescanning every large table on each Workers isolate cold start. A populated database without the proof fails fast.
|
||||
The expensive zero-invalid and logical-integrity scans are part of the maintenance transaction and D1 assertion plan;
|
||||
all future application writes use the central generator and narrowed schemas.
|
||||
The runtime does not inspect or require the normalization proof markers. A populated database with historical IDs starts
|
||||
normally. The expensive zero-invalid and logical-integrity scans run only when an administrator invokes the optional
|
||||
maintenance tool; all future application writes still use the central Base62 generator.
|
||||
|
||||
Only counts are printed to stdout. Old/new values exist only in the protected backup, map table, and optional protected
|
||||
D1 plan.
|
||||
@@ -152,7 +167,8 @@ The release rehearsal performed on 2026-08-04 used isolated temporary directorie
|
||||
`--remote` flag. It applied all 92 migrations, loaded the representative legacy fixture, completed dry-run and apply,
|
||||
created mode-`0600` backup/plan artifacts, and replayed 635 plan commands (including 496 persistent machine assertions) on a second
|
||||
fresh local D1 database. A second replay was semantically idempotent. Both SQLite and D1 returned zero foreign-key
|
||||
failures and zero invalid share/image tokens; credentials were deleted or deterministically disabled, direct,
|
||||
failures and zero invalid share/image tokens; credentials governed by the original rehearsal were deleted or
|
||||
deterministically disabled; direct,
|
||||
polymorphic, JSON, notification, and both user/downloader upload-creator references matched their mapped entities, and
|
||||
Cloud event IDs plus S3 object/image/multipart storage references stayed byte-for-byte unchanged. The backup retained
|
||||
the old IDs, public token, object keys, and session row, proving the documented full-restore rollback point.
|
||||
@@ -176,7 +192,7 @@ profile share, image hosting, and image-domain flows were then run against an is
|
||||
This Draft must not be promoted until the mandatory CONTRIBUTING preview verification is run in an approved isolated
|
||||
preview environment with its own Cloud/S3 credentials.
|
||||
|
||||
## Production maintenance procedure (requires new approval)
|
||||
## Optional production maintenance procedure (requires new approval)
|
||||
|
||||
1. Announce a write outage and prevent all API, WebDAV, background-job, Queue-consumer, and downloader writes. Stop all
|
||||
old runtime isolates/processes; their in-process caches and old writers must not run while identifiers are changing.
|
||||
@@ -197,13 +213,14 @@ preview environment with its own Cloud/S3 credentials.
|
||||
table or statement:** PK and logical-reference rewrites cross table boundaries. If the rehearsal exceeds a D1 batch,
|
||||
statement, or transaction limit, stop; a separate shadow-table/phased migration must be engineered, reviewed, and
|
||||
approved before production.
|
||||
7. Confirm both `validation_version=2` and `completed_at` exist only after all D1 assertion rows report zero. Deploy the
|
||||
new runtime only after reconciliation passes. The v2 image-domain cache namespace prevents legacy KV hits; verify the
|
||||
7. Confirm both `validation_version=2` and `completed_at` exist only after all D1 assertion rows report zero. Reopen the
|
||||
runtime only after reconciliation passes. The v2 image-domain cache namespace prevents legacy KV hits; verify the
|
||||
first domain lookup loads normalized state. Keep writes disabled until smoke tests prove share, image, object, quota,
|
||||
audit, job, WebDAV, downloader, and OAuth behavior. Re-upload reconciled avatars/logos and re-enqueue only normalized
|
||||
archive work.
|
||||
8. Notify users that old share/image/invite links are invalid, sessions require login, API keys must be recreated,
|
||||
OAuth grants require authorization, dynamic clients require registration, and downloaders must re-pair. Drain or
|
||||
8. Notify users that old share/image/invite links are invalid and sessions require login. Existing API keys and
|
||||
device-code rows remain valid according to their existing status and expiry; OAuth grants require authorization,
|
||||
dynamic clients require registration, and downloaders must re-pair. Drain or
|
||||
pause assigned/in-flight download tasks before the snapshot; after migration reconcile each task, requeue safe work,
|
||||
and issue new downloader/task-upload credentials only after the corresponding downloader has re-paired. An encrypted
|
||||
operator-only mapping export may support targeted notification; it must never power runtime fallback.
|
||||
@@ -252,4 +269,5 @@ reverse mappings in place: deleted session/OAuth/verification material cannot be
|
||||
and a partial inverse would violate the no-dual-format invariant. Re-run row counts, foreign keys, object reads, and old
|
||||
public-link checks on the restored database before reopening traffic.
|
||||
|
||||
Old public links are a deliberate breaking change. There is no runtime redirect, legacy alias table, or fallback lookup.
|
||||
Executing the optional tool deliberately rotates old public links. Without normalization, the runtime continues to
|
||||
recognize historical `ds_` direct-share tokens, `ih` image tokens, and entity IDs containing `_` or `-`.
|
||||
|
||||
@@ -61,6 +61,8 @@ const VALUE_SPECS: ValueSpec[] = [
|
||||
references: [
|
||||
ref('account', 'user_id'),
|
||||
ref('session', 'user_id'),
|
||||
ref('apikey', 'reference_id'),
|
||||
ref('deviceCode', 'user_id'),
|
||||
ref('member', 'user_id'),
|
||||
ref('invitation', 'inviter_id'),
|
||||
ref('oauthClient', 'user_id'),
|
||||
@@ -180,6 +182,7 @@ const VALUE_SPECS: ValueSpec[] = [
|
||||
...[
|
||||
'account',
|
||||
'apikey',
|
||||
'deviceCode',
|
||||
'audit_events',
|
||||
'member',
|
||||
'invitation',
|
||||
@@ -227,10 +230,8 @@ const VALUE_SPECS: ValueSpec[] = [
|
||||
]
|
||||
|
||||
const INVALIDATE_TABLES = [
|
||||
'apikey',
|
||||
'session',
|
||||
'verification',
|
||||
'deviceCode',
|
||||
'downloader_bootstrap_credentials',
|
||||
'oauthAccessToken',
|
||||
'oauthRefreshToken',
|
||||
@@ -241,11 +242,9 @@ const INVALIDATE_TABLES = [
|
||||
] as const
|
||||
|
||||
const VALIDATE_ONLY_COLUMNS = [
|
||||
ref('apikey', 'id'),
|
||||
ref('session', 'id'),
|
||||
ref('session', 'token'),
|
||||
ref('verification', 'id'),
|
||||
ref('deviceCode', 'id'),
|
||||
ref('downloader_bootstrap_credentials', 'id'),
|
||||
ref('oauthAccessToken', 'id'),
|
||||
ref('oauthRefreshToken', 'id'),
|
||||
@@ -255,6 +254,7 @@ const VALIDATE_ONLY_COLUMNS = [
|
||||
] as const
|
||||
|
||||
const JSON_COLUMNS = [
|
||||
ref('apikey', 'metadata'),
|
||||
ref('audit_events', 'metadata'),
|
||||
ref('notifications', 'metadata'),
|
||||
ref('background_jobs', 'metadata'),
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import type { Platform } from './platform/interface'
|
||||
import type { Deps } from './usecases/deps'
|
||||
|
||||
const app = { fetch: vi.fn() }
|
||||
const auth = { api: {} }
|
||||
const createAppMock = vi.hoisted(() => vi.fn(() => app))
|
||||
const createAuthMock = vi.hoisted(() => vi.fn(async () => auth))
|
||||
|
||||
vi.mock('./app', () => ({ createApp: createAppMock }))
|
||||
vi.mock('./auth', () => ({ createAuth: createAuthMock }))
|
||||
|
||||
import { createBootstrap } from './bootstrap'
|
||||
|
||||
describe('Node bootstrap', () => {
|
||||
beforeEach(() => {
|
||||
createAppMock.mockClear()
|
||||
createAuthMock.mockClear()
|
||||
})
|
||||
|
||||
it('starts without inspecting optional ID-normalization state', async () => {
|
||||
const db = { all: vi.fn(), run: vi.fn() }
|
||||
const platform = {
|
||||
db,
|
||||
getEnv: (name: string) =>
|
||||
name === 'BETTER_AUTH_SECRET' ? 'test-secret' : name === 'BETTER_AUTH_URL' ? 'https://zpan.test' : undefined,
|
||||
} as unknown as Platform
|
||||
const deps = {} as Deps
|
||||
|
||||
await expect(createBootstrap(platform, deps)).resolves.toBe(app)
|
||||
expect(createAuthMock).toHaveBeenCalledWith(platform, 'test-secret', 'https://zpan.test', ['http://localhost:5185'])
|
||||
expect(createAppMock).toHaveBeenCalledWith(platform, auth, deps)
|
||||
expect(db.all).not.toHaveBeenCalled()
|
||||
expect(db.run).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
@@ -1,12 +1,10 @@
|
||||
import { createApp } from './app'
|
||||
import { createAuth } from './auth'
|
||||
import { createDeps } from './composition'
|
||||
import { assertNormalizedIdentifiers } from './db/id-normalization'
|
||||
import type { Platform } from './platform/interface'
|
||||
import type { Deps } from './usecases/deps'
|
||||
|
||||
export async function createBootstrap(platform: Platform, deps: Deps = createDeps(platform)) {
|
||||
await assertNormalizedIdentifiers(platform.db)
|
||||
const secret = platform.getEnv('BETTER_AUTH_SECRET')
|
||||
if (!secret) {
|
||||
throw new Error('BETTER_AUTH_SECRET is required. Set it in the environment before starting the server.')
|
||||
|
||||
@@ -1,74 +0,0 @@
|
||||
import { sql } from 'drizzle-orm'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { createTestApp } from '../test/setup'
|
||||
import { assertNormalizedIdentifiers } from './id-normalization'
|
||||
|
||||
describe('normalized identifier startup contract', () => {
|
||||
it('marks a fresh database so a later restart accepts newly created records', async () => {
|
||||
const { db } = await createTestApp()
|
||||
// createTestApp initializes the configured OAuth resource; a real runtime runs
|
||||
// this gate before auth initialization, so remove that seed to model first boot.
|
||||
await db.run(sql.raw('DELETE FROM oauthResource'))
|
||||
|
||||
await expect(assertNormalizedIdentifiers(db)).resolves.toBeUndefined()
|
||||
expect(
|
||||
await db.all<{ value: string }>(
|
||||
sql.raw("SELECT value FROM _zpan_id_normalization_state WHERE key = 'validation_version'"),
|
||||
),
|
||||
).toEqual([{ value: '2' }])
|
||||
await db.run(sql`INSERT INTO user (id, name, email, email_verified, created_at, updated_at)
|
||||
VALUES ('CreatedAfterBoot', 'New', 'new@example.com', 1, 0, 0)`)
|
||||
await expect(assertNormalizedIdentifiers(db)).resolves.toBeUndefined()
|
||||
})
|
||||
|
||||
it('accepts a migrated database with a completion marker', async () => {
|
||||
const { db } = await createTestApp()
|
||||
await db.run(sql.raw('CREATE TABLE _zpan_id_normalization_state (key TEXT PRIMARY KEY, value TEXT NOT NULL)'))
|
||||
await db.run(
|
||||
sql.raw("INSERT INTO _zpan_id_normalization_state VALUES ('completed_at', '1'), ('validation_version', '2')"),
|
||||
)
|
||||
await expect(assertNormalizedIdentifiers(db)).resolves.toBeUndefined()
|
||||
})
|
||||
|
||||
it('rejects a completion marker that was not produced by the versioned validation gate', async () => {
|
||||
const { db } = await createTestApp()
|
||||
await db.run(sql.raw('CREATE TABLE _zpan_id_normalization_state (key TEXT PRIMARY KEY, value TEXT NOT NULL)'))
|
||||
await db.run(sql.raw("INSERT INTO _zpan_id_normalization_state VALUES ('completed_at', '1')"))
|
||||
|
||||
await expect(assertNormalizedIdentifiers(db)).rejects.toThrow('id_normalization_validation_marker_missing')
|
||||
})
|
||||
|
||||
it('rejects a populated database when the one-time normalization has not completed', async () => {
|
||||
const { db } = await createTestApp()
|
||||
await db.run(sql`INSERT INTO user (id, name, email, email_verified, created_at, updated_at)
|
||||
VALUES ('AlreadyBase62', 'Existing', 'existing@example.com', 1, 0, 0)`)
|
||||
|
||||
await expect(assertNormalizedIdentifiers(db)).rejects.toThrow('id_normalization_not_completed')
|
||||
})
|
||||
|
||||
it('does not mistake standalone OAuth state for a fresh empty database', async () => {
|
||||
const { db } = await createTestApp()
|
||||
|
||||
await expect(assertNormalizedIdentifiers(db)).rejects.toThrow('id_normalization_not_completed')
|
||||
})
|
||||
|
||||
it('does not bless a database that only has an existing instance identity', async () => {
|
||||
const { db } = await createTestApp()
|
||||
await db.run(sql.raw('DELETE FROM oauthResource'))
|
||||
await db.run(sql.raw("INSERT INTO system_options (key, value) VALUES ('instance_id', 'legacy-instance-id')"))
|
||||
|
||||
await expect(assertNormalizedIdentifiers(db)).rejects.toThrow('id_normalization_not_completed')
|
||||
})
|
||||
|
||||
it('accepts a populated normalized database with a completion marker', async () => {
|
||||
const { db } = await createTestApp()
|
||||
await db.run(sql`INSERT INTO user (id, name, email, email_verified, created_at, updated_at)
|
||||
VALUES ('AlreadyBase62', 'Existing', 'existing@example.com', 1, 0, 0)`)
|
||||
await db.run(sql.raw('CREATE TABLE _zpan_id_normalization_state (key TEXT PRIMARY KEY, value TEXT NOT NULL)'))
|
||||
await db.run(
|
||||
sql.raw("INSERT INTO _zpan_id_normalization_state VALUES ('completed_at', '1'), ('validation_version', '2')"),
|
||||
)
|
||||
|
||||
await expect(assertNormalizedIdentifiers(db)).resolves.toBeUndefined()
|
||||
})
|
||||
})
|
||||
@@ -1,106 +0,0 @@
|
||||
import { sql } from 'drizzle-orm'
|
||||
import type { Database } from '../platform/interface'
|
||||
|
||||
const STATE_TABLE = '_zpan_id_normalization_state'
|
||||
const VALIDATION_VERSION = '2'
|
||||
|
||||
// The release tool performs the expensive format, FK, JSON, and logical-reference
|
||||
// scans before it writes this versioned marker. Runtime startup deliberately checks
|
||||
// only that proof: repeating full-table GLOB scans on every Worker isolate would make
|
||||
// cold-start cost proportional to the whole database.
|
||||
const OCCUPANCY_TABLES = [
|
||||
'user',
|
||||
'account',
|
||||
'session',
|
||||
'verification',
|
||||
'jwks',
|
||||
'organization',
|
||||
'member',
|
||||
'invitation',
|
||||
'apikey',
|
||||
'deviceCode',
|
||||
'oauthClient',
|
||||
'oauthClientRegistration',
|
||||
'oauthResource',
|
||||
'oauthClientResource',
|
||||
'oauthRefreshToken',
|
||||
'oauthAccessToken',
|
||||
'oauthConsent',
|
||||
'oauthClientAssertion',
|
||||
'oauthPushedAuthorizationRequest',
|
||||
'oauthJwtRevocation',
|
||||
'downloader_bootstrap_credentials',
|
||||
'matters',
|
||||
'webdav_dead_properties',
|
||||
'webdav_locks',
|
||||
'storages',
|
||||
'org_quotas',
|
||||
'storage_usage_breakdowns',
|
||||
'cloud_traffic_reports',
|
||||
'org_quota_entitlements',
|
||||
'webhook_events',
|
||||
'x402_capacity_purchase_intents',
|
||||
'invite_codes',
|
||||
'site_invitations',
|
||||
'license_bindings',
|
||||
'team_invite_links',
|
||||
'shares',
|
||||
'share_recipients',
|
||||
'audit_events',
|
||||
'resource_changes',
|
||||
'stats_rollups_hourly',
|
||||
'storage_usage_ledger',
|
||||
'notifications',
|
||||
'announcements',
|
||||
'background_jobs',
|
||||
'downloaders',
|
||||
'download_tasks',
|
||||
'object_upload_sessions',
|
||||
'remote_download_usage_reports',
|
||||
'image_hosting_configs',
|
||||
'image_hostings',
|
||||
'system_options',
|
||||
] as const
|
||||
|
||||
export async function assertNormalizedIdentifiers(db: Database): Promise<void> {
|
||||
const stateTable = await db.all<{ present: number }>(
|
||||
sql.raw(`SELECT COUNT(*) AS present FROM sqlite_master WHERE type = 'table' AND name = '${STATE_TABLE}'`),
|
||||
)
|
||||
const state = stateTable[0]?.present
|
||||
? await db.all<{ key: string; value: string }>(
|
||||
sql.raw(`SELECT key, value FROM "${STATE_TABLE}" WHERE key IN ('completed_at', 'validation_version')`),
|
||||
)
|
||||
: []
|
||||
const values = new Map<string, string>(state.map((entry: { key: string; value: string }) => [entry.key, entry.value]))
|
||||
const completed = values.get('completed_at')
|
||||
|
||||
if (completed) {
|
||||
if (values.get('validation_version') !== VALIDATION_VERSION) {
|
||||
throw new Error('id_normalization_validation_marker_missing')
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// Sum scalar EXISTS expressions rather than using UNION ALL: D1's compound
|
||||
// SELECT limit is lower than the number of governed tables.
|
||||
const occupancy = await db.all<{ occupied: number }>(
|
||||
sql.raw(
|
||||
`SELECT ${OCCUPANCY_TABLES.map((table) => `EXISTS(SELECT 1 FROM "${table}" LIMIT 1)`).join(' + ')} AS occupied`,
|
||||
),
|
||||
)
|
||||
if (Number(occupancy[0]?.occupied ?? 0) > 0) {
|
||||
throw new Error('id_normalization_not_completed')
|
||||
}
|
||||
|
||||
await db.run(sql.raw(`CREATE TABLE IF NOT EXISTS "${STATE_TABLE}" (key TEXT PRIMARY KEY, value TEXT NOT NULL)`))
|
||||
await db.run(
|
||||
sql.raw(
|
||||
`INSERT INTO "${STATE_TABLE}" (key, value) VALUES ('validation_version', '${VALIDATION_VERSION}') ON CONFLICT(key) DO UPDATE SET value = excluded.value`,
|
||||
),
|
||||
)
|
||||
await db.run(
|
||||
sql.raw(
|
||||
`INSERT INTO "${STATE_TABLE}" (key, value) VALUES ('completed_at', '${Date.now()}') ON CONFLICT(key) DO UPDATE SET value = excluded.value`,
|
||||
),
|
||||
)
|
||||
}
|
||||
@@ -20,8 +20,8 @@ function context(scope: string, id: string, getBinding = vi.fn()) {
|
||||
|
||||
describe('serveAvatarBlob', () => {
|
||||
it.each([
|
||||
['user', 'legacy_user'],
|
||||
['user', 'legacy-user'],
|
||||
['user', 'invalid/user'],
|
||||
['user', 'invalid:user'],
|
||||
['organization', 'Owner123'],
|
||||
])('rejects an invalid public blob key before reading R2', async (scope, id) => {
|
||||
const ctx = context(scope, id)
|
||||
@@ -31,15 +31,15 @@ describe('serveAvatarBlob', () => {
|
||||
expect(ctx.getBinding).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('reads only the canonical scope and Base62 owner ID', async () => {
|
||||
it.each(['Owner123', 'legacy_user', 'legacy-user'])('reads a compatible owner ID %s', async (ownerId) => {
|
||||
const get = vi.fn().mockResolvedValue({
|
||||
arrayBuffer: async () => new TextEncoder().encode('avatar').buffer,
|
||||
httpMetadata: { contentType: 'image/png' },
|
||||
})
|
||||
const ctx = context('team', 'Owner123', vi.fn().mockReturnValue({ get }))
|
||||
const ctx = context('team', ownerId, vi.fn().mockReturnValue({ get }))
|
||||
const response = await serveAvatarBlob(ctx.value)
|
||||
|
||||
expect(get).toHaveBeenCalledWith('team/Owner123')
|
||||
expect(get).toHaveBeenCalledWith(`team/${ownerId}`)
|
||||
expect(response.status).toBe(200)
|
||||
expect(response.headers.get('Content-Type')).toBe('image/png')
|
||||
})
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import type { Context } from 'hono'
|
||||
import { BASE62_PATTERN } from '../../shared/ids'
|
||||
import { OPAQUE_ID_PATTERN } from '../../shared/ids'
|
||||
import type { Env } from '../middleware/platform'
|
||||
import { PUBLIC_IMAGES_BINDING, type R2BucketLike } from '../platform/interface'
|
||||
|
||||
@@ -10,7 +10,7 @@ import { PUBLIC_IMAGES_BINDING, type R2BucketLike } from '../platform/interface'
|
||||
export async function serveAvatarBlob(c: Context<Env>) {
|
||||
const scope = c.req.param('scope')
|
||||
const id = c.req.param('id')
|
||||
if ((scope !== 'user' && scope !== 'team') || !id || !BASE62_PATTERN.test(id)) return c.body(null, 404)
|
||||
if ((scope !== 'user' && scope !== 'team') || !id || !OPAQUE_ID_PATTERN.test(id)) return c.body(null, 404)
|
||||
|
||||
const bucket = c.get('platform').getBinding<R2BucketLike>(PUBLIC_IMAGES_BINDING)
|
||||
if (!bucket) return c.body(null, 404)
|
||||
|
||||
@@ -137,6 +137,24 @@ describe('GET /r/:token (direct shares)', () => {
|
||||
])
|
||||
})
|
||||
|
||||
it('keeps a historical ds_ direct-share link usable [spec: redirect/legacy-direct-share]', async () => {
|
||||
const { app, db } = await createTestApp()
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
const creatorId = await getUserId(db)
|
||||
await insertFile(db, orgId, { id: 'legacy-direct-file_', name: 'legacy-direct.bin' })
|
||||
await db.run(sql`
|
||||
INSERT INTO shares (id, token, kind, matter_id, org_id, creator_id, views, downloads, status, private, created_at)
|
||||
VALUES ('legacy-direct-share_', 'ds_legacy-token', 'direct', 'legacy-direct-file_', ${orgId}, ${creatorId}, 0, 0, 'active', 0, ${Date.now()})
|
||||
`)
|
||||
|
||||
const res = await app.request('/r/ds_legacy-token', { redirect: 'manual' })
|
||||
|
||||
expect(res.status).toBe(302)
|
||||
expect(res.headers.get('location')).toBe(MOCK_PRESIGN_URL)
|
||||
})
|
||||
|
||||
it('returns 404 for an unknown opaque token [spec: redirect/unknown-ds-token]', async () => {
|
||||
const { app } = await createTestApp()
|
||||
const res = await app.request('/r/s00000000000', { redirect: 'manual' })
|
||||
@@ -339,6 +357,19 @@ describe('GET /r/:token (image hosting)', () => {
|
||||
expect(res.headers.get('location')).toBe(MOCK_INLINE_URL)
|
||||
})
|
||||
|
||||
it('keeps a historical ih image link usable [spec: redirect/legacy-image]', async () => {
|
||||
const { app, db } = await createTestApp()
|
||||
await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
await insertImageHosting(db, orgId, { id: 'legacy-image_', token: 'ih_legacy-token' })
|
||||
|
||||
const res = await app.request('/r/ih_legacy-token.png', { redirect: 'manual' })
|
||||
|
||||
expect(res.status).toBe(302)
|
||||
expect(res.headers.get('location')).toBe(MOCK_INLINE_URL)
|
||||
})
|
||||
|
||||
it('strips .png extension and resolves same image [spec: redirect/image-strip-ext]', async () => {
|
||||
const { app, db } = await createTestApp()
|
||||
await authedHeaders(app)
|
||||
@@ -363,7 +394,7 @@ describe('GET /r/:token (image hosting)', () => {
|
||||
expect(res.headers.get('location')).toBe(MOCK_INLINE_URL)
|
||||
})
|
||||
|
||||
it('rejects a legacy punctuated image token [spec: redirect/unknown-ih-token]', async () => {
|
||||
it('returns 404 for an unknown historical image token [spec: redirect/unknown-ih-token]', async () => {
|
||||
const { app } = await createTestApp()
|
||||
const res = await app.request('/r/ih_doesnotexist', { redirect: 'manual' })
|
||||
expect(res.status).toBe(404)
|
||||
|
||||
+13
-4
@@ -1,7 +1,12 @@
|
||||
import type { Context } from 'hono'
|
||||
import { Hono } from 'hono'
|
||||
import { ZPAN_CLOUD_URL_DEFAULT } from '../../shared/constants'
|
||||
import { IMAGE_TOKEN_PATTERN, SHARE_TOKEN_PATTERN } from '../../shared/ids'
|
||||
import {
|
||||
IMAGE_TOKEN_PATTERN,
|
||||
LEGACY_DIRECT_SHARE_TOKEN_PATTERN,
|
||||
LEGACY_IMAGE_TOKEN_PATTERN,
|
||||
SHARE_TOKEN_PATTERN,
|
||||
} from '../../shared/ids'
|
||||
import { isDownloadFailureStatus, transferAuditActor, transferFailureReason } from '../middleware/audit-transfers'
|
||||
import type { Env } from '../middleware/platform'
|
||||
import { notFound } from '../usecases/ports'
|
||||
@@ -17,13 +22,17 @@ import { recordDownloadFailure, recordDownloadIssued } from '../usecases/transfe
|
||||
|
||||
type ParsedRedirectToken = { kind: 'direct_share'; token: string } | { kind: 'image_hosting'; token: string }
|
||||
|
||||
const REDIRECT_TOKEN_PATTERN = /^([si][A-Za-z0-9]{11})(?:\.[A-Za-z0-9]{1,16})?$/
|
||||
const REDIRECT_TOKEN_PATTERN = /^([A-Za-z0-9_-]+)(?:\.[A-Za-z0-9]{1,16})?$/
|
||||
|
||||
function parseRedirectToken(raw: string): ParsedRedirectToken {
|
||||
const token = REDIRECT_TOKEN_PATTERN.exec(raw)?.[1]
|
||||
if (!token) throw notFound()
|
||||
if (SHARE_TOKEN_PATTERN.test(token)) return { kind: 'direct_share', token }
|
||||
if (IMAGE_TOKEN_PATTERN.test(token)) return { kind: 'image_hosting', token }
|
||||
if (SHARE_TOKEN_PATTERN.test(token) || LEGACY_DIRECT_SHARE_TOKEN_PATTERN.test(token)) {
|
||||
return { kind: 'direct_share', token }
|
||||
}
|
||||
if (IMAGE_TOKEN_PATTERN.test(token) || LEGACY_IMAGE_TOKEN_PATTERN.test(token)) {
|
||||
return { kind: 'image_hosting', token }
|
||||
}
|
||||
throw notFound()
|
||||
}
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { eq, sql } from 'drizzle-orm'
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { generateId } from '../../shared/ids'
|
||||
import { BASE62_PATTERN, generateId, SHARE_TOKEN_PATTERN } from '../../shared/ids'
|
||||
import { S3Service } from '../adapters/gateways/s3.js'
|
||||
import { createShareRepo } from '../adapters/repos/share'
|
||||
import { auditEvents, shareRecipients, shares } from '../db/schema.js'
|
||||
@@ -144,6 +144,35 @@ describe('POST /api/shares', () => {
|
||||
vi.restoreAllMocks()
|
||||
})
|
||||
|
||||
it('rejects an unsafe externally supplied matter ID before lookup', async () => {
|
||||
const { app } = await createTestApp()
|
||||
const headers = await authedHeaders(app)
|
||||
|
||||
const res = await createShare(app, headers, { matterId: 'matter/id', kind: 'landing' })
|
||||
|
||||
expect(res.status).toBe(400)
|
||||
})
|
||||
|
||||
it('accepts a historical matter reference but creates only new-format share identities', async () => {
|
||||
const { app, db } = await createTestApp()
|
||||
const headers = await authedHeaders(app)
|
||||
await insertStorage(db)
|
||||
const orgId = await getOrgId(db)
|
||||
const matterId = 'legacy_matter-id'
|
||||
await insertFile(db, orgId, { id: matterId, name: 'legacy.txt' })
|
||||
|
||||
const res = await createShare(app, headers, { matterId, kind: 'landing' })
|
||||
|
||||
expect(res.status).toBe(201)
|
||||
const body = (await res.json()) as { token: string }
|
||||
expect(body.token).toMatch(SHARE_TOKEN_PATTERN)
|
||||
const rows = await db
|
||||
.select({ id: shares.id, matterId: shares.matterId })
|
||||
.from(shares)
|
||||
.where(eq(shares.token, body.token))
|
||||
expect(rows).toEqual([{ id: expect.stringMatching(BASE62_PATTERN), matterId }])
|
||||
})
|
||||
|
||||
it('creates a landing share without password and returns 201 with correct shape [spec: shares/create-landing]', async () => {
|
||||
const { app, db } = await createTestApp()
|
||||
const headers = await authedHeaders(app)
|
||||
|
||||
@@ -47,22 +47,28 @@ describe('buildObjectKey', () => {
|
||||
expect(a).not.toBe(b)
|
||||
})
|
||||
|
||||
it('accepts legacy owner IDs while keeping the generated key segment Base62', () => {
|
||||
const result = buildObjectKey({ ...baseVars, orgId: 'legacy_org', uid: 'legacy-user' })
|
||||
expect(result).toMatch(/^legacy_org\/legacy-user\/\d{8}\/[A-Za-z0-9]{17}\.jpg$/)
|
||||
})
|
||||
|
||||
it.each([
|
||||
['organization ID', { ...baseVars, orgId: 'legacy_org' }],
|
||||
['user ID', { ...baseVars, uid: 'legacy-user' }],
|
||||
])('rejects a non-Base62 %s before creating a key', (component, vars) => {
|
||||
['organization ID', { ...baseVars, orgId: 'invalid/org' }],
|
||||
['user ID', { ...baseVars, uid: 'invalid:user' }],
|
||||
])('rejects an unsafe %s before creating a key', (component, vars) => {
|
||||
expect(() => buildObjectKey(vars)).toThrow(`Invalid ${component} for object storage key`)
|
||||
})
|
||||
})
|
||||
|
||||
describe('buildImageStorageKey', () => {
|
||||
it('uses only Base62 ID components under the image namespace', () => {
|
||||
it('uses compatible opaque ID components under the image namespace', () => {
|
||||
expect(buildImageStorageKey('org123', 'imageABC123', 'png')).toBe('ih/org123/imageABC123.png')
|
||||
expect(buildImageStorageKey('legacy_org', 'legacy-image', 'png')).toBe('ih/legacy_org/legacy-image.png')
|
||||
})
|
||||
|
||||
it.each([
|
||||
['organization ID', 'legacy_org', 'imageABC123', 'png'],
|
||||
['image ID', 'org123', 'legacy-image', 'png'],
|
||||
['organization ID', 'invalid/org', 'imageABC123', 'png'],
|
||||
['image ID', 'org123', 'invalid:image', 'png'],
|
||||
['extension', 'org123', 'imageABC123', 'png/other'],
|
||||
])('rejects an invalid %s component', (_component, orgId, imageId, extension) => {
|
||||
expect(() => buildImageStorageKey(orgId, imageId, extension)).toThrow(/Invalid .* for image storage key/)
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { BASE62_PATTERN, generateToken } from '../../shared/ids'
|
||||
import { generateToken, OPAQUE_ID_PATTERN } from '../../shared/ids'
|
||||
|
||||
export interface TemplateVars {
|
||||
uid: string
|
||||
@@ -6,13 +6,13 @@ export interface TemplateVars {
|
||||
rawExt: string
|
||||
}
|
||||
|
||||
function requireBase62KeyComponent(keyType: string, name: string, value: string): void {
|
||||
if (!BASE62_PATTERN.test(value)) throw new Error(`Invalid ${name} for ${keyType} storage key`)
|
||||
function requireOpaqueIdKeyComponent(keyType: string, name: string, value: string): void {
|
||||
if (!OPAQUE_ID_PATTERN.test(value)) throw new Error(`Invalid ${name} for ${keyType} storage key`)
|
||||
}
|
||||
|
||||
export function assertObjectKeyOwner(vars: Pick<TemplateVars, 'orgId' | 'uid'>): void {
|
||||
requireBase62KeyComponent('object', 'organization ID', vars.orgId)
|
||||
requireBase62KeyComponent('object', 'user ID', vars.uid)
|
||||
requireOpaqueIdKeyComponent('object', 'organization ID', vars.orgId)
|
||||
requireOpaqueIdKeyComponent('object', 'user ID', vars.uid)
|
||||
}
|
||||
|
||||
/** Returns the file extension including the leading dot, or '' when there is none. */
|
||||
@@ -33,8 +33,8 @@ export function buildObjectKey(vars: TemplateVars): string {
|
||||
}
|
||||
|
||||
export function buildImageStorageKey(orgId: string, imageId: string, extension: string): string {
|
||||
requireBase62KeyComponent('image', 'organization ID', orgId)
|
||||
requireBase62KeyComponent('image', 'image ID', imageId)
|
||||
requireOpaqueIdKeyComponent('image', 'organization ID', orgId)
|
||||
requireOpaqueIdKeyComponent('image', 'image ID', imageId)
|
||||
if (!/^[A-Za-z0-9]+$/.test(extension)) throw new Error('Invalid extension for image storage key')
|
||||
return `ih/${orgId}/${imageId}.${extension}`
|
||||
}
|
||||
|
||||
+54
-12
@@ -1,31 +1,73 @@
|
||||
import { AuthorizationScope } from '@shared/authorization'
|
||||
import { BASE62_PATTERN, IMAGE_TOKEN_PATTERN, SHARE_TOKEN_PATTERN } from '@shared/ids'
|
||||
import { COMPATIBLE_IMAGE_TOKEN_PATTERN, COMPATIBLE_SHARE_TOKEN_PATTERN, OPAQUE_ID_PATTERN } from '@shared/ids'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { authRoute, findOperationsMissingAuthContract } from './http/openapi'
|
||||
import { createTestApp } from './test/setup'
|
||||
|
||||
describe('global OpenAPI document', () => {
|
||||
it('publishes the Base62 contract for ZPan-owned local ID inputs', async () => {
|
||||
it('publishes compatibility contracts for stored IDs while preserving token namespaces', async () => {
|
||||
const { app } = await createTestApp({ DOWNLOAD_TOKEN_SECRET: 'test-download-token-secret' })
|
||||
const res = await app.request('/api/openapi.json')
|
||||
const doc = (await res.json()) as {
|
||||
paths: Record<string, Record<string, { parameters?: Array<{ name: string; schema?: { pattern?: string } }> }>>
|
||||
paths: Record<
|
||||
string,
|
||||
Record<
|
||||
string,
|
||||
{
|
||||
parameters?: Array<{ name: string; schema?: { pattern?: string } }>
|
||||
requestBody?: {
|
||||
content?: {
|
||||
'application/json'?: {
|
||||
schema?: {
|
||||
properties?: Record<string, { pattern?: string; items?: { pattern?: string } }>
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
>
|
||||
>
|
||||
components?: { schemas?: Record<string, { properties?: Record<string, { pattern?: string }> }> }
|
||||
}
|
||||
const patternFor = (path: string, method: string, name: string) =>
|
||||
doc.paths[path]?.[method]?.parameters?.find((parameter) => parameter.name === name)?.schema?.pattern
|
||||
const bodyPropertyFor = (path: string, method: string, name: string) =>
|
||||
doc.paths[path]?.[method]?.requestBody?.content?.['application/json']?.schema?.properties?.[name]
|
||||
|
||||
expect(patternFor('/api/objects/{id}', 'get', 'id')).toBe(BASE62_PATTERN.source)
|
||||
for (const path of [
|
||||
'/api/objects',
|
||||
'/api/shares',
|
||||
'/api/downloads/tasks',
|
||||
'/api/downloads/downloaders',
|
||||
'/api/site/storages',
|
||||
]) {
|
||||
expect(
|
||||
bodyPropertyFor(path, 'post', 'id'),
|
||||
`${path} must not accept a caller-selected primary ID`,
|
||||
).toBeUndefined()
|
||||
}
|
||||
|
||||
expect(patternFor('/api/objects/{id}', 'get', 'id')).toBe(OPAQUE_ID_PATTERN.source)
|
||||
expect(patternFor('/api/objects/{id}/uploads/{uploadSessionId}', 'delete', 'uploadSessionId')).toBe(
|
||||
BASE62_PATTERN.source,
|
||||
OPAQUE_ID_PATTERN.source,
|
||||
)
|
||||
expect(patternFor('/api/trash/objects/{id}', 'delete', 'id')).toBe(OPAQUE_ID_PATTERN.source)
|
||||
expect(patternFor('/api/oauth-grants/{grantId}', 'delete', 'grantId')).toBe(OPAQUE_ID_PATTERN.source)
|
||||
expect(patternFor('/api/site/audit-events', 'get', 'orgId')).toBe(OPAQUE_ID_PATTERN.source)
|
||||
expect(bodyPropertyFor('/api/objects', 'post', 'storageId')?.pattern).toBe(OPAQUE_ID_PATTERN.source)
|
||||
expect(bodyPropertyFor('/api/objects/{id}/transfers', 'post', 'targetOrgId')?.pattern).toBe(
|
||||
OPAQUE_ID_PATTERN.source,
|
||||
)
|
||||
expect(bodyPropertyFor('/api/shares', 'post', 'matterId')?.pattern).toBe(OPAQUE_ID_PATTERN.source)
|
||||
expect(bodyPropertyFor('/api/oauth-consent', 'post', 'workspaceIds')?.items?.pattern).toBe(OPAQUE_ID_PATTERN.source)
|
||||
expect(patternFor('/api/shares/{token}', 'get', 'token')).toBe(COMPATIBLE_SHARE_TOKEN_PATTERN.source)
|
||||
expect(patternFor('/api/shares/{token}/objects', 'get', 'token')).toBe(COMPATIBLE_SHARE_TOKEN_PATTERN.source)
|
||||
expect(doc.components?.schemas?.ImageHosting?.properties?.token?.pattern).toBe(
|
||||
COMPATIBLE_IMAGE_TOKEN_PATTERN.source,
|
||||
)
|
||||
expect(doc.components?.schemas?.ImageHostingDraft?.properties?.token?.pattern).toBe(
|
||||
COMPATIBLE_IMAGE_TOKEN_PATTERN.source,
|
||||
)
|
||||
expect(patternFor('/api/trash/objects/{id}', 'delete', 'id')).toBe(BASE62_PATTERN.source)
|
||||
expect(patternFor('/api/oauth-grants/{grantId}', 'delete', 'grantId')).toBe(BASE62_PATTERN.source)
|
||||
expect(patternFor('/api/site/audit-events', 'get', 'orgId')).toBe(BASE62_PATTERN.source)
|
||||
expect(patternFor('/api/shares/{token}', 'get', 'token')).toBe(SHARE_TOKEN_PATTERN.source)
|
||||
expect(patternFor('/api/shares/{token}/objects', 'get', 'token')).toBe(SHARE_TOKEN_PATTERN.source)
|
||||
expect(doc.components?.schemas?.ImageHosting?.properties?.token?.pattern).toBe(IMAGE_TOKEN_PATTERN.source)
|
||||
expect(doc.components?.schemas?.ImageHostingDraft?.properties?.token?.pattern).toBe(IMAGE_TOKEN_PATTERN.source)
|
||||
})
|
||||
|
||||
it('aggregates every OpenAPIHono route at /api/openapi.json', async () => {
|
||||
|
||||
@@ -84,10 +84,19 @@ function fixture(path = ':memory:'): Database.Database {
|
||||
CREATE TABLE notifications (
|
||||
id TEXT PRIMARY KEY, user_id TEXT NOT NULL, ref_type TEXT, ref_id TEXT, metadata TEXT
|
||||
);
|
||||
CREATE TABLE apikey (id TEXT PRIMARY KEY, reference_id TEXT, key TEXT);
|
||||
CREATE TABLE apikey (id TEXT PRIMARY KEY, reference_id TEXT, key TEXT, metadata TEXT);
|
||||
CREATE TABLE session (id TEXT PRIMARY KEY, user_id TEXT NOT NULL, token TEXT NOT NULL);
|
||||
CREATE TABLE verification (id TEXT PRIMARY KEY);
|
||||
CREATE TABLE deviceCode (id TEXT PRIMARY KEY);
|
||||
CREATE TABLE deviceCode (
|
||||
id TEXT PRIMARY KEY,
|
||||
device_code TEXT NOT NULL,
|
||||
user_code TEXT NOT NULL,
|
||||
user_id TEXT,
|
||||
client_id TEXT,
|
||||
scope TEXT,
|
||||
status TEXT NOT NULL,
|
||||
expires_at INTEGER NOT NULL
|
||||
);
|
||||
CREATE TABLE oauthAccessToken (id TEXT PRIMARY KEY);
|
||||
CREATE TABLE oauthRefreshToken (id TEXT PRIMARY KEY);
|
||||
CREATE TABLE oauthPushedAuthorizationRequest (id TEXT PRIMARY KEY);
|
||||
@@ -147,10 +156,16 @@ function fixture(path = ':memory:'): Database.Database {
|
||||
'notification_old-', 'user-_legacy', 'share', 'share_old-',
|
||||
'{"shareId":"share_old-","token":"ds_direct-old","customerId":"user-_legacy"}'
|
||||
);
|
||||
INSERT INTO apikey VALUES ('api_key_old-', 'user-_legacy', 'irreversible-hash');
|
||||
INSERT INTO apikey VALUES (
|
||||
'api_key_old-', 'user-_legacy', 'irreversible-hash',
|
||||
'{"scope":{"mode":"workspace","orgId":"-org_legacy-"}}'
|
||||
);
|
||||
INSERT INTO session VALUES ('session_old-', 'user-_legacy', 'session-token');
|
||||
INSERT INTO verification VALUES ('verification_old-');
|
||||
INSERT INTO deviceCode VALUES ('device_old-');
|
||||
INSERT INTO deviceCode VALUES (
|
||||
'device_old-', 'device-code-value', 'USER-CODE', 'user-_legacy', 'static-client',
|
||||
'openid offline_access', 'pending', 1999999999999
|
||||
);
|
||||
INSERT INTO oauthAccessToken VALUES ('access_old-');
|
||||
INSERT INTO oauthRefreshToken VALUES ('refresh_old-');
|
||||
INSERT INTO oauthPushedAuthorizationRequest VALUES ('par_old-');
|
||||
@@ -215,7 +230,7 @@ describe('ID normalization backfill', () => {
|
||||
db.close()
|
||||
})
|
||||
|
||||
it('atomically maps IDs and references, rotates public tokens, rewrites JSON, and invalidates credentials', () => {
|
||||
it('atomically maps IDs and references, preserves API keys and device codes, and invalidates other credentials', () => {
|
||||
const db = fixture()
|
||||
const summary = normalizeDatabase(db, true)
|
||||
|
||||
@@ -228,6 +243,8 @@ describe('ID normalization backfill', () => {
|
||||
const shareToken = value(db, 'SELECT token AS value FROM shares')
|
||||
const imageToken = value(db, 'SELECT token AS value FROM image_hostings')
|
||||
const taskId = value(db, 'SELECT id AS value FROM download_tasks')
|
||||
const apiKeyId = value(db, 'SELECT id AS value FROM apikey')
|
||||
const deviceCodeId = value(db, 'SELECT id AS value FROM deviceCode')
|
||||
const downloaderId = value(
|
||||
db,
|
||||
"SELECT substr(created_by, 12) AS value FROM object_upload_sessions WHERE created_by LIKE 'downloader:%'",
|
||||
@@ -246,6 +263,8 @@ describe('ID normalization backfill', () => {
|
||||
shareToken,
|
||||
imageToken,
|
||||
taskId,
|
||||
apiKeyId,
|
||||
deviceCodeId,
|
||||
downloaderId,
|
||||
]) {
|
||||
expect(governedValue).toMatch(BASE62_PATTERN)
|
||||
@@ -311,6 +330,24 @@ describe('ID normalization backfill', () => {
|
||||
expect(value(db, 'SELECT event_key AS value FROM audit_events')).toContain(matterId)
|
||||
expect(value(db, 'SELECT event_key AS value FROM storage_usage_ledger')).toBe(`opening:${orgId}:${storageId}`)
|
||||
expect(value(db, 'SELECT id AS value FROM audit_events')).toMatch(BASE62_PATTERN)
|
||||
expect(db.prepare('SELECT reference_id, key, metadata FROM apikey').get()).toEqual({
|
||||
reference_id: userId,
|
||||
key: 'irreversible-hash',
|
||||
metadata: JSON.stringify({ scope: { mode: 'workspace', orgId } }),
|
||||
})
|
||||
expect(
|
||||
db.prepare('SELECT device_code, user_code, user_id, client_id, scope, status, expires_at FROM deviceCode').get(),
|
||||
).toEqual({
|
||||
device_code: 'device-code-value',
|
||||
user_code: 'USER-CODE',
|
||||
user_id: userId,
|
||||
client_id: 'static-client',
|
||||
scope: 'openid offline_access',
|
||||
status: 'pending',
|
||||
expires_at: 1999999999999,
|
||||
})
|
||||
expect(summary.invalidated.apikey).toBeUndefined()
|
||||
expect(summary.invalidated.deviceCode).toBeUndefined()
|
||||
|
||||
// Physical object keys are external storage references and intentionally do not change.
|
||||
expect(value(db, 'SELECT object AS value FROM matters')).toBe('objects/-org_legacy-/matter_old-')
|
||||
@@ -321,9 +358,7 @@ describe('ID normalization backfill', () => {
|
||||
])
|
||||
for (const table of [
|
||||
'session',
|
||||
'apikey',
|
||||
'verification',
|
||||
'deviceCode',
|
||||
'downloader_bootstrap_credentials',
|
||||
'oauthAccessToken',
|
||||
'oauthRefreshToken',
|
||||
@@ -394,12 +429,13 @@ describe('ID normalization backfill', () => {
|
||||
d1Copy.close()
|
||||
})
|
||||
|
||||
it('resolves legacy API-key task creators before credentials are invalidated', () => {
|
||||
it('resolves legacy API-key task creators while preserving their normalized API keys', () => {
|
||||
const addLegacyCreators = (db: Database.Database) => {
|
||||
db.exec(`
|
||||
ALTER TABLE download_tasks ADD COLUMN created_by_user_id TEXT;
|
||||
UPDATE download_tasks SET created_by_user_id = 'api-key:api_key_old-';
|
||||
INSERT INTO apikey VALUES ('orphan_key-', 'deleted-user_', 'orphan-hash');
|
||||
INSERT INTO user VALUES ('legacy-api-user_');
|
||||
INSERT INTO apikey VALUES ('orphan_key-', 'legacy-api-user_', 'orphan-hash', NULL);
|
||||
INSERT INTO download_tasks (id, events, status, created_by_user_id)
|
||||
VALUES ('OrphanCreatorTask', '[]', 'completed', 'api-key:orphan_key-');
|
||||
`)
|
||||
@@ -414,6 +450,9 @@ describe('ID normalization backfill', () => {
|
||||
expect(
|
||||
value(planningCopy, "SELECT created_by_user_id AS value FROM download_tasks WHERE id = 'OrphanCreatorTask'"),
|
||||
).toMatch(BASE62_PATTERN)
|
||||
expect(value(planningCopy, "SELECT reference_id AS value FROM apikey WHERE key = 'orphan-hash'")).toBe(
|
||||
value(planningCopy, "SELECT created_by_user_id AS value FROM download_tasks WHERE id = 'OrphanCreatorTask'"),
|
||||
)
|
||||
|
||||
const plan = buildD1ApplySql(planningCopy)
|
||||
const d1Copy = fixture()
|
||||
@@ -539,7 +578,7 @@ describe('ID normalization backfill', () => {
|
||||
it.each([
|
||||
{
|
||||
name: 'a newly persisted invalid API-key row ID',
|
||||
mutate: (db: Database.Database) => db.exec("INSERT INTO apikey VALUES ('invalid-api-key', NULL, 'hash')"),
|
||||
mutate: (db: Database.Database) => db.exec("INSERT INTO apikey VALUES ('invalid-api-key', NULL, 'hash', NULL)"),
|
||||
error: 'invalid_value_remaining:apikey.id:1',
|
||||
},
|
||||
{
|
||||
@@ -581,6 +620,8 @@ describe('ID normalization backfill', () => {
|
||||
const expectedShareToken = value(planningCopy, 'SELECT token AS value FROM shares')
|
||||
const plan = buildD1ApplySql(planningCopy)
|
||||
expect(plan).not.toContain('WHERE rowid')
|
||||
expect(plan).not.toContain('DELETE FROM "apikey"')
|
||||
expect(plan).not.toContain('DELETE FROM "deviceCode"')
|
||||
expect(plan).toContain('substr("token", 1, 1) != \'s\'')
|
||||
expect(plan).toContain('substr("token", 1, 1) != \'i\'')
|
||||
|
||||
@@ -612,6 +653,18 @@ describe('ID normalization backfill', () => {
|
||||
expect(value(d1Copy, 'SELECT events AS value FROM download_tasks')).toBe(
|
||||
value(planningCopy, 'SELECT events AS value FROM download_tasks'),
|
||||
)
|
||||
expect(d1Copy.prepare('SELECT id, reference_id, key, metadata FROM apikey').get()).toEqual(
|
||||
planningCopy.prepare('SELECT id, reference_id, key, metadata FROM apikey').get(),
|
||||
)
|
||||
expect(
|
||||
d1Copy
|
||||
.prepare('SELECT id, device_code, user_code, user_id, client_id, scope, status, expires_at FROM deviceCode')
|
||||
.get(),
|
||||
).toEqual(
|
||||
planningCopy
|
||||
.prepare('SELECT id, device_code, user_code, user_id, client_id, scope, status, expires_at FROM deviceCode')
|
||||
.get(),
|
||||
)
|
||||
expect((d1Copy.prepare('SELECT COUNT(*) AS count FROM session').get() as { count: number }).count).toBe(0)
|
||||
expect(d1Copy.prepare("SELECT 1 FROM oauthClient WHERE client_id = 'dynamic-client'").get()).toBeUndefined()
|
||||
expect(d1Copy.prepare("SELECT 1 FROM oauthClient WHERE client_id = 'static-client'").get()).toBeTruthy()
|
||||
|
||||
@@ -523,10 +523,28 @@ describe('webdav usecase', () => {
|
||||
expect(deleteObject).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('overwrites an existing object for compatible legacy owner IDs', async () => {
|
||||
const existing = file('m1', { object: 'legacy_/existing-file-.txt', size: 20 })
|
||||
const deps = makeDeps({ s3: { putObject: async () => 5 } })
|
||||
|
||||
await expect(
|
||||
putWebDavFile(
|
||||
deps,
|
||||
putParams({
|
||||
target: target({ matter: existing }),
|
||||
contentLength: 5,
|
||||
orgId: 'legacy_org',
|
||||
userId: 'legacy-user',
|
||||
}),
|
||||
),
|
||||
).resolves.toMatchObject({ ok: true, status: 204 })
|
||||
expect(existing.object).toBe('legacy_/existing-file-.txt')
|
||||
})
|
||||
|
||||
it.each([
|
||||
['organization ID', { orgId: 'legacy_org' }],
|
||||
['user ID', { userId: 'legacy-user' }],
|
||||
])('rejects an invalid %s before overwriting an existing S3 object', async (component, overrides) => {
|
||||
['organization ID', { orgId: 'invalid/org' }],
|
||||
['user ID', { userId: 'invalid:user' }],
|
||||
])('rejects an unsafe %s before overwriting an existing S3 object', async (component, overrides) => {
|
||||
const putObject = vi.fn(async () => 5)
|
||||
const existing = file('m1', { object: 'legacy_/existing-file-.txt', size: 20 })
|
||||
const deps = makeDeps({ s3: { putObject } })
|
||||
|
||||
@@ -12,6 +12,7 @@ import {
|
||||
PUBLIC_TOKEN_RANDOM_LENGTH,
|
||||
SHARE_TOKEN_PATTERN,
|
||||
} from './ids'
|
||||
import { base62IdSchema } from './schemas/id'
|
||||
|
||||
describe('Base62 identifiers', () => {
|
||||
it('uses the fixed 0-9A-Za-z alphabet', () => {
|
||||
@@ -23,6 +24,7 @@ describe('Base62 identifiers', () => {
|
||||
|
||||
expect(DEFAULT_ID_LENGTH * Math.log2(BASE62_ALPHABET.length)).toBeGreaterThanOrEqual(21 * Math.log2(64))
|
||||
expect(ids.every((id) => id.length === DEFAULT_ID_LENGTH && BASE62_PATTERN.test(id))).toBe(true)
|
||||
expect(ids.every((id) => base62IdSchema.safeParse(id).success)).toBe(true)
|
||||
expect(new Set(ids).size).toBe(ids.length)
|
||||
})
|
||||
|
||||
|
||||
@@ -2,6 +2,8 @@ import { customAlphabet } from 'nanoid'
|
||||
|
||||
export const BASE62_ALPHABET = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'
|
||||
export const BASE62_PATTERN = /^[A-Za-z0-9]+$/
|
||||
export const OPAQUE_ID_PATTERN = /^[A-Za-z0-9_-]+$/
|
||||
export const OPAQUE_TOKEN_PATTERN = /^[A-Za-z0-9_-]+$/
|
||||
export const DEFAULT_ID_LENGTH = 22
|
||||
export const PUBLIC_TOKEN_RANDOM_LENGTH = 11
|
||||
export const PUBLIC_TOKEN_LENGTH = 1 + PUBLIC_TOKEN_RANDOM_LENGTH
|
||||
@@ -9,6 +11,10 @@ export const SHARE_TOKEN_PREFIX = 's'
|
||||
export const IMAGE_TOKEN_PREFIX = 'i'
|
||||
export const SHARE_TOKEN_PATTERN = /^s[A-Za-z0-9]{11}$/
|
||||
export const IMAGE_TOKEN_PATTERN = /^i[A-Za-z0-9]{11}$/
|
||||
export const LEGACY_DIRECT_SHARE_TOKEN_PATTERN = /^ds_[A-Za-z0-9_-]+$/
|
||||
export const LEGACY_IMAGE_TOKEN_PATTERN = /^ih[A-Za-z0-9_-]+$/
|
||||
export const COMPATIBLE_SHARE_TOKEN_PATTERN = /^(?:s[A-Za-z0-9]{11}|ds_[A-Za-z0-9_-]+|[A-Za-z0-9_-]{10})$/
|
||||
export const COMPATIBLE_IMAGE_TOKEN_PATTERN = /^(?:i[A-Za-z0-9]{11}|ih[A-Za-z0-9_-]+)$/
|
||||
|
||||
const randomBase62 = customAlphabet(BASE62_ALPHABET)
|
||||
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { z } from 'zod'
|
||||
import { opaqueIdSchema } from './id'
|
||||
|
||||
export const backgroundJobStatusSchema = z.enum(['queued', 'running', 'completed', 'failed', 'canceled'])
|
||||
export type BackgroundJobStatusInput = z.infer<typeof backgroundJobStatusSchema>
|
||||
@@ -6,7 +7,7 @@ export type BackgroundJobStatusInput = z.infer<typeof backgroundJobStatusSchema>
|
||||
export const backgroundJobTypeSchema = z.string().min(1).max(80)
|
||||
export type BackgroundJobTypeInput = z.infer<typeof backgroundJobTypeSchema>
|
||||
|
||||
const matterIdSchema = z.string().min(1)
|
||||
const matterIdSchema = opaqueIdSchema
|
||||
|
||||
export const archiveCompressJobRequestSchema = z.object({
|
||||
type: z.literal('archive_compress'),
|
||||
|
||||
@@ -1,17 +1,29 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { imageTokenSchema, opaqueIdSchema, opaqueTokenSchema, shareTokenSchema } from './id'
|
||||
import { base62IdSchema, imageTokenSchema, opaqueIdSchema, opaqueTokenSchema, shareTokenSchema } from './id'
|
||||
|
||||
describe('opaque ID schemas', () => {
|
||||
it.each(['0', 'abcXYZ123', 'A'.repeat(64)])('accepts Base62 value %s', (value) => {
|
||||
it.each([
|
||||
'0',
|
||||
'abcXYZ123',
|
||||
'A'.repeat(64),
|
||||
'legacy_id',
|
||||
'legacy-id',
|
||||
])('accepts current and legacy value %s', (value) => {
|
||||
expect(opaqueIdSchema.parse(value)).toBe(value)
|
||||
expect(opaqueTokenSchema.parse(value)).toBe(value)
|
||||
})
|
||||
|
||||
it.each(['', 'legacy_id', 'legacy-id', 'event:value', 'urn:value', 'é'])('rejects non-Base62 value %s', (value) => {
|
||||
it.each(['', 'event:value', 'urn:value', 'path/value', 'value.with-dot', 'é'])('rejects unsafe value %s', (value) => {
|
||||
expect(opaqueIdSchema.safeParse(value).success).toBe(false)
|
||||
expect(opaqueTokenSchema.safeParse(value).success).toBe(false)
|
||||
})
|
||||
|
||||
it('keeps newly created IDs on the strict Base62 contract', () => {
|
||||
expect(base62IdSchema.parse('abcXYZ123')).toBe('abcXYZ123')
|
||||
expect(base62IdSchema.safeParse('legacy_id').success).toBe(false)
|
||||
expect(base62IdSchema.safeParse('legacy-id').success).toBe(false)
|
||||
})
|
||||
|
||||
it('keeps share and image public token namespaces disjoint', () => {
|
||||
expect(shareTokenSchema.parse('s0123456789A')).toBe('s0123456789A')
|
||||
expect(imageTokenSchema.parse('i0123456789A')).toBe('i0123456789A')
|
||||
@@ -20,4 +32,11 @@ describe('opaque ID schemas', () => {
|
||||
expect(shareTokenSchema.safeParse('s0123456789').success).toBe(false)
|
||||
expect(imageTokenSchema.safeParse('i0123456789AB').success).toBe(false)
|
||||
})
|
||||
|
||||
it('accepts legacy public tokens without weakening their resource namespace', () => {
|
||||
expect(shareTokenSchema.parse('ds_legacy-token')).toBe('ds_legacy-token')
|
||||
expect(shareTokenSchema.parse('legacy_id0')).toBe('legacy_id0')
|
||||
expect(imageTokenSchema.parse('ih_legacy-token')).toBe('ih_legacy-token')
|
||||
expect(imageTokenSchema.safeParse('ds_legacy-token').success).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
+25
-5
@@ -1,7 +1,27 @@
|
||||
import { z } from 'zod'
|
||||
import { BASE62_PATTERN, IMAGE_TOKEN_PATTERN, SHARE_TOKEN_PATTERN } from '../ids'
|
||||
import {
|
||||
BASE62_PATTERN,
|
||||
COMPATIBLE_IMAGE_TOKEN_PATTERN,
|
||||
COMPATIBLE_SHARE_TOKEN_PATTERN,
|
||||
OPAQUE_ID_PATTERN,
|
||||
OPAQUE_TOKEN_PATTERN,
|
||||
} from '../ids'
|
||||
|
||||
export const opaqueIdSchema = z.string().min(1).regex(BASE62_PATTERN, 'Must contain only ASCII letters and digits')
|
||||
export const opaqueTokenSchema = z.string().min(1).regex(BASE62_PATTERN, 'Must contain only ASCII letters and digits')
|
||||
export const shareTokenSchema = z.string().regex(SHARE_TOKEN_PATTERN, 'Must be s followed by 11 Base62 characters')
|
||||
export const imageTokenSchema = z.string().regex(IMAGE_TOKEN_PATTERN, 'Must be i followed by 11 Base62 characters')
|
||||
/** Strict contract for IDs assigned to newly created ZPan entities. */
|
||||
export const base62IdSchema = z.string().min(1).regex(BASE62_PATTERN, 'Must contain only ASCII letters and digits')
|
||||
|
||||
/** Compatibility contract for locating or referencing an already-persisted entity. */
|
||||
export const opaqueIdSchema = z
|
||||
.string()
|
||||
.min(1)
|
||||
.regex(OPAQUE_ID_PATTERN, 'Must contain only ASCII letters, digits, underscores, or hyphens')
|
||||
export const opaqueTokenSchema = z
|
||||
.string()
|
||||
.min(1)
|
||||
.regex(OPAQUE_TOKEN_PATTERN, 'Must contain only ASCII letters, digits, underscores, or hyphens')
|
||||
export const shareTokenSchema = z
|
||||
.string()
|
||||
.regex(COMPATIBLE_SHARE_TOKEN_PATTERN, 'Must be a current or historical share token')
|
||||
export const imageTokenSchema = z
|
||||
.string()
|
||||
.regex(COMPATIBLE_IMAGE_TOKEN_PATTERN, 'Must be a current or historical image token')
|
||||
|
||||
@@ -148,7 +148,7 @@ export {
|
||||
errorInfoSchema,
|
||||
errorResponseSchema,
|
||||
} from './errors'
|
||||
export { imageTokenSchema, opaqueIdSchema, opaqueTokenSchema, shareTokenSchema } from './id'
|
||||
export { base62IdSchema, imageTokenSchema, opaqueIdSchema, opaqueTokenSchema, shareTokenSchema } from './id'
|
||||
export type { ListNotificationsQuery } from './notification'
|
||||
export { listNotificationsQuerySchema } from './notification'
|
||||
export type { WorkspaceAuthorizationDetail } from './oauth-authorization'
|
||||
@@ -292,9 +292,7 @@ export const createMatterSchema = z.object({
|
||||
parent: matterParentPathSchema.default(''),
|
||||
dirtype: z.number().int().default(0),
|
||||
onConflict: conflictStrategySchema.optional(),
|
||||
storageId: z
|
||||
.string()
|
||||
.min(1)
|
||||
storageId: opaqueIdSchema
|
||||
.describe(
|
||||
'Only site administrators may set this field; omit it to let ZPan automatically select an available storage.',
|
||||
)
|
||||
@@ -410,7 +408,7 @@ export const copyObjectBodySchema = z.object({
|
||||
export type CopyObjectBodyInput = z.infer<typeof copyObjectBodySchema>
|
||||
|
||||
export const transferMatterSchema = z.object({
|
||||
targetOrgId: z.string().min(1),
|
||||
targetOrgId: opaqueIdSchema,
|
||||
targetParent: z.string().default(''),
|
||||
mode: z.enum(['copy', 'move']),
|
||||
})
|
||||
|
||||
@@ -53,7 +53,7 @@ export type OAuthConsentContextRequest = z.infer<typeof oauthConsentContextReque
|
||||
export const oauthConsentSubmitSchema = z.object({
|
||||
accept: z.boolean(),
|
||||
oauthQuery: z.string().min(1),
|
||||
workspaceIds: z.array(z.string().min(1)),
|
||||
workspaceIds: z.array(opaqueIdSchema),
|
||||
})
|
||||
export type OAuthConsentSubmit = z.infer<typeof oauthConsentSubmitSchema>
|
||||
|
||||
|
||||
@@ -1,15 +1,42 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import {
|
||||
archiveCompressJobRequestSchema,
|
||||
copyMatterSchema,
|
||||
createDownloadTaskSchema,
|
||||
createMatterSchema,
|
||||
createStorageSchema,
|
||||
oauthConsentSubmitSchema,
|
||||
signInSchema,
|
||||
signUpSchema,
|
||||
transferMatterSchema,
|
||||
updateImageDomainSettingsSchema,
|
||||
updateMatterSchema,
|
||||
} from './index.js'
|
||||
|
||||
describe('external persisted-ID references', () => {
|
||||
it.each([
|
||||
['object storage', createMatterSchema, { name: 'file.txt', storageId: 'legacy_storage-id' }],
|
||||
['object transfer target', transferMatterSchema, { targetOrgId: 'legacy_org-id', targetParent: '', mode: 'copy' }],
|
||||
['archive matters', archiveCompressJobRequestSchema, { type: 'archive_compress', matterIds: ['legacy_matter-id'] }],
|
||||
[
|
||||
'OAuth workspaces',
|
||||
oauthConsentSubmitSchema,
|
||||
{ accept: true, oauthQuery: 'request', workspaceIds: ['legacy_org-id'] },
|
||||
],
|
||||
] as const)('accepts a historical %s that references an existing record', (_name, schema, input) => {
|
||||
expect(schema.safeParse(input).success).toBe(true)
|
||||
})
|
||||
|
||||
it.each([
|
||||
['object storage', createMatterSchema, { name: 'file.txt', storageId: 'storage/id' }],
|
||||
['object transfer target', transferMatterSchema, { targetOrgId: 'org:id', targetParent: '', mode: 'copy' }],
|
||||
['archive matters', archiveCompressJobRequestSchema, { type: 'archive_compress', matterIds: ['matter/id'] }],
|
||||
['OAuth workspaces', oauthConsentSubmitSchema, { accept: true, oauthQuery: 'request', workspaceIds: ['org.id'] }],
|
||||
] as const)('rejects an unsafe %s at the request-schema boundary', (_name, schema, input) => {
|
||||
expect(schema.safeParse(input).success).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
describe('signInSchema', () => {
|
||||
it('accepts valid input', () => {
|
||||
const result = signInSchema.safeParse({ email: 'a@b.com', password: '123456' })
|
||||
|
||||
+14
-2
@@ -1,6 +1,6 @@
|
||||
Feature: Public redirects
|
||||
Short links validate a 12-character namespaced Base62 token before lookup.
|
||||
Tokens starting with s select shares and tokens starting with i select images.
|
||||
New short links use a 12-character namespaced Base62 token. Historical ds_ and ih
|
||||
links remain readable when an administrator chooses not to normalize stored data.
|
||||
Direct shares stream files, while image-hosting links serve images.
|
||||
Both meter traffic quota (refunding on failure), and images enforce the referer allowlist.
|
||||
|
||||
@@ -10,6 +10,12 @@ Feature: Public redirects
|
||||
When the link is followed
|
||||
Then it 302-redirects with attachment disposition and no-store cache
|
||||
|
||||
@redirect/legacy-direct-share @api
|
||||
Scenario: A historical direct-share link remains usable
|
||||
Given a stored ds_ direct-share token
|
||||
When the link is followed without running ID normalization
|
||||
Then it redirects to the same file
|
||||
|
||||
@redirect/unknown-ds-token @api
|
||||
Scenario: An unknown direct-share token is not found
|
||||
Given an unknown s-prefixed direct-share token
|
||||
@@ -46,6 +52,12 @@ Feature: Public redirects
|
||||
When the link is followed
|
||||
Then it 302-redirects with inline disposition and no-store cache
|
||||
|
||||
@redirect/legacy-image @api
|
||||
Scenario: A historical image link remains usable
|
||||
Given a stored ih image token
|
||||
When the link is followed without running ID normalization
|
||||
Then it redirects to the same image
|
||||
|
||||
@redirect/image-strip-ext @api
|
||||
Scenario: Image links resolve regardless of extension
|
||||
Given an image link with a file extension
|
||||
|
||||
@@ -38,21 +38,10 @@ describe('[CF] image-domain Worker fast path', () => {
|
||||
|
||||
it('serves repeated custom-domain requests without initializing Better Auth', async () => {
|
||||
const suffix = Date.now().toString(36)
|
||||
const orgId = `fastpathorg${suffix}`
|
||||
const orgId = `fast_path-org-${suffix}`
|
||||
const domain = `img-${suffix}.fast-path.test`
|
||||
const now = Date.now()
|
||||
|
||||
await env.DB.batch([
|
||||
env.DB.prepare(
|
||||
'CREATE TABLE IF NOT EXISTS _zpan_id_normalization_state (key TEXT PRIMARY KEY, value TEXT NOT NULL)',
|
||||
),
|
||||
env.DB.prepare(
|
||||
"INSERT OR REPLACE INTO _zpan_id_normalization_state (key, value) VALUES ('validation_version', '2')",
|
||||
),
|
||||
env.DB.prepare(
|
||||
"INSERT OR REPLACE INTO _zpan_id_normalization_state (key, value) VALUES ('completed_at', 'test-fixture')",
|
||||
),
|
||||
])
|
||||
await env.DB.prepare(
|
||||
`INSERT INTO organization (id, name, slug, created_at, updated_at)
|
||||
VALUES (?, 'Fast Path', ?, ?, ?)`,
|
||||
|
||||
@@ -7,7 +7,6 @@ import { createApp } from '../server/app'
|
||||
import type { Auth } from '../server/auth'
|
||||
import { createAuth } from '../server/auth'
|
||||
import { createDeps } from '../server/composition'
|
||||
import { assertNormalizedIdentifiers } from '../server/db/id-normalization'
|
||||
import { isPotentialWebDavPublicRequest } from '../server/domain/webdav-public-url'
|
||||
import { isHandledError, standaloneJsonError } from '../server/middleware/error-handler'
|
||||
import { handleImageHostingDomainRequest } from '../server/middleware/image-hosting-domain'
|
||||
@@ -46,7 +45,6 @@ interface WorkerRuntime {
|
||||
authBySlot: Map<AuthSlot, Auth>
|
||||
appBySlot: Map<AuthSlot, ReturnType<typeof createApp>>
|
||||
appInitBySlot: Map<AuthSlot, Promise<ReturnType<typeof createApp>>>
|
||||
idNormalizationCheck: Promise<void>
|
||||
}
|
||||
|
||||
let cachedRuntime: WorkerRuntime | undefined
|
||||
@@ -75,7 +73,6 @@ function runtimeFor(env: Env): WorkerRuntime {
|
||||
authBySlot: new Map(),
|
||||
appBySlot: new Map(),
|
||||
appInitBySlot: new Map(),
|
||||
idNormalizationCheck: assertNormalizedIdentifiers(platform.db),
|
||||
}
|
||||
return cachedRuntime
|
||||
}
|
||||
@@ -86,7 +83,6 @@ async function appForRequest(
|
||||
env: Env,
|
||||
): Promise<ReturnType<typeof createApp>> {
|
||||
const origin = new URL(request.url).origin
|
||||
await runtime.idNormalizationCheck
|
||||
const webDavRequest = isPotentialWebDavPublicRequest(request.url)
|
||||
const inferredOrigin = origin
|
||||
const baseURL = env.BETTER_AUTH_URL || inferredOrigin
|
||||
@@ -125,7 +121,6 @@ export default {
|
||||
throw new Error('BETTER_AUTH_SECRET is not configured for this deployment.')
|
||||
}
|
||||
const runtime = runtimeFor(env)
|
||||
await runtime.idNormalizationCheck
|
||||
const imageDomainResponse = await handleImageDomainBeforeAuth(request, env, runtime)
|
||||
if (imageDomainResponse) return imageDomainResponse
|
||||
const edgeCached = await matchConfigzResponseCache(request, runtime.cache)
|
||||
@@ -147,13 +142,11 @@ export default {
|
||||
},
|
||||
|
||||
async scheduled(event: ScheduledEvent, env: Env): Promise<void> {
|
||||
await runtimeFor(env).idNormalizationCheck
|
||||
await handleScheduled(event, env)
|
||||
},
|
||||
|
||||
async queue(batch: MessageBatch<ArchiveJobMessage>, env: Env): Promise<void> {
|
||||
const runtime = runtimeFor(env)
|
||||
await runtime.idNormalizationCheck
|
||||
const archiveJobs = createArchiveJobsGateway(runtime.platform)
|
||||
for (const message of batch.messages) {
|
||||
await archiveJobs.runMessage(message.body)
|
||||
|
||||
Reference in New Issue
Block a user