Files
zpan/scripts/run-cloud-e2e.mjs
T
Jasper Van 3402a1e099 refactor(api): RESTful resource-oriented API — drop /admin, status sub-resources, merge audience-split routers (#437)
* refactor(api): RESTful resource-oriented API — drop /admin, status sub-resources, merge audience-split routers

Reorganize the entire HTTP surface around resource abstraction instead of
business/audience abstraction.

- Auth: authMiddleware is now soft + global for /api/*; gating is per-route
  (requireAuth/requireAdmin/requireTeamRole), so one resource path serves
  public, user, and admin callers (no security change — guards moved, not dropped).
- Drop /admin from URLs; merge audience-split routers into one resource each
  (announcements, auth-providers, users, teams, quotas, invite-codes,
  site-invitations, downloaders, branding, audit).
- State transitions -> PUT /:id/status: objects (confirm/trash/restore),
  download-tasks (pause/resume/cancel), background-jobs, image-hosting confirm.
- Verbs -> noun sub-resources: objects/:id/copies, download-tasks/:id/attempts,
  background-jobs/:id/retries, site-invitations/:id/deliveries,
  licensing/pairings + /pairings/:code + refresh-runs, teams/:id/invite-links.
- Config -> /api/site/* (branding, email, options, instance, changelog);
  ihost -> image-hosting; me + profiles + admin/users -> one /api/users
  (the :username slot also resolves the internal id, so the admin UI is unchanged).
- External downloader OpenAPI contract cut over in lockstep.

Frontend (rpc.ts + api.ts) and all integration/CF/unit tests updated to match.
Typecheck (server + src), lint:http, biome, and all 4394 tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(downloader): regenerate Go client + sync spec for the new RESTful contract

The Go downloader agent (cmd/) and the BDD spec live in this repo, so they must
move with the API:

- Regenerate docs/openapi/downloader.json and cmd/internal/openapi/client.gen.go
  from the updated server OpenAPI.
- Update the hand-written Go client: heartbeat -> /downloaders/me/heartbeats,
  register -> /downloaders, object confirm -> PUT /objects/:id/status, upload
  complete -> PUT .../status, abort -> DELETE .../uploads/:sid. Drop the now-dead
  union helpers (jsonBody/decodeJSON) and the bytes import.
- spec: drop the obsolete teams invite-token-missing scenario (the route is now
  a path param) and add the auth-providers anon-public-list scenario (the merged
  GET serves the public list to anonymous callers).

gofmt clean, go test (121) pass, lint:spec passes (418 scenarios covered).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(api): cover users admin detail/entitlements + getUser wrapper

Close the patch-coverage gaps from the users-resource merge: add integration
tests for GET /api/users/:id (admin detail, success + 404) and
GET /api/users/:id/entitlements (success + 404), and a unit test for the
getUser() api.ts wrapper.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(e2e): update Playwright specs + global setup to the new RESTful paths

The e2e specs make direct API calls / response matchers that bypass the SPA, so
they need the new paths too: global-setup storage+options seeding
(/api/storages, /api/site/options), image-host (/api/image-hosting, confirm via
PUT /images/:id/status), object confirm in archive (PUT /objects/:id/status),
announcements and site-invitations (/api/announcements, /api/site-invitations,
/api/site/email). The cloud pairing action:'approve' is the external cloud API,
left as-is.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(e2e): fix cloud-store instance pairing path to /api/licensing/pairings

The cloud-store spec calls the INSTANCE pairing endpoint directly:
POST /api/licensing/pair -> /api/licensing/pairings and the poll
GET /api/licensing/pair/:code/poll -> GET /api/licensing/pairings/:code.
/api/licensing/status and /binding are unchanged; /api/pairings is the
external cloud API, left as-is.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(api): rename /api/site-invitations to /api/invitations

Avoids visual proximity with the /api/site/* config namespace. Top-level
/api/invitations is unambiguous — team invitations are nested under
/api/teams/:id/invitations and invite codes under /api/invite-codes. URL-only
change; the internal site-invitations naming stays (still the accurate concept).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(api): group resources by functional domain (URLs)

Move non-core resources under functional-domain prefixes (not permission):
- /api/site/* absorbs storages, auth-providers, audit-events, licensing,
  invitations, invite-codes (joining branding, email, options, instance, changelog)
- /api/downloads/* = tasks + downloaders (regenerated OpenAPI + Go client)
Core resources stay top-level. Updates app.ts, rpc.ts, OpenAPI doc + Go agent
client, and all integration/CF/e2e tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(server): mirror functional-domain grouping in http/ and usecases/ dirs

Reorganize source files to match the functional URL domains established for
the routes, so the directory tree reflects the same grouping as the API:

- http/{site,downloads,image-hosting}/ and usecases/{site,downloads,image-hosting}/
- dissolve the permission-based console/ dir — admin resources are grouped by
  domain (site), not by audience
- console/user -> top-level (users is a core resource, not an admin-only one)

Co-located tests move with their sources; relative imports and vi.mock paths
updated for the new depths. Pure file/directory restructure, no behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(server): finish structural cleanup — merge split admin routers, drop rename leftovers

Three follow-ups from the directory-structure review, completing the
one-file-per-resource and domain-named-file conventions:

- Merge the last two audience-split router files into their resource file as a
  second export (matching branding/quotas/invite-codes/site-invitations):
  teams-admin.ts -> teams.ts (adminTeams), licensing-admin.ts -> licensing.ts
  (licensing + licensingAdmin).
- Drop pre-rename filename leftovers now that the dirs carry the domain:
  http/image-hosting/{ihost,ihost-config} -> {images,config};
  http/site/site-invitations -> invitations;
  usecases/site/{site-invitation,site-public-origin} -> {invitation,public-origin};
  usecases/image-hosting/{image-hosting,image-hosting-config} -> {images,config}.
- Group the loose store helpers under the store domain:
  http/{cloud-store-helpers,traffic-metering-utils} -> http/cloud-store/{helpers,traffic-metering}.

Routes and exports unchanged; pure file/structure move. tests + co-located
specs move with their sources. No behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(api): move announcements under /api/site, co-locate stray tests

Announcements is instance-level, admin-authored content (like branding) — a
site resource, not a top-level one. Move it under the site domain:
- /api/announcements -> /api/site/announcements (mount, RPC base path, api.test, e2e spec)
- http/announcements -> http/site/announcements; usecases/announcement -> usecases/site/announcement

Co-locate the tests that drifted from their sources during the dir reorg
(the 1:1-paired cf-test/unit tests belong next to what they exercise):
- http/storages.cf-test.ts -> http/site/ (next to storages.ts)
- usecases/{license-certificate,license-policy,license-refresh,licensing-admin}.test
  -> usecases/site/ (next to the licensing usecase; imports simplified to ./licensing)

No behavior change beyond the announcements path. Routes/exports otherwise stable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(usecases): de-fragment the users and objects domains at the usecase layer

The HTTP layer already serves these as single resources; consolidate their
usecases to match, removing leftover files that mirrored the old split:

- Fold me.ts (avatar) + profile.ts (public lookup) into user.ts — one user
  usecase with self/public/admin sections; drop the stale /api/me/avatar and
  /api/profiles/:username doc comments. Their unit tests move into user.test.ts.
- Fold matter.ts (confirmUpload, draft→active) into object.ts — the objects
  domain is now under one "object" name (the Matter *type* stays in ports/).

Importers updated; no behavior change. server tsc + lint:http + lint:spec clean;
Node 4337 / CF 57 / libsql 6 green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(usecases): fold sub-concern usecases into their resource (one file per resource)

Consolidate the usecase layer so each resource is a single source file:

- object.ts absorbs object-upload-session, purge, and save-to-drive (its
  upload-session / recursive-purge / save-to-drive sub-concerns)
- share.ts absorbs share-notification and share-ref

External importers re-pointed (trash, redirect, entry-node, workers/scheduled,
http/share-utils, and the surviving integration/cf tests). share.ts now pulls
copyMatterToOrg/saveShareToDrive from object. share.test.ts asserts the real
notification+email fan-out now that dispatchShareCreated is intra-module.

Shared domain services (storage-usage, cloud-traffic-metering, captcha) stay
separate — they're used by many resources. 5 files removed; no behavior change.
Node 4337 / CF 57 / libsql 6 green; tsc + lint:http + lint:spec clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(http): collapse concern-split integration tests into one per resource

Each resource now has a single Node integration test file; the scenario-split
files fold into their resource's main:

- objects-quota + object-multipart-live -> objects.integration.test.ts
- me + profile -> users.integration.test.ts
- quotas-listing -> quotas.integration.test.ts
- teams-admin -> teams.integration.test.ts
- share-public -> shares.integration.test.ts (share-public.cf-test stays — CF runtime)

Helpers de-duplicated or scoped per describe; all [spec:] breadcrumbs preserved
(lint:spec still 418). 7 files removed, all 4337 tests retained. The multipart-live
block now restoreAllMocks so it exercises the real S3 gateway (latent bug fixed).
Node 4337 / CF 57 / libsql 6 green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: finish test-file reorg + convert cloud licensing to a real Playwright e2e

Directory grouping (finishing the reorg): auth tests -> http/auth/, cloud-store
test -> cloud-store/, captcha + signup-mode -> usecases/site/ (with import-depth
fixes the moves needed).

One file per resource at the test layer:
- save-to-drive.integration + purge.integration -> object.integration.test.ts
- save-to-drive.cf-test -> object.cf-test.ts
- share-notification.integration -> share.integration.test.ts
- webdav.e2e (a vitest integration test, not Playwright) -> merged into
  webdav.integration.test.ts

Cloud licensing e2e: e2e-cloud-integration.test.ts was a vitest file mostly
duplicating existing integration coverage (licensing-admin.integration +
licensing-cloud.test) and the pairing e2e already in cloud-store.spec.ts.
Replaced with a real Playwright e2e (e2e/licensing.spec.ts): pair+approve ->
assert a Pro gate opens -> unbind -> assert it closes. Shared pairing helpers
extracted to e2e/helpers.ts (cloud-store.spec now imports them). run-cloud-e2e
runs both cloud specs in one tunnel; CI grep-invert excludes the new title from
the no-cloud run.

tsc + lint:http + lint:spec clean; Node 4337 / CF 57 / libsql 6 green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(server): move the cloud-store domain under store/ (matches /api/store)

Following the dir move: http/cloud-store/* -> http/store/*, the cloud-store +
cloud-traffic-metering usecases -> usecases/store/, and the top-level
cloud-traffic-metering http integration test -> http/store/. The http/cloud-store.ts
barrel now re-exports from ./store/*. All importers + moved-file imports rewired.

tsc + lint:http + lint:spec clean; Node 4337 / CF 57 / libsql 6 green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(server): drop the cloud- prefix under store/ now that the dir carries it

- usecases/store/cloud-store -> store.ts; cloud-traffic-metering -> traffic-metering.ts
- http/store/cloud-store.integration -> store.integration; cloud-traffic-metering
  .integration -> traffic-metering.integration
- the http barrel http/cloud-store.ts -> http/store/index.ts (re-exports from
  ./storefront + ./webhooks); app.ts imports './http/store'

store/ is now uniformly named (storefront/webhooks/helpers/shared/traffic-metering
+ store + index). tsc + lint:http + lint:spec clean; Node 4337 / CF 57 / libsql 6.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(e2e): licensing spec asserts the bind/unbind lifecycle, not a pro-only gate

The cloud E2E account is business-tier; its pairing certificate does not grant
open_registration (that's why the old vitest test seeded a local pro cert for
that assertion). Assert the edition-agnostic licensing lifecycle instead:
pairAndApprove (binds + waits active) -> unbind -> /status reports bound:false.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 14:51:01 -04:00

217 lines
7.3 KiB
JavaScript

import { existsSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
import { spawn } from 'node:child_process'
import { Resolver } from 'node:dns/promises'
import { createRequire } from 'node:module'
const args = process.argv.slice(2)
const require = createRequire(import.meta.url)
const runtime = valueAfter('--runtime') ?? process.env.E2E_RUNTIME ?? 'node'
const project = valueAfter('--project') ?? 'desktop'
const specs = (valueAfter('--spec') ?? 'cloud-store.spec.ts licensing.spec.ts').split(/\s+/).filter(Boolean)
const local = args.includes('--local')
const withS3Mock = args.includes('--with-s3-mock')
const cloudflared = process.env.CLOUDFLARED_BIN ?? 'cloudflared'
const appPort = Number(process.env.E2E_APP_PORT ?? 5185)
const apiPort = Number(process.env.E2E_API_PORT ?? 9222)
const s3MockPort = Number(process.env.E2E_S3_MOCK_PORT ?? 9191)
const localBaseUrl = `http://localhost:${appPort}`
const pidFile = `.cloudflared.${runtime}.pid`
const tunnelUrlPattern = /https:\/\/[a-zA-Z0-9-]+\.trycloudflare\.com/
const publicDns = new Resolver()
publicDns.setServers(['1.1.1.1', '1.0.0.1'])
const cloudEnv = {
ZPAN_CLOUD_URL: process.env.ZPAN_CLOUD_URL ?? 'http://localhost:5186',
VITE_ZPAN_CLOUD_URL: process.env.VITE_ZPAN_CLOUD_URL ?? 'http://localhost:5186',
}
const credentialsEnv = runtimeCloudCredentials(runtime)
const tunnel = local ? null : await startTunnel(localBaseUrl)
const tunnelHost = tunnel ? new URL(tunnel.url).hostname : ''
const tunnelIp = tunnel ? await waitForPublicTunnelIp(tunnelHost) : ''
const baseUrl = tunnel?.url ?? localBaseUrl
const tunnelEnv = {
E2E_BASE_URL: baseUrl,
E2E_LOCAL_BASE_URL: localBaseUrl,
E2E_APP_PORT: String(appPort),
E2E_API_PORT: String(apiPort),
BETTER_AUTH_URL: baseUrl,
TRUSTED_ORIGINS: `${baseUrl},${localBaseUrl}`,
...(tunnel ? { E2E_CHROME_HOST_RESOLVER_RULES: `MAP ${tunnelHost} ${tunnelIp}` } : {}),
}
const e2eEnv = {
...cloudEnv,
...tunnelEnv,
...s3MockEnv(),
...credentialsEnv,
...(runtime === 'cf' ? { E2E_RUNTIME: 'cf' } : {}),
}
if (runtime === 'cf') {
if (local) rmSync('.wrangler/state/v3/d1', { recursive: true, force: true })
writeDevVars(e2eEnv)
await run('pnpm', ['exec', 'wrangler', 'd1', 'migrations', 'apply', 'DB', '--local'], e2eEnv)
}
try {
await run(process.execPath, [require.resolve('@playwright/test/cli'), 'test', ...specs, `--project=${project}`], e2eEnv)
} finally {
if (tunnel) {
try {
tunnel.process.kill()
} catch {}
}
if (existsSync(pidFile)) {
const pid = Number(readFileSync(pidFile, 'utf8'))
if (Number.isInteger(pid)) {
try {
process.kill(pid)
} catch {}
}
rmSync(pidFile, { force: true })
}
}
function valueAfter(flag) {
const index = args.indexOf(flag)
return index === -1 ? null : args[index + 1]
}
function runtimeCloudCredentials(runtime) {
const suffix = runtime === 'cf' ? '_CF' : '_NODE'
const runtimeEmail =
process.env[`E2E_CLOUD_BUSINESS_EMAIL${suffix}`]?.trim() ||
process.env[`E2E_CLOUD_PRO_EMAIL${suffix}`]?.trim()
const runtimePassword =
process.env[`E2E_CLOUD_BUSINESS_PASSWORD${suffix}`]?.trim() ||
process.env[`E2E_CLOUD_PRO_PASSWORD${suffix}`]?.trim()
if (process.env.CI && (!runtimeEmail || !runtimePassword)) {
throw new Error(`Missing E2E_CLOUD_BUSINESS_EMAIL${suffix} or E2E_CLOUD_BUSINESS_PASSWORD${suffix}`)
}
const email = runtimeEmail || process.env.E2E_CLOUD_BUSINESS_EMAIL || process.env.E2E_CLOUD_PRO_EMAIL
const password =
runtimePassword || process.env.E2E_CLOUD_BUSINESS_PASSWORD || process.env.E2E_CLOUD_PRO_PASSWORD
return email && password
? {
E2E_CLOUD_BUSINESS_EMAIL: email,
E2E_CLOUD_BUSINESS_PASSWORD: password,
}
: {}
}
function s3MockEnv() {
if (!withS3Mock) return {}
return {
E2E_S3_MOCK: '1',
E2E_S3_MOCK_PORT: String(s3MockPort),
E2E_STORAGE_ENDPOINT: `http://127.0.0.1:${s3MockPort}`,
E2E_STORAGE_BUCKET: 'e2e-test',
E2E_STORAGE_REGION: 'auto',
E2E_STORAGE_ACCESS_KEY: 'e2e-access-key',
E2E_STORAGE_SECRET_KEY: 'e2e-secret-key',
}
}
async function startTunnel(target) {
const maxAttempts = Number(process.env.E2E_TUNNEL_START_ATTEMPTS ?? 3)
let lastError = null
for (let attempt = 1; attempt <= maxAttempts; attempt += 1) {
try {
if (attempt > 1) console.log(`Retrying cloudflared quick tunnel startup (${attempt}/${maxAttempts})...`)
return await startTunnelOnce(target)
} catch (error) {
lastError = error
if (attempt === maxAttempts) break
await new Promise((resolve) => setTimeout(resolve, attempt * 3000))
}
}
throw lastError
}
function startTunnelOnce(target) {
const child = spawn(cloudflared, ['tunnel', '--url', target, '--no-autoupdate'], {
stdio: ['ignore', 'pipe', 'pipe'],
})
writeFileSync(pidFile, String(child.pid))
return new Promise((resolve, reject) => {
let tunnelUrl = null
let registered = false
const timeout = setTimeout(() => {
child.kill()
reject(new Error('Timed out waiting for cloudflared tunnel registration'))
}, 30_000)
function handleOutput(chunk) {
const text = chunk.toString()
process.stdout.write(text)
const match = text.match(tunnelUrlPattern)
if (match) tunnelUrl = match[0]
if (text.includes('Registered tunnel connection')) registered = true
if (!tunnelUrl || !registered) return
clearTimeout(timeout)
resolve({ process: child, url: tunnelUrl })
}
child.stdout.on('data', handleOutput)
child.stderr.on('data', handleOutput)
child.on('exit', (code) => {
clearTimeout(timeout)
reject(new Error(`cloudflared exited before tunnel URL was available: ${code}`))
})
})
}
function writeDevVars(env) {
const updates = {
BETTER_AUTH_SECRET: process.env.BETTER_AUTH_SECRET ?? 'ci-test-secret-that-is-at-least-32-chars',
ZPAN_CLOUD_URL: env.ZPAN_CLOUD_URL,
VITE_ZPAN_CLOUD_URL: env.VITE_ZPAN_CLOUD_URL,
BETTER_AUTH_URL: env.BETTER_AUTH_URL,
TRUSTED_ORIGINS: env.TRUSTED_ORIGINS,
}
const lines = existsSync('.dev.vars') ? readFileSync('.dev.vars', 'utf8').split(/\r?\n/) : []
const seen = new Set()
const next = lines
.filter((line) => line.trim() !== '')
.map((line) => {
const key = line.split('=')[0]
if (Object.hasOwn(updates, key)) {
seen.add(key)
return `${key}=${updates[key]}`
}
return line
})
for (const [key, value] of Object.entries(updates)) {
if (!seen.has(key)) next.push(`${key}=${value}`)
}
writeFileSync('.dev.vars', `${next.join('\n')}\n`)
}
async function waitForPublicTunnelIp(hostname) {
const deadline = Date.now() + 120_000
while (Date.now() < deadline) {
try {
const addresses = await publicDns.resolve4(hostname)
return addresses[0]
} catch {
await new Promise((resolve) => setTimeout(resolve, 1000))
}
}
throw new Error(`Timed out waiting for public tunnel DNS: ${hostname}`)
}
function run(command, commandArgs, env = {}) {
return new Promise((resolve, reject) => {
const child = spawn(command, commandArgs, {
stdio: 'inherit',
env: { ...process.env, ...env },
shell: process.platform === 'win32',
})
child.on('exit', (code) => {
if (code === 0) resolve()
else reject(new Error(`${command} ${commandArgs.join(' ')} exited with ${code}`))
})
})
}