mirror of
https://github.com/moeru-ai/airi.git
synced 2026-10-31 03:44:38 +08:00
## Summary
Imported Live2D ZIP and VRM models live only in the browser IndexedDB,
so they do not reach other devices. This PR adds private per-account
model sync: model metadata in Postgres, original files in S3, and device
caches in IndexedDB.
Cloud sync is behind the new local `display-model-sync` feature flag. It
is off by default.
Object storage is now required. The API does not start without
`S3_BUCKET` and `S3_REGION`. **Set them in every deployed environment
before merge.** The local compose stack provides RustFS.
The upload logic is generic (`upload_sessions`, `UploadSessionService`,
`TransferQueue`), so other file types can reuse it. The ADR is
`server/docs/ai/adr/2026-10-08-display-model-cloud-sync.md`. Its status
is still `proposed`. It records that a local file is a device working
copy and an uploaded file is an account asset.
This PR is independent.
## Change map
| Module | Before | After | Description |
| --- | --- | --- | --- |
| `object-store` adapter | Signs PUT with content type and metadata |
Also signs `x-amz-checksum-sha256` and `If-None-Match: *`. Reads the
stored checksum and supports ranged GET | The store rejects wrong bytes
and second writes |
| `upload-sessions` (new) | None | Server-chosen key, idempotent
reservation, size, checksum and signature check | Generic. The business
domain owns meaning and quota |
| `display-models` domain and routes (new) | None | List, reserve,
complete, download, rename, delete under `/api/v1/display-models` |
Revision checks. Delete writes a tombstone |
| Drizzle `0033` | None | `display_models` and `upload_sessions` | |
| `libs/file-transfer` (new) | None | Hash, signed PUT, verified
download, durable account-scoped queue | Resumes after restart. Drops
results after account change |
| Device storage for models | `localforage` in the store |
`database/repos/display-models.repo.ts`, still on `localforage` with the
same keys | No local data moves. The `unstorage` move is a separate pull
request |
| S3 configuration | Optional | Required at startup | Same class of
dependency as Redis |
| Local compose stack | No object storage | RustFS, a bucket, and CORS
for local web origins | `s3.localhost` resolves inside the network and
in a browser |
| `stores/display-models` | Local files only | Cloud directory, sync
status, upload queue, download on demand | Gated by the feature flag |
| `stores/settings/stage-model` | Missing model falls back to default |
A failed cloud download keeps the selection | Local-missing models still
fall back |
| Model selector | Local list | Cloud-only entries, status, upload and
delete actions | |
## Architecture and behavior
```mermaid
sequenceDiagram
participant A as Device A
participant API as API
participant DB as Postgres
participant S3 as Private S3
participant B as Device B
A->>A: Import and save File locally
A->>API: POST /uploads (id, requestId, size, sha256)
API->>DB: Lock owner, check quota, insert session
API-->>A: Signed PUT (checksum, write-once)
A->>S3: PUT bytes
A->>API: POST /uploads/:id/complete
API->>S3: HEAD checksum and size, ranged GET signature
alt mismatch
API->>S3: Delete object, abort session (422)
else match
API->>DB: Publish ready model
end
B->>API: GET /display-models
B->>API: POST /:id/download
B->>S3: GET, verify size and sha256, cache locally
```
## Boundaries and risks
| Invariant or boundary | Failure mode | Protection | Evidence or gap |
| --- | --- | --- | --- |
| Only verified bytes become ready | Client uploads other bytes |
Store-validated SHA-256, size and signature check | Unit tests with a
fake store |
| Ready bytes never change | Reused signed URL overwrites |
`If-None-Match: *` and a per-upload key | Integration test passes
against RustFS: wrong bytes get 400, a second write gets 412. Production
provider unverified |
| Quota holds under races | Parallel reservations exceed it | Advisory
lock per owner in one transaction | Sequential test only. No concurrent
test |
| Accounts stay isolated | Another account reads a model | Every query
filters by owner | Tests for upload, download and list |
| Account switch is safe | Stale upload result lands in new account |
Queue generation check | Queue unit test |
| Deleted models leave devices | Cache stays | Tombstone removes the
cache on next sync | Not tested |
Not in this PR: physical S3 cleanup after delete, Live2D package
validation, card reference check on delete, stage-pocket acceptance.
## Visual changes
The model selector gains sync status and menu items. They show only when
the flag is on, the user is signed in, and the API has S3. I could not
capture before and after images without that backend. This is
unverified.
## Verification
- `display-models.repo.browser.test.ts` runs in CI Chromium. It reads a
4 MiB record in the previous release format, and keeps a model and an
upload across repository instances.
Run after the rebase onto `main`:
- `tsc --noEmit` in `server/apps/api`: no errors.
- `vitest run server/apps/api`: 78 files pass, 2 skipped.
- `vue-tsc --noEmit` in `stage-ui`: no errors.
- `vitest run --project node` for the model repository, model store,
`settings`, `file-transfer`, and feature flags in `stage-ui`: 23 tests
pass.
- `eslint` on changed files: no errors.
Run before the rebase, with no later change to that code:
- `object-store.integration.test.ts` against local RustFS
(`TEST_S3_ENDPOINT`): 2 tests pass.
- CORS preflight from `http://localhost:5173` for a signed PUT: allowed.
Not run:
- `pnpm lint` and the root `pnpm typecheck`.
- `vue-tsc` for `stage-web` and `stage-tamagotchi` after the storage
change.
- The full compose stack. Only RustFS was started.
- Real production S3, browser or Electron end-to-end, and the two-device
flow.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
144 lines
4.1 KiB
YAML
144 lines
4.1 KiB
YAML
name: proj-airi-backend
|
|
|
|
services:
|
|
db:
|
|
image: ghcr.io/tensorchord/vchord-postgres:pg18-v1.0.0
|
|
environment:
|
|
POSTGRES_DB: postgres
|
|
POSTGRES_USER: postgres
|
|
POSTGRES_PASSWORD: example-PAssw0rd-xHjDYR.b7N
|
|
ports:
|
|
- '127.0.0.1:5435:5432'
|
|
volumes:
|
|
- ./apps/api/sql/init.sql:/docker-entrypoint-initdb.d/init.sql:ro
|
|
- db_data:/var/lib/postgresql
|
|
healthcheck:
|
|
test: ['CMD-SHELL', 'pg_isready -U $$POSTGRES_USER -d $$POSTGRES_DB']
|
|
interval: 5s
|
|
timeout: 5s
|
|
retries: 10
|
|
|
|
redis:
|
|
image: redis:7-alpine
|
|
ports:
|
|
- '127.0.0.1:6379:6379'
|
|
volumes:
|
|
- redis_data:/data
|
|
healthcheck:
|
|
test: ['CMD', 'redis-cli', 'ping']
|
|
interval: 5s
|
|
timeout: 5s
|
|
retries: 10
|
|
|
|
# Local S3 for private objects. `s3.localhost` resolves to this container inside the
|
|
# network and to the published port in a browser, so signed URLs work in both places.
|
|
rustfs:
|
|
image: rustfs/rustfs:latest
|
|
command: ['/data']
|
|
environment:
|
|
RUSTFS_ACCESS_KEY: airi-local
|
|
RUSTFS_ADDRESS: ':6113'
|
|
RUSTFS_SECRET_KEY: example-PAssw0rd-xHjDYR.b7N
|
|
networks:
|
|
default:
|
|
aliases:
|
|
- s3.localhost
|
|
ports:
|
|
- '127.0.0.1:6113:6113'
|
|
volumes:
|
|
- rustfs_data:/data
|
|
|
|
rustfs-init:
|
|
image: amazon/aws-cli:latest
|
|
depends_on:
|
|
- rustfs
|
|
environment:
|
|
AWS_ACCESS_KEY_ID: airi-local
|
|
AWS_SECRET_ACCESS_KEY: example-PAssw0rd-xHjDYR.b7N
|
|
AWS_DEFAULT_REGION: us-east-1
|
|
volumes:
|
|
- ./dev/s3/cors.json:/cors.json:ro
|
|
entrypoint: ['/bin/sh', '-c']
|
|
command:
|
|
- |
|
|
until aws --endpoint-url http://s3.localhost:6113 s3api list-buckets >/dev/null 2>&1; do sleep 1; done
|
|
aws --endpoint-url http://s3.localhost:6113 s3api head-bucket --bucket airi-private 2>/dev/null \
|
|
|| aws --endpoint-url http://s3.localhost:6113 s3api create-bucket --bucket airi-private
|
|
aws --endpoint-url http://s3.localhost:6113 s3api put-bucket-cors --bucket airi-private --cors-configuration file:///cors.json
|
|
|
|
api:
|
|
build:
|
|
context: ..
|
|
dockerfile: server/apps/api/Dockerfile
|
|
command: ['pnpm', '-F', '@proj-airi/api-server', 'start']
|
|
depends_on:
|
|
db:
|
|
condition: service_healthy
|
|
redis:
|
|
condition: service_healthy
|
|
rustfs-init:
|
|
condition: service_completed_successfully
|
|
env_file:
|
|
- path: ./apps/api/.env
|
|
required: false
|
|
- path: ./apps/api/.env.local
|
|
required: false
|
|
environment:
|
|
API_SERVER_URL: http://localhost:6112
|
|
AUTH_SERVER_INTERNAL_URL: http://auth:3000
|
|
AUTH_SERVER_URL: http://localhost:6112
|
|
AWS_ACCESS_KEY_ID: airi-local
|
|
AWS_SECRET_ACCESS_KEY: example-PAssw0rd-xHjDYR.b7N
|
|
DATABASE_URL: postgres://postgres:example-PAssw0rd-xHjDYR.b7N@db:5432/postgres
|
|
REDIS_URL: redis://redis:6379
|
|
S3_BUCKET: airi-private
|
|
S3_ENDPOINT: http://s3.localhost:6113
|
|
S3_FORCE_PATH_STYLE: 'true'
|
|
S3_REGION: us-east-1
|
|
healthcheck:
|
|
test: ['CMD-SHELL', 'wget -qO- http://localhost:3000/readyz >/dev/null']
|
|
interval: 10s
|
|
timeout: 5s
|
|
retries: 10
|
|
|
|
auth:
|
|
build:
|
|
context: ..
|
|
dockerfile: server/apps/auth/Dockerfile
|
|
command: ['pnpm', '-F', '@proj-airi/auth-server', 'start']
|
|
depends_on:
|
|
api:
|
|
condition: service_healthy
|
|
env_file:
|
|
- path: ./apps/api/.env
|
|
required: false
|
|
- path: ./apps/api/.env.local
|
|
required: false
|
|
environment:
|
|
DATABASE_URL: postgres://postgres:example-PAssw0rd-xHjDYR.b7N@db:5432/postgres
|
|
PUBLIC_URL: http://localhost:6112
|
|
REDIS_URL: redis://redis:6379
|
|
RESOURCE_SERVER_URL: http://api:3000
|
|
healthcheck:
|
|
test: ['CMD-SHELL', 'wget -qO- http://localhost:3000/readyz >/dev/null']
|
|
interval: 10s
|
|
timeout: 5s
|
|
retries: 10
|
|
|
|
caddy:
|
|
image: caddy:2-alpine
|
|
depends_on:
|
|
api:
|
|
condition: service_healthy
|
|
auth:
|
|
condition: service_healthy
|
|
ports:
|
|
- '127.0.0.1:6112:3000'
|
|
volumes:
|
|
- ./dev/caddy/Caddyfile:/etc/caddy/Caddyfile:ro
|
|
|
|
volumes:
|
|
db_data:
|
|
redis_data:
|
|
rustfs_data:
|