docs(roadmap): revise v2.7 download planning

2026-08-28 15:51:29 +08:00 · 2026-05-11 01:08:20 -04:00
parent 06c7be910b
commit 9d13ee7055
3 changed files with 52 additions and 44 deletions
@@ -34,12 +34,12 @@ Each version ships 1–2 major features. Ship small, ship often.
 | [v2.4](docs/roadmap/v2.4.md) | **Image Hosting** | Upload API, PicGo / uPic / ShareX integration, custom domain |
 | [v2.5](docs/roadmap/v2.5.md) | **Multi-Platform Deployment** | 7 first-class targets via Turso (CF, Docker, AWS Lambda, Vercel, Netlify, Azure, GCP); avatar upload |
 | [v2.6](docs/roadmap/v2.6.md) | **Pro Launch** | Cloud binding, entitlement system, quota store, white-label, audit log, site announcements, retroactive Pro gates |
-| [v2.7](docs/roadmap/v2.7.md) | **WebDAV & File Processing** | WebDAV protocol access, small-file zip compression/extraction for Community, Pro Cloud Processing for remote download and large archive jobs |
+| [v2.7](docs/roadmap/v2.7.md) | **WebDAV & File Processing** | WebDAV protocol access, small-file zip compression/extraction, and Community remote-download orchestration through Aria2, qBittorrent, and future adapters |
 | [v2.8](docs/roadmap/v2.8.md) | **Pro Analytics & Identity** | Analytics dashboard, SSO enterprise (multi-IdP OIDC + SAML), LDAP / SCIM |
 | [v2.9](docs/roadmap/v2.9.md) | **Backup** | zpan-cli (Rust) one-way backup for NAS / desktop |
 | [v2.10](docs/roadmap/v2.10.md) | **Sync & Desktop** | Bidirectional sync + Tauri desktop tray app |

-Managed cloud services (remote download, large archive processing, content moderation, server-side media processing, managed custom domains) live in a separate closed-source repo and version independently. They are consumed by ZPan via HTTP, and do not affect self-hosted deployments that don't opt in.
+Managed cloud services (large archive processing, content moderation, server-side media processing, managed custom domains) live in a separate closed-source repo and version independently. They are consumed by ZPan via HTTP, and do not affect self-hosted deployments that don't opt in. Remote download is not a Cloud execution service; ZPan integrates with user-owned download engines instead.

 ## Deployment Options

@@ -70,6 +70,7 @@ Everything an individual, family, or small team needs to self-host:
 - WebDAV access for external file managers and sync tools
 - One personal workspace plus one extra team workspace, shared folders, and basic roles
 - Small-file zip compression and extraction within local runtime limits
+- Remote-download orchestration through user-configured Aria2, qBittorrent, and future compatible adapters
 - Backup CLI + desktop app + bidirectional sync
 - All 7 deployment targets

@@ -82,7 +83,6 @@ Operator-grade features for running ZPan as a service (internally at a company,
 - Open registration (the public signup mode) + anti-abuse tooling (captcha, rate limit, email verification)
 - Audit logs + site announcements + webhook notifications + analytics dashboard
 - White-label (logo / favicon / wordmark / custom branding)
- Remote download through ZPan Cloud Processing
 - Large archive processing and future server-side media/document processing
 - SSO enterprise (multi-IdP OIDC + SAML), LDAP / SCIM provisioning
 - Advanced RBAC / custom roles, retention policies
@@ -99,7 +99,7 @@ Pro is purchased or redeemed on ZPan Cloud, then mirrored to a bound ZPan instan

 ### Managed Cloud Services (separate repo)

-A small set of Pro features depend on backend infrastructure we host: remote download (`yt-dlp` and direct URL fetching), large archive processing, content moderation (third-party NSFW scanning), server-side media/document processing, and managed custom domains (SSL provisioning). Those services live in a separate closed-source repo (`zpan-cloud`) and version independently. Self-hosted Pro users call these services via HTTPS; usage included up to a monthly quota per plan, overage billed post-paid with a user-set hard cap.
+A small set of Pro features depend on backend infrastructure we host: large archive processing, content moderation (third-party NSFW scanning), server-side media/document processing, and managed custom domains (SSL provisioning). Those services live in a separate closed-source repo (`zpan-cloud`) and version independently. Self-hosted Pro users call these services via HTTPS; usage included up to a monthly quota per plan, overage billed post-paid with a user-set hard cap. Cloud does not run remote-download workers in the v2.7 plan.

 ### Pricing

@@ -6,7 +6,7 @@ Introduce ZPan Pro. Ship the cloud-account binding flow, the entitlement verific

 **Product direction update:** v2.6 also adds a scoped storage-quota store inside ZPan. This does not move generic Pro activation into ZPan. It lets a Pro instance operator define a per-instance storage package catalog, while Cloud remains the merchant of record and Stripe integration owner. Terminal storage buyers do not need ZPan Cloud accounts.

-**Scope update:** audit log and site announcements have moved forward from the original v2.7 operator plan into v2.6. v2.7 is now reserved for file-processing workflows: small-file archive handling in Community and Cloud-backed remote download / large processing paths.
+**Scope update:** audit log and site announcements have moved forward from the original v2.7 operator plan into v2.6. v2.7 is now reserved for file-processing workflows: small-file archive handling in Community and remote-download orchestration through user-owned download engines such as Aria2 and qBittorrent.

 Pro features can be added continuously in later versions. This version's job is to make the **machinery** for paid features exist.

@@ -206,7 +206,7 @@ Everything else from v2.0–v2.5 stays Free:
 - **In-ZPan generic membership redemption input** — membership redemption lives on the cloud dashboard; only gift card checkout is accepted in the ZPan quota store
 - **In-ZPan subscription billing engine** — storage package subscriptions, metered usage pricing, invoices, and payment operations stay on Cloud; ZPan only stores entitlement deliveries and reports traffic usage
 - **Webhook notifications** — external event delivery is deferred past v2.6
- **Remote download and archive processing** — moved to v2.7
+- **Remote download and archive processing** — moved to v2.7; remote download is Community orchestration through user-owned engines, not a Pro Cloud downloader
 - **Team / Business tiers** — single Pro SKU only
 - **Multi-seat** — one Pro subscription binds one ZPan instance
 - **Multi-bind per account** — a single Pro subscription can only bind one active instance. Binding a second instance on the same Pro account requires unbinding the first, or a future Team plan
@@ -1,17 +1,17 @@
-# v2.7 — WebDAV, Remote Download & Archive Processing
+# v2.7 — WebDAV, Download Integrations & Archive Processing

 Ship WebDAV access and server-side file-processing workflows without turning basic personal tasks into Pro-only features.

-v2.6 already pulled forward the first operator features: audit log and site announcements. v2.7 therefore moves to the file workflow layer: WebDAV protocol access, archive compression/extraction for small files, plus the Cloud-backed foundation for remote download and larger processing jobs.
+v2.6 already pulled forward the first operator features: audit log and site announcements. v2.7 therefore moves to the file workflow layer: WebDAV protocol access, archive compression/extraction for small files, plus remote-download orchestration through user-owned download engines.

 ## Product Boundary

 The rule is simple:

- **Community** gets WebDAV protocol access and bounded small-file archive processing that can run safely on the existing deployment.
- **Pro** gets Cloud Processing for workloads that need hosted compute, third-party binaries, temporary disk, heavy retry logic, or abuse controls.
+- **Community** gets WebDAV protocol access, bounded small-file archive processing, and remote-download integrations with user-configured download engines.
+- **Pro** is not required for remote download. ZPan Cloud must not run a downloader execution service in v2.7.

-This keeps normal file utility features free while avoiding unstable Worker behavior for large or complex jobs.
+This keeps normal file utility features free while avoiding unstable Worker behavior for long-running downloads. ZPan remains the control plane; Aria2, qBittorrent, and future download engines perform the actual transfer work.

 ## Major Features

@@ -65,7 +65,7 @@ Add a local job model for compression and extraction.
 - **Execution**: async job queue; API requests create jobs and return immediately
 - **UI**: file action menu, batch toolbar, job drawer, retry failed job where safe

-Archive jobs are a shared foundation for both Community and Cloud Processing execution.
+Archive jobs are a shared foundation for Community execution now and possible larger processing execution later.

 ### Community Small Archive Processing

@@ -105,47 +105,49 @@ Validation rules:

 Zip extraction must fail before writing any output if validation fails. Partial success is not a valid user-visible result.

-### Cloud Remote Download
+### Remote Download Integrations

-Remote download is Pro because it depends on hosted compute and has real abuse and bandwidth cost.
+Remote download is a Community feature because the execution happens on infrastructure the instance owner configures and controls.

-Supported through ZPan Cloud Processing:
+Supported engines in v2.7:

- HTTP/HTTPS direct URL download
- `yt-dlp` download for supported sites
- Cloud-side metadata probing before execution
- Progress callbacks to the ZPan instance
- Upload result to the user's configured S3/R2 storage
- Final callback creates the file record in ZPan
+- **Aria2** — HTTP/HTTPS, FTP, magnet, and BitTorrent tasks through JSON-RPC
+- **qBittorrent** — torrent and magnet tasks through the Web API
+- **ZPan net-disk downloader adapter** — interface reserved for a future first-party downloader that can handle selected net-disk providers

-The open-source instance remains the control plane. ZPan Cloud is the execution plane.
+ZPan owns task creation, target folder selection, progress display, file import, quota checks, and final `matters` records. Download engines own fetching bytes, retrying source transfers, and keeping temporary files until ZPan imports them.

 Flow:

 1. User creates a remote download job in ZPan
-2. ZPan verifies `remote_download` or `cloud_processing`
-3. ZPan sends the job to Cloud through the existing Cloud binding
-4. Cloud resolves metadata, downloads, uploads to the target storage, and reports progress
-5. Cloud sends a signed completion callback
-6. ZPan verifies the callback and creates the final file record
+2. ZPan selects a configured download engine for the requested source type
+3. ZPan submits the task to Aria2, qBittorrent, or a compatible future adapter
+4. ZPan polls or receives progress from the engine
+5. When the engine finishes, ZPan imports the completed file into the selected storage and folder
+6. ZPan creates the final file record and clears the engine-side temporary task according to the configured cleanup policy

-### Cloud Processing Escalation
+Engine configuration:

-When a Community archive job exceeds local limits, ZPan should fail clearly or offer Cloud Processing when available.
+- Admins configure engine endpoint, credentials, default save path, and concurrency limits
+- Credentials are stored as instance secrets and are never exposed to browser clients
+- Engine health is visible in the admin settings page
+- Users can only create jobs against engines enabled by the admin

-Cloud Processing handles:
+The first implementation should define a small internal downloader interface instead of baking Aria2 or qBittorrent details into route handlers. The interface should be narrow: create task, read task status, cancel task, and resolve completed output files.
+
+### Large Processing Escalation
+
+When a Community archive job exceeds local limits, ZPan should fail clearly. A later processing service may handle large archive/media/document work, but v2.7 does not introduce Cloud downloader execution.
+
+Future processing service candidates:

 - Large zip compression
 - Large zip extraction
 - `tar`, `tar.gz`, `7z`, and `rar`
 - Password-protected archives
- Remote download
 - Future thumbnail, media probing, transcoding, and document conversion

-Feature gate:
-
- `cloud_processing` for the shared hosted processing entitlement
- `remote_download` may be exposed as a narrower gate if product packaging needs it
+Remote download stays outside this Pro processing bucket unless the product direction changes explicitly.

 ## Security and Abuse Controls

@@ -170,10 +172,11 @@ Remote download:

 - Block private IP ranges, localhost, link-local, and metadata endpoints
 - Restrict redirects through the same SSRF checks
- Apply per-instance concurrency limits
- Apply monthly bytes and job-count quotas
- Do not store user cookies longer than the job requires
+- Apply per-engine and per-instance concurrency limits
+- Apply local monthly bytes and job-count quotas
+- Do not store source credentials longer than the job requires
 - Keep source URL, output size, and failure reason in the job audit trail
+- Keep download-engine credentials server-side only

 ## Non-goals for v2.7

@@ -184,7 +187,9 @@ Remote download:
 - Full net-disk provider coverage
 - Browser-side zip generation for server files
 - Silent best-effort extraction after partial failures
- Large archive handling in plain Workers without Cloud Processing
+- Large archive handling in plain Workers
+- ZPan Cloud downloader execution
+- `yt-dlp` hosted by ZPan Cloud
 - Full media transcoding
 - Public plugin API for third-party processors

@@ -199,14 +204,17 @@ Remote download:
 **User receiving a small archive:**
 > I upload a small zip, click Extract here, and ZPan expands it into normal files after validating the archive.

-**Pro operator using remote download:**
-> I paste a video URL. ZPan creates a remote download job, Cloud runs `yt-dlp`, uploads the result to my storage backend, and the file appears in my ZPan folder.
+**Self-hosted user using remote download:**
+> I configure my Aria2 RPC endpoint in Admin settings, paste a direct URL or magnet link in ZPan, and the completed file is imported into my selected ZPan folder.
+
+**Torrent user using qBittorrent:**
+> I connect ZPan to my existing qBittorrent instance, add a magnet task from the web UI, and ZPan tracks progress until the finished content is imported into object storage.

 **Large archive user:**
-> I try to extract a 4 GB zip on a Cloudflare Workers deployment. ZPan refuses the local job and explains that large archive processing requires Cloud Processing or a Node/Docker execution path.
+> I try to extract a 4 GB zip on a Cloudflare Workers deployment. ZPan refuses the local job and explains that large archive processing is outside the local runtime limits.

 ## v1 Issues Resolved

 - WebDAV protocol access — shipped as Community external-client compatibility
- Remote URL download — shipped as Pro Cloud Processing
- Archive compression/extraction — shipped as bounded Community file processing, with Pro escalation for large jobs
+- Remote URL download — shipped as Community orchestration through Aria2, qBittorrent, and future compatible download adapters
+- Archive compression/extraction — shipped as bounded Community file processing