mirror of
https://github.com/Wei-Shaw/sub2api.git
synced 2026-09-24 16:05:44 +08:00
test: add batch image QA reports
This commit is contained in:
Turtle_Li
@@ -0,0 +1,62 @@
|
||||
# Claude Code Batch Image QA Report
|
||||
|
||||
Date: 2026-07-06
|
||||
Tester: Claude Code
|
||||
Claude model selection:
|
||||
|
||||
- Preferred for deep QA: `opus`, but the first run exceeded the initial budget before producing output.
|
||||
- Practical model used for this recorded report: `sonnet` with `--safe-mode --effort low`, because it produced a bounded independent QA report at lower cost.
|
||||
|
||||
## Original Claude Output
|
||||
|
||||
> Batch Image 功能 QA 报告（只读探查）
|
||||
>
|
||||
> ## 范围
|
||||
> 后端计费冻结/结算/退款、状态机与异常兜底、前端批量生图说明文案。基于代码走查(Explore agent)+ 2 条本地 grep 命令验证，未修改任何文件，未执行且未查看任何密钥。
|
||||
>
|
||||
> ## 执行命令
|
||||
>
|
||||
> | # | 命令 | 目的 |
|
||||
> |---|------|------|
|
||||
> | 1 | `grep -n "FOR UPDATE\|Lock(" batch_image_settlement.go batch_image_repo.go` | 验证取消/结算并发是否有行锁保护 |
|
||||
> | 2 | `grep -rn "SETTLEMENT_BILLING_FAILED\|enqueueBillingRetry\|MaxRetr" ...` | 验证结算失败重试是否有次数/退避上限 |
|
||||
>
|
||||
> ## 通过/失败表
|
||||
>
|
||||
> | 检查项 | 结果 | 依据 |
|
||||
> |---|---|---|
|
||||
> | 状态转换行锁保护(防止取消/结算竞态) | ✅ 通过 | `batch_image_repo.go:193,322,415` 均用 `SELECT ... FOR UPDATE` |
|
||||
> | 结算超额扣费保护 | ✅ 通过 | `batch_image_settlement.go:126-130`，`actualCost > holdAmount` 超万分之一即失败中止 |
|
||||
> | 冻结→结算→释放状态机完整性 | ✅ 通过 | 冻结(billing_hold.go) → 结算(settlement.go) → 释放(processor.go:225-239) 链路闭合 |
|
||||
> | 僵尸/未提交任务资金释放 | ✅ 通过 | `billing_recovery.go:22-62`，10分钟未提交自动 failed + 释放冻结 |
|
||||
> | 非法状态转换保护 | ✅ 通过 | `batch_image.go:356-401` 终态不可逆流转 |
|
||||
> | 部分失败正确计费(仅成功项扣费) | ✅ 通过 | `actualCost = successCount * unitPrice` |
|
||||
> | 结算失败重试有界（次数/超时上限） | ⚠️ 未证实 | grep 未发现 `MaxRetr`/退避上限，仅见标记 `SETTLEMENT_BILLING_FAILED` 后重新入队，逻辑分散在其他文件未定位到边界 |
|
||||
> | 前端费用/取消文案与后端逻辑一致 | ⚠️ 基本一致，措辞有偏差 | 见问题清单 P2 |
|
||||
>
|
||||
> ## 问题清单（按严重级别）
|
||||
>
|
||||
> **P1（无，未发现资金泄漏或重复扣款的确认性缺陷）**
|
||||
>
|
||||
> **P2 – 中**
|
||||
> 1. 结算失败重试缺乏可见的次数/超时上限（`SETTLEMENT_BILLING_FAILED` 后 `enqueueBillingRetry`），存在长期卡在 `settling` 状态、资金持续冻结但不释放也不完成结算的风险；需要进一步定位重试调度代码确认是否有兜底超时释放。
|
||||
> 2. 前端取消提示文案（"已生成图片仍可能结算扣费"）与后端实际计费口径（以索引完成后统计的成功项为准）表述不完全对齐，可能造成用户对扣费范围的误解，建议澄清措辞而非改变逻辑。
|
||||
>
|
||||
> **P3 – 低**
|
||||
> 3. 结算过程中途宕机（`Settle()` 执行到一半进程重启）依赖外部定时任务/人工介入恢复，未在本次探查中确认是否有自动扫描 `settling` 超时状态的兜底任务。
|
||||
>
|
||||
> ## 剩余风险
|
||||
> - 未验证"结算失败重试"的调度器代码（未在本次两条命令范围内），无法排除无限重试或永久悬挂的可能性。
|
||||
> - 未做真实并发压测，行锁存在但未验证高并发下取消+结算同时触发的实际表现（仅代码静态确认加锁点存在）。
|
||||
> - 前端文案审查仅基于关键字定位的片段，未通读整个 Guide 组件的所有分支文案。
|
||||
>
|
||||
> ## 建议后续测试
|
||||
> 1. 定位并审查 `enqueueBillingRetry` 实际调度器（重试次数、退避策略、是否有最终告警/人工介入路径），必要时补充单测覆盖"结算持续失败"场景。
|
||||
> 2. 编写并发集成测试：同一 batch_id 同时发起"取消"与"结算完成回调"，验证最终状态与金额一致性（是否只释放或只结算一次）。
|
||||
> 3. 对 `settling` 状态增加超时巡检的专项测试（类比现有 10 分钟未提交巡检），确认是否已有等价机制，如无需评估是否要补齐。
|
||||
> 4. 前端文案走查+产品确认，将"取消后扣费口径"说明与后端"仅索引完成的成功项计费"对齐后再验收。
|
||||
|
||||
## Codex Follow-Up Note
|
||||
|
||||
Codex spot-checked the first P2 after Claude's report. `enqueueBillingRetry` exists in `batch_image_public.go`, but no obvious max retry or terminal handoff was found in the quick search. Keep this as an open risk for the next implementation/test pass rather than treating it as resolved.
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
# Codex Batch Image QA Report
|
||||
|
||||
Date: 2026-07-06
|
||||
Tester: Codex
|
||||
Baseline commits:
|
||||
|
||||
- `8fab636 feat: complete batch image workflow`
|
||||
- `5553d83 fix: localize antigravity image mapping labels`
|
||||
|
||||
## Summary
|
||||
|
||||
No blocking issue remains from the Codex-run checks. One frontend regression was found during testing: Antigravity image mapping preset labels displayed English `passthrough` while the existing UI/test expectation used Chinese `透传`. It was fixed in `5553d83`, and the full frontend suite then passed.
|
||||
|
||||
## Commands Run
|
||||
|
||||
| Area | Command | Result |
|
||||
|---|---|---|
|
||||
| Backend service tests | Docker Go 1.26.4: `go test ./internal/service -run "BatchImage|AdminService_.*BatchImage|GroupBatchImage|PricingService.*Batch|UsageBilling" -count=1 -timeout=10m` | Pass |
|
||||
| Backend repository tests | Docker Go 1.26.4: `go test ./internal/repository -run "BatchImage|UsageBilling|Migrations" -count=1 -timeout=10m` | Pass |
|
||||
| Backend server tests | Docker Go 1.26.4: `go test ./internal/server/... -run "APIContract|BatchImage|APIKey" -count=1 -timeout=10m` | Pass |
|
||||
| Frontend typecheck | `pnpm --dir frontend typecheck` | Pass |
|
||||
| Frontend build | `pnpm --dir frontend build` | Pass |
|
||||
| Frontend full tests | `pnpm --dir frontend test:run` | Pass: 128 files, 803 tests |
|
||||
| Local HTTP smoke | See `smoke-summary.txt` | Pass |
|
||||
|
||||
## HTTP Smoke Result
|
||||
|
||||
Source: `smoke-summary.txt`
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| Unauthorized batch list | `401 API_KEY_REQUIRED` |
|
||||
| Model list | `200`, 2 models: `gemini-2.5-flash-image`, `gemini-3.1-flash-image` |
|
||||
| Insufficient balance submit | `402 BATCH_IMAGE_INSUFFICIENT_BALANCE` |
|
||||
| Completed batch detail | `200`, status `completed`, success `2`, fail `0`, actual cost `0.134` |
|
||||
| Completed items | `200`, item count `2` |
|
||||
| Completed download | `200 application/zip`, 1,602,237 bytes |
|
||||
| Balance restoration after smoke | Original `1.86600000 / 0.00000000`; final `1.86600000 / 0.00000000` |
|
||||
|
||||
## Findings
|
||||
|
||||
| Severity | Finding | Status |
|
||||
|---|---|---|
|
||||
| P2 | Antigravity batch edit image mapping labels were mixed English/Chinese and failed existing UI expectation. | Fixed in `5553d83`; full frontend tests pass. |
|
||||
| P3 | Frontend test output contains existing Vue/i18n warnings (`router-link`, `el-tooltip`, localstorage-file, Browserslist stale data). | Non-blocking; suite passes. |
|
||||
|
||||
## Billing And Exception Coverage
|
||||
|
||||
Covered by automated tests and smoke:
|
||||
|
||||
- Balance reserve moves available funds to frozen funds.
|
||||
- Insufficient balance returns 402 before provider submission.
|
||||
- Capture rejects actual cost greater than hold.
|
||||
- Capture below hold releases the remainder.
|
||||
- Stale pre-provider jobs can be failed and released.
|
||||
- Completed job download only returns successful outputs.
|
||||
|
||||
## Residual Risks
|
||||
|
||||
- Real provider failure combinations should still be tested with controlled fake/fixture provider outputs: malformed output JSONL, missing image bytes, provider cancelled after partial success, and delayed output indexing.
|
||||
- Concurrent cancel vs settlement needs a dedicated integration test with simultaneous requests to prove row-lock behavior under load, not only unit/static coverage.
|
||||
- Settlement billing failure retry currently needs a clearer bounded retry or operator handoff story; Claude independently flagged this too.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Proceed to broader review with Claude and/or manual exploratory testing. Before production enablement, add one integration test for cancel/settle concurrency and one for persistent settlement billing failure recovery.
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
base=http://127.0.0.1:8080
|
||||
unauthorized_status=401
|
||||
unauthorized_code=API_KEY_REQUIRED
|
||||
models_status=200
|
||||
models_count=2
|
||||
models_models=gemini-2.5-flash-image,gemini-3.1-flash-image
|
||||
insufficient_status=402
|
||||
insufficient_code=BATCH_IMAGE_INSUFFICIENT_BALANCE
|
||||
insufficient_message=insufficient balance for batch image hold
|
||||
latest_completed_batch=imgbatch_8944d988d7b92fcba158a9317fe3e699
|
||||
latest_completed_status=200
|
||||
latest_items_status=200
|
||||
latest_download_status=200 application/zip 1602237
|
||||
latest_id=imgbatch_8944d988d7b92fcba158a9317fe3e699
|
||||
latest_status=completed
|
||||
latest_success_count=2
|
||||
latest_fail_count=0
|
||||
latest_actual_cost=0.134
|
||||
latest_item_count=2
|
||||
original_balance=1.86600000
|
||||
original_frozen_balance=0.00000000
|
||||
final_balance=1.86600000
|
||||
final_frozen_balance=0.00000000
|
||||
@@ -0,0 +1,41 @@
|
||||
# Batch Image QA Test Case
|
||||
|
||||
Date: 2026-07-06
|
||||
Branch: `feature/batch-image-foundation`
|
||||
|
||||
## Scope
|
||||
|
||||
Validate the Sub2API batch image feature before broader external review:
|
||||
|
||||
- Gateway API authentication and public response shape
|
||||
- Available batch image model listing
|
||||
- Balance hold failure path before upstream submission
|
||||
- Completed job detail, item listing, and download path
|
||||
- Billing hold, release, capture, settlement, and recovery unit coverage
|
||||
- Frontend batch image page type/build/test health
|
||||
- Agent-copy instruction text for slower polling and resume records
|
||||
|
||||
## Test Data
|
||||
|
||||
- Local endpoint: `http://127.0.0.1:8080`
|
||||
- Local completed batch used for read/download smoke: `imgbatch_8944d988d7b92fcba158a9317fe3e699`
|
||||
- No API key or secret is stored in this report.
|
||||
|
||||
## Cases
|
||||
|
||||
| ID | Case | Expected |
|
||||
|---|---|---|
|
||||
| BI-API-001 | `GET /v1/images/batches` without key | `401`, `API_KEY_REQUIRED` |
|
||||
| BI-API-002 | `GET /v1/images/batches/models` with key | `200`, returns supported image batch models |
|
||||
| BI-API-003 | Submit with intentionally insufficient balance | `402`, `BATCH_IMAGE_INSUFFICIENT_BALANCE`, no provider submission |
|
||||
| BI-API-004 | Fetch completed batch detail | `200`, terminal status and cost fields present |
|
||||
| BI-API-005 | Fetch completed batch items | `200`, success/failure item summary present |
|
||||
| BI-API-006 | Download completed successful images | `200 application/zip`, non-empty archive |
|
||||
| BI-BILL-001 | Reserve balance hold | Available balance decreases, frozen balance increases |
|
||||
| BI-BILL-002 | Capture hold with actual cost below hold | Remainder released, frozen balance returns to zero |
|
||||
| BI-BILL-003 | Reject actual cost above hold | Settlement fails before over-capture |
|
||||
| BI-BILL-004 | Release stale/unsubmitted hold | Stale job fails and frozen funds are released |
|
||||
| BI-FE-001 | Frontend typecheck/build | Pass |
|
||||
| BI-FE-002 | Full frontend test suite | Pass |
|
||||
| BI-FE-003 | Batch image guide copy text | Includes slower polling and local resume-record requirements |
|
||||
|
||||
Reference in New Issue
Block a user