diff --git a/test-reports/batch-image-20260706-codex/claude-report.md b/test-reports/batch-image-20260706-codex/claude-report.md new file mode 100644 index 0000000000..ee8e93f41d --- /dev/null +++ b/test-reports/batch-image-20260706-codex/claude-report.md @@ -0,0 +1,62 @@ +# Claude Code Batch Image QA Report + +Date: 2026-07-06 +Tester: Claude Code +Claude model selection: + +- Preferred for deep QA: `opus`, but the first run exceeded the initial budget before producing output. +- Practical model used for this recorded report: `sonnet` with `--safe-mode --effort low`, because it produced a bounded independent QA report at lower cost. + +## Original Claude Output + +> Batch Image 功能 QA 报告(只读探查) +> +> ## 范围 +> 后端计费冻结/结算/退款、状态机与异常兜底、前端批量生图说明文案。基于代码走查(Explore agent)+ 2 条本地 grep 命令验证,未修改任何文件,未执行且未查看任何密钥。 +> +> ## 执行命令 +> +> | # | 命令 | 目的 | +> |---|------|------| +> | 1 | `grep -n "FOR UPDATE\|Lock(" batch_image_settlement.go batch_image_repo.go` | 验证取消/结算并发是否有行锁保护 | +> | 2 | `grep -rn "SETTLEMENT_BILLING_FAILED\|enqueueBillingRetry\|MaxRetr" ...` | 验证结算失败重试是否有次数/退避上限 | +> +> ## 通过/失败表 +> +> | 检查项 | 结果 | 依据 | +> |---|---|---| +> | 状态转换行锁保护(防止取消/结算竞态) | ✅ 通过 | `batch_image_repo.go:193,322,415` 均用 `SELECT ... FOR UPDATE` | +> | 结算超额扣费保护 | ✅ 通过 | `batch_image_settlement.go:126-130`,`actualCost > holdAmount` 超万分之一即失败中止 | +> | 冻结→结算→释放状态机完整性 | ✅ 通过 | 冻结(billing_hold.go) → 结算(settlement.go) → 释放(processor.go:225-239) 链路闭合 | +> | 僵尸/未提交任务资金释放 | ✅ 通过 | `billing_recovery.go:22-62`,10分钟未提交自动 failed + 释放冻结 | +> | 非法状态转换保护 | ✅ 通过 | `batch_image.go:356-401` 终态不可逆流转 | +> | 部分失败正确计费(仅成功项扣费) | ✅ 通过 | `actualCost = successCount * unitPrice` | +> | 结算失败重试有界(次数/超时上限) | ⚠️ 未证实 | grep 未发现 `MaxRetr`/退避上限,仅见标记 `SETTLEMENT_BILLING_FAILED` 后重新入队,逻辑分散在其他文件未定位到边界 | +> | 前端费用/取消文案与后端逻辑一致 | ⚠️ 基本一致,措辞有偏差 | 见问题清单 P2 | +> +> ## 问题清单(按严重级别) +> +> **P1(无,未发现资金泄漏或重复扣款的确认性缺陷)** +> +> **P2 – 中** +> 1. 结算失败重试缺乏可见的次数/超时上限(`SETTLEMENT_BILLING_FAILED` 后 `enqueueBillingRetry`),存在长期卡在 `settling` 状态、资金持续冻结但不释放也不完成结算的风险;需要进一步定位重试调度代码确认是否有兜底超时释放。 +> 2. 前端取消提示文案("已生成图片仍可能结算扣费")与后端实际计费口径(以索引完成后统计的成功项为准)表述不完全对齐,可能造成用户对扣费范围的误解,建议澄清措辞而非改变逻辑。 +> +> **P3 – 低** +> 3. 结算过程中途宕机(`Settle()` 执行到一半进程重启)依赖外部定时任务/人工介入恢复,未在本次探查中确认是否有自动扫描 `settling` 超时状态的兜底任务。 +> +> ## 剩余风险 +> - 未验证"结算失败重试"的调度器代码(未在本次两条命令范围内),无法排除无限重试或永久悬挂的可能性。 +> - 未做真实并发压测,行锁存在但未验证高并发下取消+结算同时触发的实际表现(仅代码静态确认加锁点存在)。 +> - 前端文案审查仅基于关键字定位的片段,未通读整个 Guide 组件的所有分支文案。 +> +> ## 建议后续测试 +> 1. 定位并审查 `enqueueBillingRetry` 实际调度器(重试次数、退避策略、是否有最终告警/人工介入路径),必要时补充单测覆盖"结算持续失败"场景。 +> 2. 编写并发集成测试:同一 batch_id 同时发起"取消"与"结算完成回调",验证最终状态与金额一致性(是否只释放或只结算一次)。 +> 3. 对 `settling` 状态增加超时巡检的专项测试(类比现有 10 分钟未提交巡检),确认是否已有等价机制,如无需评估是否要补齐。 +> 4. 前端文案走查+产品确认,将"取消后扣费口径"说明与后端"仅索引完成的成功项计费"对齐后再验收。 + +## Codex Follow-Up Note + +Codex spot-checked the first P2 after Claude's report. `enqueueBillingRetry` exists in `batch_image_public.go`, but no obvious max retry or terminal handoff was found in the quick search. Keep this as an open risk for the next implementation/test pass rather than treating it as resolved. + diff --git a/test-reports/batch-image-20260706-codex/codex-report.md b/test-reports/batch-image-20260706-codex/codex-report.md new file mode 100644 index 0000000000..a467fdd11f --- /dev/null +++ b/test-reports/batch-image-20260706-codex/codex-report.md @@ -0,0 +1,67 @@ +# Codex Batch Image QA Report + +Date: 2026-07-06 +Tester: Codex +Baseline commits: + +- `8fab636 feat: complete batch image workflow` +- `5553d83 fix: localize antigravity image mapping labels` + +## Summary + +No blocking issue remains from the Codex-run checks. One frontend regression was found during testing: Antigravity image mapping preset labels displayed English `passthrough` while the existing UI/test expectation used Chinese `透传`. It was fixed in `5553d83`, and the full frontend suite then passed. + +## Commands Run + +| Area | Command | Result | +|---|---|---| +| Backend service tests | Docker Go 1.26.4: `go test ./internal/service -run "BatchImage|AdminService_.*BatchImage|GroupBatchImage|PricingService.*Batch|UsageBilling" -count=1 -timeout=10m` | Pass | +| Backend repository tests | Docker Go 1.26.4: `go test ./internal/repository -run "BatchImage|UsageBilling|Migrations" -count=1 -timeout=10m` | Pass | +| Backend server tests | Docker Go 1.26.4: `go test ./internal/server/... -run "APIContract|BatchImage|APIKey" -count=1 -timeout=10m` | Pass | +| Frontend typecheck | `pnpm --dir frontend typecheck` | Pass | +| Frontend build | `pnpm --dir frontend build` | Pass | +| Frontend full tests | `pnpm --dir frontend test:run` | Pass: 128 files, 803 tests | +| Local HTTP smoke | See `smoke-summary.txt` | Pass | + +## HTTP Smoke Result + +Source: `smoke-summary.txt` + +| Check | Result | +|---|---| +| Unauthorized batch list | `401 API_KEY_REQUIRED` | +| Model list | `200`, 2 models: `gemini-2.5-flash-image`, `gemini-3.1-flash-image` | +| Insufficient balance submit | `402 BATCH_IMAGE_INSUFFICIENT_BALANCE` | +| Completed batch detail | `200`, status `completed`, success `2`, fail `0`, actual cost `0.134` | +| Completed items | `200`, item count `2` | +| Completed download | `200 application/zip`, 1,602,237 bytes | +| Balance restoration after smoke | Original `1.86600000 / 0.00000000`; final `1.86600000 / 0.00000000` | + +## Findings + +| Severity | Finding | Status | +|---|---|---| +| P2 | Antigravity batch edit image mapping labels were mixed English/Chinese and failed existing UI expectation. | Fixed in `5553d83`; full frontend tests pass. | +| P3 | Frontend test output contains existing Vue/i18n warnings (`router-link`, `el-tooltip`, localstorage-file, Browserslist stale data). | Non-blocking; suite passes. | + +## Billing And Exception Coverage + +Covered by automated tests and smoke: + +- Balance reserve moves available funds to frozen funds. +- Insufficient balance returns 402 before provider submission. +- Capture rejects actual cost greater than hold. +- Capture below hold releases the remainder. +- Stale pre-provider jobs can be failed and released. +- Completed job download only returns successful outputs. + +## Residual Risks + +- Real provider failure combinations should still be tested with controlled fake/fixture provider outputs: malformed output JSONL, missing image bytes, provider cancelled after partial success, and delayed output indexing. +- Concurrent cancel vs settlement needs a dedicated integration test with simultaneous requests to prove row-lock behavior under load, not only unit/static coverage. +- Settlement billing failure retry currently needs a clearer bounded retry or operator handoff story; Claude independently flagged this too. + +## Recommendation + +Proceed to broader review with Claude and/or manual exploratory testing. Before production enablement, add one integration test for cancel/settle concurrency and one for persistent settlement billing failure recovery. + diff --git a/test-reports/batch-image-20260706-codex/smoke-summary.txt b/test-reports/batch-image-20260706-codex/smoke-summary.txt new file mode 100644 index 0000000000..30696744f1 --- /dev/null +++ b/test-reports/batch-image-20260706-codex/smoke-summary.txt @@ -0,0 +1,23 @@ +base=http://127.0.0.1:8080 +unauthorized_status=401 +unauthorized_code=API_KEY_REQUIRED +models_status=200 +models_count=2 +models_models=gemini-2.5-flash-image,gemini-3.1-flash-image +insufficient_status=402 +insufficient_code=BATCH_IMAGE_INSUFFICIENT_BALANCE +insufficient_message=insufficient balance for batch image hold +latest_completed_batch=imgbatch_8944d988d7b92fcba158a9317fe3e699 +latest_completed_status=200 +latest_items_status=200 +latest_download_status=200 application/zip 1602237 +latest_id=imgbatch_8944d988d7b92fcba158a9317fe3e699 +latest_status=completed +latest_success_count=2 +latest_fail_count=0 +latest_actual_cost=0.134 +latest_item_count=2 +original_balance=1.86600000 +original_frozen_balance=0.00000000 +final_balance=1.86600000 +final_frozen_balance=0.00000000 diff --git a/test-reports/batch-image-20260706-codex/test-case.md b/test-reports/batch-image-20260706-codex/test-case.md new file mode 100644 index 0000000000..35639d5b8c --- /dev/null +++ b/test-reports/batch-image-20260706-codex/test-case.md @@ -0,0 +1,41 @@ +# Batch Image QA Test Case + +Date: 2026-07-06 +Branch: `feature/batch-image-foundation` + +## Scope + +Validate the Sub2API batch image feature before broader external review: + +- Gateway API authentication and public response shape +- Available batch image model listing +- Balance hold failure path before upstream submission +- Completed job detail, item listing, and download path +- Billing hold, release, capture, settlement, and recovery unit coverage +- Frontend batch image page type/build/test health +- Agent-copy instruction text for slower polling and resume records + +## Test Data + +- Local endpoint: `http://127.0.0.1:8080` +- Local completed batch used for read/download smoke: `imgbatch_8944d988d7b92fcba158a9317fe3e699` +- No API key or secret is stored in this report. + +## Cases + +| ID | Case | Expected | +|---|---|---| +| BI-API-001 | `GET /v1/images/batches` without key | `401`, `API_KEY_REQUIRED` | +| BI-API-002 | `GET /v1/images/batches/models` with key | `200`, returns supported image batch models | +| BI-API-003 | Submit with intentionally insufficient balance | `402`, `BATCH_IMAGE_INSUFFICIENT_BALANCE`, no provider submission | +| BI-API-004 | Fetch completed batch detail | `200`, terminal status and cost fields present | +| BI-API-005 | Fetch completed batch items | `200`, success/failure item summary present | +| BI-API-006 | Download completed successful images | `200 application/zip`, non-empty archive | +| BI-BILL-001 | Reserve balance hold | Available balance decreases, frozen balance increases | +| BI-BILL-002 | Capture hold with actual cost below hold | Remainder released, frozen balance returns to zero | +| BI-BILL-003 | Reject actual cost above hold | Settlement fails before over-capture | +| BI-BILL-004 | Release stale/unsubmitted hold | Stale job fails and frozen funds are released | +| BI-FE-001 | Frontend typecheck/build | Pass | +| BI-FE-002 | Full frontend test suite | Pass | +| BI-FE-003 | Batch image guide copy text | Includes slower polling and local resume-record requirements | +