232 Commits
Author SHA1 Message Date
myhloli 091244b55c test(file-parse): cover explicit cancellation boundaries
Verify timeout, disconnect and shutdown never send POST cancellation requests. Document explicit upstream cancellation while preserving the file_parse response contract.
2026-10-24 00:00:45 +08:00
myhloli d3dd75821f refactor(api): split task responses and unify file IDs
Generate server-owned file IDs shared by Tasks and result resources. Separate creation, detail, cancellation and upload responses, omitting null fields.

Use POST /v1/tasks/{task_id}/cancel and remove public DELETE endpoints. Update Router, SDK, file_parse recovery, examples and documentation.
2026-10-23 23:46:10 +08:00
myhloli 619765f4d8 refactor(api): unify task file states and response contracts 2026-10-23 19:19:30 +08:00
myhloli 1399b683b4 refactor(api): simplify task protocol and internalize model output 2026-10-23 16:21:59 +08:00
myhloli fb3f81df0a docs(api): specify Tasks, Files and file_parse response contracts 2026-10-23 15:25:02 +08:00
myhloli 28494e633c refactor(api): unify task file fields and generate optional client IDs 2026-10-23 11:53:26 +08:00
myhloli 412d2c9b86 refactor(gradio): require 6.21 and simplify event chains 2026-10-23 01:11:35 +08:00
myhloli f013067b96 fix(tasks): harden result leases and SDK request handling 2026-10-22 22:50:48 +08:00
myhloli 7f250e2331 feat(api)!: make Tasks the only asynchronous parsing API
Converge the SDK, Gradio, remote CLI, Doclib and Router on Tasks with hash-based uploads, per-file result URLs, task listing and optional model output. Preserve Files and the synchronous file_parse wrapper while removing standalone Uploads/Jobs and local-path HTTP inputs.

Migrate contract, lifecycle, transport and real-document regression coverage; update documentation and HTTP examples. Validation: 1452 passed, 3 conditional skips, plus targeted final PDF checks.

BREAKING CHANGE: clients and workers must use Tasks; standalone Uploads/Jobs endpoints and local-source/inline HTTP options have been removed.
2026-10-22 21:39:42 +08:00
myhloli 92613d738d chore: align Python 3.11-3.14 support and release checks 2026-10-22 21:06:05 +08:00
myhloli f59acf1ffa feat(api): add recoverable synchronous file parsing service 2026-10-22 18:42:54 +08:00
myhloli 57485941fb feat(api): process JSON tasks and deliver per-file outputs 2026-10-22 18:41:58 +08:00
Xiaomeng Zhao 1681c8e916 fix(llama): serialize Linux Xe-LPG Vulkan submissions 2026-10-22 15:08:19 +08:00
Xiaomeng Zhao 345f630eb1 feat(llama): calibrate integrated GPU routing and serialize legacy Intel Vulkan
Reuse platform ONNX decisions when no discrete GPU is available. Bind and verify the actual Vulkan devices before applying the legacy Intel submission workaround in engine and server paths.

Preserve explicit device settings and offline behavior, and include regression coverage and hardware validation records.
2026-10-22 02:43:32 +08:00
Xiaomeng Zhao 354566d8af feat(onnx): add adaptive calibration and Intel WebGPU architecture policy 2026-10-21 23:17:57 +08:00
Xiaomeng Zhao a2a4960b08 feat(onnx): add Linux WebGPU and compact OCR recognition outputs 2026-10-21 18:56:45 +08:00
myhloli 3702cd7c56 feat(api): support URL inputs in v1 tasks 2026-10-21 15:00:29 +08:00
myhloli ee3ba4dc40 feat(onnx): route UNet through shared device selection 2026-10-20 16:04:11 +08:00
myhloli c666e08dc7 feat(api): unify task uploads and result download links
Accept form manifests and per-file uploads with resumable task state and optional source hashes. Publish result URLs only after final ZIP artifacts are ready, and align Router handling with the same protocol.

Remove synchronous parsing and task result routes, drop fixed local upload limits, and update the HTTP client, documentation, and regression coverage.
2026-10-20 10:48:28 +08:00
myhloli 48524bfabb feat(onnx): automatically select CPU or DirectML devices
Add auto/cpu/gpu configuration, Windows adapter discovery, isolated rec calibration, atomic preference persistence, and vendor session presets. Route layout, rec, and full MFR batches through DirectML with CPU fallback.

Use mutually exclusive Windows x64 DirectML dependencies and guard ORT/Magika distribution identity. Include calibration fixtures, three-machine results, the 0.8 threshold with ten UHD 770 repeats, and installation documentation.

Validation: 762 MinerU tests and 71 DocVortex tests passed; dependency checks and Windows Python 3.10-3.14 resolution passed. The 19-page OCR result matches the CPU baseline. DocVortex PyPI acceptance remains pending a compatible release; pinned source acceptance passed.
2026-10-19 11:02:29 +08:00
myhloli aca9b06da3 feat(doclib): add paginated filename search and unified page selectors
Apply filename prefix matching to the final query token and paginate the
complete filtered candidate set with stable ordering and accurate totals.
Reuse input page-range syntax in read locators, including rN and all, and
preserve selected-page boundaries during block and character continuation.
Update CLI/SDK/HTTP contracts, bilingual help, documentation, and regressions.

Validation: 667 Doclib/CLI tests passed; Ruff and formatting checks passed.
Refs #5366, #5337, #5365.
2026-10-11 11:22:21 +08:00
myhloli 193d193aa8 fix(ci): require DocVortex 0.5.16 and repair docs and coverage checks 2026-10-11 04:19:10 +08:00
myhloli 78379ff635 fix(pdf): reuse shared CJK space handling for native text
Use DocVortex generated-space evidence in Python span filling and rotated-line paths, while keeping the Rust fast path aligned. Document list-tail continuation semantics and add spacing, code protection, rotation, and Python/Rust parity regressions.
2026-10-11 00:26:04 +08:00
myhloli 197795402f Record verified closure of six repaired pull requests 2026-10-10 01:30:07 +08:00
myhloli 27ecc16f0f Record authorized fix pushes and original PR closure scope 2026-10-10 01:27:27 +08:00
myhloli 5b477601a0 Document PR triage outcomes and service retention and recovery settings 2026-10-10 01:17:33 +08:00
myhloli e614751847 feat(api): add V1 task endpoints and balance Router workloads 2026-10-09 19:40:15 +08:00
myhloli f232373fb3 refactor: remove remaining OpenCV calls and calibration branch 2026-10-08 15:39:47 +08:00
myhloli e6d89a6af4 refactor: remove OpenCV from base dependencies 2026-10-08 15:08:18 +08:00
myhloli f504cff8b0 feat(api): introduce virtual model ID and description for model endpoints 2026-09-29 19:49:14 +08:00
myhloli a6be4df4bb fix(webui): display server duration for fast API tasks 2026-09-24 19:04:52 +08:00
myhloli bea1f6fac0 feat(tiers): extend file type normalization to include MHTML/MHT and update parsing rules 2026-09-24 01:58:17 +08:00
myhloli bf8a518d23 Select table ONNX CUDA automatically when available 2026-09-20 16:55:21 +08:00
myhloli 0e1124a4f1 fix(api-server): scope --log-level to server-side logs only
--log-level previously flowed into configure_global_log_level(), lowering the
process-wide Loguru stderr sink (and the LOGURU_LEVEL inherited by spawned
render workers) alongside the uvicorn/server stdlib loggers. Managed
api-server children spawned by Gradio/Router/Doclib hardcode --log-level
warning, which also silenced model INFO/DEBUG logs and made MINERU_LOG_LEVEL
ineffective in the webui scenario.

The Loguru default sink now always follows the global log.level
(MINERU_LOG_LEVEL > mineru.json > info), while --log-level controls only
Uvicorn startup, HTTP access, ASGI logs and the api-server service logger.
Explicit --log-level debug/error no longer affects Loguru; use
MINERU_LOG_LEVEL to tune model logs.
2026-09-19 17:22:49 +08:00
myhloli 99a012b146 feat(pdf): add PDF CPU gap measurement tool and documentation 2026-09-19 01:58:22 +08:00
myhloli 7ac43b362b docs(next): reference effort_for_tier() instead of renamed mapping
HYBRID_EFFORT_BY_TIER 已改名为私有 _EFFORT_BY_TIER,公开入口是
effort_for_tier();workflows 不变量 #5 与 pitfalls §1.1 同步改用
公开 API 描述。docs/plans/ 带日期的历史设计记录保持原样。

Refs #5528
2026-09-18 19:52:00 +08:00
myhloli 6a2aec38b1 docs(next): drop stale backend references after removal
backend_for_tier 派生链与 ParserRuntimeOptions.backend 已随 dd20bf0
删除,同步清理设计文档残留:sdk/parser.md 导出表删行;glossary §3.2
改写为「backend 概念整体移除,现行只有 tier 及派生 effort」;
workflows 不变量 #5/#8、pitfalls §1.1 同步改写;structured-content-schema
删去已失效的 parse.backend 开放问题;implementation-plan 的
normalize_middle_json 任务卡去掉 backend 参数与 _meta.backend 迁移目标
(schema 2.0 不记录 backend)。mineru.backend 模块仍存在,
sdk/README.md 对应行保持不变。

Refs #5528
2026-09-18 19:37:11 +08:00
myhloli 58bd959fc5 refactor: remove latex delimiters option from API and documentation 2026-09-18 15:17:04 +08:00
myhloli 464d473764 refactor: remove language option from API server and related components 2026-09-18 14:27:29 +08:00
myhloli 580a1758e9 feat: remove PARSER_BACKENDS constant from mineru.parser exports and update documentation 2026-09-17 19:08:22 +08:00
Xiaomeng Zhao 36e4fbf6de Merge pull request #5512 from myhloli/dev
feat: remove `--backend` option from `mineru-kit parse` command and update documentation
2026-09-17 18:50:47 +08:00
myhloli 2066dc6273 feat: remove --backend option from mineru-kit parse command and update documentation 2026-09-17 18:44:00 +08:00
jinzhenjandClaude Code 4ce396aa2a docs: update middle JSON compatibility boundary for 3.x remote
Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-17 18:32:05 +08:00
myhloli f470d48d14 docs: drop next-branch markers after next-to-master merge 2026-09-17 11:11:31 +08:00
myhloli 366fadb1c9 Promote TSV to an independent file suffix (ADR-0034)
- Route tsv through the shared CSV flash pipeline with an independent
  file_type and metadata.file_suffix; bump docvortex pin to 0.4.9
- Add ADR-0034 documenting the content-addressed cache identity decision,
  annotate ADR-0028, and update docs/next tier-group wording, telemetry
  and e2e specs
- Cover detection, sync/async analyze, API server jobs and doclib
  lifecycle for tsv in tests
2026-09-16 17:55:07 +08:00
myhloli 4acb913d14 Enhance language detection logic: add support for C.UTF-8 and macOS system locale fallback 2026-09-16 17:17:56 +08:00
myhloli 37a96bf110 Add i18n support for CLI messages and error handling; update README for language configuration 2026-09-16 17:09:34 +08:00
myhloli fe3e120129 Add global log level configuration support across multiple modules 2026-09-14 19:39:59 +08:00
myhloli 96c54f1254 Remove obsolete CI configurations for Gradio and Transformers 5 2026-09-14 02:21:45 +08:00
myhloli 14423a9c4f Rename 'mineru-gradio' command to 'mineru-webui' in pyproject.toml 2026-09-11 19:40:24 +08:00