211 Commits
Author SHA1 Message Date
myhloli d3dd75821f refactor(api): split task responses and unify file IDs
Generate server-owned file IDs shared by Tasks and result resources. Separate creation, detail, cancellation and upload responses, omitting null fields.

Use POST /v1/tasks/{task_id}/cancel and remove public DELETE endpoints. Update Router, SDK, file_parse recovery, examples and documentation.
2026-10-23 23:46:10 +08:00
myhloli 619765f4d8 refactor(api): unify task file states and response contracts 2026-10-23 19:19:30 +08:00
myhloli 1399b683b4 refactor(api): simplify task protocol and internalize model output 2026-10-23 16:21:59 +08:00
myhloli 28494e633c refactor(api): unify task file fields and generate optional client IDs 2026-10-23 11:53:26 +08:00
myhloli 7f250e2331 feat(api)!: make Tasks the only asynchronous parsing API
Converge the SDK, Gradio, remote CLI, Doclib and Router on Tasks with hash-based uploads, per-file result URLs, task listing and optional model output. Preserve Files and the synchronous file_parse wrapper while removing standalone Uploads/Jobs and local-path HTTP inputs.

Migrate contract, lifecycle, transport and real-document regression coverage; update documentation and HTTP examples. Validation: 1452 passed, 3 conditional skips, plus targeted final PDF checks.

BREAKING CHANGE: clients and workers must use Tasks; standalone Uploads/Jobs endpoints and local-source/inline HTTP options have been removed.
2026-10-22 21:39:42 +08:00
myhloli 92613d738d chore: align Python 3.11-3.14 support and release checks 2026-10-22 21:06:05 +08:00
myhloli c666e08dc7 feat(api): unify task uploads and result download links
Accept form manifests and per-file uploads with resumable task state and optional source hashes. Publish result URLs only after final ZIP artifacts are ready, and align Router handling with the same protocol.

Remove synchronous parsing and task result routes, drop fixed local upload limits, and update the HTTP client, documentation, and regression coverage.
2026-10-20 10:48:28 +08:00
myhloli e614751847 feat(api): add V1 task endpoints and balance Router workloads 2026-10-09 19:40:15 +08:00
myhloli bea1f6fac0 feat(tiers): extend file type normalization to include MHTML/MHT and update parsing rules 2026-09-24 01:58:17 +08:00
myhloli 2bbaf6efa8 feat(onnx): unify thread configuration for ONNX sessions with environment variable fallback 2026-09-20 23:11:53 +08:00
myhloli 3103e954a4 feat(artifacts): update image naming convention for materialized assets and improve image resolution handling 2026-09-20 22:20:44 +08:00
myhloli bf8a518d23 Select table ONNX CUDA automatically when available 2026-09-20 16:55:21 +08:00
jinzhenjandClaude Code dd20bf0052 refactor(parser): drop dead backend_for_tier chain and Hybrid-era locks
- Remove backend_for_tier / CANONICAL_HYBRID_ENGINE and the backend
  field from ParserRuntimeOptions: the string had no runtime consumer
  (doc_analyze takes effort + vlm_config, not a backend name) and
  returned the retired hybrid-engine value
- Drop the write-only application.state.backend storage
- Flip the as_kwargs/state.backend assertions to effort-based ones and
  remove the locked hybrid-backend mapping test
- Point the PARSER_BACKENDS changelog note at model.small_backend and
  model.vlm.engine instead of the removed helper

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-18 16:30:53 +08:00
myhloli 58bd959fc5 refactor: remove latex delimiters option from API and documentation 2026-09-18 15:17:04 +08:00
myhloli 2d016d626e fix: never fetch remote HTML images and keep external links
Remove fetch_remote_images=True from the API server HTML job and the
SDK default source context so remote images are never downloaded
server-side, closing the SSRF path on untrusted uploads. Bump docvortex
to >=0.4.12, which removes the fetch capability and lets saved bundles
keep external image links instead of requiring materialized assets.
Add a regression test covering parse, API-server zip output and link
retention with zero network requests.

Refs #5524
2026-09-18 03:59:09 +08:00
myhloli 580a1758e9 feat: remove PARSER_BACKENDS constant from mineru.parser exports and update documentation 2026-09-17 19:08:22 +08:00
myhloli 2066dc6273 feat: remove --backend option from mineru-kit parse command and update documentation 2026-09-17 18:44:00 +08:00
myhloli f470d48d14 docs: drop next-branch markers after next-to-master merge 2026-09-17 11:11:31 +08:00
myhloli 50108c0759 Docs: update Docker deployment instructions for non-NVIDIA devices 2026-09-16 18:45:38 +08:00
myhloli 85632fbc68 Docs: update small models Torch backend requirements to exclude XPU 2026-09-16 18:23:35 +08:00
myhloli 0334e31356 Enhance device compatibility: add support for Intel XPU and update device detection logic 2026-09-16 18:20:43 +08:00
myhloli dc44b4afc1 Docs: non-NVIDIA devices need self-installed accelerated torch plus vllm/lmdeploy; link output-formats page from READMEs and doc indexes (zh/en) 2026-09-16 17:58:04 +08:00
myhloli 97cf0f0bd6 Update docs and READMEs: remove MLX mentions, restructure README files, drop solved font setup
- Remove MLX references from user-facing docs (README, docs/zh, docs/en,
  skills/mineru/SKILL.md); macOS VLM docs now state only the llama.cpp default
- Restructure READMEs: README_en.md becomes the main README.md with the skill
  frontmatter, README_zh.md becomes README_zh-CN.md, and the Agent Guide is
  embedded as a collapsible section in both
- Link DocVortex in feature lists; update Acknowledgments (add DocVortex and
  metafile-render, remove UniMERNet, fast-langdetect, pdftext)
- Remove the MinerU-Diffusion citation entry
- Drop the development-era mineru-kit gradio wording from the 4.0 migration table
- Remove the solved Linux missing-text FAQ entry
- Drop Noto font packages and fc-cache from the 4.0 Dockerfiles (keep libgl1)
2026-09-16 16:36:49 +08:00
myhloli e18edc23c7 Update dependencies in requirements.txt and pyproject.toml; enhance documentation in quick_usage.md and cli_tools.md 2026-09-16 15:48:22 +08:00
myhloli b647f87136 Update documentation and README files for clarity and accuracy 2026-09-16 14:33:09 +08:00
myhloli 53a8bc20c8 Enhance image handling in API client: support HTML image references and externalize assets during save 2026-09-14 19:14:30 +08:00
myhloli f1f212f898 Enhance PDF layout handling and documentation for layout geometry integration 2026-09-14 00:43:59 +08:00
myhloli da88191f01 Update documentation to reflect changes in command names and structure for MinerU 4.0 migration 2026-09-12 16:04:45 +08:00
myhloli b65dbbac97 Add memory benchmarking script and enhance memory management features 2026-09-08 00:11:49 +08:00
myhloli 332bb866cf Update DocVortex dependency version and enhance Middle JSON compatibility handling 2026-09-07 16:37:44 +08:00
myhloli b31d3be7de Integrate DocVortex and preserve MinerU product contracts 2026-09-07 00:43:19 +08:00
myhloli 9b94f41352 test: move native demo coverage to DocGale and isolate Hybrid fixtures 2026-09-06 13:57:29 +08:00
myhloli 82965669bd fix: update page range syntax in documentation and codebase for compatibility with new format 2026-09-04 14:55:14 +08:00
myhloli dcc69e4e9b fix: update page range syntax from '~' to '-' across documentation and codebase 2026-09-04 11:48:37 +08:00
myhloli da5c6e01fe feat: refactor mineru-kit structure and introduce gradio command 2026-09-04 10:51:01 +08:00
myhloli a2ad5e1594 fix: remove unsupported span bbox visualization functionality 2026-09-03 14:59:18 +08:00
jinzhenjandClaude Opus 4.7 0cbbdac083 docs: cover doc/ppt/xls/ofd/html/odf formats in usage and e2e cases
- complete supported-format lists in en/zh quick_usage and cli_tools
- extend e2e fixture setup with doc/ppt/xls/rtf/ofd/epub/html/odt/ods/odp
- add FILETYPE-003/004/005 for page_range, tier and HTML source_context
- update FILETYPE-001 to cover all 14 parseable non-PDF formats

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-31 17:18:26 +08:00
myhloli 160a9b80c4 feat: add HTML parsing support and enhance image handling 2026-08-27 20:53:11 +08:00
myhloli b39de7d54d feat: refine page range handling for PDF and non-PDF inputs, ensuring only PDF supports explicit ranges 2026-08-26 19:50:40 +08:00
myhloli ef26b0fa31 feat: enhance EPUB parsing and rendering for page footnotes 2026-08-26 18:18:20 +08:00
myhloli e8ae64a85a feat: add native EPUB parsing support and update documentation 2026-08-26 15:45:51 +08:00
myhloli c0b61cee9f feat: add native OpenDocument parsing support 2026-08-26 13:17:49 +08:00
myhloli ce729ca48c feat: add native RTF parsing support 2026-08-26 12:28:55 +08:00
myhloli 5dd63a602a feat: add support for CSV file parsing and update documentation 2026-08-26 11:36:21 +08:00
myhloli 41bb5ac2da refactor: reorganize router command structure and improve CLI options 2026-08-26 01:27:00 +08:00
myhloli a343e91891 feat: remove support for low effort analysis and update related documentation 2026-08-25 16:00:10 +08:00
jinzhenjandClaude Opus 4.7 c92e6f8abc feat: support legacy Office binary formats (doc/xls/ppt) end-to-end
myhloli 的 7 个 commit 在 backend/model 层加入 DOC/XLS/PPT 原生解析,
但上层 parser/doclib/filetypes/utils/docs/tests 未同步配套。本次补齐:

- filetypes.py: OFFICE_EXTENSIONS 加入 doc/xls/ppt,补全 MIME/FILE_TYPE 映射,
  删除 LEGACY_OFFICE_EXTENSION_UPGRADES(已原生支持,不再需要"请转换为 .docx"提示)
- parser/base.py: _build_middle_json_from_legacy 补 is_full_document=True
  (修复 1.0 schema 加载必崩的 bug),_LEGACY_DEFAULT_FILE_SUFFIX Literal 扩展
- parser/mineru_parser.py: _FileSuffix/_SUPPORTED_SUFFIXES 扩展,返回类型
  tuple[MiddleJson, Any] → tuple[MiddleJson, ModelJson]
- utils/guess_suffix_or_lang.py: 新增 OLE2 容器识别(D0CF11E0 magic +
  olefile 内部 stream 区分 doc/xls/ppt),插入 OOXML 失败后、Magika 兜底前
- doclib/services/parse_svc.py: page_count 默认值分支扩展到 doc/ppt/xls,
  _unsupported_file_type_message 删除误导性"请转换"提示
- doclib/core/file_io.py: _extract_office_meta 新增 doc/ppt/xls 分支,
  用 olefile 提取 OLE SummaryInformation 属性
- doclib/background/compaction.py: model_list_to_pages → model_json_to_pages
  迁移(myhloli 已删除 model_list_to_pages)
- docs: CLAUDE.md/AGENTS.md/output_files.md/quick_usage.md 更新支持格式列表
- tests: 新增 test_filetypes.py + test_guess_suffix_ole2.py,
  test_parse_result_contract.py 补 is_full_document 断言

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-25 12:12:54 +08:00
jinzhenj d27d0a5e4a Merge branch 'next' into pr-5415-merge 2026-08-25 11:35:20 +08:00
jinzhenj fbf08ebea0 feat: use new backend and render 2026-08-25 11:04:03 +08:00
myhloli 6c23c449d9 feat: update ModelJson handling and enhance document processing with new properties 2026-08-24 02:52:03 +08:00