112 Commits
Author SHA1 Message Date
myhloli d3dd75821f refactor(api): split task responses and unify file IDs
Generate server-owned file IDs shared by Tasks and result resources. Separate creation, detail, cancellation and upload responses, omitting null fields.

Use POST /v1/tasks/{task_id}/cancel and remove public DELETE endpoints. Update Router, SDK, file_parse recovery, examples and documentation.
2026-10-23 23:46:10 +08:00
myhloli 619765f4d8 refactor(api): unify task file states and response contracts 2026-10-23 19:19:30 +08:00
myhloli 1399b683b4 refactor(api): simplify task protocol and internalize model output 2026-10-23 16:21:59 +08:00
myhloli 28494e633c refactor(api): unify task file fields and generate optional client IDs 2026-10-23 11:53:26 +08:00
myhloli 7f250e2331 feat(api)!: make Tasks the only asynchronous parsing API
Converge the SDK, Gradio, remote CLI, Doclib and Router on Tasks with hash-based uploads, per-file result URLs, task listing and optional model output. Preserve Files and the synchronous file_parse wrapper while removing standalone Uploads/Jobs and local-path HTTP inputs.

Migrate contract, lifecycle, transport and real-document regression coverage; update documentation and HTTP examples. Validation: 1452 passed, 3 conditional skips, plus targeted final PDF checks.

BREAKING CHANGE: clients and workers must use Tasks; standalone Uploads/Jobs endpoints and local-source/inline HTTP options have been removed.
2026-10-22 21:39:42 +08:00
myhloli c666e08dc7 feat(api): unify task uploads and result download links
Accept form manifests and per-file uploads with resumable task state and optional source hashes. Publish result URLs only after final ZIP artifacts are ready, and align Router handling with the same protocol.

Remove synchronous parsing and task result routes, drop fixed local upload limits, and update the HTTP client, documentation, and regression coverage.
2026-10-20 10:48:28 +08:00
myhloli e614751847 feat(api): add V1 task endpoints and balance Router workloads 2026-10-09 19:40:15 +08:00
myhloli bea1f6fac0 feat(tiers): extend file type normalization to include MHTML/MHT and update parsing rules 2026-09-24 01:58:17 +08:00
myhloli 2bbaf6efa8 feat(onnx): unify thread configuration for ONNX sessions with environment variable fallback 2026-09-20 23:11:53 +08:00
myhloli bf8a518d23 Select table ONNX CUDA automatically when available 2026-09-20 16:55:21 +08:00
myhloli 58bd959fc5 refactor: remove latex delimiters option from API and documentation 2026-09-18 15:17:04 +08:00
myhloli 2066dc6273 feat: remove --backend option from mineru-kit parse command and update documentation 2026-09-17 18:44:00 +08:00
myhloli f470d48d14 docs: drop next-branch markers after next-to-master merge 2026-09-17 11:11:31 +08:00
myhloli 85632fbc68 Docs: update small models Torch backend requirements to exclude XPU 2026-09-16 18:23:35 +08:00
myhloli 0334e31356 Enhance device compatibility: add support for Intel XPU and update device detection logic 2026-09-16 18:20:43 +08:00
myhloli 97cf0f0bd6 Update docs and READMEs: remove MLX mentions, restructure README files, drop solved font setup
- Remove MLX references from user-facing docs (README, docs/zh, docs/en,
  skills/mineru/SKILL.md); macOS VLM docs now state only the llama.cpp default
- Restructure READMEs: README_en.md becomes the main README.md with the skill
  frontmatter, README_zh.md becomes README_zh-CN.md, and the Agent Guide is
  embedded as a collapsible section in both
- Link DocVortex in feature lists; update Acknowledgments (add DocVortex and
  metafile-render, remove UniMERNet, fast-langdetect, pdftext)
- Remove the MinerU-Diffusion citation entry
- Drop the development-era mineru-kit gradio wording from the 4.0 migration table
- Remove the solved Linux missing-text FAQ entry
- Drop Noto font packages and fc-cache from the 4.0 Dockerfiles (keep libgl1)
2026-09-16 16:36:49 +08:00
myhloli e18edc23c7 Update dependencies in requirements.txt and pyproject.toml; enhance documentation in quick_usage.md and cli_tools.md 2026-09-16 15:48:22 +08:00
myhloli b647f87136 Update documentation and README files for clarity and accuracy 2026-09-16 14:33:09 +08:00
myhloli da88191f01 Update documentation to reflect changes in command names and structure for MinerU 4.0 migration 2026-09-12 16:04:45 +08:00
myhloli b65dbbac97 Add memory benchmarking script and enhance memory management features 2026-09-08 00:11:49 +08:00
myhloli 332bb866cf Update DocVortex dependency version and enhance Middle JSON compatibility handling 2026-09-07 16:37:44 +08:00
myhloli b31d3be7de Integrate DocVortex and preserve MinerU product contracts 2026-09-07 00:43:19 +08:00
myhloli 9b94f41352 test: move native demo coverage to DocGale and isolate Hybrid fixtures 2026-09-06 13:57:29 +08:00
myhloli 82965669bd fix: update page range syntax in documentation and codebase for compatibility with new format 2026-09-04 14:55:14 +08:00
myhloli dcc69e4e9b fix: update page range syntax from '~' to '-' across documentation and codebase 2026-09-04 11:48:37 +08:00
myhloli da5c6e01fe feat: refactor mineru-kit structure and introduce gradio command 2026-09-04 10:51:01 +08:00
jinzhenjandClaude Opus 4.7 0cbbdac083 docs: cover doc/ppt/xls/ofd/html/odf formats in usage and e2e cases
- complete supported-format lists in en/zh quick_usage and cli_tools
- extend e2e fixture setup with doc/ppt/xls/rtf/ofd/epub/html/odt/ods/odp
- add FILETYPE-003/004/005 for page_range, tier and HTML source_context
- update FILETYPE-001 to cover all 14 parseable non-PDF formats

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-31 17:18:26 +08:00
myhloli b39de7d54d feat: refine page range handling for PDF and non-PDF inputs, ensuring only PDF supports explicit ranges 2026-08-26 19:50:40 +08:00
myhloli ef26b0fa31 feat: enhance EPUB parsing and rendering for page footnotes 2026-08-26 18:18:20 +08:00
myhloli e8ae64a85a feat: add native EPUB parsing support and update documentation 2026-08-26 15:45:51 +08:00
myhloli c0b61cee9f feat: add native OpenDocument parsing support 2026-08-26 13:17:49 +08:00
myhloli ce729ca48c feat: add native RTF parsing support 2026-08-26 12:28:55 +08:00
myhloli 5dd63a602a feat: add support for CSV file parsing and update documentation 2026-08-26 11:36:21 +08:00
myhloli 41bb5ac2da refactor: reorganize router command structure and improve CLI options 2026-08-26 01:27:00 +08:00
jinzhenjandClaude Opus 4.7 c92e6f8abc feat: support legacy Office binary formats (doc/xls/ppt) end-to-end
myhloli 的 7 个 commit 在 backend/model 层加入 DOC/XLS/PPT 原生解析,
但上层 parser/doclib/filetypes/utils/docs/tests 未同步配套。本次补齐:

- filetypes.py: OFFICE_EXTENSIONS 加入 doc/xls/ppt,补全 MIME/FILE_TYPE 映射,
  删除 LEGACY_OFFICE_EXTENSION_UPGRADES(已原生支持,不再需要"请转换为 .docx"提示)
- parser/base.py: _build_middle_json_from_legacy 补 is_full_document=True
  (修复 1.0 schema 加载必崩的 bug),_LEGACY_DEFAULT_FILE_SUFFIX Literal 扩展
- parser/mineru_parser.py: _FileSuffix/_SUPPORTED_SUFFIXES 扩展,返回类型
  tuple[MiddleJson, Any] → tuple[MiddleJson, ModelJson]
- utils/guess_suffix_or_lang.py: 新增 OLE2 容器识别(D0CF11E0 magic +
  olefile 内部 stream 区分 doc/xls/ppt),插入 OOXML 失败后、Magika 兜底前
- doclib/services/parse_svc.py: page_count 默认值分支扩展到 doc/ppt/xls,
  _unsupported_file_type_message 删除误导性"请转换"提示
- doclib/core/file_io.py: _extract_office_meta 新增 doc/ppt/xls 分支,
  用 olefile 提取 OLE SummaryInformation 属性
- doclib/background/compaction.py: model_list_to_pages → model_json_to_pages
  迁移(myhloli 已删除 model_list_to_pages)
- docs: CLAUDE.md/AGENTS.md/output_files.md/quick_usage.md 更新支持格式列表
- tests: 新增 test_filetypes.py + test_guess_suffix_ole2.py,
  test_parse_result_contract.py 补 is_full_document 断言

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-25 12:12:54 +08:00
jinzhenj d27d0a5e4a Merge branch 'next' into pr-5415-merge 2026-08-25 11:35:20 +08:00
myhloli 6c23c449d9 feat: update ModelJson handling and enhance document processing with new properties 2026-08-24 02:52:03 +08:00
jinzhenj 32d90209bd Merge branch 'next' into pr-5415-merge 2026-08-23 17:56:27 +08:00
myhloli 8fdebbd198 feat: enforce strict paragraph title levels between 2 and 6 and update related tests 2026-08-22 03:01:11 +08:00
myhloli c0165ac492 feat: enhance table merging functionality by refining cross-page detection and improving metadata handling 2026-08-12 23:54:52 +08:00
jinzhenj 57aa965d41 refactor: rename parsing tiers 2026-07-21 12:28:30 +08:00
jinzhenj 468848b705 Add model download completion markers
Refs #5273
2026-07-17 13:46:06 +08:00
jinzhenj 528ea04375 Refine model download required paths 2026-07-14 11:51:38 +08:00
jinzhenj 70125ef5fe Refactor model download storage
Add model repo registry and tier-based model downloads.

Move model source/base_dir handling into config.yaml and remove legacy mineru.json model path handling.

Update runtime model path resolution, Docker/docs, and tests.
2026-07-13 22:44:35 +08:00
myhloli 642ba0b177 feat: remove legacy formula and table options from CLI and API 2026-07-07 01:23:27 +08:00
myhloli ec8c462d15 feat: update backend references from pipeline to hybrid and adjust related documentation 2026-07-02 17:55:53 +08:00
myhloli 36f656fc96 feat: update hybrid backend effort levels to include low option and adjust related descriptions 2026-07-02 16:50:43 +08:00
myhloli 51b401f2f4 feat: refactor VLM backend handling to unify with hybrid engine and improve compatibility 2026-07-02 11:23:17 +08:00
myhloli 1703b3cd95 feat: update model source accessibility checks and improve documentation links 2026-07-01 11:24:08 +08:00
myhloli f1c87edaa2 feat: update model source handling to support auto detection and improve configuration management 2026-07-01 02:15:13 +08:00