opensource/MinerU
3
0
Fork 0
mirror of https://github.com/opendatalab/MinerU.git synced 2026-10-25 09:35:17 +08:00
Code Issues Packages Projects Releases Wiki Activity
6,686 Commits 91 Branches 226 Tags
next
Commit Graph
304 Commits
Author SHA1 Message Date
myhloli f470d48d14 docs: drop next-branch markers after next-to-master merge 2026-09-17 11:11:31 +08:00
myhloli 50108c0759 Docs: update Docker deployment instructions for non-NVIDIA devices 2026-09-16 18:45:38 +08:00
myhloli 85632fbc68 Docs: update small models Torch backend requirements to exclude XPU 2026-09-16 18:23:35 +08:00
myhloli 0334e31356 Enhance device compatibility: add support for Intel XPU and update device detection logic 2026-09-16 18:20:43 +08:00
myhloli dc44b4afc1 Docs: non-NVIDIA devices need self-installed accelerated torch plus vllm/lmdeploy; link output-formats page from READMEs and doc indexes (zh/en) 2026-09-16 17:58:04 +08:00
myhloli 97cf0f0bd6 Update docs and READMEs: remove MLX mentions, restructure README files, drop solved font setup 
- Remove MLX references from user-facing docs (README, docs/zh, docs/en,
  skills/mineru/SKILL.md); macOS VLM docs now state only the llama.cpp default
- Restructure READMEs: README_en.md becomes the main README.md with the skill
  frontmatter, README_zh.md becomes README_zh-CN.md, and the Agent Guide is
  embedded as a collapsible section in both
- Link DocVortex in feature lists; update Acknowledgments (add DocVortex and
  metafile-render, remove UniMERNet, fast-langdetect, pdftext)
- Remove the MinerU-Diffusion citation entry
- Drop the development-era mineru-kit gradio wording from the 4.0 migration table
- Remove the solved Linux missing-text FAQ entry
- Drop Noto font packages and fc-cache from the 4.0 Dockerfiles (keep libgl1)
 2026-09-16 16:36:49 +08:00
myhloli e18edc23c7 Update dependencies in requirements.txt and pyproject.toml; enhance documentation in quick_usage.md and cli_tools.md 2026-09-16 15:48:22 +08:00
myhloli b647f87136 Update documentation and README files for clarity and accuracy 2026-09-16 14:33:09 +08:00
myhloli 53a8bc20c8 Enhance image handling in API client: support HTML image references and externalize assets during save 2026-09-14 19:14:30 +08:00
myhloli f1f212f898 Enhance PDF layout handling and documentation for layout geometry integration 2026-09-14 00:43:59 +08:00
myhloli da88191f01 Update documentation to reflect changes in command names and structure for MinerU 4.0 migration 2026-09-12 16:04:45 +08:00
myhloli b65dbbac97 Add memory benchmarking script and enhance memory management features 2026-09-08 00:11:49 +08:00
myhloli 332bb866cf Update DocVortex dependency version and enhance Middle JSON compatibility handling 2026-09-07 16:37:44 +08:00
myhloli b31d3be7de Integrate DocVortex and preserve MinerU product contracts 2026-09-07 00:43:19 +08:00
myhloli 9b94f41352 test: move native demo coverage to DocGale and isolate Hybrid fixtures 2026-09-06 13:57:29 +08:00
myhloli 82965669bd fix: update page range syntax in documentation and codebase for compatibility with new format 2026-09-04 14:55:14 +08:00
myhloli dcc69e4e9b fix: update page range syntax from '~' to '-' across documentation and codebase 2026-09-04 11:48:37 +08:00
myhloli da5c6e01fe feat: refactor mineru-kit structure and introduce gradio command 2026-09-04 10:51:01 +08:00
myhloli a2ad5e1594 fix: remove unsupported span bbox visualization functionality 2026-09-03 14:59:18 +08:00
jinzhenjandClaude Opus 4.7 0cbbdac083 docs: cover doc/ppt/xls/ofd/html/odf formats in usage and e2e cases 
- complete supported-format lists in en/zh quick_usage and cli_tools
- extend e2e fixture setup with doc/ppt/xls/rtf/ofd/epub/html/odt/ods/odp
- add FILETYPE-003/004/005 for page_range, tier and HTML source_context
- update FILETYPE-001 to cover all 14 parseable non-PDF formats

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-08-31 17:18:26 +08:00
myhloli b67bbd49c9 feat: add OFD document parsing support and update Middle JSON schema to 3.0 2026-08-28 18:35:53 +08:00
myhloli 160a9b80c4 feat: add HTML parsing support and enhance image handling 2026-08-27 20:53:11 +08:00
myhloli b39de7d54d feat: refine page range handling for PDF and non-PDF inputs, ensuring only PDF supports explicit ranges 2026-08-26 19:50:40 +08:00
myhloli ef26b0fa31 feat: enhance EPUB parsing and rendering for page footnotes 2026-08-26 18:18:20 +08:00
myhloli e8ae64a85a feat: add native EPUB parsing support and update documentation 2026-08-26 15:45:51 +08:00
myhloli c0b61cee9f feat: add native OpenDocument parsing support 2026-08-26 13:17:49 +08:00
myhloli ce729ca48c feat: add native RTF parsing support 2026-08-26 12:28:55 +08:00
myhloli 5dd63a602a feat: add support for CSV file parsing and update documentation 2026-08-26 11:36:21 +08:00
myhloli 41bb5ac2da refactor: reorganize router command structure and improve CLI options 2026-08-26 01:27:00 +08:00
myhloli a343e91891 feat: remove support for low effort analysis and update related documentation 2026-08-25 16:00:10 +08:00
jinzhenjandClaude Opus 4.7 c92e6f8abc feat: support legacy Office binary formats (doc/xls/ppt) end-to-end 
myhloli 的 7 个 commit 在 backend/model 层加入 DOC/XLS/PPT 原生解析，
但上层 parser/doclib/filetypes/utils/docs/tests 未同步配套。本次补齐：

- filetypes.py: OFFICE_EXTENSIONS 加入 doc/xls/ppt，补全 MIME/FILE_TYPE 映射，
  删除 LEGACY_OFFICE_EXTENSION_UPGRADES（已原生支持，不再需要"请转换为 .docx"提示）
- parser/base.py: _build_middle_json_from_legacy 补 is_full_document=True
  （修复 1.0 schema 加载必崩的 bug），_LEGACY_DEFAULT_FILE_SUFFIX Literal 扩展
- parser/mineru_parser.py: _FileSuffix/_SUPPORTED_SUFFIXES 扩展，返回类型
  tuple[MiddleJson, Any] → tuple[MiddleJson, ModelJson]
- utils/guess_suffix_or_lang.py: 新增 OLE2 容器识别（D0CF11E0 magic +
  olefile 内部 stream 区分 doc/xls/ppt），插入 OOXML 失败后、Magika 兜底前
- doclib/services/parse_svc.py: page_count 默认值分支扩展到 doc/ppt/xls，
  _unsupported_file_type_message 删除误导性"请转换"提示
- doclib/core/file_io.py: _extract_office_meta 新增 doc/ppt/xls 分支，
  用 olefile 提取 OLE SummaryInformation 属性
- doclib/background/compaction.py: model_list_to_pages → model_json_to_pages
  迁移（myhloli 已删除 model_list_to_pages）
- docs: CLAUDE.md/AGENTS.md/output_files.md/quick_usage.md 更新支持格式列表
- tests: 新增 test_filetypes.py + test_guess_suffix_ole2.py，
  test_parse_result_contract.py 补 is_full_document 断言

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-08-25 12:12:54 +08:00
jinzhenj d27d0a5e4a Merge branch 'next' into pr-5415-merge 2026-08-25 11:35:20 +08:00
jinzhenj fbf08ebea0 feat: use new backend and render 2026-08-25 11:04:03 +08:00
myhloli 6c23c449d9 feat: update ModelJson handling and enhance document processing with new properties 2026-08-24 02:52:03 +08:00
jinzhenj 32d90209bd Merge branch 'next' into pr-5415-merge 2026-08-23 17:56:27 +08:00
myhloli 8fdebbd198 feat: enforce strict paragraph title levels between 2 and 6 and update related tests 2026-08-22 03:01:11 +08:00
myhloli c0165ac492 feat: enhance table merging functionality by refining cross-page detection and improving metadata handling 2026-08-12 23:54:52 +08:00
jinzhenj d138d20170 build: enable hf-xet model downloads 2026-07-21 18:24:46 +08:00
jinzhenj 57aa965d41 refactor: rename parsing tiers 2026-07-21 12:28:30 +08:00
jinzhenj 468848b705 Add model download completion markers 
Refs #5273
 2026-07-17 13:46:06 +08:00
jinzhenj 528ea04375 Refine model download required paths 2026-07-14 11:51:38 +08:00
jinzhenj 70125ef5fe Refactor model download storage 
Add model repo registry and tier-based model downloads.

Move model source/base_dir handling into config.yaml and remove legacy mineru.json model path handling.

Update runtime model path resolution, Docker/docs, and tests.
 2026-07-13 22:44:35 +08:00
jinzhenj 10ea049abe feat: rearrange skill 2026-07-10 15:35:53 +08:00
jinzhenj a43d380d91 Merge remote-tracking branch 'opendatalab/next' into next 2026-07-08 12:09:52 +08:00
myhloli 642ba0b177 feat: remove legacy formula and table options from CLI and API 2026-07-07 01:23:27 +08:00
jinzhenj 0635bc7fc6 feat: remove data writer, refactor vlm server. 2026-07-06 11:33:39 +08:00
myhloli ec8c462d15 feat: update backend references from pipeline to hybrid and adjust related documentation 2026-07-02 17:55:53 +08:00
myhloli 36f656fc96 feat: update hybrid backend effort levels to include low option and adjust related descriptions 2026-07-02 16:50:43 +08:00
myhloli 51b401f2f4 feat: refactor VLM backend handling to unify with hybrid engine and improve compatibility 2026-07-02 11:23:17 +08:00
myhloli 1703b3cd95 feat: update model source accessibility checks and improve documentation links 2026-07-01 11:24:08 +08:00