opensource/MinerU
3
0
Fork 0
mirror of https://github.com/opendatalab/MinerU.git synced 2026-10-25 17:42:08 +08:00
Code Issues Packages Projects Releases Wiki Activity
6,931 Commits 92 Branches 226 Tags
dev
Commit Graph
53 Commits
Author SHA1 Message Date
myhloli 1399b683b4 refactor(api): simplify task protocol and internalize model output 2026-10-23 16:21:59 +08:00
myhloli bea1f6fac0 feat(tiers): extend file type normalization to include MHTML/MHT and update parsing rules 2026-09-24 01:58:17 +08:00
myhloli 58bd959fc5 refactor: remove latex delimiters option from API and documentation 2026-09-18 15:17:04 +08:00
myhloli 85632fbc68 Docs: update small models Torch backend requirements to exclude XPU 2026-09-16 18:23:35 +08:00
myhloli 0334e31356 Enhance device compatibility: add support for Intel XPU and update device detection logic 2026-09-16 18:20:43 +08:00
myhloli 97cf0f0bd6 Update docs and READMEs: remove MLX mentions, restructure README files, drop solved font setup 
- Remove MLX references from user-facing docs (README, docs/zh, docs/en,
  skills/mineru/SKILL.md); macOS VLM docs now state only the llama.cpp default
- Restructure READMEs: README_en.md becomes the main README.md with the skill
  frontmatter, README_zh.md becomes README_zh-CN.md, and the Agent Guide is
  embedded as a collapsible section in both
- Link DocVortex in feature lists; update Acknowledgments (add DocVortex and
  metafile-render, remove UniMERNet, fast-langdetect, pdftext)
- Remove the MinerU-Diffusion citation entry
- Drop the development-era mineru-kit gradio wording from the 4.0 migration table
- Remove the solved Linux missing-text FAQ entry
- Drop Noto font packages and fc-cache from the 4.0 Dockerfiles (keep libgl1)
 2026-09-16 16:36:49 +08:00
myhloli e18edc23c7 Update dependencies in requirements.txt and pyproject.toml; enhance documentation in quick_usage.md and cli_tools.md 2026-09-16 15:48:22 +08:00
myhloli b647f87136 Update documentation and README files for clarity and accuracy 2026-09-16 14:33:09 +08:00
myhloli da88191f01 Update documentation to reflect changes in command names and structure for MinerU 4.0 migration 2026-09-12 16:04:45 +08:00
myhloli b31d3be7de Integrate DocVortex and preserve MinerU product contracts 2026-09-07 00:43:19 +08:00
myhloli 9b94f41352 test: move native demo coverage to DocGale and isolate Hybrid fixtures 2026-09-06 13:57:29 +08:00
myhloli 82965669bd fix: update page range syntax in documentation and codebase for compatibility with new format 2026-09-04 14:55:14 +08:00
myhloli dcc69e4e9b fix: update page range syntax from '~' to '-' across documentation and codebase 2026-09-04 11:48:37 +08:00
myhloli da5c6e01fe feat: refactor mineru-kit structure and introduce gradio command 2026-09-04 10:51:01 +08:00
jinzhenjandClaude Opus 4.7 0cbbdac083 docs: cover doc/ppt/xls/ofd/html/odf formats in usage and e2e cases 
- complete supported-format lists in en/zh quick_usage and cli_tools
- extend e2e fixture setup with doc/ppt/xls/rtf/ofd/epub/html/odt/ods/odp
- add FILETYPE-003/004/005 for page_range, tier and HTML source_context
- update FILETYPE-001 to cover all 14 parseable non-PDF formats

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-08-31 17:18:26 +08:00
myhloli e8ae64a85a feat: add native EPUB parsing support and update documentation 2026-08-26 15:45:51 +08:00
myhloli c0b61cee9f feat: add native OpenDocument parsing support 2026-08-26 13:17:49 +08:00
myhloli ce729ca48c feat: add native RTF parsing support 2026-08-26 12:28:55 +08:00
myhloli 5dd63a602a feat: add support for CSV file parsing and update documentation 2026-08-26 11:36:21 +08:00
myhloli 41bb5ac2da refactor: reorganize router command structure and improve CLI options 2026-08-26 01:27:00 +08:00
jinzhenjandClaude Opus 4.7 c92e6f8abc feat: support legacy Office binary formats (doc/xls/ppt) end-to-end 
myhloli 的 7 个 commit 在 backend/model 层加入 DOC/XLS/PPT 原生解析，
但上层 parser/doclib/filetypes/utils/docs/tests 未同步配套。本次补齐：

- filetypes.py: OFFICE_EXTENSIONS 加入 doc/xls/ppt，补全 MIME/FILE_TYPE 映射，
  删除 LEGACY_OFFICE_EXTENSION_UPGRADES（已原生支持，不再需要"请转换为 .docx"提示）
- parser/base.py: _build_middle_json_from_legacy 补 is_full_document=True
  （修复 1.0 schema 加载必崩的 bug），_LEGACY_DEFAULT_FILE_SUFFIX Literal 扩展
- parser/mineru_parser.py: _FileSuffix/_SUPPORTED_SUFFIXES 扩展，返回类型
  tuple[MiddleJson, Any] → tuple[MiddleJson, ModelJson]
- utils/guess_suffix_or_lang.py: 新增 OLE2 容器识别（D0CF11E0 magic +
  olefile 内部 stream 区分 doc/xls/ppt），插入 OOXML 失败后、Magika 兜底前
- doclib/services/parse_svc.py: page_count 默认值分支扩展到 doc/ppt/xls，
  _unsupported_file_type_message 删除误导性"请转换"提示
- doclib/core/file_io.py: _extract_office_meta 新增 doc/ppt/xls 分支，
  用 olefile 提取 OLE SummaryInformation 属性
- doclib/background/compaction.py: model_list_to_pages → model_json_to_pages
  迁移（myhloli 已删除 model_list_to_pages）
- docs: CLAUDE.md/AGENTS.md/output_files.md/quick_usage.md 更新支持格式列表
- tests: 新增 test_filetypes.py + test_guess_suffix_ole2.py，
  test_parse_result_contract.py 补 is_full_document 断言

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-08-25 12:12:54 +08:00
jinzhenj d27d0a5e4a Merge branch 'next' into pr-5415-merge 2026-08-25 11:35:20 +08:00
myhloli 6c23c449d9 feat: update ModelJson handling and enhance document processing with new properties 2026-08-24 02:52:03 +08:00
jinzhenj 32d90209bd Merge branch 'next' into pr-5415-merge 2026-08-23 17:56:27 +08:00
myhloli 8fdebbd198 feat: enforce strict paragraph title levels between 2 and 6 and update related tests 2026-08-22 03:01:11 +08:00
jinzhenj 70125ef5fe Refactor model download storage 
Add model repo registry and tier-based model downloads.

Move model source/base_dir handling into config.yaml and remove legacy mineru.json model path handling.

Update runtime model path resolution, Docker/docs, and tests.
 2026-07-13 22:44:35 +08:00
myhloli ec8c462d15 feat: update backend references from pipeline to hybrid and adjust related documentation 2026-07-02 17:55:53 +08:00
myhloli 51b401f2f4 feat: refactor VLM backend handling to unify with hybrid engine and improve compatibility 2026-07-02 11:23:17 +08:00
myhloli 4e9e0a918d feat: update model download command to use mineru-kit and refactor related documentation 2026-06-24 15:49:17 +08:00
myhloli e96faf224e feat: update documentation to include support for PPTX and XLSX file types 2026-04-14 18:52:33 +08:00
myhloli a3b65470cf fix: correct formatting of usage instructions in quick_usage.md 2026-04-01 20:50:07 +08:00
myhloli ce3575ec8b feat: add --enable-vlm-preload option to CLI for VLM model preloading during startup 2026-03-30 22:46:03 +08:00
myhloli 016169ce2e feat: add backend dependency checks for hybrid clients and update documentation 2026-03-30 15:26:03 +08:00
myhloli fb2049f2de feat: update quick_usage.md and README files to include mineru-router command and enhance document clarity 2026-03-29 03:03:28 +08:00
myhloli 42c78c0700 feat: update quick_usage.md to enhance document parsing instructions and add mineru-router usage details 2026-03-29 00:53:17 +08:00
myhloli 40a52da3cf feat: add API protocol version and default processing window size to client 2026-03-25 20:44:46 +08:00
myhloli e4995cfd84 feat: implement synchronous parsing endpoint and enhance task management 2026-03-23 16:11:47 +08:00
myhloli 1d96a77a19 Refactor FastAPI to add asynchronous task upload and download interfaces. 2026-03-19 19:05:54 +08:00
myhloli 984b303dfa refactor: update default backend to hybrid-auto-engine and enhance documentation for parsing options 2025-12-25 19:17:08 +08:00
myhloli 376d1e38d5 fix: update quick_usage.md to clarify support for vllm and lmdeploy acceleration 2025-11-19 19:43:11 +08:00
myhloli 14ca71eed0 docs: enhance quick usage documentation with configuration examples and improve mac environment check 2025-10-31 11:42:37 +08:00
myhloli 8bf407a5e5 docs: update quick_usage.md to format parameter name for clarity 2025-10-31 11:13:36 +08:00
myhloli 59b5ffaf95 docs: update default model in llm-aided-config and clarify enable_thinking parameter usage in quick_usage.md 2025-10-30 17:54:21 +08:00
myhloli a6cc8fa90d fix: update Gradio API access instructions in quick_usage.md 2025-10-22 22:50:36 +08:00
myhloli 2ba1c35fbd update docs 2025-09-19 03:03:06 +08:00
myhloli e47b19c416 update docs 2025-09-19 02:16:17 +08:00
myhloli ec3549702f update docs 2025-09-19 01:55:35 +08:00
myhloli 4864a086ce update docs 2025-09-19 01:48:50 +08:00
myhloli 745954ca08 docs: update references from sglang to vllm in documentation and configuration files 2025-09-15 01:45:35 +08:00
myhloli e120a90d11 docs: update documentation for vllm integration and parameter optimization 2025-09-15 01:25:23 +08:00