Commit Graph
6307 Commits
Author SHA1 Message Date
Lubanandmultica-agent 100f1ded56 docs: add CONTRIBUTING.md with verification loop, link from CLAUDE.md
Co-authored-by: multica-agent <github@multica.ai>
2026-09-01 17:30:02 +08:00
jinzhenjandClaude Opus 4.7 317428b8e8 fix: crop Middle JSON 0-1 bbox against rendered image size
crop_pil_image multiplied the normalized bbox by the render scale,
yielding ~2px crops. Map it to the actual rendered image dimensions
instead and drop the now-unused scale parameter from the crop chain.
Rename _normalize_bbox_to_thousandths to _normalize_bbox_to_unit since
it already emits 0-1 coordinates.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-01 09:58:03 +08:00
jinzhenjandClaude Opus 4.7 28d068fac2 refactor: drop unused ParseResult page-index fields
_retained_page_indices and _broken_page_indices had no active consumer;
broken pages are already materialized as empty PageInfo placeholders and
page selection is expressed via is_full_document. Remove the fields,
their serialization keys, and the now-unused _parse_optional_int_list.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-01 09:57:55 +08:00
jinzhenj e6f5e74c17 feat: change default json indent of doclib to 2 2026-09-01 09:18:01 +08:00
jinzhenjandClaude Opus 4.7 4e56adb394 refactor: pin Middle JSON schema version to 2.0
3.0 was never released; use 2.0 as the public version. Rename the
reader helper to _build_middle_json_from_current and update tests and
docs that referenced 3.0. EPUB OPF "version": "3.0" is left intact.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-31 18:26:16 +08:00
jinzhenjandClaude Opus 4.7 28a2c8044e fix: resolve stale paths when locating files by sha256
Selecting the newest active file row whose path still exists fixes
FileNotFoundError after a source file is moved. Applies to parse
execution and source-image rendering; ingest now retires duplicate
rows whose files no longer exist. FTS rebuild keeps any active row
(sorted deterministically) and reads title/author from docs, which
the files table never had.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-31 18:02:56 +08:00
jinzhenjandClaude Opus 4.7 0cbbdac083 docs: cover doc/ppt/xls/ofd/html/odf formats in usage and e2e cases
- complete supported-format lists in en/zh quick_usage and cli_tools
- extend e2e fixture setup with doc/ppt/xls/rtf/ofd/epub/html/odt/ods/odp
- add FILETYPE-003/004/005 for page_range, tier and HTML source_context
- update FILETYPE-001 to cover all 14 parseable non-PDF formats

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-31 17:18:26 +08:00
jinzhenjandClaude Opus 4.7 b9bfa4bda3 fix: accept legacy pdf_info middle_json in api client
Remote V1 APIs can still return MinerU 3.4.5 pdf_info payloads.
Route them through ParseResult.from_dict legacy migration instead of
rejecting, and wrap ValueError as invalid_middle_json_output so unsafe
image sidecar paths surface as API errors.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-31 17:18:18 +08:00
Xiaomeng Zhao e64c8353aa Merge pull request #5440 from myhloli/next
fix: recover internal PDF punctuation during span fill
2026-08-31 10:43:32 +08:00
myhloli d4ede7dff3 fix: recover internal PDF punctuation during span fill 2026-08-30 19:36:18 +08:00
myhloli 0899e76b8c fix: update text_sha256 values in PDF table manifests and improve cell glyph handling 2026-08-30 19:28:11 +08:00
Xiaomeng Zhao 15c0247a31 Merge pull request #5435 from myhloli/next
feat: add OFD document parsing support and update Middle JSON schema to 3.0
2026-08-30 12:39:50 +08:00
myhloli 463dd1354c fix: bound PDF formulas and OFD text decoding 2026-08-30 11:31:45 +08:00
myhloli 5394960cf9 fix: unify native PDF script detection across efforts 2026-08-30 10:17:58 +08:00
myhloli f17da42e2f docs: clarify recoverable OFD template references 2026-08-30 03:26:40 +08:00
myhloli 9cf1921567 fix: bound OFD parsing and standardize table styles 2026-08-30 02:58:21 +08:00
myhloli c61b2f06a4 feat: enhance handling of compound model suffixes in PDF table rendering 2026-08-30 02:03:11 +08:00
myhloli c50995e439 feat: enhance table rendering with geometry-aware HTML serialization for native tables 2026-08-30 01:26:27 +08:00
myhloli b07554ea5a feat: enhance PDF text analysis with formula candidate detection and inline math region handling 2026-08-29 23:19:48 +08:00
myhloli bec1c2c85b fix: bound PDFPageTextGeometry handling in PDF analysis and rendering 2026-08-29 17:07:57 +08:00
myhloli a0879fa6b3 fix: bound OFD document and glyph declarations 2026-08-29 15:16:09 +08:00
myhloli 29f0483ceb fix: bound OFD pages and measure text gaps 2026-08-29 15:00:34 +08:00
myhloli 2ce0ad042d fix: bound OFD glyph mapping and DOCX equation offsets 2026-08-29 13:00:35 +08:00
myhloli eeb59d34da fix: bound OFD and renderer resource handling 2026-08-29 11:41:53 +08:00
myhloli cdc82f9724 fix: bound raster decoding and malformed OFD paths 2026-08-29 04:50:27 +08:00
myhloli 52c31ffe70 feat: implement native script markup detection and handling for superscripts and subscripts 2026-08-29 04:29:50 +08:00
myhloli 2354d5d463 feat: remove mode parameter from DOCX, EPUB, and PDF render options for unified handling 2026-08-29 03:42:16 +08:00
myhloli cd38eb9e2f feat: add EPUB 3.3 and PDF rendering support with strict MiddleJson handling 2026-08-29 03:26:07 +08:00
myhloli ec5210e533 feat: implement rendering for Content List V1 and V2 with strict MiddleJson support 2026-08-29 02:32:50 +08:00
myhloli 5b8f6cf6f2 feat: enhance algorithm rendering with HTML inline syntax and support for additional styles 2026-08-29 02:17:37 +08:00
myhloli 24d16acd77 feat: update Middle JSON schema to 3.0 and enhance document parsing support 2026-08-29 01:43:24 +08:00
myhloli 56d51942c5 feat: implement legacy schema adapter for MinerU 3.4.5 Middle JSON conversion 2026-08-28 23:42:33 +08:00
myhloli b67bbd49c9 feat: add OFD document parsing support and update Middle JSON schema to 3.0 2026-08-28 18:35:53 +08:00
Xiaomeng Zhao 9aa938dc60 Merge pull request #5431 from myhloli/next
feat: add HTML parsing support and enhance image handling
2026-08-28 11:25:02 +08:00
myhloli b5ce7b008f fix: enforce HTML source policies 2026-08-28 11:15:36 +08:00
myhloli d43ba8820b fix: isolate HTML styles and pruning metrics 2026-08-28 10:58:22 +08:00
myhloli 95a92978c3 fix: preserve HTML visibility and URL typing 2026-08-28 10:42:04 +08:00
myhloli ead99aef1d add MinerU HTML v1 wire decoding and materialization logic 2026-08-28 10:21:16 +08:00
myhloli 03c802378b remove outdated principles on document information projection 2026-08-28 09:54:02 +08:00
myhloli 9db13fcfc1 add anchor management for HTML/XHTML documents with visibility and identity policies 2026-08-28 09:39:55 +08:00
myhloli b5d0a1dc55 address latest HTML parser review findings 2026-08-28 09:09:24 +08:00
myhloli 482346ef84 preserve content outside HTML wire root 2026-08-28 08:53:13 +08:00
myhloli 6665024706 avoid dangling structured note anchors 2026-08-28 03:52:44 +08:00
myhloli 58036ae00e precompute html penalties and local links 2026-08-28 03:34:23 +08:00
myhloli ba0ab9059a preserve edited algorithms and remote links 2026-08-28 03:15:59 +08:00
myhloli 703ab7b221 harden html decoding and flowchart wires 2026-08-28 03:00:51 +08:00
myhloli 206c3bb193 harden html selection hot paths 2026-08-28 02:50:17 +08:00
myhloli a0625f6264 validate html table wire bodies 2026-08-28 02:39:39 +08:00
myhloli f620c28110 fix html review edge cases 2026-08-28 02:31:02 +08:00
myhloli 27e8e9c585 fix: validate edited MinerU HTML wire content 2026-08-28 02:13:32 +08:00