Commit Graph
45 Commits
Author SHA1 Message Date
myhloli 31553d29ea ci: add PR gates and separate optional backend tests 2026-10-24 01:16:43 +08:00
myhloli 28494e633c refactor(api): unify task file fields and generate optional client IDs 2026-10-23 11:53:26 +08:00
myhloli 7f250e2331 feat(api)!: make Tasks the only asynchronous parsing API
Converge the SDK, Gradio, remote CLI, Doclib and Router on Tasks with hash-based uploads, per-file result URLs, task listing and optional model output. Preserve Files and the synchronous file_parse wrapper while removing standalone Uploads/Jobs and local-path HTTP inputs.

Migrate contract, lifecycle, transport and real-document regression coverage; update documentation and HTTP examples. Validation: 1452 passed, 3 conditional skips, plus targeted final PDF checks.

BREAKING CHANGE: clients and workers must use Tasks; standalone Uploads/Jobs endpoints and local-source/inline HTTP options have been removed.
2026-10-22 21:39:42 +08:00
myhloli 92613d738d chore: align Python 3.11-3.14 support and release checks 2026-10-22 21:06:05 +08:00
myhloli f59acf1ffa feat(api): add recoverable synchronous file parsing service 2026-10-22 18:42:54 +08:00
myhloli 57485941fb feat(api): process JSON tasks and deliver per-file outputs 2026-10-22 18:41:58 +08:00
Xiaomeng Zhao 354566d8af feat(onnx): add adaptive calibration and Intel WebGPU architecture policy 2026-10-21 23:17:57 +08:00
Xiaomeng Zhao a2a4960b08 feat(onnx): add Linux WebGPU and compact OCR recognition outputs 2026-10-21 18:56:45 +08:00
myhloli c666e08dc7 feat(api): unify task uploads and result download links
Accept form manifests and per-file uploads with resumable task state and optional source hashes. Publish result URLs only after final ZIP artifacts are ready, and align Router handling with the same protocol.

Remove synchronous parsing and task result routes, drop fixed local upload limits, and update the HTTP client, documentation, and regression coverage.
2026-10-20 10:48:28 +08:00
myhloli 48524bfabb feat(onnx): automatically select CPU or DirectML devices
Add auto/cpu/gpu configuration, Windows adapter discovery, isolated rec calibration, atomic preference persistence, and vendor session presets. Route layout, rec, and full MFR batches through DirectML with CPU fallback.

Use mutually exclusive Windows x64 DirectML dependencies and guard ORT/Magika distribution identity. Include calibration fixtures, three-machine results, the 0.8 threshold with ten UHD 770 repeats, and installation documentation.

Validation: 762 MinerU tests and 71 DocVortex tests passed; dependency checks and Windows Python 3.10-3.14 resolution passed. The 19-page OCR result matches the CPU baseline. DocVortex PyPI acceptance remains pending a compatible release; pinned source acceptance passed.
2026-10-19 11:02:29 +08:00
myhloli 77698670a5 feat: update issue template and sync published release version 2026-10-11 18:40:17 +08:00
myhloli e614751847 feat(api): add V1 task endpoints and balance Router workloads 2026-10-09 19:40:15 +08:00
myhloli e3d7addafe feat: add support for docker repository flags and enhance alias handling in command-line arguments 2026-09-18 17:24:43 +08:00
myhloli e18edc23c7 Update dependencies in requirements.txt and pyproject.toml; enhance documentation in quick_usage.md and cli_tools.md 2026-09-16 15:48:22 +08:00
myhloli 6881cadedc Refactor PDF analysis to support native asynchronous execution and enhance resource management 2026-09-12 04:59:17 +08:00
myhloli 349f94e294 Update packaging documentation to reflect changes in backend installation and clarify dependency management 2026-09-11 18:52:50 +08:00
myhloli 37bc36e297 Refactor model configuration to replace 'stack' with 'small_backend' and 'vlm.engine' for improved backend selection 2026-09-11 18:38:41 +08:00
myhloli 74dc0fac70 Add shared OCR resources and update model stack references for ONNX and Torch 2026-09-11 00:56:45 +08:00
myhloli a63acd74f2 Validate resolved dependency Python metadata independently 2026-09-08 23:33:31 +08:00
myhloli e8a3fe1d37 Merge Transformers 5 migration and integrate Python 3.14 document and backend support 2026-09-08 23:05:45 +08:00
myhloli 73d16e6be7 Add new SVG icons for toolbar buttons and update Gradio header styles 2026-09-08 10:27:44 +08:00
myhloli 74c6db5a49 Upgrade inference stack to Transformers 5.10+ 2026-09-08 06:24:16 +08:00
myhloli b65dbbac97 Add memory benchmarking script and enhance memory management features 2026-09-08 00:11:49 +08:00
myhloli f1c9546985 refactor: adopt DocGale native parsing and shared document pipeline 2026-09-06 03:52:53 +08:00
myhloli d9b0c46e4c feat: implement semantic fingerprinting and geometry summary for Flash PDF review 2026-09-03 15:09:02 +08:00
myhloli a565806c34 fix: canonicalize DOCX anchors and harden review script 2026-09-03 13:42:44 +08:00
myhloli 3942ce08be fix: preserve document ordering and review invariants 2026-09-03 03:04:29 +08:00
myhloli 91eaf879da feat: enhance document geometry analysis with layout and style risk detection 2026-09-03 00:24:25 +08:00
myhloli 156b70c98f fix: update PDF fixture handling to support encrypted samples and improve geometry validation 2026-09-01 19:43:07 +08:00
myhloli 448ad54050 fix: stabilize Flash layout with dual bbox geometry 2026-08-31 22:04:21 +08:00
myhloli c50995e439 feat: enhance table rendering with geometry-aware HTML serialization for native tables 2026-08-30 01:26:27 +08:00
myhloli b07554ea5a feat: enhance PDF text analysis with formula candidate detection and inline math region handling 2026-08-29 23:19:48 +08:00
myhloli 0904e810f1 feat: improve CSS minification to preserve descendant combinators before pseudo-classes 2026-08-20 15:23:18 +08:00
myhloli ad3dbb1688 feat: implement CSS minification and build script for MinerU HTML rendering 2026-08-20 11:49:29 +08:00
myhloli cf5c8f47f4 docs: remove outdated documentation files
- Deleted .readthedocs.yaml files from multiple directories
- Removed outdated API and user guide documentation files
- Deleted command line usage examples
- Removed CUDA acceleration guide
2025-06-13 19:11:55 +08:00
myhloli 3bd1e0e46f refactor: comment out paddleocr model copying code
- Commented out the code that copies the paddleocr model to user directory
- This change affects both download_models.py and download_models_hf.py scripts
2025-04-02 15:56:12 +08:00
myhloli 5237a3851c fix(scripts): update model download scripts for OCR
- Update download_models.py and download_models_hf.py scripts
- Change OCR model path from paddleocr to paddleocr_torch
2025-04-02 15:55:27 +08:00
myhloli 6e35e382df feat(models): add PaddleOCR model and update download scripts
- Add PaddleOCR model to the list of models to download
- Update download_models.py and download_models_hf.py scripts to include PaddleOCR
- Create user directory for PaddleOCR and copy downloaded models
2025-03-20 15:19:03 +08:00
myhloli 2f3b66a5d0 refactor(model): update model initialization and dependencies
- Update config version to1.2.0
- Refactor model initialization in model_init.py- Update dependencies in requirements.txt files
- Remove unused imports and models
- Add conditional imports for table models
2025-03-20 11:45:32 +08:00
myhloli 2a3a006f4d fix(models): update unimernet_small model path
- Update model path from 'unimernet_small' to 'unimernet_small_2501' in multiple scripts and configuration files
- This change affects download_models.py, download_models_hf.py, and model_configs.yaml
2025-01-21 16:29:29 +08:00
myhloli 59502e5290 refactor(model): update config version check to 1.1.1
- Update the version check in download_models.py and download_models_hf.py
- Change the threshold from '1.1.0' to '1.1.1' for model configuration updates
2025-01-17 18:15:17 +08:00
myhloli a1c4f22a8c fix: update resource URLs to jsdelivr
- Replace Gitee URLs with jsdelivr URLs in multiple files
- Update Dockerfile instructions to use new URLs
- Modify documentation to reflect new resource location
- Adjust scripts and READMEs to point to new URL for downloads
2025-01-12 03:43:25 +08:00
myhloli c660fdc8f0 feat(llm): add LLM-aided formula and text correction
- Add LLM-aided formula and text correction functionality
- Update config reader to include LLM-aided settings
- Create new LLM-aided processing module
- Update main processing script to incorporate LLM-aided corrections
- Modify download scripts to check for new config version
2024-12-24 14:33:35 +08:00
myhloli 9496c6c4cb refactor(model download script)
- Remove the download script from the `docs` directory and unify it in the `scripts` directory.
- Update the JSON template URL in the script to the master branch.
- Modify the script download link provided in the documentation.
- Update the table recognition model in the configuration template to `rapid_table`.
2024-11-11 15:10:38 +08:00
xu rui 065bf993ec fix: download model file scripts path 2024-11-01 15:16:03 +08:00