opensource/MinerU
3
0
Fork 0
mirror of https://github.com/opendatalab/MinerU.git synced 2026-09-21 12:42:22 +08:00
Code Issues Packages Projects Releases Wiki Activity
2,068 Commits 90 Branches 219 Tags
0281048d83a97daceba802b7a33e703daec5cc77
Commit Graph
2068 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Xiaomeng Zhao 0281048d83 Merge pull request #1338 from myhloli/dev 
refactor(pre_proc): improve character overlap handling in spans
 2024-12-20 17:36:10 +08:00
myhloli 24dfd1a0cf Merge remote-tracking branch 'origin/dev' into dev 2024-12-20 17:34:29 +08:00
myhloli 15e876677d refactor(pre_proc): improve character overlap handling in spans 
- Remove remove_overlaps_chars function
- Add check_chars_is_overlap_in_span function
- Update span processing logic to handle character overlaps- Improve efficiency and readability of overlap detection
 2024-12-20 17:33:50 +08:00
Xiaomeng Zhao 58b2e78dd3 Merge pull request #1336 from icecraft/docs/add_more_method_usage 
docs: add more method description
 2024-12-20 17:10:35 +08:00
xu rui 24ee9c41e2 docs: add more method description 2024-12-20 15:41:00 +08:00
Xiaomeng Zhao a9dea5f0f8 Merge pull request #1330 from myhloli/dev 
feat(demo): add demo script for PDF processing
 2024-12-19 18:24:37 +08:00
myhloli d6a291623b feat(demo): add demo script for PDF processing 
- Create demo.py script for PDF file processing
- Implement PDF reading, classification, and inference usingOpendatalab's magic_pdf library- Add pipelines for OCR and text modes
- Include result visualization and markdown export
 2024-12-19 18:23:52 +08:00
Xiaomeng Zhao 5eb9feee11 Merge pull request #1329 from myhloli/dev 
feat(pre_proc): add function to remove overlapping characters in spans
 2024-12-19 15:22:51 +08:00
myhloli 2f4d4b0c80 feat(pre_proc): add function to remove overlapping characters in spans 
- Implement remove_overlaps_chars function to detect and remove overlapping characters within spans
- Integrate remove_overlaps_chars function into the PDF parsing process
- Improve character-level processing and reduce redundancy in OCR results
 2024-12-19 15:20:51 +08:00
Xiaomeng Zhao 7248676d8f Merge pull request #1325 from pangguosheng1106/dev 
fix: skip the char corresponding to invalid bounding boxes
 2024-12-19 14:46:51 +08:00
pangguosheng 1e6de549a8 fix: drop reason append error 2024-12-19 01:10:59 +08:00
pangguosheng 51b8c57df2 fix: skip the char corresponding to invalid bounding boxes 2024-12-19 00:35:44 +08:00
Xiaomeng Zhao b71993a97f Merge pull request #1323 from myhloli/dev 
feat(gradio-app): improve PDF conversion and UI functionalities
 2024-12-18 19:08:18 +08:00
myhloli bf2ff5a241 feat(gradio-app): improve PDF conversion and UI functionalities 
- Add automatic conversion of uploaded files to PDF
- Update max page slider range and default value- Prevent interaction with PDF preview to avoid errors
- Increase Markdown rendering height for better visibility
- Update file change event handling for PDF conversion
- Modify supported image suffixes for file upload
 2024-12-18 19:07:15 +08:00
Xiaomeng Zhao e1d69928ba Merge pull request #1322 from myhloli/dev 
refactor(magic_pdf): move model config variables
 2024-12-18 17:26:45 +08:00
myhloli 489f70e91d refactor(magic_pdf): move model config variables 
- Move __use_inside_model__ and __model_mode__ from operators/__init__.py to model/__init__.py
- These variables are more appropriately located in the model module since they relate to model configuration
 2024-12-18 17:26:06 +08:00
Xiaomeng Zhao 34cd1350a9 Merge pull request #1321 from icecraft/docs/fix_docs 
Docs/fix docs
 2024-12-18 15:36:17 +08:00
icecraft c968ce860d fix: remove pipe_auto_method 2024-12-18 15:33:41 +08:00
xu rui cd11ddcd6b docs: make sure the generate process of docs work properly 2024-12-18 15:22:56 +08:00
Xiaomeng Zhao d94ddcf81c Merge pull request #1318 from icecraft/refactor/code_struture 
refactor: refactor code
 2024-12-18 14:30:22 +08:00
icecraft b2887ca0aa refactor: refactor code 2024-12-18 14:19:41 +08:00
Xiaomeng Zhao 303a4b01ec Merge pull request #1313 from icecraft/fix/abs_pipe_dev 2024-12-17 19:25:13 +08:00
icecraft 78f56a1ed0 fix: AbsPipe initial method 2024-12-17 19:22:54 +08:00
Xiaomeng Zhao 3f64c16ba9 Merge pull request #1311 from myhloli/add-auto-lang 
feat(language-detection): add YOLOv11 language detection model
 2024-12-17 19:11:21 +08:00
myhloli 20438bd2b7 feat(language-detection): add YOLOv11 language detection model 
- Add YOLOv11 language detection model for PDF documents
- Implement language detection in PymuDocDataset
- Update app.py to include 'auto' language option
- Create language detection utilities and constants
 2024-12-17 19:06:50 +08:00
Xiaomeng Zhao fb6db2d9cf Merge pull request #1304 from icecraft/feat/add_get_middle_json 
feat: add get_middle_json method
 2024-12-17 14:13:13 +08:00
icecraft e9d36221dd feat: add get_middle_json method 2024-12-17 10:46:17 +08:00
Xiaomeng Zhao 51fd53d9af Merge pull request #1301 from myhloli/add-auto-lang 
refactor(magic_pdf): remove YOLO_VERBOSE setting and update YOLOv8 prediction verbosity
 2024-12-16 17:28:37 +08:00
myhloli 9e4ebea939 refactor(magic_pdf): remove YOLO_VERBOSE setting and update YOLOv8 prediction verbosity 
- Remove YOLO_VERBOSE environment variable from multiple files
- Set verbose=False in YOLOv8 prediction method to suppress logger output
 2024-12-16 17:27:19 +08:00
Xiaomeng Zhao 0d5a1c74ad Merge pull request #1292 from dt-yy/dev 2024-12-14 01:11:17 +08:00
dt-yy 905c4ae5aa update case 2024-12-13 19:05:37 +08:00
dt-yy 97e11346e5 update test case 2024-12-13 18:59:21 +08:00
dt-yy 54c23a5351 update test case 2024-12-13 18:58:10 +08:00
Xiaomeng Zhao 379cf15013 Merge pull request #1291 from myhloli/add-llm-aided 
fix(pdf): improve ligature handling and text extraction
 2024-12-13 15:44:19 +08:00
myhloli c638fc5d1f fix(pdf): improve ligature handling and text extraction 
- Move ligature replacement function to pdf_parse_union_core_v2.py
- Optimize ligature replacement using a more efficient approach
- Modify text extraction flags to preserve ligatures in PDF content
- Remove unnecessary function from ocr_mkcontent.py
 2024-12-13 15:43:05 +08:00
Xiaomeng Zhao 5e3890f560 Merge pull request #1289 from icecraft/docs/fix_image_path 
docs: fix image path
 2024-12-13 11:31:47 +08:00
xu rui 3e734d6c0a docs: fix image path 2024-12-13 11:21:38 +08:00
Xiaomeng Zhao 55e9bb9b94 Merge pull request #1284 from IMSUVEN/dev 2024-12-13 09:36:30 +08:00
Suven be01039495 feat: add logging for detection time in BatchAnalyze when OCR is not applied 2024-12-13 08:37:11 +08:00
Suven 49bfdf07c8 feat: enhance batch processing in BatchAnalyze with layout and OCR timing logs 2024-12-13 08:25:23 +08:00
Suven 4fd1e41e0e fix: batch methods in DocLayoutYOLO and YOLOv8 models 2024-12-12 18:31:01 +08:00
Xiaomeng Zhao 8ccfff6fd5 Merge pull request #1281 from IMSUVEN/dev 
feat: add batch prediction methods for YOLOv8 and Unimernet models
 2024-12-12 17:13:39 +08:00
Xiaomeng Zhao d0a3058ba8 Merge pull request #1280 from icecraft/docs/tune_docs 
Docs/tune docs
 2024-12-12 17:04:42 +08:00
xu rui f49d261d4f docs: assume image can not using **txt** mode 2024-12-12 17:01:47 +08:00
xu rui b7a65e890a docs: update docs 2024-12-12 16:58:14 +08:00
Suven 7ce9edc674 feat: add batch prediction methods for YOLOv8 and Unimernet models 2024-12-12 16:55:49 +08:00
icecraft 440fd0c75b fix: projects 2024-12-12 16:52:27 +08:00
xu rui 6750b53d20 feat: 1.update demos, 2.update docs 2024-12-12 15:05:59 +08:00
Xiaomeng Zhao 91aa7e5685 Merge pull request #1277 from opendatalab/master 
remove old demo code
 2024-12-12 14:51:43 +08:00
Xiaomeng Zhao c65c0478cd Update README_zh-CN.md 2024-12-12 14:46:24 +08:00