Commit Graph
25 Commits
Author SHA1 Message Date
sawmice 87af738ab1 fix: 1. ocr txt mode error 2. lose pdf_parse_type field 2024-12-07 18:03:39 +08:00
icecraft a3a720ea87 refactor: isolate inference and pipeline 2024-12-03 14:06:58 +08:00
myhloli 309be741e8 refactor(txt_parse): improve text extraction accuracy with new algorithm
- Implement new text extraction method (txt_spans_extract_v2) to enhance accuracy
- Add character filling in spans for better text reconstruction
- Introduce empty span handling using OCR for missed text
- Optimize span filtering and overlap removal
2024-11-21 18:08:23 +08:00
icecraft 6a481320ea fix: using new data api replace old rw api 2024-11-18 17:03:53 +08:00
myhloli 1279f2cd0f feat(model): add support for DocLayout-YOLO model
- Add new layout model option: DocLayout-YOLO
- Implement model initialization and prediction for DocLayout-YOLO
- Update configuration options to include new model- Modify existing code to support both LayoutLMv3 and DocLayout-YOLO models
- Update Gradio app to support more Custom Switch
2024-10-23 17:07:39 +08:00
myhloli 6062862c96 feat(pipeline): pass language parameter for parsing and markdown conversion
The pipeline now supports passing the language parameter to parsing functions and
during markdown conversion to optimize processing based on the specified language.
This enhancement allows for more accurate parsing and markdown generation, particularly
when dealing with non-English content.
2024-09-12 12:13:51 +08:00
myhloli ef39b8251c fix(UNIPipe): add language parameter support for document analysis
Introduce a new parameter `lang` to the `UNIPipe` class's `parse_union_pdf` function to allow
specifying the language for document analysis. This enhancement enables users to customizethe language setting for better processing of multilingual documents.
2024-09-09 22:01:59 +08:00
Xiaomeng Zhao 0f91fcf61f feat(cli&analyze&pipeline): add start_page and end_page args for pagination (#507)
* feat(cli&analyze&pipeline): add start_page and end_page args for paginationAdd start_page_id and end_page_id arguments to various components of the PDF parsing
pipeline to support pagination functionality. This feature allows users to specify the
range of pages to be processed, enhancing the efficiency and flexibility of the system.

* feat(cli&analyze&pipeline): add start_page and end_page args for paginationAdd start_page_id and end_page_id arguments to various components of the PDF parsing
pipeline to support pagination functionality. This feature allows users to specify the
range of pages to be processed, enhancing the efficiency and flexibility of the system.

* feat(cli&analyze&pipeline): add start_page and end_page args for paginationAdd start_page_id and end_page_id arguments to various components of the PDF parsing
pipeline to support pagination functionality. This feature allows users to specify the
range of pages to be processed, enhancing the efficiency and flexibility of the system.
2024-08-30 13:59:42 +08:00
赵小蒙 959b8d82d8 renamed pipeline file name 2024-06-25 14:42:43 +08:00
赵小蒙 df14c61f6f update: Enhance the capability to detect garbled document issues 2024-06-19 16:03:21 +08:00
赵小蒙 389826c5fe update custom model framework 2024-06-18 14:45:23 +08:00
赵小蒙 0606301412 make paddle analyze mode adaptation cli input mode to improve analyze speed 2024-06-17 11:37:48 +08:00
赵小蒙 9b5b116369 fix: change garbled_rate 0.1 -> 0.02 2024-06-05 15:21:14 +08:00
赵小蒙 1de37e4c65 add version_name to middle json 2024-06-04 11:15:52 +08:00
赵小蒙 bd1834284e add version_name to middle json 2024-06-03 18:51:38 +08:00
赵小蒙 496045f361 update annotation 2024-05-31 10:27:23 +08:00
赵小蒙 97a4e47319 change garbled rate check from not_common_character_rate to not_printable_rate 2024-05-28 18:22:01 +08:00
赵小蒙 3711a33345 add garbled_rate too large process logic 2024-05-24 15:47:06 +08:00
赵小蒙 7d04ed6e78 add garbled_rate too large process logic 2024-05-24 15:24:11 +08:00
赵小蒙 7f0874de26 change parse_union_pdf error output 2024-04-25 14:04:04 +08:00
许瑞 b16599cdf2 feat: update cli 2024-04-22 18:44:10 +08:00
赵小蒙 dcf6e7127a 将ocr_parse逻辑切换到v2,并解决几个parse过程中的error 2024-04-22 14:29:39 +08:00
kernel.h@qq.com f702defe5a 修正几个常量 2024-04-17 12:06:23 +08:00
kernel.h@qq.com ef03c906a1 io --> rw 2024-04-16 20:50:41 +08:00
赵小蒙 709a65008a 中间态dict结构调整
部分函数重构
2024-04-15 18:51:58 +08:00