Commit Graph
33 Commits
Author SHA1 Message Date
xu rui f6bd47de6a docs: add dataset method description 2024-12-03 14:06:58 +08:00
icecraft 4a82d6a07a feat: add function definitions 2024-12-03 14:06:58 +08:00
icecraft a3a720ea87 refactor: isolate inference and pipeline 2024-12-03 14:06:58 +08:00
icecraft b492c19c4c refactor: move some constants or enums defs to config folder 2024-11-19 16:56:53 +08:00
icecraft 6a481320ea fix: using new data api replace old rw api 2024-11-18 17:03:53 +08:00
myhloli 1279f2cd0f feat(model): add support for DocLayout-YOLO model
- Add new layout model option: DocLayout-YOLO
- Implement model initialization and prediction for DocLayout-YOLO
- Update configuration options to include new model- Modify existing code to support both LayoutLMv3 and DocLayout-YOLO models
- Update Gradio app to support more Custom Switch
2024-10-23 17:07:39 +08:00
myhloli fc49f5c446 refactor(magic_pdf): remove unused parameters and simplify functions
- Remove unused parameters parse_type and lang from various functions
- Simplify function calls by removing unnecessary arguments
- Update related files to reflect these changes
2024-10-18 18:51:45 +08:00
myhloli 23b621e05a feat(UNIPipe): change default drop_mode to NONE_WITH_REASON 2024-09-18 17:43:16 +08:00
myhloli 6062862c96 feat(pipeline): pass language parameter for parsing and markdown conversion
The pipeline now supports passing the language parameter to parsing functions and
during markdown conversion to optimize processing based on the specified language.
This enhancement allows for more accurate parsing and markdown generation, particularly
when dealing with non-English content.
2024-09-12 12:13:51 +08:00
myhloli ef39b8251c fix(UNIPipe): add language parameter support for document analysis
Introduce a new parameter `lang` to the `UNIPipe` class's `parse_union_pdf` function to allow
specifying the language for document analysis. This enhancement enables users to customizethe language setting for better processing of multilingual documents.
2024-09-09 22:01:59 +08:00
myhloli 4b372f3f7e feat(ocr): pass language parameter for custom model init
Pass the `lang` parameter to `custom_model_init` in `doc_analyze` to support language-specific OCR configurations. This enhancement allows the use of language information to improve OCR accuracy when processing PDFs.
2024-09-09 14:31:04 +08:00
Xiaomeng Zhao 0f91fcf61f feat(cli&analyze&pipeline): add start_page and end_page args for pagination (#507)
* feat(cli&analyze&pipeline): add start_page and end_page args for paginationAdd start_page_id and end_page_id arguments to various components of the PDF parsing
pipeline to support pagination functionality. This feature allows users to specify the
range of pages to be processed, enhancing the efficiency and flexibility of the system.

* feat(cli&analyze&pipeline): add start_page and end_page args for paginationAdd start_page_id and end_page_id arguments to various components of the PDF parsing
pipeline to support pagination functionality. This feature allows users to specify the
range of pages to be processed, enhancing the efficiency and flexibility of the system.

* feat(cli&analyze&pipeline): add start_page and end_page args for paginationAdd start_page_id and end_page_id arguments to various components of the PDF parsing
pipeline to support pagination functionality. This feature allows users to specify the
range of pages to be processed, enhancing the efficiency and flexibility of the system.
2024-08-30 13:59:42 +08:00
myhloli f8f6ba6fd3 update:Add md make mode config in do_parse.You can control whether the produced md is for NLP or MM by changing the value of f_make_md_mode 2024-07-11 16:31:10 +08:00
赵小蒙 df14c61f6f update: Enhance the capability to detect garbled document issues 2024-06-19 16:03:21 +08:00
赵小蒙 389826c5fe update custom model framework 2024-06-18 14:45:23 +08:00
赵小蒙 0606301412 make paddle analyze mode adaptation cli input mode to improve analyze speed 2024-06-17 11:37:48 +08:00
赵小蒙 3955a3b303 update some annotation 2024-05-08 16:33:41 +08:00
赵小蒙 413a9df2c4 remove invalid references 2024-05-07 10:43:08 +08:00
赵小蒙 4bd31ceda4 change content make logic to union_make 2024-04-30 12:44:57 +08:00
赵小蒙 e980d2efa0 fix UNIPipe and spans space with language 2024-04-29 14:47:30 +08:00
赵小蒙 5f3cf14afc pipe初始化移除img_parent_path参数 2024-04-26 18:40:36 +08:00
赵小蒙 d3542f6a71 add para_to_standard_format logic 2024-04-25 18:47:41 +08:00
赵小蒙 1340a97ae2 统一使用ocr组装markdown 2024-04-22 22:18:09 +08:00
赵小蒙 7e8e9cabee 重构parse_by_ocr_v2 2024-04-18 21:05:20 +08:00
kernel.h@qq.com 1f45e0ab4b 添加debug模式 2024-04-17 14:41:32 +08:00
kernel.h@qq.com f702defe5a 修正几个常量 2024-04-17 12:06:23 +08:00
kernel.h@qq.com 698c4a83d2 修改pipe模块 2024-04-17 11:54:54 +08:00
kernel.h@qq.com ffc20db7f3 修改包引用问题 2024-04-16 21:30:39 +08:00
drunkpig 435ab922c6 Merge branch 'master' into master 2024-04-16 21:16:29 +08:00
kernel.h@qq.com ef03c906a1 io --> rw 2024-04-16 20:50:41 +08:00
赵小蒙 18b02ae3bd Pipe类重构 2024-04-16 18:26:34 +08:00
赵小蒙 7b937d58b8 fix 参数名称 2024-04-15 22:15:30 +08:00
赵小蒙 709a65008a 中间态dict结构调整
部分函数重构
2024-04-15 18:51:58 +08:00