mirror of
https://github.com/PaddlePaddle/PaddleOCR.git
synced 2026-09-24 23:33:08 +08:00
add layout and seal docs (#15176)
* fix layout docs * fix layout docs * fix layout docs
This commit is contained in:
Sunflower7788
@@ -0,0 +1,622 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Layout Detection Module Tutorial
|
||||
|
||||
## I. Overview
|
||||
The core task of structure analysis is to parse and segment the content of input document images. By identifying different elements in the image (such as text, charts, images, etc.), they are classified into predefined categories (e.g., pure text area, title area, table area, image area, list area, etc.), and the position and size of these regions in the document are determined.
|
||||
|
||||
## II. Supported Model List
|
||||
|
||||
* <b>The layout detection model includes 20 common categories: document title, paragraph title, text, page number, abstract, table, references, footnotes, header, footer, algorithm, formula, formula number, image, table, seal, figure_table title, chart, and sidebar text and lists of references</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (M)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayout_plus-L</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout_plus-L_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout_plus-L_pretrained.pdparams">Training Model</a></td>
|
||||
<td>83.2</td>
|
||||
<td>34.6244 / 10.3945</td>
|
||||
<td>510.57 / - </td>
|
||||
<td>126.01 M</td>
|
||||
<td>A higher-precision layout area localization model trained on a self-built dataset containing Chinese and English papers, PPT, multi-layout magazines, contracts, books, exams, ancient books and research reports using RT-DETR-L</td>
|
||||
</tr>
|
||||
<tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* <b>The layout detection model includes 1 category: Block:</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (M)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocBlockLayout</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBlockLayout_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocBlockLayout_pretrained.pdparams">Training Model</a></td>
|
||||
<td>95.9</td>
|
||||
<td>34.6244 / 10.3945</td>
|
||||
<td>510.57 / - </td>
|
||||
<td>123.92 M</td>
|
||||
<td>A layout block localization model trained on a self-built dataset containing Chinese and English papers, PPT, multi-layout magazines, contracts, books, exams, ancient books and research reports using RT-DETR-L</td>
|
||||
</tr>
|
||||
<tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* <b>The layout detection model includes 23 common categories: document title, paragraph title, text, page number, abstract, table of contents, references, footnotes, header, footer, algorithm, formula, formula number, image, figure caption, table, table caption, seal, figure title, figure, header image, footer image, and sidebar text</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (M)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayout-L</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-L_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-L_pretrained.pdparams">Training Model</a></td>
|
||||
<td>90.4</td>
|
||||
<td>34.6244 / 10.3945</td>
|
||||
<td>510.57 / -</td>
|
||||
<td>123.76 M</td>
|
||||
<td>A high-precision layout area localization model trained on a self-built dataset containing Chinese and English papers, magazines, contracts, books, exams, and research reports using RT-DETR-L.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocLayout-M</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-M_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-M_pretrained.pdparams">Training Model</a></td>
|
||||
<td>75.2</td>
|
||||
<td>13.3259 / 4.8685</td>
|
||||
<td>44.0680 / 44.0680</td>
|
||||
<td>22.578</td>
|
||||
<td>A layout area localization model with balanced precision and efficiency, trained on a self-built dataset containing Chinese and English papers, magazines, contracts, books, exams, and research reports using PicoDet-L.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocLayout-S</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-S_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-S_pretrained.pdparams">Training Model</a></td>
|
||||
<td>70.9</td>
|
||||
<td>8.3008 / 2.3794</td>
|
||||
<td>10.0623 / 9.9296</td>
|
||||
<td>4.834</td>
|
||||
<td>A high-efficiency layout area localization model trained on a self-built dataset containing Chinese and English papers, magazines, contracts, books, exams, and research reports using PicoDet-S.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
> ❗ The above list includes the <b>4 core models</b> that are key supported by the text recognition module. The module actually supports a total of <b>12 full models</b>, including several predefined models with different categories. The complete model list is as follows:
|
||||
|
||||
<details><summary> 👉 Details of Model List</summary>
|
||||
|
||||
* <b>Table Layout Detection Model</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (M)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet_layout_1x_table</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet_layout_1x_table_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet_layout_1x_table_pretrained.pdparams">Training Model</a></td>
|
||||
<td>97.5</td>
|
||||
<td>8.02 / 3.09</td>
|
||||
<td>23.70 / 20.41</td>
|
||||
<td>7.4 M</td>
|
||||
<td>A high-efficiency layout area localization model trained on a self-built dataset using PicoDet-1x, capable of detecting table regions.</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>3-Class Layout Detection Model, including Table, Image, and Stamp</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (M)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet-S_layout_3cls</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-S_layout_3cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-S_layout_3cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>88.2</td>
|
||||
<td>8.99 / 2.22</td>
|
||||
<td>16.11 / 8.73</td>
|
||||
<td>4.8</td>
|
||||
<td>A high-efficiency layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-S.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PicoDet-L_layout_3cls</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-L_layout_3cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-L_layout_3cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>89.0</td>
|
||||
<td>13.05 / 4.50</td>
|
||||
<td>41.30 / 41.30</td>
|
||||
<td>22.6</td>
|
||||
<td>A balanced efficiency and precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-L.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-H_layout_3cls</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-H_layout_3cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-H_layout_3cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>95.8</td>
|
||||
<td>114.93 / 27.71</td>
|
||||
<td>947.56 / 947.56</td>
|
||||
<td>470.1</td>
|
||||
<td>A high-precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using RT-DETR-H.</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>5-Class English Document Area Detection Model, including Text, Title, Table, Image, and List</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (M)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet_layout_1x</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet_layout_1x_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet_layout_1x_pretrained.pdparams">Training Model</a></td>
|
||||
<td>97.8</td>
|
||||
<td>9.03 / 3.10</td>
|
||||
<td>25.82 / 20.70</td>
|
||||
<td>7.4</td>
|
||||
<td>A high-efficiency English document layout area localization model trained on the PubLayNet dataset using PicoDet-1x.</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>17-Class Area Detection Model, including 17 common layout categories: Paragraph Title, Image, Text, Number, Abstract, Content, Figure Caption, Formula, Table, Table Caption, References, Document Title, Footnote, Header, Algorithm, Footer, and Stamp</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (M)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet-S_layout_17cls</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-S_layout_17cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-S_layout_17cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>87.4</td>
|
||||
<td>9.11 / 2.12</td>
|
||||
<td>15.42 / 9.12</td>
|
||||
<td>4.8</td>
|
||||
<td>A high-efficiency layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-S.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PicoDet-L_layout_17cls</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-L_layout_17cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-L_layout_17cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>89.0</td>
|
||||
<td>13.50 / 4.69</td>
|
||||
<td>43.32 / 43.32</td>
|
||||
<td>22.6</td>
|
||||
<td>A balanced efficiency and precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-L.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-H_layout_17cls</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-H_layout_17cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-H_layout_17cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>98.3</td>
|
||||
<td>115.29 / 104.09</td>
|
||||
<td>995.27 / 995.27</td>
|
||||
<td>470.2</td>
|
||||
<td>A high-precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using RT-DETR-H.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset：</strong>
|
||||
<ul>
|
||||
<li>20 types of layout detection models: PaddleOCR's self built layout area detection dataset, including Chinese and English papers, magazines, newspapers, research papers PPT、 1300 images of document types such as test papers and textbooks. </li>
|
||||
<li>Type 1 version face region detection model: PaddleOCR's self built version face region detection dataset, including Chinese and English papers, magazines, newspapers, research reports PPT、 1000 document type images such as test papers and textbooks. </li>
|
||||
<li>23 categories Layout Detection Model: A self-built layout area detection dataset by PaddleOCR, containing 500 common document type images such as Chinese and English papers, magazines, contracts, books, exam papers, and research reports.</li>
|
||||
<li>Table Layout Detection Model: A self-built table area detection dataset by PaddleOCR, including 7,835 Chinese and English paper document type images with tables.</li>
|
||||
<li> 3-Class Layout Detection Model: A self-built layout area detection dataset by PaddleOCR, comprising 1,154 common document type images such as Chinese and English papers, magazines, and research reports.</li>
|
||||
<li>5-Class English Document Area Detection Model: The evaluation dataset of <a href="https://developer.ibm.com/exchanges/data/all/publaynet">PubLayNet</a>, containing 11,245 images of English documents.</li>
|
||||
<li>17-Class Area Detection Model: A self-built layout area detection dataset by PaddleOCR, including 892 common document type images such as Chinese and English papers, magazines, and research reports.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Hardware Configuration：</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
<li>Other Environments: Ubuntu 20.04 / cuDNN 8.6 / TensorRT 8.5.2.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration </th>
|
||||
<th>CPU Configuration </th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of pre-selected precision types and acceleration strategies</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Pre-selected optimal backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
</details>
|
||||
|
||||
|
||||
## III. Quick Integration <a id="quick"> </a>
|
||||
|
||||
> ❗ Before quick integration, please install the PaddleOCR wheel package. For detailed instructions, refer to [PaddleOCR Local Installation Tutorial](../ppocr/installation.en.md)。
|
||||
|
||||
Quickly experience with just one command:
|
||||
|
||||
```bash
|
||||
paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg
|
||||
```
|
||||
|
||||
You can also integrate the model inference from the layout area detection module into your project. Before running the following code, please download [Example Image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg) Go to the local area.
|
||||
|
||||
```python
|
||||
from paddleocr import LayoutDetection
|
||||
|
||||
model = LayoutDetection(model_name="PP-DocLayout_plus-L")
|
||||
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result obtained is:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'layout.jpg', 'page_index': None, 'boxes': [{'cls_id': 2, 'label': 'text', 'score': 0.9870226979255676, 'coordinate': [34.101906, 349.85275, 358.59213, 611.0772]}, {'cls_id': 2, 'label': 'text', 'score': 0.9866003394126892, 'coordinate': [34.500324, 647.1585, 358.29367, 848.66797]}, {'cls_id': 2, 'label': 'text', 'score': 0.9846674203872681, 'coordinate': [385.71445, 497.40973, 711.2261, 697.84265]}, {'cls_id': 8, 'label': 'table', 'score': 0.984126091003418, 'coordinate': [73.76879, 105.94899, 321.95303, 298.84888]}, {'cls_id': 8, 'label': 'table', 'score': 0.9834211468696594, 'coordinate': [436.95642, 105.81531, 662.7168, 313.48462]}, {'cls_id': 2, 'label': 'text', 'score': 0.9832247495651245, 'coordinate': [385.62787, 346.2288, 710.10095, 458.77127]}, {'cls_id': 2, 'label': 'text', 'score': 0.9816061854362488, 'coordinate': [385.7802, 735.1931, 710.56134, 849.9764]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9577341079711914, 'coordinate': [34.421448, 20.055151, 358.71283, 76.53663]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9505634307861328, 'coordinate': [385.72278, 20.053688, 711.29333, 74.92744]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.9001723527908325, 'coordinate': [386.46344, 477.03488, 699.4023, 490.07474]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8845751285552979, 'coordinate': [35.413048, 627.73596, 185.58383, 640.52264]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8837394118309021, 'coordinate': [387.17603, 716.3423, 524.7841, 729.258]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8508939743041992, 'coordinate': [35.50064, 331.18445, 141.6444, 344.81097]}]}}
|
||||
```
|
||||
|
||||
The meanings of the parameters are as follows:
|
||||
- `input_path`: The path to the input image for prediction.
|
||||
- `page_index`: If the input is a PDF file, it indicates which page of the PDF it is; otherwise, it is `None`.
|
||||
- `boxes`: Information about the predicted bounding boxes, a list of dictionaries. Each dictionary represents a detected object and contains the following information:
|
||||
- `cls_id`: Class ID, an integer.
|
||||
- `label`: Class label, a string.
|
||||
- `score`: Confidence score of the bounding box, a float.
|
||||
- `coordinate`: Coordinates of the bounding box, a list of floats in the format <code>[xmin, ymin, xmax, ymax]</code>.
|
||||
|
||||
|
||||
The visualized image is as follows:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/layout_det/layout_res.jpg"/>
|
||||
|
||||
Relevant methods, parameters, and explanations are as follows:
|
||||
|
||||
* `LayoutDetection` instantiates a target detection model (here, `PP-DocLayout_plus-L` is used as an example). The detailed explanation is as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Options</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Name of the model</td>
|
||||
<td><code>str</code></td>
|
||||
<td>None</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Path to store the model</td>
|
||||
<td><code>str</code></td>
|
||||
<td>None</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>The device used for model inference</td>
|
||||
<td><code>str</code></td>
|
||||
<td>It supports specifying specific GPU card numbers, such as "gpu:0", other hardware card numbers, such as "npu:0", or CPU, such as "cpu".</td>
|
||||
<td><code>gpu:0</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>img_size</code></td>
|
||||
<td>Size of the input image; if not specified, the default PaddleX official model configuration will be used</td>
|
||||
<td><code>int/list/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>int</b>, e.g., 640, means resizing the input image to 640x640</li>
|
||||
<li><b>List</b>, e.g., [640, 512], means resizing the input image to a width of 640 and a height of 512</li>
|
||||
<li><b>None</b>, not specified, will use the default PaddleX official model configuration</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td>Threshold for filtering low-confidence prediction results; if not specified, the default PaddleX official model configuration will be used</td>
|
||||
<td><code>float/dict/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>float</b>, e.g., 0.2, means filtering out all bounding boxes with a confidence score less than 0.2</li>
|
||||
<li><b>Dictionary</b>, with keys as <b>int</b> representing <code>cls_id</code> and values as <b>float</b> thresholds. For example, <code>{0: 0.45, 2: 0.48, 7: 0.4}</code> means applying a threshold of 0.45 for cls_id 0, 0.48 for cls_id 2, and 0.4 for cls_id 7</li>
|
||||
<li><b>None</b>, not specified, will use the default PaddleX official model configuration</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_nms</code></td>
|
||||
<td>Whether to use NMS post-processing to filter overlapping boxes; if not specified, the default PaddleX official model configuration will be used</td>
|
||||
<td><code>bool/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>bool</b>, True/False, indicates whether to use NMS for post-processing to filter overlapping boxes</li>
|
||||
<li><b>None</b>, not specified, will use the default PaddleX official model configuration</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_unclip_ratio</code></td>
|
||||
<td>Scaling factor for the side length of the detection box; if not specified, the default PaddleX official model configuration will be used</td>
|
||||
<td><code>float/list/dict/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>float</b>, a positive float number, e.g., 1.1, means expanding the width and height of the detection box by 1.1 times while keeping the center unchanged</li>
|
||||
<li><b>List</b>, e.g., [1.2, 1.5], means expanding the width by 1.2 times and the height by 1.5 times while keeping the center unchanged</li>
|
||||
<li><b>dict</b>, keys as <b>int</b> representing <code>cls_id</code>, values as float scaling factors, e.g., <code>{0: (1.1, 2.0)}</code> means cls_id 0 expanding the width by 1.1 times and the height by 2.0 times while keeping the center unchanged</li>
|
||||
<li><b>None</b>, not specified, will use the default PaddleX official model configuration</li>
|
||||
</ul>
|
||||
</td>
|
||||
<tr>
|
||||
<td><code>layout_merge_bboxes_mode</code></td>
|
||||
<td>Merging mode for the detection boxes output by the model; if not specified, the default PaddleX official model configuration will be used</td>
|
||||
<td><code>string/dict/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>large</b>, when set to large, only the largest external box will be retained for overlapping detection boxes, and the internal overlapping boxes will be deleted</li>
|
||||
<li><b>small</b>, when set to small, only the smallest internal box will be retained for overlapping detection boxes, and the external overlapping boxes will be deleted</li>
|
||||
<li><b>union</b>, no filtering of boxes will be performed, and both internal and external boxes will be retained</li>
|
||||
<li><b>dict</b>, keys as <b>int</b> representing <code>cls_id</code> and values as merging modes, e.g., <code>{0: "large", 2: "small"}</li>
|
||||
<li><b>None</b>, not specified, will use the default PaddleX official model configuration</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_hpip</code></td>
|
||||
<td>Whether to enable the high-performance inference plugin</td>
|
||||
<td><code>bool</code></td>
|
||||
<td>None</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>hpi_config</code></td>
|
||||
<td>High-performance inference configuration</td>
|
||||
<td><code>dict</code> | <code>None</code></td>
|
||||
<td>None</td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Note that `model_name` must be specified. After specifying `model_name`, the default PaddleX built-in model parameters will be used. If `model_dir` is specified, the user-defined model will be used.
|
||||
|
||||
* The `predict()` method of the target detection model is called for inference prediction. The parameters of the `predict()` method are `input`, `batch_size`, and `threshold`, which are explained as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Options</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Data for prediction, supporting multiple input types</td>
|
||||
<td><code>Python Var</code>/<code>str</code>/<code>list</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>Python Variable</b>, such as image data represented by <code>numpy.ndarray</code></li>
|
||||
<li><b>File Path</b>, such as the local path of an image file: <code>/root/data/img.jpg</code></li>
|
||||
<li><b>URL link</b>, such as the network URL of an image file: <a href = "https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg">示例</a></li>
|
||||
<li><b>Local Directory</b>, the directory should contain the data files to be predicted, such as the local path: <code>/root/data/</code></li>
|
||||
<li><b>List</b>, the elements of the list should be of the above-mentioned data types, such as <code>[numpy.ndarray, numpy.ndarray]</code>, <code>[\"/root/data/img1.jpg\", \"/root/data/img2.jpg\"]</code>, <code>[\"/root/data1\", \"/root/data2\"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size</td>
|
||||
<td><code>int</code></td>
|
||||
<td>Any integer greater than 0</td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td>Threshold for filtering low-confidence prediction results</td>
|
||||
<td><code>float/dict/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>float</b>, e.g., 0.2, means filtering out all bounding boxes with a confidence score less than 0.2</li>
|
||||
<li><b>Dictionary</b>, with keys as <b>int</b> representing <code>cls_id</code> and values as <b>float</b> thresholds. For example, <code>{0: 0.45, 2: 0.48, 7: 0.4}</code> means applying a threshold of 0.45 for cls_id 0, 0.48 for cls_id 2, and 0.4 for cls_id 7</li>
|
||||
<li><b>None</b>, not specified, will use the <code>threshold</code> parameter specified in <code>create_model</code>. If not specified in <code>create_model</code>, the default PaddleX official model configuration will be used</li>
|
||||
</ul>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_nms</code></td>
|
||||
<td>Whether to use NMS post-processing to filter overlapping boxes; if not specified, the default PaddleX official model configuration will be used</td>
|
||||
<td><code>bool/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>bool</b>, True/False, indicates whether to use NMS for post-processing to filter overlapping boxes</li>
|
||||
<li><b>None</b>, not specified, will use the <code>layout_nms</code> parameter specified in <code>create_model</code>. If not specified in <code>create_model</code>, the default PaddleX official model configuration will be used</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_unclip_ratio</code></td>
|
||||
<td>Scaling factor for the side length of the detection box; if not specified, the default PaddleX official model configuration will be used</td>
|
||||
<td><code>float/list/dict/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>float</b>, a positive float number, e.g., 1.1, means expanding the width and height of the detection box by 1.1 times while keeping the center unchanged</li>
|
||||
<li><b>List</b>, e.g., [1.2, 1.5], means expanding the width by 1.2 times and the height by 1.5 times while keeping the center unchanged</li>
|
||||
<li><b>dict</b>, keys as <b>int</b> representing <code>cls_id</code>, values as float scaling factors, e.g., <code>{0: (1.1, 2.0)}</code> means cls_id 0 expanding the width by 1.1 times and the height by 2.0 times while keeping the center unchanged</li>
|
||||
<li><b>None</b>, not specified, will use the <code>layout_unclip_ratio</code> parameter specified in <code>create_model</code>. If not specified in <code>create_model</code>, the default PaddleX official model configuration will be used</li>
|
||||
</ul>
|
||||
</td>
|
||||
<tr>
|
||||
<td><code>layout_merge_bboxes_mode</code></td>
|
||||
<td>Merging mode for the detection boxes output by the model; if not specified, the default PaddleX official model configuration will be used</td>
|
||||
<td><code>string/dict/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>large</b>, when set to large, only the largest external box will be retained for overlapping detection boxes, and the internal overlapping boxes will be deleted</li>
|
||||
<li><b>small</b>, when set to small, only the smallest internal box will be retained for overlapping detection boxes, and the external overlapping boxes will be deleted</li>
|
||||
<li><b>union</b>, no filtering of boxes will be performed, and both internal and external boxes will be retained</li>
|
||||
<li><b>dict</b>, keys as <b>int</b> representing <code>cls_id</code> and values as merging modes, e.g., <code>{0: "large", 2: "small"}</li>
|
||||
<li><b>None</b>, not specified, will use the <code>layout_merge_bboxes_mode</code> parameter specified in <code>create_model</code>. If not specified in <code>create_model</code>, the default PaddleX official model configuration will be used</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</tr></table>
|
||||
|
||||
* Process the prediction results, with each sample's prediction result being the corresponding Result object, and supporting operations such as printing, saving as an image, and saving as a 'json' file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Method Description</th>
|
||||
<th>Parameters</th>
|
||||
<th>Parameter type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print the result to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Do you want to use <code>JSON</code> indentation formatting for the output content</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to enhance the readability of the <code>JSON</code> data output, only valid when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non ASCII characters to Unicode characters. When set to <code>True</code>, all non ASCII </code>characters will be escaped; <code>False</code> preserves the original characters and is only valid when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save the result as a JSON format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The saved file path, when it is a directory, the name of the saved file is consistent with the name of the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to enhance the readability of the <code>JSON</code> data output, only valid when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non ASCII characters to Unicode characters. When set to <code>True</code>, all non <code>ASCII</code> characters will be escaped; <code>False</code> preserves the original characters and is only valid when<code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the results as an image format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The saved file path, when it is a directory, the name of the saved file is consistent with the name of the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
* Additionally, it also supports obtaining the visualized image with results and the prediction results via attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">Get the visualized image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
## IV. Custom Development
|
||||
|
||||
Since PaddleOCR does not directly provide training for the layout detection module, if you need to train the layout area detection model, you can refer to [PaddleX Layout Detection Module Secondary Development](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/layout_detection.html#_5)Partially conduct training. The trained model can be seamlessly integrated into PaddleOCR's API for inference.
|
||||
@@ -10,6 +10,60 @@ comments: true
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
* <b>版面检测模型，包含20个常见的类别：文档标题、段落标题、文本、页码、摘要、目录、参考文献、脚注、页眉、页脚、算法、公式、公式编号、图像、表格、图和表标题（图标题、表格标题和图表标题）、印章、图表、侧栏文本和参考文献内容</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（M）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayout_plus-L</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout_plus-L_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout_plus-L_pretrained.pdparams">训练模型</a></td>
|
||||
<td>83.2</td>
|
||||
<td>34.6244 / 10.3945</td>
|
||||
<td>510.57 / - </td>
|
||||
<td>126.01 M</td>
|
||||
<td>基于RT-DETR-L在包含中英文论文、多栏杂志、报纸、PPT、合同、书本、试卷、研报、古籍、日文文档、竖版文字文档等场景的自建数据集训练的更高精度版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<b>注：以上精度指标的评估集是自建的版面区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1300 张文档类型图片。</b>
|
||||
|
||||
* <b>文档图像版面子模块检测，包含1个 版面区域 类别，能检测多栏的报纸、杂志的每个子文章的文本区域：</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（M）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocBlockLayout</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBlockLayout_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocBlockLayout_pretrained.pdparams">训练模型</a></td>
|
||||
<td>95.9</td>
|
||||
<td>34.6244 / 10.3945</td>
|
||||
<td>510.57 / - </td>
|
||||
<td>123.92 M</td>
|
||||
<td>基于RT-DETR-L在包含中英文论文、多栏杂志、报纸、PPT、合同、书本、试卷、研报、古籍、日文文档、竖版文字文档等场景的自建数据集训练的文档图像版面子模块检测模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<b>注：以上精度指标的评估集是自建的版面子区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1000 张文档类型图片。</b>
|
||||
|
||||
* <b>版面检测模型，包含23个常见的类别：文档标题、段落标题、文本、页码、摘要、目录、参考文献、脚注、页眉、页脚、算法、公式、公式编号、图像、图表标题、表格、表格标题、印章、图表标题、图表、页眉图像、页脚图像、侧栏文本</b>
|
||||
<table>
|
||||
<thead>
|
||||
@@ -50,8 +104,9 @@ comments: true
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<b>注：以上精度指标的评估集是自建的版面区域检测数据集，包含中英文论文、报纸、研报和试卷等 500 张文档类型图片。</b>
|
||||
|
||||
> ❗ 以上列出的是版面检测模块重点支持的<b>3个核心模型</b>，该模块总共支持<b>11个全量模型</b>，包含多个预定义了不同类别的模型，完整的模型列表如下：
|
||||
> ❗ 以上列出的是版面检测模块重点支持的<b>5个核心模型</b>，该模块总共支持<b>13个全量模型</b>，包含多个预定义了不同类别的模型，完整的模型列表如下：
|
||||
|
||||
<details><summary> 👉模型列表详情</summary>
|
||||
|
||||
@@ -187,8 +242,9 @@ comments: true
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>
|
||||
<ul>
|
||||
<li>版面检测模型： PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志、合同、书本、试卷和研报等常见的 500 张文档类型图片。</li>
|
||||
<li>表格版面检测模型：PaddleOCR 自建的版面表格区域检测数据集，包含中英文 7835 张带有表格的论文文档类型图片。</li>
|
||||
<li>20类版面检测模型： PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1300 张文档类型图片。</li>
|
||||
<li>1类版面子区域检测模型： PaddleOCR 自建的版面子区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1000 张文档类型图片。</li>
|
||||
<li>23类版面检测模型： PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志、合同、书本、试卷和研报等常见的 500 张文档类型图片。</li>
|
||||
<li>3类版面检测模型：PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志和研报等常见的 1154 张文档类型图片。</li>
|
||||
<li>5类英文文档区域检测模型： <a href="https://developer.ibm.com/exchanges/data/all/publaynet" target="_blank">PubLayNet</a> 的评估数据集，包含英文文档的 11245 张图片。</li>
|
||||
<li>17类区域检测模型：PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志和研报等常见的 892 张文档类型图片。</li>
|
||||
@@ -247,7 +303,7 @@ paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex
|
||||
```python
|
||||
from paddleocr import LayoutDetection
|
||||
|
||||
model = LayoutDetection(model_name="PP-DocLayout-L")
|
||||
model = LayoutDetection(model_name="PP-DocLayout_plus-L")
|
||||
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
|
||||
for res in output:
|
||||
res.print()
|
||||
@@ -258,7 +314,7 @@ for res in output:
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'layout.jpg', 'page_index': None, 'boxes': [{'cls_id': 8, 'label': 'table', 'score': 0.9866452813148499, 'coordinate': [74.30952, 105.71375, 321.98676, 299.11072]}, {'cls_id': 2, 'label': 'text', 'score': 0.9859839081764221, 'coordinate': [34.65901, 349.91003, 358.33832, 611.3427]}, {'cls_id': 2, 'label': 'text', 'score': 0.9850624799728394, 'coordinate': [34.945335, 647.378, 358.32523, 849.23413]}, {'cls_id': 8, 'label': 'table', 'score': 0.9850127696990967, 'coordinate': [438.06952, 105.37871, 662.88666, 313.88693]}, {'cls_id': 2, 'label': 'text', 'score': 0.9847850799560547, 'coordinate': [385.97076, 497.04156, 710.9561, 697.6779]}, {'cls_id': 2, 'label': 'text', 'score': 0.9805672764778137, 'coordinate': [385.79672, 345.93826, 710.07385, 459.14514]}, {'cls_id': 2, 'label': 'text', 'score': 0.9799845814704895, 'coordinate': [386.07553, 735.38086, 710.6084, 850.1987]}, {'cls_id': 9, 'label': 'table_title', 'score': 0.9376267194747925, 'coordinate': [35.27453, 19.852173, 358.92462, 77.81253]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8755457997322083, 'coordinate': [386.6317, 476.607, 699.7845, 490.11603]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8617177605628967, 'coordinate': [387.27615, 715.9574, 524.3855, 729.2082]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8607730865478516, 'coordinate': [35.451878, 627.4962, 185.63542, 640.4025]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8575080633163452, 'coordinate': [35.334385, 330.80493, 141.46948, 344.40747]}, {'cls_id': 9, 'label': 'table_title', 'score': 0.7959735989570618, 'coordinate': [385.93796, 19.755125, 711.5128, 75.00555]}]}}
|
||||
{'res': {'input_path': 'layout.jpg', 'page_index': None, 'boxes': [{'cls_id': 2, 'label': 'text', 'score': 0.9870226979255676, 'coordinate': [34.101906, 349.85275, 358.59213, 611.0772]}, {'cls_id': 2, 'label': 'text', 'score': 0.9866003394126892, 'coordinate': [34.500324, 647.1585, 358.29367, 848.66797]}, {'cls_id': 2, 'label': 'text', 'score': 0.9846674203872681, 'coordinate': [385.71445, 497.40973, 711.2261, 697.84265]}, {'cls_id': 8, 'label': 'table', 'score': 0.984126091003418, 'coordinate': [73.76879, 105.94899, 321.95303, 298.84888]}, {'cls_id': 8, 'label': 'table', 'score': 0.9834211468696594, 'coordinate': [436.95642, 105.81531, 662.7168, 313.48462]}, {'cls_id': 2, 'label': 'text', 'score': 0.9832247495651245, 'coordinate': [385.62787, 346.2288, 710.10095, 458.77127]}, {'cls_id': 2, 'label': 'text', 'score': 0.9816061854362488, 'coordinate': [385.7802, 735.1931, 710.56134, 849.9764]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9577341079711914, 'coordinate': [34.421448, 20.055151, 358.71283, 76.53663]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9505634307861328, 'coordinate': [385.72278, 20.053688, 711.29333, 74.92744]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.9001723527908325, 'coordinate': [386.46344, 477.03488, 699.4023, 490.07474]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8845751285552979, 'coordinate': [35.413048, 627.73596, 185.58383, 640.52264]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8837394118309021, 'coordinate': [387.17603, 716.3423, 524.7841, 729.258]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8508939743041992, 'coordinate': [35.50064, 331.18445, 141.6444, 344.81097]}]}}
|
||||
```
|
||||
|
||||
参数含义如下：
|
||||
@@ -277,7 +333,7 @@ for res in output:
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `LayoutDetection`实例化目标检测模型（此处以`PP-DocLayout-L`为例），具体说明如下：
|
||||
* `LayoutDetection`实例化目标检测模型（此处以`PP-DocLayout_plus-L`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
@@ -564,4 +620,6 @@ for res in output:
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
......
|
||||
由于 PaddleOCR 并不直接提供版面区域检测模块的训练，因此，如果需要训练版面区域测模型，可以参考 [PaddleX 版面区域检测模块二次开发](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/layout_detection.html#_5)部分进行训练。训练后的模型可以无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
## 五、FAQ
|
||||
|
||||
@@ -0,0 +1,516 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Seal Text Detection Module Tutorial
|
||||
|
||||
## I. Overview
|
||||
The seal text detection module typically outputs multi-point bounding boxes around text regions, which are then passed as inputs to the distortion correction and text recognition modules for subsequent processing to identify the textual content of the seal. Recognizing seal text is an integral part of document processing and finds applications in various scenarios such as contract comparison, inventory access auditing, and invoice reimbursement verification. The seal text detection module serves as a subtask within OCR (Optical Character Recognition), responsible for locating and marking the regions containing seal text within an image. The performance of this module directly impacts the accuracy and efficiency of the entire seal text OCR system.
|
||||
|
||||
## II. Supported Model List
|
||||
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model Name</th><th>Model Download Link</th>
|
||||
<th>Hmean（%）</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Size (M)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_seal_det</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_seal_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_seal_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>98.21</td>
|
||||
<td>74.75 / 67.72</td>
|
||||
<td>382.55 / 382.55</td>
|
||||
<td>109 M</td>
|
||||
<td>The server-side seal text detection model of PP-OCRv4 boasts higher accuracy and is suitable for deployment on better-equipped servers.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_seal_det</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_seal_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_seal_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>96.47</td>
|
||||
<td>7.82 / 3.09</td>
|
||||
<td>48.28 / 23.97</td>
|
||||
<td>4.6 M</td>
|
||||
<td>The mobile-side seal text detection model of PP-OCRv4, on the other hand, offers greater efficiency and is suitable for deployment on end devices.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset：</strong> PaddleX Custom Dataset, Containing 500 Images of Circular Stamps.</li>
|
||||
<li><strong>Hardware Configuration：</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
<li>Other Environments: Ubuntu 20.04 / cuDNN 8.6 / TensorRT 8.5.2.2</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration </th>
|
||||
<th>CPU Configuration </th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of pre-selected precision types and acceleration strategies</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Pre-selected optimal backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## III. Quick Integration <a id="quick"> </a>
|
||||
|
||||
> ❗ Before quick integration, please install the PaddleOCR wheel package. For detailed instructions, refer to [PaddleOCR Local Installation Tutorial](../ppocr/installation.en.md)。
|
||||
|
||||
Quickly experience with just one command:
|
||||
|
||||
```bash
|
||||
paddleocr seal_text_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png
|
||||
```
|
||||
|
||||
|
||||
You can also integrate the model inference from the layout area detection module into your project. Before running the following code, please download [Example Image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png) Go to the local area.
|
||||
|
||||
```python
|
||||
from paddleocr import SealTextDetection
|
||||
model = SealTextDetection(model_name="PP-OCRv4_server_seal_det")
|
||||
output = model.predict("seal_text_det.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result is:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'seal_text_det.png', 'page_index': None, 'dt_polys': [array([[463, 477],
|
||||
...,
|
||||
[428, 505]]), array([[297, 444],
|
||||
...,
|
||||
[230, 443]]), array([[457, 346],
|
||||
...,
|
||||
[267, 345]]), array([[325, 38],
|
||||
...,
|
||||
[322, 37]])], 'dt_scores': [0.9912680344777314, 0.9906849624837963, 0.9847219455533163, 0.9914791724153904]}}
|
||||
```
|
||||
|
||||
The meanings of the parameters are as follows:
|
||||
- `input_path`: represents the path of the input image to be predicted
|
||||
- `dt_polys`: represents the predicted text detection boxes, where each text detection box contains multiple vertices of a polygon. Each vertex is a list of two elements, representing the x and y coordinates of the vertex respectively
|
||||
- `dt_scores`: represents the confidence scores of the predicted text detection boxes
|
||||
|
||||
The visualization image is as follows:
|
||||
|
||||
<img alt="Visualization Image" src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/seal_text_det/seal_text_det_res.png"/>
|
||||
|
||||
The explanations of related methods and parameters are as follows:
|
||||
|
||||
* `SealTextDetection` instantiates a text detection model (here we take `PP-OCRv4_server_seal_det` as an example), and the specific explanations are as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Parameter Type</th>
|
||||
<th>Options</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Name of the model</td>
|
||||
<td><code>str</code></td>
|
||||
<td>All model names supported by PaddleX for seal text detection</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Path to store the model</td>
|
||||
<td><code>str</code></td>
|
||||
<td>None</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>The device used for model inference</td>
|
||||
<td><code>str</code></td>
|
||||
<td>It supports specifying specific GPU card numbers, such as "gpu:0", other hardware card numbers, such as "npu:0", or CPU, such as "cpu".</td>
|
||||
<td><code>gpu:0</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td>Limit on the side length of the image for detection</td>
|
||||
<td><code>int/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>int</b>: Any integer greater than 0
|
||||
<li><b>None</b>: If set to None, the default value from the official PaddleX model configuration will be used</li></li></ul></td>
|
||||
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td>Type of side length limit for detection</td>
|
||||
<td><code>str/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>str</b>: Supports min and max. min ensures the shortest side of the image is not less than det_limit_side_len, max ensures the longest side is not greater than limit_side_len
|
||||
<li><b>None</b>: If set to None, the default value from the official PaddleX model configuration will be used</li></li></ul></td>
|
||||
|
||||
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td>In the output probability map, pixels with scores greater than this threshold will be considered as text pixels</td>
|
||||
<td><code>float/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>float</b>: Any float greater than 0
|
||||
<li><b>None</b>: If set to None, the default value from the official PaddleX model configuration will be used</li></li></ul></td>
|
||||
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td>If the average score of all pixels within a detection result box is greater than this threshold, the result will be considered as a text region</td>
|
||||
<td><code>float/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>float</b>: Any float greater than 0
|
||||
<li><b>None</b>: If set to None, the default value from the official PaddleX model configuration will be used</li></li></ul></td>
|
||||
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>max_candidates</code></td>
|
||||
<td>Maximum number of text boxes to output</td>
|
||||
<td><code>int/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>int</b>: Any integer greater than 0
|
||||
<li><b>None</b>: If set to None, the default value from the official PaddleX model configuration will be used</li></li></ul></td>
|
||||
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td>Expansion ratio for the Vatti clipping algorithm, used to expand the text region</td>
|
||||
<td><code>float/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>float</b>: Any float greater than 0
|
||||
<li><b>None</b>: If set to None, the default value from the official PaddleX model configuration will be used</li></li></ul></td>
|
||||
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_dilation</code></td>
|
||||
<td>Whether to dilate the segmentation result</td>
|
||||
<td><code>bool/None</code></td>
|
||||
<td>True/False/None</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_hpip</code></td>
|
||||
<td>Whether to enable the high-performance inference plugin</td>
|
||||
<td><code>bool</code></td>
|
||||
<td>None</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>hpi_config</code></td>
|
||||
<td>High-performance inference configuration</td>
|
||||
<td><code>dict</code> | <code>None</code></td>
|
||||
<td>None</td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* The `model_name` must be specified. After specifying `model_name`, the built-in model parameters of PaddleX will be used by default. On this basis, if `model_dir` is specified, the user-defined model will be used.
|
||||
|
||||
* The `predict()` method of the seal text detection model is called for inference prediction. The parameters of the `predict()` method include `input`, `batch_size`, `limit_side_len`, `limit_type`, `thresh`, `box_thresh`, `max_candidates`, `unclip_ratio`, and `use_dilation`. The specific descriptions are as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Parameter Type</th>
|
||||
<th>Options</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Data to be predicted, supporting multiple input types</td>
|
||||
<td><code>Python Var</code>/<code>str</code>/<code>dict</code>/<code>list</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>Python Variable</b>, such as image data represented by <code>numpy.ndarray</code></li>
|
||||
<li><b>File Path</b>, such as the local path of an image file: <code>/root/data/img.jpg</code></li>
|
||||
<li><b>URL Link</b>, such as the web URL of an image file: <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png">Example</a></li>
|
||||
<li><b>Local Directory</b>, the directory should contain the data files to be predicted, such as the local path: <code>/root/data/</code></li>
|
||||
<li><b>List</b>, the elements of the list should be of the above-mentioned data types, such as <code>[numpy.ndarray, numpy.ndarray]</code>, <code>[\"/root/data/img1.jpg\", \"/root/data/img2.jpg\"]</code>, <code>[\"/root/data1\", \"/root/data2\"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size</td>
|
||||
<td><code>int</code></td>
|
||||
<td>Any integer greater than 0</td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td>Side length limit for detection</td>
|
||||
<td><code>int/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>int</b>: Any integer greater than 0
|
||||
<li><b>None</b>: If set to None, the parameter value initialized by the model will be used by default</li></li></ul></td>
|
||||
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td>Type of side length limit for detection</td>
|
||||
<td><code>str/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>str</b>: Supports min and max. min indicates that the shortest side of the image is not less than det_limit_side_len, max indicates that the longest side of the image is not greater than limit_side_len
|
||||
<li><b>None</b>: If set to None, the parameter value initialized by the model will be used by default</li></li></ul></td>
|
||||
|
||||
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td>In the output probability map, pixels with scores greater than this threshold will be considered as text pixels</td>
|
||||
<td><code>float/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>float</b>: Any float greater than 0
|
||||
<li><b>None</b>: If set to None, the parameter value initialized by the model will be used by default</li></li></ul></td>
|
||||
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td>If the average score of all pixels within the detection result box is greater than this threshold, the result will be considered as a text area</td>
|
||||
<td><code>float/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>float</b>: Any float greater than 0
|
||||
<li><b>None</b>: If set to None, the parameter value initialized by the model will be used by default</li></li></ul></td>
|
||||
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>max_candidates</code></td>
|
||||
<td>Maximum number of text boxes to be output</td>
|
||||
<td><code>int/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>int</b>: Any integer greater than 0
|
||||
<li><b>None</b>: If set to None, the parameter value initialized by the model will be used by default</li></li></ul></td>
|
||||
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td>Expansion coefficient of the Vatti clipping algorithm, used to expand the text area</td>
|
||||
<td><code>float/None</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>float</b>: Any float greater than 0
|
||||
<li><b>None</b>: If set to None, the parameter value initialized by the model will be used by default</li></li></ul></td>
|
||||
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_dilation</code></td>
|
||||
<td>Whether to dilate the segmentation result</td>
|
||||
<td><code>bool/None</code></td>
|
||||
<td>True/False/None</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Process the prediction results. Each sample's prediction result is a corresponding Result object, and it supports operations such as printing, saving as an image, and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Method Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Parameter Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print the result to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable. This is only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters to <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters. This is only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save the result as a file in JSON format</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The file path for saving. When it is a directory, the saved file name will be consistent with the input file name</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable. This is only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters to <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters. This is only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the result as a file in image format</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The file path for saving. When it is a directory, the saved file name will be consistent with the input file name</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* In addition, it also supports obtaining visual images with results and prediction results through attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Attribute Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">Get the visual image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
For more information on using PaddleX's single-model inference API, refer to the [PaddleX Single Model Python Script Usage Instructions](../../instructions/model_python_API.en.md).
|
||||
|
||||
## IV. Custom Development
|
||||
|
||||
If the above model is still not performing well in your scenario, you can try the following steps for secondary development. Here, we'll use training `PP-OCRv4_server_seal_det` as an example; you can replace it with the corresponding configuration files for other models. First, you need to prepare a text detection dataset. You can refer to the format of the [seal text detection demo data](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar) for preparation. Once prepared, you can follow the steps below for model training and export. After export, you can quickly integrate the model into the above API. This example uses a seal text detection demo dataset. Before training the model, please ensure that you have installed the dependencies required by PaddleOCR as per the [installation documentation](xxx).
|
||||
|
||||
### 4.1 Dataset and Pre-trained Model Preparation
|
||||
|
||||
### 4.1.1 Preparing the Dataset
|
||||
|
||||
```shell
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar -P ./dataset
|
||||
tar -xf ./dataset/ocr_curve_det_dataset_examples.tar -C ./dataset/
|
||||
```
|
||||
### 4.1.1 Preparing the pre-trained model
|
||||
|
||||
```shell
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 Model Training
|
||||
|
||||
PaddleOCR has modularized the code, and when training the `PP-OCRv4_server_seal_det` model, you need to use the [configuration file](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml) for `PP-OCRv4_server_seal_det`.
|
||||
|
||||
The training commands are as follows:
|
||||
|
||||
```bash
|
||||
# Single GPU training (default training method)
|
||||
python3 tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
|
||||
# Multi-GPU training, specify GPU ids using the --gpus parameter
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.4 Model Evaluation
|
||||
|
||||
You can evaluate the trained weights, such as `output/xxx/xxx.pdparams`, using the following command:
|
||||
|
||||
```bash
|
||||
# Make sure to set the pretrained_model path to the local path. If using a model that was trained and saved by yourself, be sure to modify the path and filename to {path/to/weights}/{model_name}.
|
||||
# Demo test set evaluation
|
||||
python3 tools/eval.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams
|
||||
```
|
||||
|
||||
### 4.5 Model Export
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams \
|
||||
save_inference_dir="./PP-OCRv4_server_seal_det_infer/"
|
||||
```
|
||||
|
||||
After exporting the model, the static graph model will be stored in the `./PP-OCRv4_server_seal_det_infer/` directory. In this directory, you will see the following files:
|
||||
```
|
||||
./PP-OCRv4_server_seal_det_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
With this, the secondary development is complete, and the static graph model can be directly integrated into PaddleOCR's API.
|
||||
|
||||
## 5. FAQ
|
||||
@@ -6,7 +6,7 @@ comments: true
|
||||
|
||||
## 一、概述
|
||||
|
||||
印章文本检测模块通常会输出文本区域的多点边界框（Bounding Boxes），这些边界框将作为输入传递给弯曲矫正和文本识别模块进行后续处理，识别出印章的文字内容。印章文本的识别是文档处理的一部分，在很多场景都有用途，例如合同比对，出入库审核以及发票报销审核等场景。印章文本检测模块是OCR（光学字符识别）中的子任务，负责在图像中定位和标记出包含印章文本的区域。该模块的性能直接影响到整个印章文本OCR系统的准确性和效率。
|
||||
印章文本检测模块通常会输出文本区域的多点边界框（Bounding Boxes），这些边界框将作为输入传递给弯曲矫正和文本检测模块进行后续处理，识别出印章的文字内容。印章文本的识别是文档处理的一部分，在很多场景都有用途，例如合同比对，出入库审核以及发票报销审核等场景。印章文本检测模块是OCR（光学字符识别）中的子任务，负责在图像中定位和标记出包含印章文本的区域。该模块的性能直接影响到整个印章文本OCR系统的准确性和效率。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
@@ -448,4 +448,70 @@ for res in output:
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
......
|
||||
如果以上模型在您的场景上效果仍然不理想，您可以尝试以下步骤进行二次开发，此处以训练 `PP-OCRv4_server_seal_det` 举例，其他模型替换对应配置文件即可。首先，您需要准备文本检测的数据集，可以参考[印章文本检测 Demo 数据](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar)的格式准备，准备好后，即可按照以下步骤进行模型训练和导出，导出后，可以将模型快速集成到上述 API 中。此处以印章文本检测 Demo 数据示例。在训练模型之前，请确保已经按照[安装文档](xxx)安装了 PaddleOCR 所需要的依赖。
|
||||
|
||||
|
||||
## 4.1 数据集、预训练模型准备
|
||||
|
||||
### 4.1.1 准备数据集
|
||||
|
||||
```shell
|
||||
# 下载示例数据集
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar -P ./dataset
|
||||
tar -xf ./dataset/ocr_curve_det_dataset_examples.tar -C ./dataset/
|
||||
```
|
||||
|
||||
### 4.1.2 下载预训练模型
|
||||
|
||||
```shell
|
||||
# 下载 PP-OCRv4_server_seal_det 预训练模型
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 模型训练
|
||||
|
||||
PaddleOCR 对代码进行了模块化，训练 `PP-OCRv4_server_seal_det` 模型时需要使用 `PP-OCRv4_server_seal_det` 的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml)。
|
||||
|
||||
|
||||
训练命令如下：
|
||||
|
||||
```bash
|
||||
#单卡训练 (默认训练方式)
|
||||
python3 tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
|
||||
#多卡训练，通过--gpus参数指定卡号
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
|
||||
### 4.4 模型评估
|
||||
|
||||
您可以评估已经训练好的权重，如，`output/xxx/xxx.pdprams`，使用如下命令进行评估：
|
||||
|
||||
```bash
|
||||
# 注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
|
||||
# demo 测试集评估
|
||||
python3 tools/eval.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdprams
|
||||
```
|
||||
|
||||
### 4.5 模型导出
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdprams \
|
||||
save_inference_dir="./PP-OCRv4_server_seal_det_infer/"
|
||||
```
|
||||
|
||||
导出模型后，静态图模型会存放于当前目录的`./PP-OCRv4_server_seal_det_infer/`中，在该目录下，您将看到如下文件：
|
||||
```
|
||||
./PP-OCRv4_server_seal_det_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
至此，二次开发完成，该静态图模型可以直接集成到 PaddleOCR 的 API 中。
|
||||
|
||||
## 五、FAQ
|
||||
|
||||
@@ -641,6 +641,7 @@ PaddleOCR 对代码进行了模块化，训练 `PP-OCRv5_server_rec` 识别模
|
||||
#单卡训练 (默认训练方式)
|
||||
python3 tools/train.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_rec_pretrained.pdparams
|
||||
|
||||
#多卡训练，通过--gpus参数指定卡号
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_rec_pretrained.pdparams
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -24,6 +24,31 @@ comments: true
|
||||
<details>
|
||||
<summary> <b>版面区域检测模块（可选）：</b></summary>
|
||||
|
||||
* <b>版面检测模型，包含20个常见的类别：文档标题、段落标题、文本、页码、摘要、目录、参考文献、脚注、页眉、页脚、算法、公式、公式编号、图像、表格、图和表标题（图标题、表格标题和图表标题）、印章、图表、侧栏文本和参考文献内容</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（M）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayout_plus-L</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout_plus-L_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout_plus-L_pretrained.pdparams">训练模型</a></td>
|
||||
<td>83.2</td>
|
||||
<td>34.6244 / 10.3945</td>
|
||||
<td>510.57 / - </td>
|
||||
<td>126.01 M</td>
|
||||
<td>基于RT-DETR-L在包含中英文论文、多栏杂志、报纸、PPT、合同、书本、试卷、研报、古籍、日文文档、竖版文字文档等场景的自建数据集训练的更高精度版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* <b>版面检测模型，包含23个常见的类别：文档标题、段落标题、文本、页码、摘要、目录、参考文献、脚注、页眉、页脚、算法、公式、公式编号、图像、图表标题、表格、表格标题、印章、图表标题、图表、页眉图像、页脚图像、侧栏文本</b>
|
||||
<table>
|
||||
<thead>
|
||||
@@ -65,7 +90,7 @@ comments: true
|
||||
</table>
|
||||
|
||||
|
||||
>❗ 以上列出的是版面检测模块重点支持的<b>3个核心模型</b>，该模块总共支持<b>11个全量模型</b>，包含多个预定义了不同类别的模型，其中包含印章类别的模型有9个，除上述3个核心模型外，其余模型列表如下：
|
||||
>❗ 以上列出的是版面检测模块重点支持的<b>4个核心模型</b>，该模块总共支持<b>13个全量模型</b>，包含多个预定义了不同类别的模型，其中包含印章类别的模型有9个，除上述3个核心模型外，其余模型列表如下：
|
||||
|
||||
|
||||
<details><summary> 👉模型列表详情</summary>
|
||||
@@ -947,9 +972,9 @@ paddleocr seal_recognition -i ./seal_text_det.png --device gpu
|
||||
from paddleocr import SealRecognition
|
||||
|
||||
pipeline = SealRecognition()
|
||||
# ocr = TableRecognitionPipelineV2(use_doc_orientation_classify=True) # 通过 use_doc_orientation_classify 指定是否使用文档方向分类模型
|
||||
# ocr = TableRecognitionPipelineV2(use_doc_unwarping=True) # 通过 use_doc_unwarping 指定是否使用文本图像矫正模块
|
||||
# ocr = TableRecognitionPipelineV2(device="gpu") # 通过 device 指定模型推理时使用 GPU
|
||||
# ocr = SealRecognition(use_doc_orientation_classify=True) # 通过 use_doc_orientation_classify 指定是否使用文档方向分类模型
|
||||
# ocr = SealRecognition(use_doc_unwarping=True) # 通过 use_doc_unwarping 指定是否使用文本图像矫正模块
|
||||
# ocr = SealRecognition(device="gpu") # 通过 device 指定模型推理时使用 GPU
|
||||
output = pipeline.predict("./seal_text_det.png")
|
||||
for res in output:
|
||||
res.print() ## 打印预测的结构化输出
|
||||
@@ -1751,4 +1776,41 @@ for i, res in enumerate(result["sealRecResults"]):
|
||||
## 4. 二次开发
|
||||
如果印章文本识别产线提供的默认模型权重在您的场景中，精度或速度不满意，您可以尝试利用<b>您自己拥有的特定领域或应用场景的数据</b>对现有模型进行进一步的<b>微调</b>，以提升印章文本识别产线的在您的场景中的识别效果。
|
||||
|
||||
......
|
||||
由于印章文本识别产线包含若干模块，模型产线的效果如果不及预期，可能来自于其中任何一个模块。您可以对识别效果差的图片进行分析，进而确定是哪个模块存在问题，并参考以下表格中对应的微调教程链接进行模型微调。
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>情形</th>
|
||||
<th>微调模块</th>
|
||||
<th>微调参考链接</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>印章位置检测不准或未检出</td>
|
||||
<td>版面检测模块</td>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/layout_detection.html">链接</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>印章文本存在漏检</td>
|
||||
<td>印章文本检测模块</td>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/seal_text_detection.html">链接</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>文本内容不准</td>
|
||||
<td>文本识别模块</td>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/text_recognition.html">链接</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>整图旋转矫正不准</td>
|
||||
<td>文档图像方向分类模块</td>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/doc_img_orientation_classification.html">链接</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>图像扭曲矫正不准</td>
|
||||
<td>文本图像矫正模块</td>
|
||||
<td>暂不支持微调</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
Reference in New Issue
Block a user