+
+Relevant methods, parameters, and explanations are as follows:
+
+* `LayoutDetection` instantiates a target detection model (here, `PP-DocLayout_plus-L` is used as an example). The detailed explanation is as follows:
+
👉模型列表详情
@@ -187,8 +242,9 @@ comments: true- 测试数据集：
-
-
- 版面检测模型： PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志、合同、书本、试卷和研报等常见的 500 张文档类型图片。 -
- 表格版面检测模型：PaddleOCR 自建的版面表格区域检测数据集，包含中英文 7835 张带有表格的论文文档类型图片。 +
- 20类版面检测模型： PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1300 张文档类型图片。 +
- 1类版面子区域检测模型： PaddleOCR 自建的版面子区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1000 张文档类型图片。 +
- 23类版面检测模型： PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志、合同、书本、试卷和研报等常见的 500 张文档类型图片。
- 3类版面检测模型：PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志和研报等常见的 1154 张文档类型图片。
- 5类英文文档区域检测模型： PubLayNet 的评估数据集，包含英文文档的 11245 张图片。
- 17类区域检测模型：PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志和研报等常见的 892 张文档类型图片。 @@ -247,7 +303,7 @@ paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex ```python from paddleocr import LayoutDetection -model = LayoutDetection(model_name="PP-DocLayout-L") +model = LayoutDetection(model_name="PP-DocLayout_plus-L") output = model.predict("layout.jpg", batch_size=1, layout_nms=True) for res in output: res.print() @@ -258,7 +314,7 @@ for res in output: 运行后，得到的结果为： ```bash -{'res': {'input_path': 'layout.jpg', 'page_index': None, 'boxes': [{'cls_id': 8, 'label': 'table', 'score': 0.9866452813148499, 'coordinate': [74.30952, 105.71375, 321.98676, 299.11072]}, {'cls_id': 2, 'label': 'text', 'score': 0.9859839081764221, 'coordinate': [34.65901, 349.91003, 358.33832, 611.3427]}, {'cls_id': 2, 'label': 'text', 'score': 0.9850624799728394, 'coordinate': [34.945335, 647.378, 358.32523, 849.23413]}, {'cls_id': 8, 'label': 'table', 'score': 0.9850127696990967, 'coordinate': [438.06952, 105.37871, 662.88666, 313.88693]}, {'cls_id': 2, 'label': 'text', 'score': 0.9847850799560547, 'coordinate': [385.97076, 497.04156, 710.9561, 697.6779]}, {'cls_id': 2, 'label': 'text', 'score': 0.9805672764778137, 'coordinate': [385.79672, 345.93826, 710.07385, 459.14514]}, {'cls_id': 2, 'label': 'text', 'score': 0.9799845814704895, 'coordinate': [386.07553, 735.38086, 710.6084, 850.1987]}, {'cls_id': 9, 'label': 'table_title', 'score': 0.9376267194747925, 'coordinate': [35.27453, 19.852173, 358.92462, 77.81253]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8755457997322083, 'coordinate': [386.6317, 476.607, 699.7845, 490.11603]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8617177605628967, 'coordinate': [387.27615, 715.9574, 524.3855, 729.2082]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8607730865478516, 'coordinate': [35.451878, 627.4962, 185.63542, 640.4025]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8575080633163452, 'coordinate': [35.334385, 330.80493, 141.46948, 344.40747]}, {'cls_id': 9, 'label': 'table_title', 'score': 0.7959735989570618, 'coordinate': [385.93796, 19.755125, 711.5128, 75.00555]}]}} +{'res': {'input_path': 'layout.jpg', 'page_index': None, 'boxes': [{'cls_id': 2, 'label': 'text', 'score': 0.9870226979255676, 'coordinate': [34.101906, 349.85275, 358.59213, 611.0772]}, {'cls_id': 2, 'label': 'text', 'score': 0.9866003394126892, 'coordinate': [34.500324, 647.1585, 358.29367, 848.66797]}, {'cls_id': 2, 'label': 'text', 'score': 0.9846674203872681, 'coordinate': [385.71445, 497.40973, 711.2261, 697.84265]}, {'cls_id': 8, 'label': 'table', 'score': 0.984126091003418, 'coordinate': [73.76879, 105.94899, 321.95303, 298.84888]}, {'cls_id': 8, 'label': 'table', 'score': 0.9834211468696594, 'coordinate': [436.95642, 105.81531, 662.7168, 313.48462]}, {'cls_id': 2, 'label': 'text', 'score': 0.9832247495651245, 'coordinate': [385.62787, 346.2288, 710.10095, 458.77127]}, {'cls_id': 2, 'label': 'text', 'score': 0.9816061854362488, 'coordinate': [385.7802, 735.1931, 710.56134, 849.9764]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9577341079711914, 'coordinate': [34.421448, 20.055151, 358.71283, 76.53663]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9505634307861328, 'coordinate': [385.72278, 20.053688, 711.29333, 74.92744]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.9001723527908325, 'coordinate': [386.46344, 477.03488, 699.4023, 490.07474]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8845751285552979, 'coordinate': [35.413048, 627.73596, 185.58383, 640.52264]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8837394118309021, 'coordinate': [387.17603, 716.3423, 524.7841, 729.258]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8508939743041992, 'coordinate': [35.50064, 331.18445, 141.6444, 344.81097]}]}} ``` 参数含义如下： @@ -277,7 +333,7 @@ for res in output: 相关方法、参数等说明如下： -* `LayoutDetection`实例化目标检测模型（此处以`PP-DocLayout-L`为例），具体说明如下： +* `LayoutDetection`实例化目标检测模型（此处以`PP-DocLayout_plus-L`为例），具体说明如下：
- Performance Test Environment
+
-
+
- Test Dataset： PaddleX Custom Dataset, Containing 500 Images of Circular Stamps. +
- Hardware Configuration：
+
-
+
- GPU: NVIDIA Tesla T4 +
- CPU: Intel Xeon Gold 6271C @ 2.60GHz +
- Other Environments: Ubuntu 20.04 / cuDNN 8.6 / TensorRT 8.5.2.2 +
+
+ - Inference Mode Description +
- int: Any integer greater than 0 +
- None: If set to None, the default value from the official PaddleX model configuration will be used
- str: Supports min and max. min ensures the shortest side of the image is not less than det_limit_side_len, max ensures the longest side is not greater than limit_side_len +
- None: If set to None, the default value from the official PaddleX model configuration will be used
- float: Any float greater than 0 +
- None: If set to None, the default value from the official PaddleX model configuration will be used
- float: Any float greater than 0 +
- None: If set to None, the default value from the official PaddleX model configuration will be used
- int: Any integer greater than 0 +
- None: If set to None, the default value from the official PaddleX model configuration will be used
- float: Any float greater than 0 +
- None: If set to None, the default value from the official PaddleX model configuration will be used
- Python Variable, such as image data represented by
numpy.ndarray
+ - File Path, such as the local path of an image file:
/root/data/img.jpg
+ - URL Link, such as the web URL of an image file: Example +
- Local Directory, the directory should contain the data files to be predicted, such as the local path:
/root/data/
+ - List, the elements of the list should be of the above-mentioned data types, such as
[numpy.ndarray, numpy.ndarray],[\"/root/data/img1.jpg\", \"/root/data/img2.jpg\"],[\"/root/data1\", \"/root/data2\"]
+ - int: Any integer greater than 0 +
- None: If set to None, the parameter value initialized by the model will be used by default
- str: Supports min and max. min indicates that the shortest side of the image is not less than det_limit_side_len, max indicates that the longest side of the image is not greater than limit_side_len +
- None: If set to None, the parameter value initialized by the model will be used by default
- float: Any float greater than 0 +
- None: If set to None, the parameter value initialized by the model will be used by default
- float: Any float greater than 0 +
- None: If set to None, the parameter value initialized by the model will be used by default
- int: Any integer greater than 0 +
- None: If set to None, the parameter value initialized by the model will be used by default
- float: Any float greater than 0 +
- None: If set to None, the parameter value initialized by the model will be used by default
- Performance Test Environment
+
-
+
- Test Dataset：
+
-
+
- Document Image Orientation Classification Module: A self-built dataset using PaddleX, covering multiple scenarios such as ID cards and documents, containing 1000 images. +
- Text Image Rectification Model: DocUNet +
- Layout Detection Model: A self-built layout detection dataset using PaddleOCR, including 500 images of common document types such as Chinese and English papers, magazines, contracts, books, exam papers, and research reports. +
- 3-Category Layout Detection Model: A self-built layout detection dataset using PaddleOCR, containing 1154 images of common document types such as Chinese and English papers, magazines, and research reports. +
- 17-Category Region Detection Model: A self-built layout detection dataset using PaddleOCR, including 892 images of common document types such as Chinese and English papers, magazines, and research reports. +
- Text Detection Model: A self-built Chinese dataset using PaddleOCR, covering multiple scenarios such as street scenes, web images, documents, and handwriting, with 500 images for detection. +
- Chinese Recognition Model: A self-built Chinese dataset using PaddleOCR, covering multiple scenarios such as street scenes, web images, documents, and handwriting, with 11,000 images for text recognition. +
- ch_SVTRv2_rec: Evaluation set A for "OCR End-to-End Recognition Task" in the PaddleOCR Algorithm Model Challenge +
- ch_RepSVTR_rec: Evaluation set B for "OCR End-to-End Recognition Task" in the PaddleOCR Algorithm Model Challenge. +
- English Recognition Model: A self-built English dataset using PaddleX. +
- Multilingual Recognition Model: A self-built multilingual dataset using PaddleX. +
- Text Line Orientation Classification Model: A self-built dataset using PaddleX, covering various scenarios such as ID cards and documents, containing 1000 images. +
- Seal Text Detection Model: A self-built dataset using PaddleX, containing 500 images of circular seal textures. +
+ - Hardware Configuration：
+
-
+
- GPU: NVIDIA Tesla T4 +
- CPU: Intel Xeon Gold 6271C @ 2.60GHz +
- Other Environments: Ubuntu 20.04 / cuDNN 8.6 / TensorRT 8.5.2.2 +
+
+ - Test Dataset：
+
- Inference Mode Description +
- Python Var: Image data represented by
numpy.ndarray
+ - str: Local path of an image or PDF file, e.g.,
/root/data/img.jpg; URL link, e.g., the network URL of an image or PDF file: Example; Local directory, containing images to be predicted, e.g.,/root/data/(currently does not support prediction of PDF files in directories; PDF files must be specified with an exact file path)
+ - List: Elements of the list must be of the above types, e.g.,
[numpy.ndarray, numpy.ndarray],[\"/root/data/img1.jpg\", \"/root/data/img2.jpg\"],[\"/root/data1\", \"/root/data2\"]
+ - CPU: e.g.,
cpufor CPU inference;
+ - GPU: e.g.,
gpu:0for inference using the first GPU;
+ - NPU: e.g.,
npu:0for inference using the first NPU;
+ - XPU: e.g.,
xpu:0for inference using the first XPU;
+ - MLU: e.g.,
mlu:0for inference using the first MLU;
+ - DCU: e.g.,
dcu:0for inference using the first DCU;
+ - None: If set to
None, the default value from the pipeline initialization will be used. During initialization, the local GPU device 0 will be prioritized; if unavailable, the CPU device will be used.
+ - bool:
TrueorFalse;
+ - None: If set to
None, the default value from the pipeline initialization will be used, initialized asTrue.
+ - bool:
TrueorFalse;
+ - None: If set to
None, the default value from the pipeline initialization will be used, initialized asTrue.
+ - bool:
TrueorFalse;
+ - None: If set to
None, the default value from the pipeline initialization will be used, initialized asTrue.
+ - float: Any float greater than
0
+ - dict: Key is the int category ID, value is any float greater than
0
+ - None: If set to
None, the default value from the pipeline initialization will be used, initialized as0.5
+ - bool:
TrueorFalse;
+ - None: If set to
None, the default value from the pipeline initialization will be used, initialized asTrue.
+ - float: Any float greater than 0, e.g., 1.1, which means expanding the width and height of the detection box by 1.1 times while keeping the center unchanged +
- list: e.g., [1.2, 1.5], which means expanding the width of the detection box by 1.2 times and the height by 1.5 times while keeping the center unchanged +
- None: If set to
None, the default value from the pipeline initialization will be used, initialized as 1.0
+ - large: When set to
large, only the largest external box will be retained for overlapping detection boxes, and the internal overlapping boxes will be removed.
+ - small: When set to
small, only the smallest internal box will be retained for overlapping detection boxes, and the external overlapping boxes will be removed.
+ - union: No filtering of boxes will be performed; both internal and external boxes will be retained. +
- None: If set to
None, the default value from the pipeline initialization will be used, initialized aslarge.
+ - int: Any integer greater than
0
+ - None: If set to
None, the default value from the pipeline initialization will be used, initialized as736
+ - float: Any float greater than
0
+ - None: If set to
None, the default value from the pipeline initialization will be used, initialized as0.0. This means no threshold is applied.
+ - float: Any floating point number greater than
0
+ - dict: Keys are int category IDs, values are any floating point number greater than
0
+ - None: If set to
None, the default value initialized by the production line,0.5, will be used
+ - float: A floating point number greater than 0, e.g., 1.1, indicates that the width and height of the detection box output by the model will be expanded by 1.1 times, keeping the center unchanged +
- list: e.g., [1.2, 1.5], indicates that the width will be expanded by 1.2 times and the height by 1.5 times, keeping the center unchanged +
- None: If set to
None, the default value initialized by the production line, initialized to 1.0, will be used
+ - large: When set to large, only the largest external box will be retained for overlapping detection boxes, and overlapping internal boxes will be deleted +
- small: When set to small, only the small internal box will be retained, and overlapping external boxes will be deleted +
- union: No filtering will be performed, and both internal and external boxes will be retained +
- None: If set to
None, the default value initialized by the production line, initialized tolarge, will be used
+ - int: Any integer greater than
0
+ - None: If set to
None, the default value initialized by the production line, initialized to736, will be used
+ - str: Supports
minandmax;minensures that the shortest side of the image is not less thandet_limit_side_len, whilemaxensures that the longest side of the image is not greater thanlimit_side_len
+ - None: If set to
None, the default value initialized by the production line, initialized tomin, will be used
+ - float: Any floating point number greater than
0
+ - None: If set to
None, the default value initialized by the production line,0.2, will be used
+ - float: Any floating point number greater than
0
+ - None: If set to
None, the default value initialized by the production line,0.6, will be used
+ - float: Any floating point number greater than
0
+ - None: If set to
None, the default value initialized by the production line,0.5, will be used
+ - float: Any floating point number greater than
0
+ - None: If set to
None, the default value initialized by the production line,0.0, will be used, meaning no threshold is set
+ - CPU: e.g.,
cpuindicates using the CPU for inference
+ - GPU: e.g.,
gpu:0indicates using the first GPU for inference
+ - NPU: e.g.,
npu:0indicates using the first NPU for inference
+ - XPU: e.g.,
xpu:0indicates using the first XPU for inference
+ - MLU: e.g.,
mlu:0indicates using the first MLU for inference
+ - DCU: e.g.,
dcu:0indicates using the first DCU for inference
+ - None: If set to
None, the default value initialized by the production line will be used, which will prioritize using local GPU 0 if available, otherwise it will use the CPU
+ - Python Var: Image data represented by
numpy.ndarray
+ - str: Local path of an image or PDF file, e.g.,
/root/data/img.jpg; URL link, e.g., the network URL of an image or PDF file: Example; Local directory, containing images to be predicted, e.g.,/root/data/(currently does not support prediction of PDF files in directories; PDF files must be specified with an exact file path)
+ - List: Elements of the list must be of the above types, e.g.,
[numpy.ndarray, numpy.ndarray],[\"/root/data/img1.jpg\", \"/root/data/img2.jpg\"],[\"/root/data1\", \"/root/data2\"]
+ - CPU: e.g.,
cpufor CPU inference;
+ - GPU: e.g.,
gpu:0for inference using the first GPU;
+ - NPU: e.g.,
npu:0for inference using the first NPU;
+ - XPU: e.g.,
xpu:0for inference using the first XPU;
+ - MLU: e.g.,
mlu:0for inference using the first MLU;
+ - DCU: e.g.,
dcu:0for inference using the first DCU;
+ - None: If set to
None, the default value from the pipeline initialization will be used. During initialization, the local GPU device 0 will be prioritized; if unavailable, the CPU device will be used.
+ - bool:
TrueorFalse;
+ - None: If set to
None, the default value from the pipeline initialization will be used, initialized asTrue.
+ - bool:
TrueorFalse;
+ - None: If set to
None, the default value from the pipeline initialization will be used, initialized asTrue.
+ - bool:
TrueorFalse;
+ - None: If set to
None, the default value from the pipeline initialization will be used, initialized asTrue.
+ - float: Any float greater than
0
+ - dict: Key is the int category ID, value is any float greater than
0
+ - None: If set to
None, the default value from the pipeline initialization will be used, initialized as0.5
+ - bool:
TrueorFalse;
+ - None: If set to
None, the default value from the pipeline initialization will be used, initialized asTrue.
+ - float: Any float greater than 0, e.g., 1.1, which means expanding the width and height of the detection box by 1.1 times while keeping the center unchanged +
- list: e.g., [1.2, 1.5], which means expanding the width of the detection box by 1.2 times and the height by 1.5 times while keeping the center unchanged +
- None: If set to
None, the default value from the pipeline initialization will be used, initialized as 1.0
+ - large: When set to
large, only the largest external box will be retained for overlapping detection boxes, and the internal overlapping boxes will be removed.
+ - small: When set to
small, only the smallest internal box will be retained for overlapping detection boxes, and the external overlapping boxes will be removed.
+ - union: No filtering of boxes will be performed; both internal and external boxes will be retained. +
- None: If set to
None, the default value from the pipeline initialization will be used, initialized aslarge.
+ - int: Any integer greater than
0
+ - None: If set to
None, the default value from the pipeline initialization will be used, initialized as736
+ - float: Any float greater than
0
+ - None: If set to
None, the default value from the pipeline initialization will be used, initialized as0.0. This means no threshold is applied.
+ - The HTTP request method is POST. +
- The request body and response body are both JSON data (JSON objects). +
- When the request is processed successfully, the response status code is
200, and the attributes of the response body are as follows:
+ - When the request is not processed successfully, the attributes of the response body are as follows: +
infer
+- The attributes of the request body are as follows: +
- When the request is processed successfully, the
resultin the response body has the following properties:
+
@@ -564,4 +620,6 @@ for res in output: ## 四、二次开发 -...... +由于 PaddleOCR 并不直接提供版面区域检测模块的训练，因此，如果需要训练版面区域测模型，可以参考 [PaddleX 版面区域检测模块二次开发](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/layout_detection.html#_5)部分进行训练。训练后的模型可以无缝集成到 PaddleOCR 的 API 中进行推理。 + +## 五、FAQ diff --git a/docs/version3.x/module_usage/seal_text_detection.en.md b/docs/version3.x/module_usage/seal_text_detection.en.md new file mode 100644 index 0000000000..bb8fbf8e59 --- /dev/null +++ b/docs/version3.x/module_usage/seal_text_detection.en.md @@ -0,0 +1,516 @@ +--- +comments: true +--- + +# Seal Text Detection Module Tutorial + +## I. Overview +The seal text detection module typically outputs multi-point bounding boxes around text regions, which are then passed as inputs to the distortion correction and text recognition modules for subsequent processing to identify the textual content of the seal. Recognizing seal text is an integral part of document processing and finds applications in various scenarios such as contract comparison, inventory access auditing, and invoice reimbursement verification. The seal text detection module serves as a subtask within OCR (Optical Character Recognition), responsible for locating and marking the regions containing seal text within an image. The performance of this module directly impacts the accuracy and efficiency of the entire seal text OCR system. + +## II. Supported Model List + + + + +
+ +Test Environment Description: + ++ + + +Model Name Model Download Link +Hmean（%） +GPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]CPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]Model Size (M) +Description ++ +PP-OCRv4_server_seal_det Inference Model/Training Model +98.21 +74.75 / 67.72 +382.55 / 382.55 +109 M +The server-side seal text detection model of PP-OCRv4 boasts higher accuracy and is suitable for deployment on better-equipped servers. ++ + +PP-OCRv4_mobile_seal_det Inference Model/Training Model +96.47 +7.82 / 3.09 +48.28 / 23.97 +4.6 M +The mobile-side seal text detection model of PP-OCRv4, on the other hand, offers greater efficiency and is suitable for deployment on end devices. +-
+
+ +
+ + +## III. Quick Integration + +> ❗ Before quick integration, please install the PaddleOCR wheel package. For detailed instructions, refer to [PaddleOCR Local Installation Tutorial](../ppocr/installation.en.md)。 + +Quickly experience with just one command: + +```bash +paddleocr seal_text_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png +``` + + +You can also integrate the model inference from the layout area detection module into your project. Before running the following code, please download [Example Image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png) Go to the local area. + +```python +from paddleocr import SealTextDetection +model = SealTextDetection(model_name="PP-OCRv4_server_seal_det") +output = model.predict("seal_text_det.png", batch_size=1) +for res in output: + res.print() + res.save_to_img(save_path="./output/") + res.save_to_json(save_path="./output/res.json") +``` + +After running, the result is: + +```bash +{'res': {'input_path': 'seal_text_det.png', 'page_index': None, 'dt_polys': [array([[463, 477], + ..., + [428, 505]]), array([[297, 444], + ..., + [230, 443]]), array([[457, 346], + ..., + [267, 345]]), array([[325, 38], + ..., + [322, 37]])], 'dt_scores': [0.9912680344777314, 0.9906849624837963, 0.9847219455533163, 0.9914791724153904]}} +``` + +The meanings of the parameters are as follows: +- `input_path`: represents the path of the input image to be predicted +- `dt_polys`: represents the predicted text detection boxes, where each text detection box contains multiple vertices of a polygon. Each vertex is a list of two elements, representing the x and y coordinates of the vertex respectively +- `dt_scores`: represents the confidence scores of the predicted text detection boxes + +The visualization image is as follows: + ++ + + +Mode +GPU Configuration +CPU Configuration +Acceleration Technology Combination ++ +Normal Mode +FP32 Precision / No TRT Acceleration +FP32 Precision / 8 Threads +PaddleInference ++ + +High-Performance Mode +Optimal combination of pre-selected precision types and acceleration strategies +FP32 Precision / 8 Threads +Pre-selected optimal backend (Paddle/OpenVINO/TRT, etc.) +
+
+The explanations of related methods and parameters are as follows:
+
+* `SealTextDetection` instantiates a text detection model (here we take `PP-OCRv4_server_seal_det` as an example), and the specific explanations are as follows:
++ +
+ +* The `model_name` must be specified. After specifying `model_name`, the built-in model parameters of PaddleX will be used by default. On this basis, if `model_dir` is specified, the user-defined model will be used. + +* The `predict()` method of the seal text detection model is called for inference prediction. The parameters of the `predict()` method include `input`, `batch_size`, `limit_side_len`, `limit_type`, `thresh`, `box_thresh`, `max_candidates`, `unclip_ratio`, and `use_dilation`. The specific descriptions are as follows: + ++ + +Parameter +Parameter Description +Parameter Type +Options +Default Value ++ +model_nameName of the model +strAll model names supported by PaddleX for seal text detection +None ++ +model_dirPath to store the model +strNone +None ++ +deviceThe device used for model inference +strIt supports specifying specific GPU card numbers, such as "gpu:0", other hardware card numbers, such as "npu:0", or CPU, such as "cpu". +gpu:0+ +limit_side_lenLimit on the side length of the image for detection +int/None+ + +-
+
None ++ +limit_typeType of side length limit for detection +str/None+ + + +-
+
None ++ +threshIn the output probability map, pixels with scores greater than this threshold will be considered as text pixels +float/None+ + +-
+
None ++ +box_threshIf the average score of all pixels within a detection result box is greater than this threshold, the result will be considered as a text region +float/None+ + +-
+
None ++ +max_candidatesMaximum number of text boxes to output +int/None+ + +-
+
None ++ +unclip_ratioExpansion ratio for the Vatti clipping algorithm, used to expand the text region +float/None+ + +-
+
None ++ +use_dilationWhether to dilate the segmentation result +bool/NoneTrue/False/None +None ++ +use_hpipWhether to enable the high-performance inference plugin +boolNone +False+ +hpi_configHigh-performance inference configuration +dict|NoneNone +None+ +
+ +* Process the prediction results. Each sample's prediction result is a corresponding Result object, and it supports operations such as printing, saving as an image, and saving as a `json` file: + ++ + +Parameter +Parameter Description +Parameter Type +Options +Default Value ++ +inputData to be predicted, supporting multiple input types +Python Var/str/dict/list+ +-
+
None ++ +batch_sizeBatch size +intAny integer greater than 0 +1 ++ +limit_side_lenSide length limit for detection +int/None+ + +-
+
None ++ +limit_typeType of side length limit for detection +str/None+ + + +-
+
None ++ +threshIn the output probability map, pixels with scores greater than this threshold will be considered as text pixels +float/None+ + +-
+
None ++ +box_threshIf the average score of all pixels within the detection result box is greater than this threshold, the result will be considered as a text area +float/None+ + +-
+
None ++ +max_candidatesMaximum number of text boxes to be output +int/None+ + +-
+
None ++ +unclip_ratioExpansion coefficient of the Vatti clipping algorithm, used to expand the text area +float/None+ + +-
+
None ++ +use_dilationWhether to dilate the segmentation result +bool/NoneTrue/False/None +None ++ +
+ +* In addition, it also supports obtaining visual images with results and prediction results through attributes, as follows: + ++ + +Method +Method Description +Parameter +Parameter Type +Parameter Description +Default Value ++ +print()Print the result to the terminal +format_jsonboolWhether to format the output content using +JSONindentationTrue+ +indentintSpecify the indentation level to beautify the output +JSONdata, making it more readable. This is only effective whenformat_jsonisTrue4 ++ +ensure_asciiboolControl whether to escape non- +ASCIIcharacters toUnicode. When set toTrue, all non-ASCIIcharacters will be escaped;Falseretains the original characters. This is only effective whenformat_jsonisTrueFalse+ +save_to_json()Save the result as a file in JSON format +save_pathstrThe file path for saving. When it is a directory, the saved file name will be consistent with the input file name +None ++ +indentintSpecify the indentation level to beautify the output +JSONdata, making it more readable. This is only effective whenformat_jsonisTrue4 ++ +ensure_asciiboolControl whether to escape non- +ASCIIcharacters toUnicode. When set toTrue, all non-ASCIIcharacters will be escaped;Falseretains the original characters. This is only effective whenformat_jsonisTrueFalse+ +save_to_img()Save the result as a file in image format +save_pathstrThe file path for saving. When it is a directory, the saved file name will be consistent with the input file name +None ++ +
+ +For more information on using PaddleX's single-model inference API, refer to the [PaddleX Single Model Python Script Usage Instructions](../../instructions/model_python_API.en.md). + +## IV. Custom Development + +If the above model is still not performing well in your scenario, you can try the following steps for secondary development. Here, we'll use training `PP-OCRv4_server_seal_det` as an example; you can replace it with the corresponding configuration files for other models. First, you need to prepare a text detection dataset. You can refer to the format of the [seal text detection demo data](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar) for preparation. Once prepared, you can follow the steps below for model training and export. After export, you can quickly integrate the model into the above API. This example uses a seal text detection demo dataset. Before training the model, please ensure that you have installed the dependencies required by PaddleOCR as per the [installation documentation](xxx). + +### 4.1 Dataset and Pre-trained Model Preparation + +### 4.1.1 Preparing the Dataset + +```shell +wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar -P ./dataset +tar -xf ./dataset/ocr_curve_det_dataset_examples.tar -C ./dataset/ +``` +### 4.1.1 Preparing the pre-trained model + +```shell +wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_seal_det_pretrained.pdparams +``` + +### 4.2 Model Training + +PaddleOCR has modularized the code, and when training the `PP-OCRv4_server_seal_det` model, you need to use the [configuration file](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml) for `PP-OCRv4_server_seal_det`. + +The training commands are as follows: + +```bash +# Single GPU training (default training method) +python3 tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \ + -o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams + +# Multi-GPU training, specify GPU ids using the --gpus parameter +python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \ + -o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams +``` + +### 4.4 Model Evaluation + +You can evaluate the trained weights, such as `output/xxx/xxx.pdparams`, using the following command: + +```bash +# Make sure to set the pretrained_model path to the local path. If using a model that was trained and saved by yourself, be sure to modify the path and filename to {path/to/weights}/{model_name}. +# Demo test set evaluation +python3 tools/eval.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \ +Global.pretrained_model=output/xxx/xxx.pdparams +``` + +### 4.5 Model Export + +```bash +python3 tools/export_model.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \ +Global.pretrained_model=output/xxx/xxx.pdparams \ +save_inference_dir="./PP-OCRv4_server_seal_det_infer/" +``` + +After exporting the model, the static graph model will be stored in the `./PP-OCRv4_server_seal_det_infer/` directory. In this directory, you will see the following files: +``` +./PP-OCRv4_server_seal_det_infer/ +├── inference.json +├── inference.pdiparams +├── inference.yml +``` +With this, the secondary development is complete, and the static graph model can be directly integrated into PaddleOCR's API. + +## 5. FAQ diff --git a/docs/version3.x/module_usage/seal_text_detection.md b/docs/version3.x/module_usage/seal_text_detection.md index eb6bb7705c..c457aa679f 100644 --- a/docs/version3.x/module_usage/seal_text_detection.md +++ b/docs/version3.x/module_usage/seal_text_detection.md @@ -6,7 +6,7 @@ comments: true ## 一、概述 -印章文本检测模块通常会输出文本区域的多点边界框（Bounding Boxes），这些边界框将作为输入传递给弯曲矫正和文本识别模块进行后续处理，识别出印章的文字内容。印章文本的识别是文档处理的一部分，在很多场景都有用途，例如合同比对，出入库审核以及发票报销审核等场景。印章文本检测模块是OCR（光学字符识别）中的子任务，负责在图像中定位和标记出包含印章文本的区域。该模块的性能直接影响到整个印章文本OCR系统的准确性和效率。 +印章文本检测模块通常会输出文本区域的多点边界框（Bounding Boxes），这些边界框将作为输入传递给弯曲矫正和文本检测模块进行后续处理，识别出印章的文字内容。印章文本的识别是文档处理的一部分，在很多场景都有用途，例如合同比对，出入库审核以及发票报销审核等场景。印章文本检测模块是OCR（光学字符识别）中的子任务，负责在图像中定位和标记出包含印章文本的区域。该模块的性能直接影响到整个印章文本OCR系统的准确性和效率。 ## 二、支持模型列表 @@ -448,4 +448,70 @@ for res in output: ## 四、二次开发 -...... +如果以上模型在您的场景上效果仍然不理想，您可以尝试以下步骤进行二次开发，此处以训练 `PP-OCRv4_server_seal_det` 举例，其他模型替换对应配置文件即可。首先，您需要准备文本检测的数据集，可以参考[印章文本检测 Demo 数据](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar)的格式准备，准备好后，即可按照以下步骤进行模型训练和导出，导出后，可以将模型快速集成到上述 API 中。此处以印章文本检测 Demo 数据示例。在训练模型之前，请确保已经按照[安装文档](xxx)安装了 PaddleOCR 所需要的依赖。 + + +## 4.1 数据集、预训练模型准备 + +### 4.1.1 准备数据集 + +```shell +# 下载示例数据集 +wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar -P ./dataset +tar -xf ./dataset/ocr_curve_det_dataset_examples.tar -C ./dataset/ +``` + +### 4.1.2 下载预训练模型 + +```shell +# 下载 PP-OCRv4_server_seal_det 预训练模型 +wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_seal_det_pretrained.pdparams +``` + +### 4.2 模型训练 + +PaddleOCR 对代码进行了模块化，训练 `PP-OCRv4_server_seal_det` 模型时需要使用 `PP-OCRv4_server_seal_det` 的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml)。 + + +训练命令如下： + +```bash +#单卡训练 (默认训练方式) +python3 tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \ + -o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams + +#多卡训练，通过--gpus参数指定卡号 +python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \ + -o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams +``` + + +### 4.4 模型评估 + +您可以评估已经训练好的权重，如，`output/xxx/xxx.pdprams`，使用如下命令进行评估： + +```bash +# 注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。 + # demo 测试集评估 + python3 tools/eval.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \ + Global.pretrained_model=output/xxx/xxx.pdprams + ``` + +### 4.5 模型导出 + +```bash + python3 tools/export_model.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \ + Global.pretrained_model=output/xxx/xxx.pdprams \ + save_inference_dir="./PP-OCRv4_server_seal_det_infer/" + ``` + + 导出模型后，静态图模型会存放于当前目录的`./PP-OCRv4_server_seal_det_infer/`中，在该目录下，您将看到如下文件： + ``` + ./PP-OCRv4_server_seal_det_infer/ + ├── inference.json + ├── inference.pdiparams + ├── inference.yml + ``` +至此，二次开发完成，该静态图模型可以直接集成到 PaddleOCR 的 API 中。 + +## 五、FAQ diff --git a/docs/version3.x/module_usage/text_recognition.md b/docs/version3.x/module_usage/text_recognition.md index c346bfd5b8..919c427803 100644 --- a/docs/version3.x/module_usage/text_recognition.md +++ b/docs/version3.x/module_usage/text_recognition.md @@ -641,6 +641,7 @@ PaddleOCR 对代码进行了模块化，训练 `PP-OCRv5_server_rec` 识别模 #单卡训练 (默认训练方式) python3 tools/train.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml \ -o Global.pretrained_model=./PP-OCRv5_server_rec_pretrained.pdparams + #多卡训练，通过--gpus参数指定卡号 python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml \ -o Global.pretrained_model=./PP-OCRv5_server_rec_pretrained.pdparams diff --git a/docs/version3.x/pipeline_usage/seal_recognition.en.md b/docs/version3.x/pipeline_usage/seal_recognition.en.md new file mode 100644 index 0000000000..297886b0cb --- /dev/null +++ b/docs/version3.x/pipeline_usage/seal_recognition.en.md @@ -0,0 +1,1899 @@ +--- +comments: true +--- + +# Seal Text Recognition Pipeline Tutorial + +## 1. Introduction to Seal Text Recognition Pipeline +Seal text recognition is a technology that automatically extracts and recognizes the content of seals from documents or images. The recognition of seal text is part of document processing and has many applications in various scenarios, such as contract comparison, warehouse entry and exit review, and invoice reimbursement review. + +The seal text recognition pipeline is used to recognize the text content of seals, extracting the text information from seal images and outputting it in text form. This pipeline integrates the industry-renowned end-to-end OCR system PP-OCRv4, supporting the detection and recognition of curved seal text. Additionally, this pipeline integrates an optional layout region localization module, which can accurately locate the layout position of the seal within the entire document. It also includes optional document image orientation correction and distortion correction functions. Based on this pipeline, millisecond-level accurate text content prediction can be achieved on a CPU. This pipeline also provides flexible service deployment methods, supporting the use of multiple programming languages on various hardware. Moreover, it offers custom development capabilities, allowing you to train and fine-tune on your own dataset based on this pipeline, and the trained model can be seamlessly integrated. + ++ + +Attribute +Attribute Description ++ +jsonGet the prediction result in +jsonformat+ +imgGet the visual image in +dictformat
+
+The seal text recognition pipeline includes a seal text detection module and a text recognition module, as well as optional layout detection module, document image orientation classification module, and text image correction module.
+
+- [Seal Text Detection Module](../module_usage/seal_text_detection.en.md)
+- [Text Recognition Module](../module_usage/text_recognition.en.md)
+- [Layout Detection Module](../module_usage/layout_detection.en.mdmd) (Optional)
+- [Document Image Orientation Classification Module](../module_usage/doc_img_orientation_classification.en.md) (Optional)
+- [Text Image Unwarping Module](../module_usage/text_image_unwarping.en.md) (Optional)
+
+If you prioritize model accuracy, choose a model with higher accuracy. If you prioritize inference speed, choose a model with faster inference speed. If you prioritize model storage size, choose a model with smaller storage size.
+
+Layout Region Detection Module (Optional):
+ +* The layout detection model includes 20 common categories: document title, paragraph title, text, page number, abstract, table, references, footnotes, header, footer, algorithm, formula, formula number, image, table, seal, figure_table title, chart, and sidebar text and lists of references ++ +
+ +* Layout detection model, including 23 common categories: document title, paragraph title, text, page number, abstract, table of contents, references, footnotes, header, footer, algorithm, formula, formula number, image, chart title, table, table title, seal, chart title, chart, header image, footer image, sidebar text ++ + + +Model Model Download Link +mAP(0.5) (%) +GPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]CPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]Model Storage Size (M) +Introduction ++ +PP-DocLayout_plus-L Inference Model/Training Model +83.2 +34.6244 / 10.3945 +510.57 / - +126.01 M +A higher-precision layout area localization model trained on a self-built dataset containing Chinese and English papers, PPT, multi-layout magazines, contracts, books, exams, ancient books and research reports using RT-DETR-L ++ ++ +
+ + +> ❗ The above list includes the 4 core models that are key supported by the text recognition module. The module actually supports a total of 13 full models. It includes multiple predefined models of different categories, among which there are 10 models specifically for the seal category. Apart from the three core models mentioned above, the remaining models are listed as follows: + ++ + + +Model Model Download Link +mAP(0.5) (%) +GPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]CPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]Model Storage Size (M) +Introduction ++ +PP-DocLayout-L Inference Model/Training Model +90.4 +34.6244 / 10.3945 +510.57 / - +123.76 M +A high-precision layout area localization model trained on a self-built dataset containing Chinese and English papers, magazines, contracts, books, exams, and research reports using RT-DETR-L. ++ +PP-DocLayout-M Inference Model/Training Model +75.2 +13.3259 / 4.8685 +44.0680 / 44.0680 +22.578 +A layout area localization model with balanced precision and efficiency, trained on a self-built dataset containing Chinese and English papers, magazines, contracts, books, exams, and research reports using PicoDet-L. ++ + +PP-DocLayout-S Inference Model/Training Model +70.9 +8.3008 / 2.3794 +10.0623 / 9.9296 +4.834 +A high-efficiency layout area localization model trained on a self-built dataset containing Chinese and English papers, magazines, contracts, books, exams, and research reports using PicoDet-S. ++ +👉 Details of Model List
+ +* 3-Class Layout Detection Model, including Table, Image, and Stamp ++ +
+ +* 17-Class Area Detection Model, including 17 common layout categories: Paragraph Title, Image, Text, Number, Abstract, Content, Figure Caption, Formula, Table, Table Caption, References, Document Title, Footnote, Header, Algorithm, Footer, and Stamp ++ + + +Model Model Download Link +mAP(0.5) (%) +GPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]CPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]Model Storage Size (M) +Introduction ++ +PicoDet-S_layout_3cls Inference Model/Training Model +88.2 +8.99 / 2.22 +16.11 / 8.73 +4.8 +A high-efficiency layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-S. ++ +PicoDet-L_layout_3cls Inference Model/Training Model +89.0 +13.05 / 4.50 +41.30 / 41.30 +22.6 +A balanced efficiency and precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-L. ++ +RT-DETR-H_layout_3cls Inference Model/Training Model +95.8 +114.93 / 27.71 +947.56 / 947.56 +470.1 +A high-precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using RT-DETR-H. ++ +
++ + + +Model Model Download Link +mAP(0.5) (%) +GPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]CPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]Model Storage Size (M) +Introduction ++ +PicoDet-S_layout_17cls Inference Model/Training Model +87.4 +9.11 / 2.12 +15.42 / 9.12 +4.8 +A high-efficiency layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-S. ++ +PicoDet-L_layout_17cls Inference Model/Training Model +89.0 +13.50 / 4.69 +43.32 / 43.32 +22.6 +A balanced efficiency and precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-L. ++ + +RT-DETR-H_layout_17cls Inference Model/Training Model +98.3 +115.29 / 104.09 +995.27 / 995.27 +470.2 +A high-precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using RT-DETR-H. +Document Image Orientation Classification Module (Optional):
++ +
++ + + +Model Model Download Link +Top-1 Acc (%) +GPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]CPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]Model Storage Size (M) +Description ++ + +PP-LCNet_x1_0_doc_ori Inference Model/Training Model +99.06 +2.31 / 0.43 +3.37 / 1.27 +7 +A document image classification model based on PP-LCNet_x1_0, containing four categories: 0 degrees, 90 degrees, 180 degrees, and 270 degrees +Text Image Correction Module (Optional):
++ +
+ ++ + + +Model Model Download Link +CER +Model Storage Size (M) +Description ++ + +UVDoc Inference Model/Training Model +0.179 +30.3 M +High-precision text image correction model +Text Detection Module:
++ +
+ ++ + + +Model Model Download Link +Detection Hmean (%) +GPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]CPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]Model Storage Size (M) +Description ++ +PP-OCRv4_server_seal_det Inference Model/Training Model +98.21 +74.75 / 67.72 +382.55 / 382.55 +109 +PP-OCRv4 server-side seal text detection model, with higher accuracy, suitable for deployment on better servers ++ + +PP-OCRv4_mobile_seal_det Inference Model/Training Model +96.47 +7.82 / 3.09 +48.28 / 23.97 +4.6 +PP-OCRv4 mobile-side seal text detection model, with higher efficiency, suitable for deployment on the edge +Text Recognition Module:
+ ++
+ +> ❗ The above list features the 4 core models that the text recognition module primarily supports. In total, this module supports 18 models. The complete list of models is as follows: + ++ +Model Model Download Link +Recognition Avg Accuracy(%) +GPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]CPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]Model Storage Size (M) +Introduction ++ +PP-OCRv4_server_rec_doc Inference Model/Training Model +81.53 +6.65 / 2.38 +32.92 / 32.92 +74.7 M +PP-OCRv4_server_rec_doc is trained on a mixed dataset of more Chinese document data and PP-OCR training data based on PP-OCRv4_server_rec. It has added the ability to recognize some traditional Chinese characters, Japanese, and special characters, and can support the recognition of more than 15,000 characters. In addition to improving the text recognition capability related to documents, it also enhances the general text recognition capability. ++ +PP-OCRv4_mobile_rec Inference Model/Training Model +78.74 +4.82 / 1.20 +16.74 / 4.64 +10.6 M ++The lightweight recognition model of PP-OCRv4 has high inference efficiency and can be deployed on various hardware devices, including edge devices. ++ +PP-OCRv4_server_rec Inference Model/Training Model +80.61 +6.58 / 2.43 +33.17 / 33.17 +71.2 M +The server-side model of PP-OCRv4 offers high inference accuracy and can be deployed on various types of servers. ++ +en_PP-OCRv4_mobile_rec Inference Model/Training Model +70.39 +4.81 / 0.75 +16.10 / 5.31 +6.8 M +The ultra-lightweight English recognition model, trained based on the PP-OCRv4 recognition model, supports the recognition of English letters and numbers. ++ +Test Environment Description: + +👉Model List Details
+ +* Chinese Recognition Model ++
+ ++ +Model Model Download Link +Recognition Avg Accuracy(%) +CPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]CPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]Model Storage Size (M) +Introduction ++ +PP-OCRv4_server_rec_doc Inference Model/Training Model +81.53 +6.65 / 2.38 +32.92 / 32.92 +74.7 M +PP-OCRv4_server_rec_doc is trained on a mixed dataset of more Chinese document data and PP-OCR training data based on PP-OCRv4_server_rec. It has added the recognition capabilities for some traditional Chinese characters, Japanese, and special characters. The number of recognizable characters is over 15,000. In addition to the improvement in document-related text recognition, it also enhances the general text recognition capability. ++ +PP-OCRv4_mobile_rec Inference Model/Training Model +78.74 +4.82 / 1.20 +16.74 / 4.64 +10.6 M +The lightweight recognition model of PP-OCRv4 has high inference efficiency and can be deployed on various hardware devices, including edge devices. ++ +PP-OCRv4_server_rec Inference Model/Training Model +80.61 +6.58 / 2.43 +33.17 / 33.17 +71.2 M +The server-side model of PP-OCRv4 offers high inference accuracy and can be deployed on various types of servers. ++ +PP-OCRv3_mobile_rec Inference Model/Training Model +72.96 +5.87 / 1.19 +9.07 / 4.28 +9.2 M +PP-OCRv3’s lightweight recognition model is designed for high inference efficiency and can be deployed on a variety of hardware devices, including edge devices. ++
+ ++ +Model Model Download Link +Recognition Avg Accuracy(%) +GPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]CPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]Model Storage Size (M) +Introduction ++ +ch_SVTRv2_rec Inference Model/Training Model +68.81 +8.08 / 2.74 +50.17 / 42.50 +73.9 M ++SVTRv2 is a server text recognition model developed by the OpenOCR team of Fudan University's Visual and Learning Laboratory (FVL). It won the first prize in the PaddleOCR Algorithm Model Challenge - Task One: OCR End-to-End Recognition Task. The end-to-end recognition accuracy on the A list is 6% higher than that of PP-OCRv4. + ++
+ +* English Recognition Model ++ +Model Model Download Link +Recognition Avg Accuracy(%) +GPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]CPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]Model Storage Size (M) +Introduction ++ +ch_RepSVTR_rec Inference Model/Training Model +65.07 +5.93 / 1.62 +20.73 / 7.32 +22.1 M +The RepSVTR text recognition model is a mobile text recognition model based on SVTRv2. It won the first prize in the PaddleOCR Algorithm Model Challenge - Task One: OCR End-to-End Recognition Task. The end-to-end recognition accuracy on the B list is 2.5% higher than that of PP-OCRv4, with the same inference speed. ++
+ +* Multilingual Recognition Model ++ +Model Model Download Link +Recognition Avg Accuracy(%) +GPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]CPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]Model Storage Size (M) +Introduction ++ +en_PP-OCRv4_mobile_rec Inference Model/Training Model +70.39 +4.81 / 0.75 +16.10 / 5.31 +6.8 M +The ultra-lightweight English recognition model trained based on the PP-OCRv4 recognition model supports the recognition of English and numbers. ++ +en_PP-OCRv3_mobile_rec Inference Model/Training Model +70.69 +5.44 / 0.75 +8.65 / 5.57 +7.8 M +The ultra-lightweight English recognition model trained based on the PP-OCRv3 recognition model supports the recognition of English and numbers. ++
++ +Model Model Download Link +Recognition Avg Accuracy(%) +GPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]CPU Inference Time (ms) +
[Normal Mode / High-Performance Mode]Model Storage Size (M) +Introduction ++ +korean_PP-OCRv3_mobile_rec Inference Model/Training Model +60.21 +5.40 / 0.97 +9.11 / 4.05 +8.6 M +The ultra-lightweight Korean recognition model trained based on the PP-OCRv3 recognition model supports the recognition of Korean and numbers. ++ +japan_PP-OCRv3_mobile_rec Inference Model/Training Model +45.69 +5.70 / 1.02 +8.48 / 4.07 +8.8 M +The ultra-lightweight Japanese recognition model trained based on the PP-OCRv3 recognition model supports the recognition of Japanese and numbers. ++ +chinese_cht_PP-OCRv3_mobile_rec Inference Model/Training Model +82.06 +5.90 / 1.28 +9.28 / 4.34 +9.7 M +The ultra-lightweight Traditional Chinese recognition model trained based on the PP-OCRv3 recognition model supports the recognition of Traditional Chinese and numbers. ++ +te_PP-OCRv3_mobile_rec Inference Model/Training Model +95.88 +5.42 / 0.82 +8.10 / 6.91 +7.8 M +The ultra-lightweight Telugu recognition model trained based on the PP-OCRv3 recognition model supports the recognition of Telugu and numbers. ++ +ka_PP-OCRv3_mobile_rec Inference Model/Training Model +96.96 +5.25 / 0.79 +9.09 / 3.86 +8.0 M +The ultra-lightweight Kannada recognition model trained based on the PP-OCRv3 recognition model supports the recognition of Kannada and numbers. ++ +ta_PP-OCRv3_mobile_rec Inference Model/Training Model +76.83 +5.23 / 0.75 +10.13 / 4.30 +8.0 M +The ultra-lightweight Tamil recognition model trained based on the PP-OCRv3 recognition model supports the recognition of Tamil and numbers. ++ +latin_PP-OCRv3_mobile_rec Inference Model/Training Model +76.93 +5.20 / 0.79 +8.83 / 7.15 +7.8 M +The ultra-lightweight Latin recognition model trained based on the PP-OCRv3 recognition model supports the recognition of Latin script and numbers. ++ +arabic_PP-OCRv3_mobile_rec Inference Model/Training Model +73.55 +5.35 / 0.79 +8.80 / 4.56 +7.8 M +The ultra-lightweight Arabic script recognition model trained based on the PP-OCRv3 recognition model supports the recognition of Arabic script and numbers. ++ +cyrillic_PP-OCRv3_mobile_rec Inference Model/Training Model +94.28 +5.23 / 0.76 +8.89 / 3.88 +7.9 M ++The ultra-lightweight cyrillic alphabet recognition model trained based on the PP-OCRv3 recognition model supports the recognition of cyrillic letters and numbers. ++ +devanagari_PP-OCRv3_mobile_rec Inference Model/Training Model +96.44 +5.22 / 0.79 +8.56 / 4.06 +7.9 M +The ultra-lightweight Devanagari script recognition model trained based on the PP-OCRv3 recognition model supports the recognition of Devanagari script and numbers. +-
+
+ +
+ + + +## 2. Quick Start + +Before using the seal text recognition production line locally, please ensure that you have completed the installation of the wheel package according to the [installation tutorial](../ppocr/installation.md). Once the installation is complete, you can experience it locally via the command line or integrate it with Python. + +### 2.1 命令行方式体验 + +You can quickly experience the seal_recognition production line effect with a single command: + +```bash +paddleocr seal_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png + +# Specify whether to use the document orientation classification model with --use_doc_orientation_classify +paddleocr seal_recognition -i ./seal_text_det.png --use_doc_orientation_classify True + +# Specify whether to use the text image correction module with --use_doc_unwarping. +paddleocr seal_recognition -i ./seal_text_det.png --use_doc_unwarping True + +# Use --device to specify the use of GPU for model inference. +paddleocr seal_recognition -i ./seal_text_det.png --device gpu +``` + + + +After running, the results will be printed to the terminal, as follows: + ++ + + +Mode +GPU Configuration +CPU Configuration +Acceleration Technology Combination ++ +Normal Mode +FP32 Precision / No TRT Acceleration +FP32 Precision / 8 Threads +PaddleInference ++ + +High-Performance Mode +Optimal combination of pre-selected precision types and acceleration strategies +FP32 Precision / 8 Threads +Pre-selected optimal backend (Paddle/OpenVINO/TRT, etc.) ++ +The explanation of the result parameters can be found in [2.1.2 Python Script Integration](#212-python-script-integration). + +The visualized results are saved under `save_path`, and the visualized result of seal OCR is as follows: + +👉Click to Expand
+ +```bash +{'res': {'input_path': 'seal_text_det.png', 'model_settings': {'use_doc_preprocessor': False, 'use_layout_detection': True}, 'layout_det_res': {'input_path': None, 'page_index': None, 'boxes': [{'cls_id': 16, 'label': 'seal', 'score': 0.975531280040741, 'coordinate': [6.195526, 0.1579895, 634.3982, 628.84595]}]}, 'seal_res_list': [{'input_path': None, 'page_index': None, 'model_settings': {'use_doc_preprocessor': False, 'use_textline_orientation': False}, 'dt_polys': [array([[320, 38], + ..., + [315, 38]]), array([[461, 347], + ..., + [456, 346]]), array([[439, 445], + ..., + [434, 444]]), array([[158, 468], + ..., + [154, 466]])], 'text_det_params': {'limit_side_len': 736, 'limit_type': 'min', 'thresh': 0.2, 'box_thresh': 0.6, 'unclip_ratio': 0.5}, 'text_type': 'seal', 'textline_orientation_angles': array([-1, ..., -1]), 'text_rec_score_thresh': 0, 'rec_texts': ['天津君和缘商贸有限公司', '发票专用章', '吗繁物', '5263647368706'], 'rec_scores': array([0.9934051 , ..., 0.99139398]), 'rec_polys': [array([[320, 38], + ..., + [315, 38]]), array([[461, 347], + ..., + [456, 346]]), array([[439, 445], + ..., + [434, 444]]), array([[158, 468], + ..., + [154, 466]])], 'rec_boxes': array([], dtype=float64)}]}} +``` + +
+
+#### 2.2.2 Python Script Integration
+
+* The above command line is for quickly experiencing and viewing the effect. Generally, in a project, you often need to integrate through code. You can complete the quick inference of the pipeline with just a few lines of code. The inference code is as follows:
+
+```python
+from paddlex import create_pipeline
+
+pipeline = create_pipeline(pipeline="seal_recognition")
+
+output = pipeline.predict(
+ "seal_text_det.png",
+ use_doc_orientation_classify=False,
+ use_doc_unwarping=False,
+)
+for res in output:
+ res.print()
+ res.save_to_img("./output/")
+ res.save_to_json("./output/")
+```
+
+In the above Python script, the following steps were executed:
+
+(1) The seal recognition pipeline object was instantiated via `create_pipeline()`, with the specific parameters described as follows:
+
++ +
+ +(2) Call the `predict()` method of the Seal Text Recognition pipeline object for inference prediction. This method will return a `generator`. Below are the parameters and their descriptions for the `predict()` method: + ++ + + +Parameter +Description +Type +Default Value ++ +pipelineThe name of the pipeline or the path to the pipeline configuration file. If it is a pipeline name, it must be supported by PaddleX. +strNone+ +configSpecific configuration information for the pipeline (if set simultaneously with +pipeline, it has higher priority thanpipeline, and the pipeline name must be consistent withpipeline).dict[str, Any]None+ +deviceThe device used for pipeline inference. It supports specifying the specific card number of the GPU, such as "gpu:0", other hardware card numbers, such as "npu:0", or CPU, such as "cpu". Supports specifying multiple devices simultaneously for parallel inference. For details, please refer to Pipeline Parallel Inference. +strgpu:0+ +use_hpipWhether to enable the high-performance inference plugin. If set to +None, the setting from the configuration file orconfigwill be used.boolNone +None+ + +hpi_configHigh-performance inference configuration +dict|NoneNone +None+ +
+ +(3) Process the prediction results. The prediction result for each sample is of `dict` type and supports operations such as printing, saving as an image, and saving as a `json` file: + ++ + +Parameter +Description +Type +Options +Default Value ++ +inputData to be predicted, supports multiple input types (required) +Python Var|str|list+ +-
+
None+ +deviceInference device for the pipeline +str|None+ +-
+
None+ +use_doc_orientation_classifyWhether to use the document orientation classification module +bool|None+ +-
+
None+ +use_doc_unwarpingWhether to use the document unwarping module +bool|None+ +-
+
None+ +use_layout_detectionWhether to use the layout detection module +bool|None+ +-
+
None+ +layout_thresholdConfidence threshold for layout detection; only scores above this threshold will be output +float|dict|None+ +-
+
None+ +layout_nmsWhether to use Non-Maximum Suppression (NMS) for layout detection post-processing +bool|None+ +-
+
None+ +layout_unclip_ratioExpansion ratio of detection box edges; if not specified, the default value from the PaddleX official model configuration will be used +float|list|None+ +-
+
+ +layout_merge_bboxes_modeMerging mode for detection boxes in layout detection output; if not specified, the default value from the PaddleX official model configuration will be used +string|None+ +-
+
None ++ +seal_det_limit_side_lenSide length limit for seal text detection +int|None+ +-
+
None+ +seal_rec_score_threshText recognition threshold; text results with scores above this threshold will be retained +float|None+ +-
+
None+ +
+ +- Calling the `print()` method will print the results to the terminal, and the explanations of the printed content are as follows: + + - `input_path`: `(str)` The input path of the image to be predicted. + + - `model_settings`: `(Dict[str, bool])` The model parameters required for pipeline configuration. + + - `use_doc_preprocessor`: `(bool)` Controls whether to enable the document preprocessing sub-pipeline. + - `use_layout_detection`: `(bool)` Controls whether to enable the layout detection sub-module. + + - `layout_det_res`: `(Dict[str, Union[List[numpy.ndarray], List[float]]])` The output result of the layout detection sub-module. Only exists when `use_layout_detection=True`. + + - `input_path`: `(Union[str, None])` The image path accepted by the layout detection module. Saved as `None` when the input is a `numpy.ndarray`. + - `page_index`: `(Union[int, None])` Indicates the current page number of the PDF if the input is a PDF file; otherwise, it is `None`. + - `boxes`: `(List[Dict])` A list of detected layout seal regions, with each element containing the following fields: + - `cls_id`: `(int)` The class ID of the detected seal region. + - `score`: `(float)` The confidence score of the detected region. + - `coordinate`: `(List[float])` The coordinates of the four corners of the detection box, in the order of x1, y1, x2, y2, representing the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, the x-coordinate of the bottom-right corner, and the y-coordinate of the bottom-right corner. + + - `seal_res_list`: `List[Dict]` A list of seal text recognition results, with each element containing the following fields: + + - `input_path`: `(Union[str, None])` The image path accepted by the seal text recognition pipeline. Saved as `None` when the input is a `numpy.ndarray`. + - `page_index`: `(Union[int, None])` Indicates the current page number of the PDF if the input is a PDF file; otherwise, it is `None`. + - `model_settings`: `(Dict[str, bool])` The model configuration parameters for the seal text recognition pipeline. + - `use_doc_preprocessor`: `(bool)` Controls whether to enable the document preprocessing sub-pipeline. + - `use_textline_orientation`: `(bool)` Controls whether to enable the text line orientation classification sub-module. + + - `doc_preprocessor_res`: `(Dict[str, Union[str, Dict[str, bool], int]])` The output result of the document preprocessing sub-pipeline. Only exists when `use_doc_preprocessor=True`. + + - `input_path`: `(Union[str, None])` The image path accepted by the document preprocessing sub-pipeline. Saved as `None` when the input is a `numpy.ndarray`. + - `model_settings`: `(Dict)` The model configuration parameters for the preprocessing sub-pipeline. + - `use_doc_orientation_classify`: `(bool)` Controls whether to enable document orientation classification. + - `use_doc_unwarping`: `(bool)` Controls whether to enable document unwarping. + - `angle`: `(int)` The predicted result of document orientation classification. When enabled, it takes values [0, 1, 2, 3], corresponding to [0°, 90°, 180°, 270°]; when disabled, it is -1. + + - `dt_polys`: `(List[numpy.ndarray])` A list of polygon boxes for seal text detection. Each detection box is represented by a numpy array of multiple vertex coordinates, with the array shape being (n, 2). + + - `dt_scores`: `(List[float])` A list of confidence scores for text detection boxes. + + - `text_det_params`: `(Dict[str, Dict[str, int, float]])` Configuration parameters for the text detection module. + - `limit_side_len`: `(int)` The side length limit value during image preprocessing. + - `limit_type`: `(str)` The handling method for side length limits. + - `thresh`: `(float)` The confidence threshold for text pixel classification. + - `box_thresh`: `(float)` The confidence threshold for text detection boxes. + - `unclip_ratio`: `(float)` The expansion ratio for text detection boxes. + - `text_type`: `(str)` The type of seal text detection, currently fixed as "seal". + + - `text_rec_score_thresh`: `(float)` The filtering threshold for text recognition results. + + - `rec_texts`: `(List[str])` A list of text recognition results, containing only texts with confidence scores above `text_rec_score_thresh`. + + - `rec_scores`: `(List[float])` A list of confidence scores for text recognition, filtered by `text_rec_score_thresh`. + + - `rec_polys`: `(List[numpy.ndarray])` A list of text detection boxes filtered by confidence score, in the same format as `dt_polys`. + + - `rec_boxes`: `(numpy.ndarray)` An array of rectangular bounding boxes for detection boxes; the seal recognition pipeline returns an empty array. + +- Calling the `save_to_json()` method will save the above content to the specified `save_path`. If a directory is specified, the saved path will be `save_path/{your_img_basename}_res.json`. If a file is specified, it will be saved directly to that file. Since JSON files do not support saving numpy arrays, `numpy.array` types will be converted to list format. + +- Calling the `save_to_img()` method will save the visualization results to the specified `save_path`. If a directory is specified, the saved path will be `save_path/{your_img_basename}_seal_res_region1.{your_img_extension}`. If a file is specified, it will be saved directly to that file. (The pipeline usually contains multiple result images, so it is not recommended to specify a specific file path directly, as multiple images will be overwritten, and only the last image will be retained.) + +* Additionally, you can obtain visualized images with results and prediction results through attributes, as follows: + ++ + +Method +Description +Parameter +Parameter Type +Parameter Description +Default Value ++ +print()Print results to the terminal +format_jsonboolWhether to format the output content using +JSONindentationTrue+ +indentintSpecify the indentation level to beautify the output +JSONdata for better readability, effective only whenformat_jsonisTrue4 ++ +ensure_asciiboolControl whether to escape non- +ASCIIcharacters toUnicode. When set toTrue, all non-ASCIIcharacters will be escaped;Falsewill retain the original characters, effective only whenformat_jsonisTrueFalse+ +save_to_json()Save results as a json file +save_pathstrThe file path to save the results. When it is a directory, the saved file name will be consistent with the input file type +None ++ +indentintSpecify the indentation level to beautify the output +JSONdata for better readability, effective only whenformat_jsonisTrue4 ++ +ensure_asciiboolControl whether to escape non- +ASCIIcharacters toUnicode. When set toTrue, all non-ASCIIcharacters will be escaped;Falsewill retain the original characters, effective only whenformat_jsonisTrueFalse+ +save_to_img()Save results as an image file +save_pathstrThe file path to save the results, supports directory or file path +None ++ +
+ +- The prediction results obtained through the `json` attribute are of dict type, with content consistent with what is saved by calling the `save_to_json()` method. +- The prediction results returned by the `img` attribute are of dict type. The keys are `layout_det_res`, `seal_res_region1`, and `preprocessed_img`, corresponding to three `Image.Image` objects: one for visualizing layout detection, one for visualizing seal text recognition results, and one for visualizing image preprocessing. If the image preprocessing sub-module is not used, `preprocessed_img` will not be included in the dictionary. If the layout region detection module is not used, `layout_det_res` will not be included. + +Additionally, you can obtain the configuration file for the seal text recognition pipeline and load the configuration file for prediction. You can execute the following command to save the results in `my_path`: + +``` +paddlex --get_pipeline_config seal_recognition --save_path ./my_path +``` + +If you have obtained the configuration file, you can customize the settings for the seal text recognition pipeline by simply modifying the `pipeline` parameter value in the `create_pipeline` method to the path of the pipeline configuration file. The example is as follows: + +```python +from paddlex import create_pipeline +pipeline = create_pipeline(pipeline="./my_path/seal_recognition.yaml") +output = pipeline.predict("seal_text_det.png") +for res in output: + res.print() ## 打印预测的结构化输出 + res.save_to_img("./output/") ## 保存可视化结果 + res.save_to_json("./output/") ## 保存预测结果的json文件 +``` + +(1) Instantiate the seal text recognition production object through `SealRecognition()`. The specific parameter descriptions are as follows: + +Here's the translation of the table into English: + ++ + +Attribute +Description ++ +jsonGet the prediction results in +jsonformat.+ +imgGet the visualization results in +dictformat.+ +
+ + +(2) Call the `predict()` method of the seal text recognition production object for inference prediction. This method will return a list of results. + +Additionally, the production line also offers the `predict_iter()` method. Both methods are identical in terms of parameter acceptance and result return; the difference lies in that `predict_iter()` returns a `generator`, allowing for gradual processing and retrieval of prediction results. This is suitable for handling large datasets or scenarios where memory saving is desired. You can choose either method based on your actual needs. + +Below are the parameters for the `predict()` method and their descriptions: + ++ + + +Parameter +Description +Type +Default Value ++ +doc_orientation_classify_model_nameName of the document orientation classification model. If set to +None, the default model will be used.strNone+ +doc_orientation_classify_model_dirDirectory path of the document orientation classification model. If set to +None, the official model will be downloaded.strNone+ +doc_unwarping_model_nameName of the text image correction model. If set to +None, the default model will be used.strNone+ +doc_unwarping_model_dirDirectory path of the text image correction model. If set to +None, the official model will be downloaded.strNone+ +layout_detection_model_nameName of the layout detection model. If set to +None, the default model will be used.strNone+ +layout_detection_model_dirDirectory path of the layout detection model. If set to +None, the official model will be downloaded.strNone+ +seal_text_detection_model_nameName of the seal text detection model. If set to +None, the default model will be used.strNone+ +seal_text_detection_model_dirDirectory path of the seal text detection model. If set to +None, the official model will be downloaded.strNone+ +text_recognition_model_nameName of the text recognition model. If set to +None, the default model will be used.strNone+ +text_recognition_model_dirDirectory path of the text recognition model. If set to +None, the official model will be downloaded.strNone+ +text_recognition_batch_sizeBatch size for the text recognition model. If set to +None, the default batch size is1.intNone+ +use_doc_orientation_classifyWhether to load the document orientation classification module. If set to +None, the default value initialized by the production line will be used, initialized toTrue.boolNone+ +use_doc_unwarpingWhether to load the text image correction module. If set to +None, the default value initialized by the production line will be used, initialized toTrue.boolNone+ +use_layout_detectionWhether to load the layout detection module. If set to +None, the default value initialized by the production line will be used, initialized toTrue.boolNone+ +layout_thresholdLayout detection confidence threshold; only scores greater than this threshold will be output. + +-
+
float|dictNone+ +layout_nmsWhether to use post-processing NMS in layout detection. If set to +None, the default value initialized by the production line, initialized toTrue, will be used.boolNone+ +layout_unclip_ratioScale factor for the sides of the detection box. + +-
+
float|listNone+ +layout_merge_bboxes_modeMode for merging detection boxes output by the model in layout detection. + +-
+
strNone+ +seal_det_limit_side_lenImage side length limit for seal text detection. + +-
+
intNone+ +seal_det_limit_typeImage side length limit type for seal text detection. + +-
+
strNone+ +seal_det_threshDetection pixel threshold; only pixels with scores greater than this threshold in the output probability map will be considered as text pixels. + +-
+
floatNone+ +seal_det_box_threshDetection box threshold; when the average score of all pixels within a detection result box is greater than this threshold, the result is considered a text area. + +-
+
floatNone+ +seal_det_unclip_ratioSeal text detection expansion coefficient; the larger the value, the greater the expansion area. + +-
+
floatNone+ +seal_rec_score_threshText recognition threshold; text results with scores greater than this threshold will be retained. + +-
+
floatNone+ +deviceThe device used for inference. Supports specifying a specific card number. + +-
+
strNone+ +enable_hpiWhether to enable high-performance inference. +boolFalse+ +use_tensorrtWhether to use TensorRT for inference acceleration. +boolFalse+ +min_subgraph_sizeMinimum subgraph size, used for optimizing model subgraph computation. +int3+ +precisionComputation precision, such as fp32 or fp16. +strfp32+ +enable_mkldnnWhether to enable the MKL-DNN acceleration library. If set to +None, it will be enabled by default.boolNone+ + +cpu_threadsThe number of threads to use when performing inference on the CPU. +int8+ +
+ +(3) Process the prediction results. The prediction result for each sample is of `dict` type and supports operations such as printing, saving as an image, and saving as a `json` file: + ++ + +Parameter +Description +Type +Options +Default Value ++ +inputData to be predicted, supports multiple input types (required) +Python Var|str|list+ +-
+
None+ +deviceInference device for the pipeline +str|None+ +-
+
None+ +use_doc_orientation_classifyWhether to use the document orientation classification module +bool|None+ +-
+
None+ +use_doc_unwarpingWhether to use the document unwarping module +bool|None+ +-
+
None+ +use_layout_detectionWhether to use the layout detection module +bool|None+ +-
+
None+ +layout_thresholdConfidence threshold for layout detection; only scores above this threshold will be output +float|dict|None+ +-
+
None+ +layout_nmsWhether to use Non-Maximum Suppression (NMS) for layout detection post-processing +bool|None+ +-
+
None+ +layout_unclip_ratioExpansion ratio of detection box edges; if not specified, the default value from the PaddleX official model configuration will be used +float|list|None+ +-
+
+ +layout_merge_bboxes_modeMerging mode for detection boxes in layout detection output; if not specified, the default value from the PaddleX official model configuration will be used +string|None+ +-
+
None ++ +seal_det_limit_side_lenSide length limit for seal text detection +int|None+ +-
+
None+ +seal_rec_score_threshText recognition threshold; text results with scores above this threshold will be retained +float|None+ +-
+
None+ +
+ +Note: The parameters in the configuration file are the pipeline initialization parameters. If you wish to change the initialization parameters of the seal text recognition pipeline, you can directly modify the parameters in the configuration file and load the configuration file for prediction. Additionally, CLI prediction also supports passing in a configuration file. Simply specify the path of the configuration file with `--pipeline`. + +- Calling the `print()` method will print the results to the terminal, and the explanations of the printed content are as follows: + + - `input_path`: `(str)` The input path of the image to be predicted. + + - `model_settings`: `(Dict[str, bool])` The model parameters required for pipeline configuration. + + - `use_doc_preprocessor`: `(bool)` Controls whether to enable the document preprocessing sub-pipeline. + - `use_layout_detection`: `(bool)` Controls whether to enable the layout detection sub-module. + + - `layout_det_res`: `(Dict[str, Union[List[numpy.ndarray], List[float]]])` The output result of the layout detection sub-module. Only exists when `use_layout_detection=True`. + + - `input_path`: `(Union[str, None])` The image path accepted by the layout detection module. Saved as `None` when the input is a `numpy.ndarray`. + - `page_index`: `(Union[int, None])` Indicates the current page number of the PDF if the input is a PDF file; otherwise, it is `None`. + - `boxes`: `(List[Dict])` A list of detected layout seal regions, with each element containing the following fields: + - `cls_id`: `(int)` The class ID of the detected seal region. + - `score`: `(float)` The confidence score of the detected region. + - `coordinate`: `(List[float])` The coordinates of the four corners of the detection box, in the order of x1, y1, x2, y2, representing the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, the x-coordinate of the bottom-right corner, and the y-coordinate of the bottom-right corner. + + - `seal_res_list`: `List[Dict]` A list of seal text recognition results, with each element containing the following fields: + + - `input_path`: `(Union[str, None])` The image path accepted by the seal text recognition pipeline. Saved as `None` when the input is a `numpy.ndarray`. + - `page_index`: `(Union[int, None])` Indicates the current page number of the PDF if the input is a PDF file; otherwise, it is `None`. + - `model_settings`: `(Dict[str, bool])` The model configuration parameters for the seal text recognition pipeline. + - `use_doc_preprocessor`: `(bool)` Controls whether to enable the document preprocessing sub-pipeline. + - `use_textline_orientation`: `(bool)` Controls whether to enable the text line orientation classification sub-module. + + - `doc_preprocessor_res`: `(Dict[str, Union[str, Dict[str, bool], int]])` The output result of the document preprocessing sub-pipeline. Only exists when `use_doc_preprocessor=True`. + + - `input_path`: `(Union[str, None])` The image path accepted by the document preprocessing sub-pipeline. Saved as `None` when the input is a `numpy.ndarray`. + - `model_settings`: `(Dict)` The model configuration parameters for the preprocessing sub-pipeline. + - `use_doc_orientation_classify`: `(bool)` Controls whether to enable document orientation classification. + - `use_doc_unwarping`: `(bool)` Controls whether to enable document unwarping. + - `angle`: `(int)` The predicted result of document orientation classification. When enabled, it takes values [0, 1, 2, 3], corresponding to [0°, 90°, 180°, 270°]; when disabled, it is -1. + + - `dt_polys`: `(List[numpy.ndarray])` A list of polygon boxes for seal text detection. Each detection box is represented by a numpy array of multiple vertex coordinates, with the array shape being (n, 2). + + - `dt_scores`: `(List[float])` A list of confidence scores for text detection boxes. + + - `text_det_params`: `(Dict[str, Dict[str, int, float]])` Configuration parameters for the text detection module. + - `limit_side_len`: `(int)` The side length limit value during image preprocessing. + - `limit_type`: `(str)` The handling method for side length limits. + - `thresh`: `(float)` The confidence threshold for text pixel classification. + - `box_thresh`: `(float)` The confidence threshold for text detection boxes. + - `unclip_ratio`: `(float)` The expansion ratio for text detection boxes. + - `text_type`: `(str)` The type of seal text detection, currently fixed as "seal". + + - `text_rec_score_thresh`: `(float)` The filtering threshold for text recognition results. + + - `rec_texts`: `(List[str])` A list of text recognition results, containing only texts with confidence scores above `text_rec_score_thresh`. + + - `rec_scores`: `(List[float])` A list of confidence scores for text recognition, filtered by `text_rec_score_thresh`. + + - `rec_polys`: `(List[numpy.ndarray])` A list of text detection boxes filtered by confidence score, in the same format as `dt_polys`. + + - `rec_boxes`: `(numpy.ndarray)` An array of rectangular bounding boxes for detection boxes; the seal recognition pipeline returns an empty array. + +- Calling the `save_to_json()` method will save the above content to the specified `save_path`. If a directory is specified, the saved path will be `save_path/{your_img_basename}_res.json`. If a file is specified, it will be saved directly to that file. Since JSON files do not support saving numpy arrays, `numpy.array` types will be converted to list format. + +- Calling the `save_to_img()` method will save the visualization results to the specified `save_path`. If a directory is specified, the saved path will be `save_path/{your_img_basename}_seal_res_region1.{your_img_extension}`. If a file is specified, it will be saved directly to that file. (The pipeline usually contains multiple result images, so it is not recommended to specify a specific file path directly, as multiple images will be overwritten, and only the last image will be retained.) + +* Additionally, you can obtain visualized images with results and prediction results through attributes, as follows: + ++ + +Method +Description +Parameter +Parameter Type +Parameter Description +Default Value ++ +print()Print results to the terminal +format_jsonboolWhether to format the output content using +JSONindentationTrue+ +indentintSpecify the indentation level to beautify the output +JSONdata for better readability, effective only whenformat_jsonisTrue4 ++ +ensure_asciiboolControl whether to escape non- +ASCIIcharacters toUnicode. When set toTrue, all non-ASCIIcharacters will be escaped;Falsewill retain the original characters, effective only whenformat_jsonisTrueFalse+ +save_to_json()Save results as a json file +save_pathstrThe file path to save the results. When it is a directory, the saved file name will be consistent with the input file type +None ++ +indentintSpecify the indentation level to beautify the output +JSONdata for better readability, effective only whenformat_jsonisTrue4 ++ +ensure_asciiboolControl whether to escape non- +ASCIIcharacters toUnicode. When set toTrue, all non-ASCIIcharacters will be escaped;Falsewill retain the original characters, effective only whenformat_jsonisTrueFalse+ +save_to_img()Save results as an image file +save_pathstrThe file path to save the results, supports directory or file path +None ++ +
+ +- The prediction results obtained through the `json` attribute are of dict type, with content consistent with what is saved by calling the `save_to_json()` method. +- The prediction results returned by the `img` attribute are of dict type. The keys are `layout_det_res`, `seal_res_region1`, and `preprocessed_img`, corresponding to three `Image.Image` objects: one for visualizing layout detection, one for visualizing seal text recognition results, and one for visualizing image preprocessing. If the image preprocessing sub-module is not used, `preprocessed_img` will not be included in the dictionary. If the layout region detection module is not used, `layout_det_res` will not be included. + + +## 3. Development Integration/Deployment +If the pipeline meets your requirements for inference speed and accuracy, you can proceed directly with development integration/deployment. + +If you need to integrate the pipeline into your Python project, you can refer to the example code in [2.2 Python Script Method](#22-python脚本方式集成). + +In addition, PaddleX also provides three other deployment methods, which are detailed as follows: + +🚀 High-Performance Inference: In real-world production environments, many applications have stringent performance requirements for deployment strategies, especially in terms of response speed, to ensure efficient system operation and a smooth user experience. To address this, PaddleOCR offers high-performance inference capabilities aimed at deeply optimizing the performance of model inference and pre/post-processing, thereby significantly accelerating the end-to-end process. For detailed high-performance inference procedures, please refer to the [High-Performance Inference Guide](../deployment/high_performance_inference.md). + +☁️ Service Deployment: Service deployment is a common form of deployment in real-world production environments. By encapsulating inference functionality into a service, clients can access these services via network requests to obtain inference results. For detailed production service deployment procedures, please refer to the [Service Deployment Guide](../deployment/serving.md). + +Below are the API references for basic serving deployment and multi-language service invocation examples: + ++ + +Attribute +Description ++ +jsonGet the prediction results in +jsonformat.+ +imgGet the visualization results in +dictformat.+API Reference
+For the main operations provided by the service:
+-
+
+ +
++ + + +Name +Type +Description ++ +logIdstringThe UUID of the request. ++ +errorCodeintegerError code. Fixed as +0.+ +errorMsgstringError message. Fixed as +"Success".+ + +resultobjectThe result of the operation. +-
+
+ +
++ + + +Name +Type +Description ++ +logIdstringThe UUID of the request. ++ +errorCodeintegerError code. Same as the response status code. ++ + +errorMsgstringError message. +The main operations provided by the service are as follows:
+-
+
Obtain the seal text recognition result.
+
+POST /seal-recognition-
+
+ +
++ + + +Name +Type +Description +Required ++ +filestringThe URL of an image or PDF file accessible by the server, or the Base64-encoded content of the file. By default, for PDF files exceeding 10 pages, only the content of the first 10 pages will be processed. +
+To remove the page limit, please add the following configuration to the pipeline configuration file: +Serving: + extra: + max_num_input_imgs: null +Yes ++ +fileTypeinteger|nullThe type of file. +0indicates a PDF file,1indicates an image file. If this attribute is not present in the request body, the file type will be inferred from the URL.No ++ +useDocOrientationClassifyboolean|nullPlease refer to the description of the +use_doc_orientation_classifyparameter of the pipeline object'spredictmethod.No ++ +useDocUnwarpingboolean|nullPlease refer to the description of the +use_doc_unwarpingparameter of the pipeline object'spredictmethod.No ++ +useLayoutDetectionboolean|nullPlease refer to the description of the +use_layout_detectionparameter of the pipeline object'spredictmethod.No ++ +layoutThresholdnumber|nullPlease refer to the description of the +layout_thresholdparameter of the pipeline object'spredictmethod.No ++ +layoutNmsboolean|nullPlease refer to the description of the +layout_nmsparameter of the pipeline object'spredictmethod.No ++ +layoutUnclipRationumber|array|nullPlease refer to the description of the +layout_unclip_ratioparameter of the pipeline object'spredictmethod.No ++ +layoutMergeBboxesModestring|nullPlease refer to the description of the +layout_merge_bboxes_modeparameter of the pipeline object'spredictmethod.No ++ +sealDetLimitSideLeninteger|nullPlease refer to the description of the +seal_det_limit_side_lenparameter of the pipeline object'spredictmethod.No ++ +sealDetLimitTypestring|nullPlease refer to the description of the +seal_det_limit_typeparameter of the pipeline object'spredictmethod.No ++ +sealDetThreshnumber|nullPlease refer to the description of the +seal_det_threshparameter of the pipeline object'spredictmethod.No ++ +sealDetBoxThreshnumber|nullPlease refer to the description of the +seal_det_box_threshparameter of the pipeline object'spredictmethod.No ++ +sealDetUnclipRationumber|nullPlease refer to the description of the +seal_det_unclip_ratioparameter of the pipeline object'spredictmethod.No ++ + +sealRecScoreThreshnumber|nullPlease refer to the description of the +seal_rec_score_threshparameter of the pipeline object'spredictmethod.No +-
+
+ +
++ + + +Name +Type +Meaning ++ +sealRecResultsobjectThe seal text recognition result. The array length is 1 (for image input) or the actual number of document pages processed (for PDF input). For PDF input, each element in the array represents the result of each page actually processed in the PDF file. ++ + +dataInfoobjectInformation about the input data. +Each element in
+sealRecResultsis anobjectwith the following properties:+ +
+ + + +Name +Type +Meaning ++ +prunedResultobjectA simplified version of the +resfield in the JSON representation generated by thepredictmethod of the production object, where theinput_pathand thepage_indexfields are removed.+ +outputImagesobject|nullSee the description of the +imgattribute of the result of the pipeline prediction. The images are in JPEG format and encoded in Base64.+ + +inputImagestring|nullThe input image. The image is in JPEG format and encoded in Base64. ++Multi-language Service Invocation Example
+++Python
+ +import base64 +import requests + +API_URL = "http://localhost:8080/seal-recognition" +file_path = "./demo.jpg" + +with open(file_path, "rb") as file: + file_bytes = file.read() + file_data = base64.b64encode(file_bytes).decode("ascii") + +payload = {"file": file_data, "fileType": 1} + +response = requests.post(API_URL, json=payload) + +assert response.status_code == 200 +result = response.json()["result"] +for i, res in enumerate(result["sealRecResults"]): + print(res["prunedResult"]) + for img_name, img in res["outputImages"].items(): + img_path = f"{img_name}_{i}.jpg" + with open(img_path, "wb") as f: + f.write(base64.b64decode(img)) + print(f"Output image saved at {img_path}") +
+ + +## 4. Custom Development +If the default model weights provided by the seal text recognition pipeline do not meet your requirements in terms of accuracy or speed, you can try to fine-tune the existing models using your own domain-specific or application data to improve the recognition performance of the seal text recognition pipeline in your scenario. + +Since the seal text recognition pipeline consists of several modules, if the pipeline's performance does not meet expectations, the issue may arise from any one of these modules. You can analyze images with poor recognition results to identify which module is problematic and refer to the corresponding fine-tuning tutorial links in the table below for model fine-tuning. + ++ +
diff --git a/docs/version3.x/pipeline_usage/seal_recognition.md b/docs/version3.x/pipeline_usage/seal_recognition.md index 23fb90dd11..055efe4e5c 100644 --- a/docs/version3.x/pipeline_usage/seal_recognition.md +++ b/docs/version3.x/pipeline_usage/seal_recognition.md @@ -24,6 +24,31 @@ comments: true+ + + +Scenario +Fine-Tuning Module +Fine-Tuning Reference Link ++ +Inaccurate or missing seal position detection +Layout Detection Module +Link ++ +Missing text detection +Text Detection Module +Link ++ +Inaccurate text content +Text Recognition Module +Link ++ +Inaccurate full-image rotation correction +Document Image Orientation Classification Module +Link ++ + +Inaccurate image distortion correction +Text Image Correction Module +Not supported for fine-tuning +版面区域检测模块（可选）：
+* 版面检测模型，包含20个常见的类别：文档标题、段落标题、文本、页码、摘要、目录、参考文献、脚注、页眉、页脚、算法、公式、公式编号、图像、表格、图和表标题（图标题、表格标题和图表标题）、印章、图表、侧栏文本和参考文献内容 ++ +
+ * 版面检测模型，包含23个常见的类别：文档标题、段落标题、文本、页码、摘要、目录、参考文献、脚注、页眉、页脚、算法、公式、公式编号、图像、图表标题、表格、表格标题、印章、图表标题、图表、页眉图像、页脚图像、侧栏文本+ + + +模型 模型下载链接 +mAP(0.5)（%） +GPU推理耗时（ms） +
[常规模式 / 高性能模式]CPU推理耗时（ms） +
[常规模式 / 高性能模式]模型存储大小（M） +介绍 ++ +PP-DocLayout_plus-L 推理模型/训练模型 +83.2 +34.6244 / 10.3945 +510.57 / - +126.01 M +基于RT-DETR-L在包含中英文论文、多栏杂志、报纸、PPT、合同、书本、试卷、研报、古籍、日文文档、竖版文字文档等场景的自建数据集训练的更高精度版面区域定位模型 ++ +@@ -65,7 +90,7 @@ comments: true
->❗ 以上列出的是版面检测模块重点支持的3个核心模型，该模块总共支持11个全量模型，包含多个预定义了不同类别的模型，其中包含印章类别的模型有9个，除上述3个核心模型外，其余模型列表如下： +>❗ 以上列出的是版面检测模块重点支持的4个核心模型，该模块总共支持13个全量模型，包含多个预定义了不同类别的模型，其中包含印章类别的模型有9个，除上述3个核心模型外，其余模型列表如下：👉模型列表详情
@@ -947,9 +972,9 @@ paddleocr seal_recognition -i ./seal_text_det.png --device gpu from paddleocr import SealRecognition pipeline = SealRecognition() -# ocr = TableRecognitionPipelineV2(use_doc_orientation_classify=True) # 通过 use_doc_orientation_classify 指定是否使用文档方向分类模型 -# ocr = TableRecognitionPipelineV2(use_doc_unwarping=True) # 通过 use_doc_unwarping 指定是否使用文本图像矫正模块 -# ocr = TableRecognitionPipelineV2(device="gpu") # 通过 device 指定模型推理时使用 GPU +# ocr = SealRecognition(use_doc_orientation_classify=True) # 通过 use_doc_orientation_classify 指定是否使用文档方向分类模型 +# ocr = SealRecognition(use_doc_unwarping=True) # 通过 use_doc_unwarping 指定是否使用文本图像矫正模块 +# ocr = SealRecognition(device="gpu") # 通过 device 指定模型推理时使用 GPU output = pipeline.predict("./seal_text_det.png") for res in output: res.print() ## 打印预测的结构化输出 @@ -1751,4 +1776,41 @@ for i, res in enumerate(result["sealRecResults"]): ## 4. 二次开发 如果印章文本识别产线提供的默认模型权重在您的场景中，精度或速度不满意，您可以尝试利用您自己拥有的特定领域或应用场景的数据对现有模型进行进一步的微调，以提升印章文本识别产线的在您的场景中的识别效果。 -...... +由于印章文本识别产线包含若干模块，模型产线的效果如果不及预期，可能来自于其中任何一个模块。您可以对识别效果差的图片进行分析，进而确定是哪个模块存在问题，并参考以下表格中对应的微调教程链接进行模型微调。 + ++ +
+ + + +情形 +微调模块 +微调参考链接 ++ +印章位置检测不准或未检出 +版面检测模块 +链接 ++ +印章文本存在漏检 +印章文本检测模块 +链接 ++ +文本内容不准 +文本识别模块 +链接 ++ +整图旋转矫正不准 +文档图像方向分类模块 +链接 ++ + +图像扭曲矫正不准 +文本图像矫正模块 +暂不支持微调 +
