update PaddleOCR-VL docs (#17692)

* update PaddleOCR-VL docs (#17702) Co-authored-by: Lin Manhui <mhlin425@whu.edu.cn> * add llama.cpp support (#17676) * add llama.cpp support * update * update * update * update * update * update --------- Co-authored-by: Lin Manhui <mhlin425@whu.edu.cn> --------- Co-authored-by: Lin Manhui <mhlin425@whu.edu.cn>
2026-08-31 01:45:27 +08:00 · 2026-02-12 17:36:48 +08:00
parent ce7a96e9bb
commit d7e428a105
5 changed files with 133 additions and 42 deletions
@@ -49,7 +49,6 @@ Install the MLX-VLM inference framework:

 ```shell
 python -m pip install -U mlx-vlm
-python -m pip install "transformers<5.0.0"
 ```

 Start the MLX-VLM inference service:
@@ -49,7 +49,6 @@ python -m pip install -U "paddleocr[doc-parser]"

 ```shell
 python -m pip install -U mlx-vlm
-python -m pip install "transformers<5.0.0"
 ```

 启动 MLX-VLM 推理服务：
@@ -40,7 +40,7 @@ For some inference hardware, you may need to refer to other environment configur

 ## Inference Device Support for PaddleOCR-VL

-Currently, PaddleOCR-VL offers five inference methods, with varying levels of support for different inference devices. Please confirm that your inference device meets the requirements in the table below before proceeding with PaddleOCR-VL deployment:
+Currently, PaddleOCR-VL offers six inference methods, with varying levels of support for different inference devices. Please confirm that your inference device meets the requirements in the table below before proceeding with PaddleOCR-VL deployment:

 <table border="1">
 <thead>
@@ -76,8 +76,8 @@ Currently, PaddleOCR-VL offers five inference methods, with varying levels of su
    <td>🚧</td>
    <td>🚧</td>
    <td>✅</td>
-    <td>❌</td>
-    <td>❌</td>
+    <td>-</td>
+    <td>-</td>
  </tr>
  <tr style="text-align: center;">
    <td>PaddlePaddle + SGLang</td>
@@ -87,8 +87,8 @@ Currently, PaddleOCR-VL offers five inference methods, with varying levels of su
    <td>🚧</td>
    <td>🚧</td>
    <td>🚧</td>
-    <td>❌</td>
-    <td>❌</td>
+    <td>-</td>
+    <td>-</td>
  </tr>
  <tr style="text-align: center;">
    <td>PaddlePaddle + FastDeploy</td>
@@ -98,20 +98,31 @@ Currently, PaddleOCR-VL offers five inference methods, with varying levels of su
    <td>✅</td>
    <td>✅</td>
    <td>🚧</td>
-    <td>❌</td>
-    <td>❌</td>
+    <td>-</td>
+    <td>-</td>
  </tr>
  <tr style="text-align: center;">
    <td>PaddlePaddle + MLX-VLM</td>
-    <td>❌</td>
-    <td>❌</td>
-    <td>❌</td>
-    <td>❌</td>
-    <td>❌</td>
-    <td>❌</td>
-    <td>❌</td>
+    <td>-</td>
+    <td>-</td>
+    <td>-</td>
+    <td>-</td>
+    <td>-</td>
+    <td>-</td>
+    <td>-</td>
    <td>✅</td>
  </tr>
+  <tr style="text-align: center;">
+    <td>PaddlePaddle + llama.cpp</td>
+    <td>✅</td>
+    <td>🚧</td>
+    <td>🚧</td>
+    <td>🚧</td>
+    <td>🚧</td>
+    <td>🚧</td>
+    <td>✅</td>
+    <td>🚧</td>
+  </tr>
 </tbody>
 </table>

@@ -133,7 +144,7 @@ Since different hardware requires different dependencies, if your hardware meets

 | Hardware Type  | Environment Configuration Tutorial                                                                                           |
 |----------------|------------------------------------------------------------------------------------------------------------------------------|
-| x64 CPU        | This tutorial                                                                                                                |
+| x64 CPU        | This tutorial (Dependencies must be installed manually for now)                                                                                                                |
 | NVIDIA GPU     | - NVIDIA Blackwell architecture GPU (e.g., RTX 50 series) refer to [PaddleOCR-VL NVIDIA Blackwell Architecture GPU Environment Configuration Tutorial](./PaddleOCR-VL-NVIDIA-Blackwell.en.md) <br/> - Other NVIDIA GPUs refer to this tutorial |
 | KUNLUNXIN XPU  | [PaddleOCR-VL KUNLUNXIN XPU Environment Configuration Tutorial](./PaddleOCR-VL-KUNLUNXIN-XPU.en.md)                                              |
 | HYGON DCU      | [PaddleOCR-VL HYGON DCU Environment Configuration Tutorial](./PaddleOCR-VL-HYGON-DCU.en.md)                                              |
@@ -218,7 +229,7 @@ python -m pip install -U "paddleocr[doc-parser]"

 PaddleOCR-VL supports two usage methods: CLI command line and Python API. The CLI command line method is simpler and suitable for quickly verifying functionality, while the Python API method is more flexible and suitable for integration into existing projects.

-> TIP:
+> IMPORTANT:
 > The methods introduced in this section are primarily for rapid validation. Their inference speed, memory usage, and stability may not meet the requirements of a production environment. **If deployment to a production environment is needed, we strongly recommend using a dedicated inference acceleration framework**. For specific methods, please refer to the next section.

 ### 2.1 Command Line Usage
@@ -1492,9 +1503,20 @@ The inference performance under default configurations is not fully optimized an

 There are two methods to launch the VLM inference service; choose either one:

- Method 1: Launch the service using the official Docker image.
+- Method 1: Launch the service using the official Docker image. Currently supported:
+    - FastDeploy
+    - vLLM

- Method 2: Launch the service by manually installing dependencies via the PaddleOCR CLI.
+- Method 2: Launch the service by manually installing dependencies via the PaddleOCR CLI. Currently supported:
+    - FastDeploy
+    - vLLM
+    - SGLang
+
+- Method 3: Launch service directly using inference acceleration frameworks. Currently supported:
+    - FastDeploy
+    - vLLM
+    - MLX-VLM
+    - llama.cpp

 **We strongly recommend using the Docker image to minimize potential environment-related issues.**

@@ -1552,6 +1574,8 @@ docker run \

 #### 3.1.2 Method 2: Installation and Usage via PaddleOCR CLI

+**The PaddleOCR CLI has already resolved complex version compatibility issues. Instead of spending time studying framework documentation, you can install the necessary environment with a single command.**
+
 Due to potential dependency conflicts between inference acceleration frameworks and PaddlePaddle, it is recommended to install them in a virtual environment:

 ```shell
@@ -1590,13 +1614,34 @@ The parameters supported by this command are as follows:
 | `--backend`        | Backend name, i.e., the name of the inference acceleration framework used; options are `vllm` or `sglang` |
 | `--backend_config` | Can specify a YAML file containing backend configurations      |

+#### 3.1.3 Launch Service Directly Using Inference Acceleration Frameworks
+
+**If you need to install a custom version of an inference framework and launch the service natively, please refer to the following guidelines. Please note that when launching natively, the pre-configured performance tuning parameters provided by PaddleOCR will not be applied.**
+
+- FastDeploy: [Refer to this document](https://paddlepaddle.github.io/FastDeploy/best_practices/PaddleOCR-VL-0.9B/)
+- vLLM: [Refer to this document](https://docs.vllm.ai/projects/recipes/en/latest/PaddlePaddle/PaddleOCR-VL.html)
+- MLX-VLM: [Refer to this document](./PaddleOCR-VL-Apple-Silicon.en.md)
+- llama.cpp:
+    1. Install llama.cpp by referring to the `Quick start` section in the [llama.cpp github](https://github.com/ggml-org/llama.cpp).
+    2. Download the model files in gguf format: [megemini/PaddleOCR-VL-1.5-GGUF](https://modelscope.cn/models/megemini/PaddleOCR-VL-1.5-GGUF/files) or [megemini/PaddleOCR-VL-GGUF](https://modelscope.cn/models/megemini/PaddleOCR-VL-GGUF/files).
+    3. Execute the following command to start the inference service. For an introduction to the parameters, please refer to [LLaMA.cpp HTTP Server](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md):
+
+        ```shell
+        ./build/bin/llama-server \
+          -m /path/to/PaddleOCR-VL-1.5-GGUF.gguf \
+          --mmproj /path/to/PaddleOCR-VL-1.5-GGUF-mmproj.gguf  \
+          --port 8111  \
+          --host 0.0.0.0 \
+          --temp 0
+        ```
+
 ### 3.2 Client Usage Methods

 After launching the VLM inference service, the client can call the service through PaddleOCR. **Please note that because the client needs to call the layout detection model, it is still recommended to run the client on GPU or other acceleration devices to achieve more stable and efficient performance. Please refer to Section 1 for the client-side environment configuration. The configuration described in Section 3.1 applies only to starting the service and is not applicable to the client.**

 #### 3.2.1 CLI Invocation

-Specify the backend type (`vllm-server` or `sglang-server`) using `--vl_rec_backend` and the service address using `--vl_rec_server_url`, for example:
+Specify the backend type (`vllm-server`, `sglang-server`, `fastdeploy-server`, `mlx-vlm-server` or `llama-cpp-server`) using `--vl_rec_backend` and the service address using `--vl_rec_server_url`, for example:

 ```shell
 paddleocr doc_parser --input paddleocr_vl_demo.png --vl_rec_backend vllm-server --vl_rec_server_url http://localhost:8118/v1
@@ -1640,7 +1685,7 @@ paddleocr doc_parser \

 #### 3.2.2 Python API Invocation

-When creating a `PaddleOCRVL` object, pass the `vl_rec_backend` and `vl_rec_server_url` parameters to specify the backend type and the service endpoint, respectively:
+When creating a `PaddleOCRVL` object, specify the backend type (`vllm-server`, `sglang-server`, `fastdeploy-server`, `mlx-vlm-server` or `llama-cpp-server`) using `vl_rec_backend` and the service address using `vl_rec_server_url`, for example:

 ```python
 pipeline = PaddleOCRVL(vl_rec_backend="vllm-server", vl_rec_server_url="http://localhost:8118/v1")
@@ -40,7 +40,7 @@ PaddleOCR-VL 是一款先进、高效的文档解析模型，专为文档中的

 ## PaddleOCR-VL 对推理设备的支持情况

-目前 PaddleOCR-VL 有五种推理方式，支持的推理设备不完全相同，请确认您的推理设备是否满足下表要求再进行 PaddleOCR-VL 的推理部署：
+目前 PaddleOCR-VL 有六种推理方式，支持的推理设备不完全相同，请确认您的推理设备是否满足下表要求再进行 PaddleOCR-VL 的推理部署：

 <table border="1">
 <thead>
@@ -76,8 +76,8 @@ PaddleOCR-VL 是一款先进、高效的文档解析模型，专为文档中的
    <td>🚧</td>
    <td>🚧</td>
    <td>✅</td>
-    <td>❌</td>
-    <td>❌</td>
+    <td>-</td>
+    <td>-</td>
  </tr>
  <tr style="text-align: center;">
    <td>PaddlePaddle + SGLang</td>
@@ -87,8 +87,8 @@ PaddleOCR-VL 是一款先进、高效的文档解析模型，专为文档中的
    <td>🚧</td>
    <td>🚧</td>
    <td>🚧</td>
-    <td>❌</td>
-    <td>❌</td>
+    <td>-</td>
+    <td>-</td>
  </tr>
  <tr style="text-align: center;">
    <td>PaddlePaddle + FastDeploy</td>
@@ -98,20 +98,31 @@ PaddleOCR-VL 是一款先进、高效的文档解析模型，专为文档中的
    <td>✅</td>
    <td>✅</td>
    <td>🚧</td>
-    <td>❌</td>
-    <td>❌</td>
+    <td>-</td>
+    <td>-</td>
  </tr>
  <tr style="text-align: center;">
    <td>PaddlePaddle + MLX-VLM</td>
-    <td>❌</td>
-    <td>❌</td>
-    <td>❌</td>
-    <td>❌</td>
-    <td>❌</td>
-    <td>❌</td>
-    <td>❌</td>
+    <td>-</td>
+    <td>-</td>
+    <td>-</td>
+    <td>-</td>
+    <td>-</td>
+    <td>-</td>
+    <td>-</td>
    <td>✅</td>
  </tr>
+  <tr style="text-align: center;">
+    <td>PaddlePaddle + llama.cpp</td>
+    <td>✅</td>
+    <td>🚧</td>
+    <td>🚧</td>
+    <td>🚧</td>
+    <td>🚧</td>
+    <td>🚧</td>
+    <td>✅</td>
+    <td>🚧</td>
+  </tr>
 </tbody>
 </table>

@@ -133,7 +144,7 @@ PaddleOCR-VL 是一款先进、高效的文档解析模型，专为文档中的

 | 硬件类型         | 环境配置教程 |
 |-----------------|--------------------------------------------------|
-| x64 CPU         | 本教程                                           |
+| x64 CPU         | 本教程（当前仅支持手动安装依赖） |
 | 英伟达 GPU      | - NVIDIA Blackwell 架构 GPU（如RTX 50 系）参考 [PaddleOCR-VL NVIDIA Blackwell 架构 GPU 环境配置教程](./PaddleOCR-VL-NVIDIA-Blackwell.md) <br/> - 其他 NVIDIA GPU 参考本教程 |
 | 昆仑芯 XPU      | [PaddleOCR-VL 昆仑芯 XPU 环境配置教程](./PaddleOCR-VL-KUNLUNXIN-XPU.md) |
 | 海光 DCU        | [PaddleOCR-VL 海光 DCU 环境配置教程](./PaddleOCR-VL-HYGON-DCU.md) |
@@ -220,7 +231,7 @@ python -m pip install -U "paddleocr[doc-parser]"

 PaddleOCR-VL 支持 CLI 命令行方式和 Python API 两种使用方式，其中 CLI 命令行方式更简单，适合快速验证功能，而 Python API 方式更灵活，适合集成到现有项目中。

-> TIP:
+> IMPORTANT:
 > 本节所介绍的方法主要用于快速验证，其推理速度、显存占用及稳定性表现未必能满足生产环境的要求。**若需部署至生产环境，我们强烈建议使用专门的推理加速框架** ，具体方法请参考下一节。

 ### 2.1 命令行方式体验
@@ -1461,9 +1472,20 @@ output = pipeline.predict(["imgs/file1.png", "imgs/file2.png", "imgs/file3.png"]

 启动 VLM 推理服务有以下两种方式，任选一种即可：

- 方法一：使用官方 Docker 镜像启动服务。
+- 方法一：使用官方 Docker 镜像启动服务，目前支持：
+    - FastDeploy
+    - vLLM

- 方法二：通过 PaddleOCR CLI 手动安装依赖后启动服务。
+- 方法二：通过 PaddleOCR CLI 手动安装依赖后启动服务，目前支持：
+    - FastDeploy
+    - vLLM
+    - SGLang
+
+- 方法三：直接使用推理加速框架启动服务，目前支持：
+    - FastDeploy
+    - vLLM
+    - MLX-VLM
+    - llama.cpp

 **我们强烈推荐采用 Docker 镜像的方式，以最大程度减少可能出现的环境问题。**

@@ -1521,6 +1543,8 @@ docker run \

 #### 3.1.2 方法二：通过 PaddleOCR CLI 安装和使用

+**PaddleOCR CLI 已经为您解决了复杂的版本兼容性问题。您无需花费时间研究推理框架的文档，只需一条简单的命令即可安装推理框架所需的依赖环境。**
+
 由于推理加速框架可能与飞桨框架存在依赖冲突，建议在虚拟环境中安装：

 ```shell
@@ -1569,13 +1593,36 @@ paddleocr genai_server --model_name PaddleOCR-VL-1.5-0.9B --backend vllm --port
 | `--backend`        | 后端名称，即使用的推理加速框架名称，可选 `vllm` 或 `sglang` |
 | `--backend_config` | 可指定 YAML 文件，包含后端配置        |

+#### 3.1.3 方法三：直接使用推理加速框架启动服务
+
+**如果您需要安装自定义版本的推理框架并使用原生方式启动服务，请参考以下指引。请注意，使用原生方式启动时，将无法应用 PaddleOCR 预置的性能调优参数。**
+
+- FastDeploy：[参考此文档](https://paddlepaddle.github.io/FastDeploy/zh/best_practices/PaddleOCR-VL-0.9B/)
+- vLLM：[参考此文档](https://docs.vllm.ai/projects/recipes/en/latest/PaddlePaddle/PaddleOCR-VL.html)
+- MLX-VLM：[参考此文档](./PaddleOCR-VL-Apple-Silicon.md)
+- llama.cpp：
+    1. 参考 [llama.cpp github](https://github.com/ggml-org/llama.cpp) 中的 `Quick start` 安装 llama.cpp。
+    2. 下载 gguf 格式的模型文件：[megemini/PaddleOCR-VL-1.5-GGUF](https://modelscope.cn/models/megemini/PaddleOCR-VL-1.5-GGUF/files) 或 [megemini/PaddleOCR-VL-GGUF](https://modelscope.cn/models/megemini/PaddleOCR-VL-GGUF/files)。
+    3. 执行以下命令启动推理服务，参数介绍可参考 [LLaMA.cpp HTTP Server](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md)：
+
+        ```shell
+        ./build/bin/llama-server \
+          -m /path/to/PaddleOCR-VL-1.5-GGUF.gguf \
+          --mmproj /path/to/PaddleOCR-VL-1.5-GGUF-mmproj.gguf  \
+          --port 8111  \
+          --host 0.0.0.0 \
+          --temp 0
+        ```
+
+</details>
+
 ### 3.2 客户端使用方法

 启动 VLM 推理服务后，客户端即可通过 PaddleOCR 调用该服务。**请注意，由于客户端需要调用版面检测模型，仍建议在 GPU 等加速设备上运行客户端，以获得更稳定和高效的性能。客户端环境配置请参考第 1 节，3.1 节介绍的环境配置仅适用于启动服务，不适用于客户端。**

 #### 3.2.1 CLI 调用

-可通过 `--vl_rec_backend` 指定后端类型（`vllm-server` 或 `sglang-server`），通过 `--vl_rec_server_url` 指定服务地址，例如：
+可通过 `--vl_rec_backend` 指定后端类型（`vllm-server`、`sglang-server`、`fastdeploy-server`、`mlx-vlm-server` 或 `llama-cpp-server`），通过 `--vl_rec_server_url` 指定服务地址，例如：

 ```shell
 paddleocr doc_parser --input paddleocr_vl_demo.png --vl_rec_backend vllm-server --vl_rec_server_url http://localhost:8118/v1
@@ -1619,7 +1666,7 @@ paddleocr doc_parser \

 #### 3.2.2 Python API 调用

-创建 `PaddleOCRVL` 对象时传入 `vl_rec_backend` 和 `vl_rec_server_url` 参数，分别指定后端类型和服务地址：
+创建 `PaddleOCRVL` 对象时传入 `vl_rec_backend` 指定后端类型（`vllm-server`、`sglang-server`、`fastdeploy-server`、`mlx-vlm-server` 或 `llama-cpp-server`）， `vl_rec_server_url` 指定服务地址，例如：

 ```python
 pipeline = PaddleOCRVL(vl_rec_backend="vllm-server", vl_rec_server_url="http://localhost:8118/v1")
@@ -30,6 +30,7 @@ _SUPPORTED_VL_BACKENDS = [
    "sglang-server",
    "fastdeploy-server",
    "mlx-vlm-server",
+    "llama-cpp-server",
 ]