mirror of
https://github.com/opendatalab/MinerU.git
synced 2026-10-25 17:42:08 +08:00
Converge the SDK, Gradio, remote CLI, Doclib and Router on Tasks with hash-based uploads, per-file result URLs, task listing and optional model output. Preserve Files and the synchronous file_parse wrapper while removing standalone Uploads/Jobs and local-path HTTP inputs. Migrate contract, lifecycle, transport and real-document regression coverage; update documentation and HTTP examples. Validation: 1452 passed, 3 conditional skips, plus targeted final PDF checks. BREAKING CHANGE: clients and workers must use Tasks; standalone Uploads/Jobs endpoints and local-source/inline HTTP options have been removed.
68 lines
5.5 KiB
Markdown
68 lines
5.5 KiB
Markdown
# Migrating from 3.x to 4.0
|
|
|
|
4.0 changes command entrypoints, dependency groups, configuration, and APIs. Validate in an isolated environment before migrating production callers. AMD and vendor accelerators stay on `mineru<4` using the [legacy guides](../usage/compatibility.md).
|
|
|
|
## Entrypoints and installation
|
|
|
|
| Previous usage | 4.0 usage |
|
|
| --- | --- |
|
|
| `mineru -p input.pdf -o output` | Library: `mineru parse input.pdf --pages all -o output.md`; one-off conversion: `mineru-kit parse input.pdf -o output.md` |
|
|
| Directory batch conversion | `mineru-kit parse ./documents -o ./output --format zip` |
|
|
| `mineru-gradio` | `mineru-webui` / `mineru-kit webui` |
|
|
| Model download option `-m all` | `mineru-kit models download --tier standard` |
|
|
| Old HTTP parsing calls | V1 uploads → parse jobs → output files |
|
|
| Extras such as `mineru[core]` and `mineru[pipeline]` | Base package, `mineru[torch]`, or `mineru[full]` |
|
|
|
|
Install stable with `"mineru>=4.0,<5"`, inserting extras after the package name when needed. The package requires Python `>=3.11,<3.15`; optional engines also require suitable wheels and drivers. Stop services in the original environment before upgrading and check `mineru version --json` afterwards. Do not replace legacy platform images or installation commands with mainline 4.0 instructions.
|
|
|
|
## Tiers, pages, and outputs
|
|
|
|
- Select parsing quality with `flash/basic/standard/advanced`. Basic uses small models; Standard / Advanced use a VLM. Do not infer new quality defaults from old backend names.
|
|
- `mineru parse` uses the library, cache, and continuation protocol, defaulting to the first 10 PDF pages. `mineru-kit parse` and the Python SDK default to all pages.
|
|
- PDF ranges use `1-5,8,r3-r1`; `r1` is the last page and `all` selects the whole document. Non-PDF native documents are parsed as a whole.
|
|
- `-o` file/directory semantics depend on the entrypoint. `mineru-kit parse` defaults to Markdown for one file; directory or multi-file inputs write to a directory.
|
|
- Fields such as `_backend` and `pdf_info` in legacy results are not templates for 4.0. Read actual JSON using the [current output contract](output_files.md). Legacy artifacts are accepted only by explicitly supported compatibility readers; not every old JSON document can be passed to `ParseResult.from_dict()`.
|
|
|
|
Legacy-result handling is scoped as follows:
|
|
|
|
| Data | How 4.0 handles it |
|
|
| --- | --- |
|
|
| Current shared protocol with a `schema` identity (`2.0`) | Read strictly by `ParseResult`, HTTP/ZIP, and Gradio; a payload that carries the current identity but fails validation is not retried as a legacy format |
|
|
| Recognizable Doclib-cached history (3.4.5 `pdf_info` pages, `schema_version` `1.0`, or `2.0` without a `schema` key) | Converted in memory by the dedicated Doclib cache reader only; this does not extend to other entrypoints, and history files are not rewritten |
|
|
| Old result JSON saved outside Doclib | Not guaranteed to load via `ParseResult.from_dict()`; re-parse the source document |
|
|
| Old artifacts missing assets or new geometry data | Regenerate by re-parsing; changing a version number alone does not migrate a payload |
|
|
|
|
"Can be read" (Doclib cache), "migrated automatically" (nothing rewrites files), and "can re-export every format" (requires current assets and geometry) are separate claims; treat them independently instead of inferring one from another.
|
|
|
|
## Configuration and models
|
|
|
|
Configuration lives in `$MINERU_HOME/config.yaml`, optionally overridden by `MINERU_CONFIG`. Old `mineru.json` configuration is not read. `model.stack`, `MINERU_MODEL_STACK`, and `--stack` have been removed; configure the two components independently:
|
|
|
|
```yaml
|
|
model:
|
|
source: auto
|
|
base_dir: ~/.mineru/models
|
|
small_backend: auto
|
|
vlm:
|
|
engine: auto
|
|
```
|
|
|
|
Two 3.x environment variables are still recognized and rejected on conflict: when a remote VLM is configured (`model.vlm.server_url` set), a non-empty `MINERU_VL_API_KEY` or `MINERU_VL_MODEL_NAME` that disagrees with the new configuration raises an error asking you to unset the legacy variable. Move these values to the new locations:
|
|
|
|
| Legacy variable | 4.0 configuration field | 4.0 environment variable |
|
|
| --- | --- | --- |
|
|
| `MINERU_VL_API_KEY` | `model.vlm.api_key` | `MINERU_MODEL_VLM_API_KEY` |
|
|
| `MINERU_VL_MODEL_NAME` | `model.vlm.model` | `MINERU_MODEL_VLM_MODEL` |
|
|
|
|
Automatic selection follows the platform. Explicit `onnx + llama-cpp` can serve lighter environments. Torch and VLM engines can also be combined when their dependencies are available. Apple Silicon defaults to llama.cpp for the VLM.
|
|
|
|
Restart relevant services after configuration changes, and download and verify models with matching backend choices. 4.0 has separate small-model bundles: the presence of an old model directory does not prove readiness. See [Model Source](../usage/model_source.md).
|
|
|
|
## API and WebUI
|
|
|
|
Asynchronous parsing uses `/v1/tasks` JSON manifests, optional uploads and per-file result URLs. Upgrade the SDK and WebUI together with the service. Files downloads and the separate synchronous wrapper remain available; loopback callers also use hash preflight. See [HTTP API](../usage/http_api.md).
|
|
|
|
Existing Gradio clients also need to follow the new event interface; renaming the command alone is insufficient. Retained third-party integration guides describe their original adaptations and do not imply V1 compatibility.
|
|
|
|
Validate migration with a native document and a text-layer PDF. Check pages, Markdown, structured JSON, and assets, then validate the required quality tiers and API calls on the actual target hardware. See [SDK and API](../usage/sdk_api.md).
|