* fix(ocr_mkcontent): revise table caption output
- Ensuring that
table captions are properly included in the output.
- Remove the redundant `table_caption` variable。
* Update cla.yml
* Update bug_report.yml
* feat(cli): add debug option for detailed error handling
Enable users to invoke the CLI command with a new debug flag to get detailed debugging information.
* fix(pdf-extract-kit): adjust crop_paste parameters for better accuracyThe crop_paste_x and crop_paste_y values in the pdf_extract_kit.py have been modified
to improve the accuracy and consistency of OCR processing. The new values are set to 25
to ensure more precise image cropping and pasting which leads to better OCR recognition
results.
* Update README_zh-CN.md (#404)
correct FAQ url
* Update README_zh-CN.md (#404) (#409) (#410)
correct FAQ url
Co-authored-by: sfk <18810651050@163.com>
* Update FAQ_zh_cn.md
add new issue
* Update FAQ_en_us.md
* Update README_Windows_CUDA_Acceleration_zh_CN.md
* Update README_zh-CN.md
* @Thepathakarpit has signed the CLA in opendatalab/MinerU#418
* fix(pdf-extract-kit): increase crop_paste margin for OCR processingDouble the crop_paste margin from25 to 50 to ensure better OCR accuracy and
handling of border cases. This change will help in improving the overall quality of
OCR'ed text by providing more context around the detected text areas.
* fix(common): deep copy model list before drawing model bbox
Use a deep copy of the original model list in `drow_model_bbox` to avoid potential
modifications to the source data. This ensures the integrity of the original models
is maintained while generating the model bounding boxes visualization.
---------
Co-authored-by: sfk <18810651050@163.com>
Co-authored-by: drunkpig <60862764+drunkpig@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
* feat: rename the file generated by command line tools
* feat: add pdf filename as prefix to {span,layout,model}.pdf
---------
Co-authored-by: icecraft <tmortred@gmail.com>
Co-authored-by: icecraft <xurui1@pjlab.org.cn>
Add FAQ entries in both English and Chinese to address the issue where the
libGL.so.1 library is missing on Ubuntu22.04 when running under WSL2. The
FAQ now includes instructions on how to install the missing library, resolvingthe corresponding ImportError.Closes https://github.com/opendatalab/MinerU/issues/388
Add notes in the Ubuntu and Windows CUDA acceleration guides on how to
determine if CUDA acceleration is working. This includes checking for
significant reductions in `layout detection cost`, `mfr time`, and `ocr cost`
as indicators of successful acceleration.
Add an exclamation mark to the section title to stress the importance of completing the
additional steps after downloading a model. This change is made in the Chinese
documentation to ensure users are aware of the necessary post-download actions.
Adjust the print statement in the how_to_download_models_zh_cn.md guide to reflect
the correct model download location. The path has been updated to specify the 'models'
directory where the model is actually downloaded.
Add a print statement to the example code in 'how_to_download_models_zh_cn.md' to
output the downloaded model directory path. This enhancement aids users in locating
the model files as it provides a clear indication of where they are saved on the
user's file system.
unnecessary sections and simplifying the steps. The updated guide now includesclearer instructions on installing Git LFS, downloading models from Hugging Face,and additional checks for model file completeness. This change ensures that the
documentation is up-to-date and provides a streamlined experience for users
downloading models.
Update the URLs for downloading the `magic-pdf.template.json` and `small_ocr.pdf`
files in the Windows CUDA acceleration guides. The links now point to the giteerepository instead of GitHub, ensuring users have access to the necessary files
from the correct source.
- Streamline the installation process by removing the redundant apt update step.
- Adjust the numbering of installation steps throughout the document.
- Update download URLs to gitee for the configuration template and demo file.
- Ensure consistency in the model directory configuration advice.
Add a new README_Ubuntu_CUDA_Acceleration_en_US.md document to provide users with a
setup guide for enabling and testing CUDA acceleration on Ubuntu 22.04 LTS. The guideincludes steps to check and install NVIDIA drivers, install Anaconda, create a conda
environment, install required applications, download and verify models, configure theenvironment, and test CUDA acceleration.
This addition addresses the need for clear, concise instructions on achieving better
performance with CUDA-enabled graphics cards and
- Note the fix in version 0.6.2b1 for the network error during the first run of offline deployment and clarify the model download requirement.
- Update the dependency installation guide for users on macOS with Intel CPUs.
- Indicate the resolution in version 0.6.2b1 for compatibility issues with paddlepaddle
version 2.6.1 on certain Linux systems.
This change aims to make the FAQ more informative and easier to navigate for users
experiencing similar issues, providing direct solutions and links where applicable.
Update the download links for Anaconda in both Ubuntu and Windows CUDA
Acceleration documents to use the Tuna mirror. This change helps ensure that
users in China have faster access to the Anaconda distribution.
Update the README documents to clarify that the "models-dir" in the
configuration should be an absolute path. Also, provide additional guidance
for Windows users on how to correctly format the path to avoid common issues
with path escaping in JSON files.
Add instructions to download the magic-pdf.template.json file using wget on
Ubuntu and a PowerShell script on Windows in the respective README files.
This is to facilitate the setup process by providing direct download options,
replacing manual file transfers.
Update the install commands in both Ubuntu and Windows CUDA Acceleration
guides to specify Magic-PDF version 0.6.2b1, ensuring consistency andavoiding potential version mismatches.
Update the FAQ to clarify the dependency installation issue when using magic-pdf. Ensure
users are directed to install the specific version of magic-pdf that resolves the dependency
error, rather than listing all individual dependencies. This simplifies the troubleshooting process
and provides a direct solution for users encountering the "Required dependency not installed"
error.
Update the download methods for models in the Chinese documentation to reflect
the latest options available from ModelScope. Simplify the section titles and
revise download instructions for clarity and consistency.
Update the documentation to reflect the latest model download procedures, emphasis on
model file integrity checks, and expanded instructions for setting up CUDA accelerationon Ubuntu and Windows environments. The README files for various OS have been
enhanced with additional details to assist users in configuring and verifying their
environments for optimal performance.
* # add table recognition using struct-eqtable
## Changelog
31/07/20204
- Support table recognition. Table images will be converted into html.
### how to use the new feature:
set the attribute 'table-mode' to 'true' in magic-pdf.json
### caution:
it takes 200s to 500s to convert a single table image using cpu
* # add table recognition using struct-eqtable
## Changelog
31/07/20204
- Support table recognition. Table images will be converted into LaTex.
### how to use the new feature:
set the attribute 'table-mode' to 'true' in magic-pdf.json
### caution:
it takes 200s to 500s to convert a single table image using cpu
* # feat(model inference): add table recognition and convertion to LaTeX
# What's Changed
### New Features
- Add table content recognition, we use weights of [StructEqTable](https://github.com/UniModal4Reasoning/StructEqTable-Deploy) to convert table image to LaTex.
### Instruction
- pip install pypandoc struct-eqtable==0.1.0
- Download [StructEqTable weights](https://huggingface.co/wanderkid/PDF-Extract-Kit/tree/main/models/TabRec) and put it under models/ directory.
- Edit 'table-mode' value to turn on table recognition function which is turned off by default.
- If you did not download any models before, refer to [how to download models](docs/how_to_download_models_zh_cn.md)。
* add table recognition and convertion to LaTeX
* add table recognition and conversion to LaTeX
* add table recognition and conversion to LaTeX
* add table recognition and conversion to LaTeX
---------
Co-authored-by: liukaiwen <liukaiwen@pjlab.org.cn>
- Fix the description of the 'poly' coordinate format in the output file documentation to correctly reflect the order of coordinates: left-top, right-top, right-bottom,
left-bottom.
- Update various table-related descriptions for clarity and consistency, including
field names and their corresponding explanations.
- Add version name field description in 'middle.json' structure to document the
version of the magic-pdf used in the parsing process.
- Refactor the block and line description tables to improve readability and alignment
with the rest of the documentation.
Update the Ubuntu CUDA Acceleration setup guide to reflect the correct user directory
path and improve the clarity of instructions. Remove references to Windows and macOS
as they are out of scope for this document. Ensure the configuration file copying
command is correctly represented for Linux users.
Update the link to the magic-pdf.template.json configuration template file in the
README_Ubuntu_CUDA_Acceleration_zh_CN.md document. The file path was previously
incorrect and has been amended to point to the correct location.
Update the instruction in README_Ubuntu_CUDA_Acceleration_zh_CN.md to reference
the correct section number for downloading the model weights. This change ensures
that users are directed to the correct location in the document for setting up the
model directory in the magic-pdf.json configuration.
- Detail the process of checking, installing, and configuring NVIDIA drivers.
- Include instructions for installing Anaconda and creating a specific environment.
- Provide guidance on installing magic-pdf and its dependencies.
- Add a note to verify magic-pdf version and report issues if necessary.
- Describe the process of downloading models and configuring the application.
- Include a sample command to run the application with CUDA acceleration.
- Add a note for enabling OCR CUDA acceleration with specific GPU requirements.
This update ensures users have the latest information for setting up CUDA accelerationwith magic-pdf on Ubuntu 22.04 LTS, specifically for version 0.6.2, and provides clearer
instructions on the installation and configuration process.
Restructured the how-to download models document for better clarity and
added sections on downloading models from ModelScope, including SDK and
Git download methods. Provided detailed steps for installing Git LFS and
checking model integrity after download. Also included recommendations
for moving the models to an SSD for better performance.
Add detailed instructions for installing git lfs on Linux, macOS, and Windows
to facilitate users in downloading models from ModelScope repository. The guide
is included in the `how_to_download_models_zh_cn.md` document.
