OvisOCR2

--- license: apache-2.0 library_name: transformers pipeline_tag: image-text-to-text base_model: Qwen/Qwen3.5-0.8B tags: - ocr - document-parsing - multimodal - markdown - tables - formulas - vllm - qwen3_5 --- # OvisOCR2 <p align="center"> <img src="https://cdn-uploads.huggingface.co/production/uploads/658a8a837959448ef5500ce5/vRCIu5QD8VuIJolkC_ZHQ.png" alt="Ovis" width="30%" /> </p> ## Introduction We are pleased to announce the release of OvisOCR2, a compact 0.8B end-to-end model for page-level document parsing. Given a document page image, OvisOCR2 generates a Markdown representation in natural reading order, covering text, formulas, tables, and visual regions. OvisOCR2 is developed by post-training Qwen3.5-0.8B using a carefully designed data engine that combines real-world and synthetic data, together with a multi-stage training recipe integrating SFT, RL, and OPD. The model delivers strong document parsing performance while maintaining a small deployment footprint. OvisOCR2 achieves an overall score of 96.58 on OmniDocBench v1.6, establishing a new state of the art and **becoming the first end-to-end model to top this leaderboard previously dominated by pipeline methods**. On PureDocBench, OvisOCR2 also achieves the highest Avg3 score of 75.06. <p align="center"> <img src="./performance.png" alt="Performance of OvisOCR2 on OmniDocBench v1.6" width="100%" /> </p> ## Performance <p align="center"> <img src="./performance_omnidocbench_v16.png" alt="OmniDocBench v1.6 comparison" width="100%" /> </p> <p align="center"> <img src="./performance_puredocbench.png" alt="PureDocBench comparison" width="100%" /> </p> ## Inference ```bash pip install "vllm==0.22.1" pillow ``` ```python from PIL import Image from vllm import LLM, SamplingParams class OvisOCR2Parser: def __init__(self, model_name_or_path: str): self.model = LLM( model=model_name_or_path, tensor_parallel_size=1, gpu_memory_utilization=0.8, gdn_prefill_backend="triton" ) prompt = '\nExtract all readable content from the image in natural human reading order and output the result as a single Markdown document. For charts or images, represent them using an HTML image tag: <' + 'img src="images/bbox_{left}_{top}_{right}_{bottom}.jpg" />, where left, top, right, bottom are bounding box coordinates scaled to [0, 1000). Format formulas as LaTeX. Format tables as HTML: <table>...<...

Source context: 12 downloads · 0 likes · Pipeline image-text-to-text · Library transformers · Repo TechnoBaptist/OvisOCR2