Falcon OCR ONNX

--- pipeline_tag: image-to-text library_name: onnxruntime tags: - falcon - ocr - vision-language - document-understanding license: apache-2.0 --- # ONNX model for [Falcon-OCR](https://huggingface.co/tiiuae/Falcon-OCR) ## try with [ningpp/flux](https://github.com/ningpp/flux) Flux is a Java-based OCR # Falcon OCR Falcon OCR is a 300M parameter early-fusion vision-language model for document OCR. Given an image, it can produce plain text, LaTeX for formulas, or HTML for tables, depending on the requested output format. Most OCR VLM systems are built as a pipeline with a vision encoder feeding a separate text decoder, plus additional task-specific glue. Falcon OCR takes a different approach: a single Transformer processes image patches and text tokens in a shared parameter space from the first layer, using a hybrid attention mask where image tokens attend bidirectionally and text tokens decode causally conditioned on the image. We built it this way for two practical reasons. First, it keeps the interface simple: one backbone, one decoding path, and task switching through prompts rather than a growing set of modules. Second, a 0.3B model has a lower latency and cost footprint than 0.9B-class OCR VLMs, and in our vLLM-based serving setup this translates into higher throughput, often 2–3× faster depending on sequence lengths and batch configuration. To our knowledge, this is one of the first attempts to apply this early-fusion single-stack recipe directly to competitive document OCR at this scale.

Source context: 19 downloads · 0 likes · Pipeline image-to-text · Library onnxruntime · Repo ningpp/Falcon-OCR-ONNX

Source context: 10 downloads · 0 likes · Pipeline image-to-text · Library onnxruntime · Repo ningpp/Falcon-OCR-ONNX