Math Ocr
This is a LoRA adapter fine-tuned on the Qwen2.5-VL-3B-Instruct base model for Optical Character Recognition (OCR) of mathematical formulas, converting images directly into LaTeX code.
Modellquelle
- Art
- LoRA
- Dateigröße
- 142 MB
- Basismodell
- Qwen/Qwen2.5-VL-3B-Instruct
- Version
- vadapter_model
- Aktualisiert
- 30. Okt. 2026
- Downloads
- 3
- Quelle
- Hugging Face
Quellenbeschreibung
Model Card for Model ID
This is a LoRA adapter fine-tuned on the Qwen2.5-VL-3B-Instruct base model for Optical Character Recognition (OCR) of mathematical formulas, converting images directly into LaTeX code.
Model Details
Model Description
This model is a Vision-Language Model (VLM) adapter fine-tuned to transcribe images of math equations into their corresponding LaTeX representation. It uses 4-bit QLoRA quantization to achieve efficient fine-tuning on a single consumer-grade GPU (e.g., Tesla T4).
- Developed by: ericmrib
- Model type: Vision-Language Model Adapter (LoRA)
- Language(s) (NLP): English, LaTeX
- Finetuned from model: Qwen/Qwen2.5-VL-3B-Instruct
Out-of-Scope Use
The model is highly specialized for LaTeX formula extraction and may not perform well on general image captioning, document parsing without equations, or natural language reasoning tasks outside its fine-tuned scope.
Bias, Risks, and Limitations
As with any vision-language model, the performance relies heavily on the quality, clarity, and handwriting style (if applicable) of the input images. Complex or highly stylized formatting might lead to inaccuracies in the generated LaTeX.
How to Get Started with the Model
Use the code below to get started with the model.
from peft import PeftModel
from transformers import AutoProcessor, AutoModelForImageTextToText
# Load base model and processor
base_model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct", device_map="auto")
processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct")
# Load adapter
model = PeftModel.from_pretrained(base_model, "ericmrib/math-ocr")
Training Details
Training Data
The model was fine-tuned on the unsloth/LaTeX_OCR dataset, which contains approximately 76,000 samples of image-to-LaTeX pairs.
Training Hyperparameters
- Training regime: fp16 mixed precision
- Quantization: 4-bit (QLoRA) with NF4
- Batch size: 1 (with 16 gradient accumulation steps; effective batch size = 16)
- Learning rate: 2e-4
- Optimizer: adamw_torch
- Scheduler: cosine
- Max Steps: 500
- LoRA configuration: r=16, alpha=32, dropout=0.05, target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
Technical Specifications
Hardware
- Hardware Type: Tesla T4 (1x)
- Hours used: ~4.2 hours
- Cloud Provider: Google Cloud (Google Colab)
Quellen
1 Quelle
Quellen
1 QuelleVerifiziert 30. Okt.
Download files
Repository files
File inventory only; completeness is unverified.
Modellartefakte
1 Artefakt
Modellartefakte
1 Artefaktadapter_model.safetensors
safetensors · 142 MB · SHA-256 7a4c6ab6712c…56f2 · Hugging Face
Download fileQuellenauszüge
3 Auszüge
Quellenauszüge
3 Auszüge--- base_model: Qwen/Qwen2.5-VL-3B-Instruct library_name: peft pipeline_tag: image-to-text tags: - base_model:adapter:Qwen/Qwen2.5-VL-3B-Instruct - lora - transformers --- # Model Card for Model ID This is a LoRA adapter fine-tuned on the Qwen2.5-VL-3B-Instruct base model for Optical Character Recognition (OCR) of mathematical formulas, converting images directly into LaTeX code. ## Model Details ### Model Description <!-- Provide a longer summary of what this model is. --> This model is a Vision-Language Model (VLM) adapter fine-tuned to transcribe images of math equations into their corresponding LaTeX representation. It uses 4-bit QLoRA quantization to achieve efficient fine-tuning on a single consumer-grade GPU (e.g., Tesla T4). - **Developed by:** ericmrib - **Model type:** Vision-Language Model Adapter (LoRA) - **Language(s) (NLP):** English, LaTeX - **Finetuned from model:** Qwen/Qwen2.5-VL-3B-Instruct ### Out-of-Scope Use The model is highly specialized for LaTeX formula extraction and may not perform well on general image captioning, document parsing without equations, or natural language reasoning tasks outside its fine-tuned scope. ## Bias, Risks, and Limitations As with any vision-language model, the performance relies heavily on the quality, clarity, and handwriting style (if applicable) of the input images. Complex or highly stylized formatting might lead to inaccuracies in the generated LaTeX. ## How to Get Started with the Model Use the code below to get started with the model. ```python from peft import PeftModel from transformers import AutoProcessor, AutoModelForImageTextToText # Load base model and processor base_model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct", device_map="auto") processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct") # Load adapter model = PeftModel.from_pretrained(base_model, "ericmrib/math-ocr") ``` ## Training Details ### Training Data The model was fine-tuned on the `unsloth/LaTeX_OCR` dataset, which contains approximately 76,000 samples of image-to-LaTeX pairs. #### Training Hyperparameters - **Training regime:** fp16 mixed precision - **Quantization:** 4-bit (QLoRA) with NF4 - **Batch size:** 1 (with 16 gradient accumulation steps; effective batch size = 16) - **Learning rate:** 2e-4 - **Optimizer:** adamw_torch - **Scheduler:** cosine - **M...
Source context: 3 downloads · 0 likes · Pipeline image-to-text · Library peft · Repo ericmrib/math-ocr
Source context: 9 downloads · 0 likes · Pipeline image-to-text · Library peft · Repo ericmrib/math-ocr
