Trocr Traditional Chinese Historical Finetune

--- language: - zh tags: - trocr - ocr - traditional-chinese - historical-documents - vision - image-to-text datasets: - ZihCiLin/traditional-chinese-historical-ocr-lo-chia-luen base_model: - ZihCiLin/trocr-traditional-chinese-baseline metrics: - cer library_name: transformers --- # TrOCR Traditional Chinese Historical Finetune ## Model Description This is a TrOCR model finetuned on Lo Chia-Luen historical manuscripts for Traditional Chinese historical document recognition. - **Base Model**: [ZihCiLin/trocr-traditional-chinese-baseline](https://huggingface.co/ZihCiLin/trocr-traditional-chinese-baseline) - **Finetuning Data**: 690 annotated historical manuscript samples - **Task**: Historical OCR for Traditional Chinese documents ## Intended Use This finetuned model is optimized for: - Historical manuscript recognition - Semi-classical Chinese text - Degraded document images - Archaic character variants ## Training Details ### Finetuning Data - **Dataset**: [ZihCiLin/traditional-chinese-historical-ocr-lo-chia-luen](https://huggingface.co/datasets/ZihCiLin/traditional-chinese-historical-ocr-lo-chia-luen) - **Train**: 690 samples - **Validation**: 46 samples - **Source**: Lo Chia-Luen Manuscripts (20th century Chinese historical documents) ### Finetuning Hyperparameters - Batch size: 8 (per device) - Learning rate: 2e-5 - Epochs: 30 - Data augmentation: First 20 epochs (ColorJitter, RandomAffine, GaussianBlur) - Clean training: Last 10 epochs ## Usage ```python from transformers import TrOCRProcessor, VisionEncoderDecoderModel from PIL import Image # Load model and processor processor = TrOCRProcessor.from_pretrained("ZihCiLin/trocr-traditional-chinese-historical-finetune") model = VisionEncoderDecoderModel.from_pretrained("ZihCiLin/trocr-traditional-chinese-historical-finetune") # Load historical document image image = Image.open("historical_manuscript.jpg").convert("RGB") # Generate text pixel_values = processor(image, return_tensors="pt").pixel_values generated_ids = model.generate(pixel_values) generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0] print(generated_text) ``` ### With Decoding-Time Fusion (Recommended) For best results, use with LLM-guided decoding: ```python # See: https://github.com/Jason9339/traditional-chinese-historical-document-ocr-llm-fusion # Fusion with Breeze-7B improves CER by 10%...