TROCR Chess

--- language: en license: mit tags: - chess - ocr - handwritten-text-recognition - trocr - transformers - computer-vision - image-to-text datasets: - handwritten-chess-notation metrics: - accuracy - cer widget: - src: https://huggingface.co/datasets/mishig/sample_images/resolve/main/chess_example.png example_title: Chess Move Example pipeline_tag: image-to-text --- # Handwritten Chess Notation Recognition with TrOCR ## Model Description This is a fine-tuned version of Microsoft's TrOCR-Large model, specifically trained to recognize handwritten chess notation from images of chess scoresheets. The model can accurately transcribe chess moves in Standard Algebraic Notation (SAN) from handwritten images. **Model Type:** Vision Encoder-Decoder **Architecture:** TrOCR (Transformer-based Optical Character Recognition) **Base Model:** `microsoft/trocr-large-handwritten` ## Intended Uses & Limitations ### Intended Use - Transcription of handwritten chess moves from scoresheet images - Digitization of historical chess games - Chess notation recognition in mobile apps - Educational tools for chess analysis ### Limitations - Works best with clear handwriting - Trained specifically on chess notation (not general text) - May struggle with extremely cursive handwriting - Requires well-lit, focused images ## Training Data The model was trained on a custom dataset of 13,731 handwritten chess move images with corresponding text labels in Standard Algebraic Notation. **Dataset Characteristics:** - Format: PNG images with move text annotations - Content: Chess moves in SAN format (e.g., "e4", "Nf3", "O-O", "Qxf7#") - Split: 88% training, 12% validation - Handwriting styles: Multiple variations ## Training Procedure ### Preprocessing Images were resized to 384x384 pixels and converted to RGB format. Text was tokenized with chess-specific vocabulary and padded/truncated to a maximum length of 16 tokens. ### Training Hyperparameters - **Epochs:** 6 - **Batch Size:** 1 (with gradient accumulation of 8) - **Learning Rate:** 3e-5 - **Optimizer:** AdamW - **Weight Decay:** 0.01 - **Warmup Steps:** 200 - **Mixed Precision:** FP16 ### Hardware - **GPU:** NVIDIA (8+ GB VRAM recommended) - **Training Time:** ~6 hours ## Evaluation Results | Metric | Value | |--------|-------| | Accuracy | ~92% | | Character Error Rate (CER) | ~3% | | Inference Speed | ~100 ms/image | #...