--- base_model: deepseek-ai/DeepSeek-OCR tags: - text-generation-inference - transformers - unsloth - deepseek_vl_v2 license: apache-2.0 language: - en - km datasets: - Kimang18/WildKhmerST-Warped metrics: - cer --- # Objective Fine-tune `deepseek-ai/DeepSeek-OCR` on [`WildKhmerST`](https://l3i-share.univ-lr.fr/2025WildKhmerST/). The setting for image processing is the following ```python base_size=512 image_size=512 crop_mode=False ``` The setting for LoRA is the following ```python model = FastVisionModel.get_peft_model( model, target_modules = [ "q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj", ], r = 16, # The larger, the higher the accuracy, but might overfit lora_alpha = 16, # Recommended alpha == r at least lora_dropout = 0, bias = "none", random_state = 3407, use_rslora = False, # We support rank stabilized LoRA loftq_config = None, # And LoftQ # target_modules = "all-linear", # Optional now! Can specify a list if needed ) ``` For future benchmarking in Khmer OCR task, [`WildKhmerST`](https://l3i-share.univ-lr.fr/2025WildKhmerST/) is processed to contains pairs of 'text region image' and 'text'. The processed dataset is uploaded to huggingface platform [here](Kimang18/WildKhmerST-Warped). There are 3 splits. `train` split is used to fine-tune `unsloth/DeepSeek-OCR`. `test` split is used to evaluate the fine-tuned model. The fine-tuned model performs better on Khmer text, yet it still completely misses some images in the `test` split. Concretely, its `cer` on the whole `test` split is `237%` (result file can be found [here](https://huggingface.co/Kimang18/deepseek_ocr_Khmer_finetuned/resolve/main/deepseek_ocr_lora_eval.txt)) This is because when the fine-tuned completely misses decoding some images, it hallucinates and then repeats the same text token until the number of maximum tokens is met. Removing hallucinating cases, `cer` drops to `67.66%` (result file can be found [here](https://huggingface.co/Kimang18/deepseek_ocr_Khmer_finetuned/resolve/main/deepseek_ocr_lora_eval_remove_hallucination.txt)). # Uploaded finetuned model - **Developed by:** Kimang18 - **License:** apache-2.0 - **Finetuned from model :** deepseek-ai/DeepSeek-OCR This deepseek_vl_v2 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library. [ ](https://github.com/unslothai/unsloth) # Usage TBA
