Rico Blip LoRA Model

--- library_name: transformers tags: - transformers - image-to-text --- # RICO BLIP LoRA Model ## Model Description This model is a BLIP image captioning model fine-tuned using **LoRA (Low Rank Adaptation)** on the **RICO Screen2Words dataset**. The goal of the model is to generate textual descriptions of mobile user interface screenshots. The model takes an image of a UI screen as input and produces a caption describing the interface. --- ## Model Details * **Developed by:** suhas2468 * **Model type:** Vision-Language Model (Image Captioning) * **Language(s):** English * **License:** Apache 2.0 * **Finetuned from model:** Salesforce/blip-image-captioning-base --- ## Model Sources * **Repository:** https://github.com/suhassh2/orange-multimodal-slm * **Base Model:** https://huggingface.co/Salesforce/blip-image-captioning-base * **Dataset:** https://huggingface.co/datasets/rootsautomation/RICO-Screen2Words --- ## Intended Use ### Direct Use This model can be used to generate captions describing mobile UI screenshots. It can help automatically summarize UI layouts and components present in mobile application interfaces. Example use cases include: * UI understanding * UI documentation * Automated interface description ### Out-of-Scope Use The model is not designed for: * general image captioning outside UI screenshots * safety-critical applications * decision-making systems --- ## How to Use the Model Example inference code: ```python from transformers import pipeline pipe = pipeline( "image-to-text", model="suhas2468/rico-blip-lora-model" ) result = pipe("example_image.png") print(result) ``` --- ## Training Details ### Training Data The model was fine-tuned using the **RICO Screen2Words dataset**. This dataset contains mobile UI screenshots paired with captions describing the interface. Dataset link: https://huggingface.co/datasets/rootsautomation/RICO-Screen2Words --- ### Training Setup * **GPU:** NVIDIA Tesla T4 * **Epochs:** 1 * **Batch size:** 4 * **Learning rate:** 2e-5 * **Training samples:** 500 Training was performed using Hugging Face **Transformers**, **PEFT**, and **PyTorch**. --- ## Training Method The model was fine-tuned using **LoRA (Low Rank Adaptation)**. LoRA enables efficient fine-tuning by adding small trainable adapters to the base model while keeping most of the original model parameters frozen....