--- base_model: Qwen/Qwen2.5-VL-3B-Instruct library_name: peft --- # VLMPed-CoT A LoRA fine-tuned version of [Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct) for pedestrian crossing intention prediction, trained **with** Chain-of-Thought supervision. This model is part of the ECE 228 final project at UCSD (Spring 2026): **"How Do Vision Language Models Utilize Multi-Frame Temporal Information for Pedestrian Intention Prediction?"** Project repository: [ece228_VLMPed-CoT](https://github.com/chiawen0104/ece228_VLMPed-CoT) --- ## Model Details - **Developed by:** chiawen0104 - **Model type:** Vision-Language Model (LoRA fine-tuned) - **Finetuned from:** [Qwen/Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct) - **Task:** Pedestrian crossing intention prediction (binary: cross / not cross) - **Training datasets:** JAAD, PIE - **Framework:** PEFT 0.15.1 --- ## How to Get Started ```python from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration from peft import PeftModel base_model = Qwen2_5_VLForConditionalGeneration.from_pretrained( "Qwen/Qwen2.5-VL-3B-Instruct" ) model = PeftModel.from_pretrained(base_model, "chiawen0104/VLMPed-CoT") processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct") ``` --- ## Training Details - **Base model:** Qwen2.5-VL-3B-Instruct - **Fine-tuning method:** LoRA (via PEFT) - **Training regime:** bf16 mixed precision - **Training data:** JAAD and PIE pedestrian crossing intention datasets - **CoT supervision:** ✅ Chain-of-Thought reasoning generated via Gemini API --- ## Intended Use This model takes multi-frame pedestrian images as input and predicts whether a pedestrian intends to cross the street. The CoT supervision encourages the model to reason step-by-step before making a prediction. It is intended for research purposes in autonomous driving and pedestrian behavior analysis. --- ## Differences from VLMPed-wo-CoT | | VLMPed-CoT | VLMPed-wo-CoT | |---|---|---| | CoT supervision | ✅ | ❌ | | Direct prediction | ✅ | ✅ | --- ## Reference - Original Paper: [VLMPed-CoT: A large vision-language model with a chain-of-thought mechanism for pedestrian crossing intention prediction](https://www.sciopen.com/article/10.26599/COMMTR.2026.9640009) - Original implementation: [lyc2121/VLMPed-CoT-for-Pedestrian-Crossing-Intention-Prediction](https://github.com/lyc2121/VLMPed-CoT-for-Pedestrian-Crossing-Intention-Prediction) - Companion model: [chiawen0104/VLMPed-wo-CoT](https://huggingface.co/chiawen0104/VLMPed-wo-CoT) ### Framework versions - PEFT 0.15.1
