Ir para o conteúdo
ModelVerificadoExterno
Baixar

VLMPed CoT

A LoRA fine-tuned version of Qwen2.5-VL-3B-Instruct for pedestrian crossing intention prediction, trained with Chain-of-Thought supervision.

Fontehuggingface.co/chiawen0104/VLMPed-CoT/tree/30cc326fbf99dbcefc6a76557a334a1f1f1fd6ae
TipoLoRAModelo baseQwen/Qwen2.5-VL-3B-InstructVersãoadapter_modelPublicador@chiawen0104Cnota

Fonte do modelo

Tipo
LoRA
Tamanho do arquivo
142 MB
Modelo base
Qwen/Qwen2.5-VL-3B-Instruct
Versão
vadapter_model
Atualizado
3 de out. de 2026
Publicador
@chiawen0104
Downloads
4
Fonte
Hugging Face

Descrição da fonte

VLMPed-CoT

A LoRA fine-tuned version of Qwen2.5-VL-3B-Instruct for pedestrian crossing intention prediction, trained with Chain-of-Thought supervision.

This model is part of the ECE 228 final project at UCSD (Spring 2026): "How Do Vision Language Models Utilize Multi-Frame Temporal Information for Pedestrian Intention Prediction?"

Project repository: ece228_VLMPed-CoT

Model Details

  • Developed by: chiawen0104
  • Model type: Vision-Language Model (LoRA fine-tuned)
  • Finetuned from: Qwen/Qwen2.5-VL-3B-Instruct
  • Task: Pedestrian crossing intention prediction (binary: cross / not cross)
  • Training datasets: JAAD, PIE
  • Framework: PEFT 0.15.1

How to Get Started

from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration
from peft import PeftModel

base_model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2.5-VL-3B-Instruct"
)
model = PeftModel.from_pretrained(base_model, "chiawen0104/VLMPed-CoT")
processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct")

Training Details

  • Base model: Qwen2.5-VL-3B-Instruct
  • Fine-tuning method: LoRA (via PEFT)
  • Training regime: bf16 mixed precision
  • Training data: JAAD and PIE pedestrian crossing intention datasets
  • CoT supervision: ✅ Chain-of-Thought reasoning generated via Gemini API

Intended Use

This model takes multi-frame pedestrian images as input and predicts whether a pedestrian intends to cross the street. The CoT supervision encourages the model to reason step-by-step before making a prediction. It is intended for research purposes in autonomous driving and pedestrian behavior analysis.

Differences from VLMPed-wo-CoT

VLMPed-CoTVLMPed-wo-CoT
CoT supervision✅❌
Direct prediction✅✅

Reference

Framework versions

  • PEFT 0.15.1

Fontes

1 fonte
Hugging FacePrimárioVersão 30cc326fbf99dbcefc6a76557a334a1f1f1fd6ae
huggingface.co/chiawen0104/VLMPed-CoT/tree/30cc326fbf99dbcefc6a76557a334a1f1f1fd6ae

Verificado 3 de out.

Artefatos de modelo

1 artefato

adapter_model.safetensors

safetensors · 142 MB · SHA-256 2ada072ccdbc…22ac · Hugging Face

Baixar

Trechos de fonte

3 trechos
Readme11 de jul.huggingface.co/chiawen0104/VLMPed-CoT

--- base_model: Qwen/Qwen2.5-VL-3B-Instruct library_name: peft --- # VLMPed-CoT A LoRA fine-tuned version of [Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct) for pedestrian crossing intention prediction, trained **with** Chain-of-Thought supervision. This model is part of the ECE 228 final project at UCSD (Spring 2026): **"How Do Vision Language Models Utilize Multi-Frame Temporal Information for Pedestrian Intention Prediction?"** Project repository: [ece228_VLMPed-CoT](https://github.com/chiawen0104/ece228_VLMPed-CoT) --- ## Model Details - **Developed by:** chiawen0104 - **Model type:** Vision-Language Model (LoRA fine-tuned) - **Finetuned from:** [Qwen/Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct) - **Task:** Pedestrian crossing intention prediction (binary: cross / not cross) - **Training datasets:** JAAD, PIE - **Framework:** PEFT 0.15.1 --- ## How to Get Started ```python from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration from peft import PeftModel base_model = Qwen2_5_VLForConditionalGeneration.from_pretrained( "Qwen/Qwen2.5-VL-3B-Instruct" ) model = PeftModel.from_pretrained(base_model, "chiawen0104/VLMPed-CoT") processor = AutoProcessor.from_pretrained("Qwen/Qwen2.5-VL-3B-Instruct") ``` --- ## Training Details - **Base model:** Qwen2.5-VL-3B-Instruct - **Fine-tuning method:** LoRA (via PEFT) - **Training regime:** bf16 mixed precision - **Training data:** JAAD and PIE pedestrian crossing intention datasets - **CoT supervision:** ✅ Chain-of-Thought reasoning generated via Gemini API --- ## Intended Use This model takes multi-frame pedestrian images as input and predicts whether a pedestrian intends to cross the street. The CoT supervision encourages the model to reason step-by-step before making a prediction. It is intended for research purposes in autonomous driving and pedestrian behavior analysis. --- ## Differences from VLMPed-wo-CoT | | VLMPed-CoT | VLMPed-wo-CoT | |---|---|---| | CoT supervision | ✅ | ❌ | | Direct prediction | ✅ | ✅ | --- ## Reference - Original Paper: [VLMPed-CoT: A large vision-language model with a chain-of-thought mechanism for pedestrian crossing intention prediction](https://www.sciopen.com/article/10.26599/COMMTR.2026.9640009) - Original implementation: [lyc2121/VLMPed-CoT-for-Pedestrian-Crossing-Intention-Predic...

Note3 de out.huggingface.co/chiawen0104/VLMPed-CoT

Source context: 4 downloads · 0 likes · Library peft · Repo chiawen0104/VLMPed-CoT

Note17 de jul.huggingface.co/chiawen0104/VLMPed-CoT

Source context: 2 downloads · 0 likes · Library peft · Repo chiawen0104/VLMPed-CoT