Aller au contenu
ModelPrésentéExterne
Browse files

Petvlm

A video-language model fine-tuned to detect naughty vs nice pet behavior with a sarcastic pet detective persona.

Sourcehuggingface.co/Raullen/petvlm/tree/3fc2b40433672cd95215aa28fc8e3ae098541a6d
TypeOtherModèle de baseQwen/Qwen2-VL-2B-InstructVersionRevision 3fc2b40BClasse

Source du modèle

Type
Other
Taille du fichier
70,5 MB
Modèle de base
Qwen/Qwen2-VL-2B-Instruct
Version
vRevision 3fc2b40
Mis à jour
24 oct. 2026
Téléchargements
4
Source
Hugging Face

Description de la source

PetVLM v1.1 - Sherlock Pet

A video-language model fine-tuned to detect naughty vs nice pet behavior with a sarcastic pet detective persona.

Accuracy: 87.5% on held-out test set (7/8 correct)

Model Description

PetVLM analyzes pet videos and classifies behavior:

  • ALERT! - Naughty behavior (climbing furniture, scratching, stealing food)
  • All Clear - Good behavior (sleeping, eating properly, playing nicely)

Training Data

  • Dataset Size: 175 videos (cleaned and balanced)
  • Label Balance: 51% ALERT / 49% All Clear
  • Best Checkpoint: epoch 3.6
  • Training Time: ~13 minutes on RTX 4090

Intended Uses

  • Home pet monitoring
  • Pet behavior analysis
  • Entertainment (sarcastic detective persona)

Running on Mac (Apple Silicon)

PetVLM runs on Mac computers with Apple Silicon (M1/M2/M3) using Metal Performance Shaders (MPS).

Requirements

ComponentRequirement
macOS13.0 (Ventura) or later
Python3.10+
RAM8GB minimum (16GB recommended)

Memory Usage by Precision

PrecisionMemorySpeed (M2 Pro)
FP16 + MPS~6 GB3-5s/video
INT4 + MPS~2.5 GB5-8s/video
FP32 + CPU~10 GB30-60s/video

Quick Start

# Install dependencies
pip install torch torchvision torchaudio
pip install transformers peft accelerate
pip install qwen-vl-utils imageio[ffmpeg]

# Run inference
python inference_mac.py your_video.mp4

Example Code

import torch
from transformers import Qwen2VLForConditionalGeneration, AutoProcessor
from peft import PeftModel

# Use MPS for Apple Silicon
device = "mps" if torch.backends.mps.is_available() else "cpu"

# Load model
model = Qwen2VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2-VL-2B-Instruct",
    torch_dtype=torch.float16
).to(device)

model = PeftModel.from_pretrained(model, "Raullen/petvlm")
model.eval()

processor = AutoProcessor.from_pretrained("Raullen/petvlm")

For complete Mac installation guide and optimized inference script, see:

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 1
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 4
  • optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.05
  • num_epochs: 15.0

Training results

Training LossEpochStepValidation Loss
3.28590.6369252.8292
2.01431.2548501.9940
1.68291.8917751.7856
1.47742.50961001.7516
1.19083.12741251.6817
1.20463.76431501.6712
0.80124.38221751.7618
0.95955.02001.7216
0.51155.63692251.8212
0.46666.25482502.1777
0.40826.89172752.0293
0.22627.50963002.3396
0.17748.12743252.3459
0.15128.76433502.4745
0.07199.38223752.7719
0.074110.04002.7706
0.067910.63694252.8739
0.03611.25484503.0192
0.0311.89174753.0430
0.022612.50965003.1513
0.015513.12745253.1610
0.01513.76435503.1778
0.016914.38225753.2098
0.022315.06003.2078

Framework versions

  • PEFT 0.17.1
  • Transformers 4.57.1
  • Pytorch 2.10.0+cu128
  • Datasets 4.0.0
  • Tokenizers 0.22.2

Sources

1 source
Hugging FacePrincipalVersion 3fc2b40433672cd95215aa28fc8e3ae098541a6d
huggingface.co/Raullen/petvlm/tree/3fc2b40433672cd95215aa28fc8e3ae098541a6d

Vérifié 24 oct.

Download files

Artefacts du modèle

4 artefacts

adapter_model.safetensors

safetensors · 70,5 MB · SHA-256 6e6ca2140cc4…3003 · Hugging Face

Download file

checkpoint-150/adapter_model.safetensors

safetensors · 70,5 MB · SHA-256 6e6ca2140cc4…3003 · Hugging Face

Download file

checkpoint-550/adapter_model.safetensors

safetensors · 70,5 MB · SHA-256 c6ea1d743da9…5dc7 · Hugging Face

Download file

checkpoint-600/adapter_model.safetensors

safetensors · 70,5 MB · SHA-256 25f3841e9633…60ef · Hugging Face

Download file

Extraits de sources

3 extraits
Readme7 aoûthuggingface.co/Raullen/petvlm

--- library_name: peft license: mit base_model: Qwen/Qwen2-VL-2B-Instruct tags: - llama-factory - lora - transformers - video-language-model - pet-detection pipeline_tag: video-text-to-text model-index: - name: petvlm-v1.1 results: [] --- # PetVLM v1.1 - Sherlock Pet A video-language model fine-tuned to detect naughty vs nice pet behavior with a sarcastic pet detective persona. **Accuracy: 87.5%** on held-out test set (7/8 correct) ## Model Description PetVLM analyzes pet videos and classifies behavior: - **ALERT!** - Naughty behavior (climbing furniture, scratching, stealing food) - **All Clear** - Good behavior (sleeping, eating properly, playing nicely) ## Training Data - **Dataset Size**: 175 videos (cleaned and balanced) - **Label Balance**: 51% ALERT / 49% All Clear - **Best Checkpoint**: epoch 3.6 - **Training Time**: ~13 minutes on RTX 4090 ## Intended Uses - Home pet monitoring - Pet behavior analysis - Entertainment (sarcastic detective persona) ## Running on Mac (Apple Silicon) PetVLM runs on Mac computers with Apple Silicon (M1/M2/M3) using Metal Performance Shaders (MPS). ### Requirements | Component | Requirement | |-----------|-------------| | macOS | 13.0 (Ventura) or later | | Python | 3.10+ | | RAM | 8GB minimum (16GB recommended) | ### Memory Usage by Precision | Precision | Memory | Speed (M2 Pro) | |-----------|--------|----------------| | FP16 + MPS | ~6 GB | 3-5s/video | | INT4 + MPS | ~2.5 GB | 5-8s/video | | FP32 + CPU | ~10 GB | 30-60s/video | ### Quick Start ```bash # Install dependencies pip install torch torchvision torchaudio pip install transformers peft accelerate pip install qwen-vl-utils imageio[ffmpeg] # Run inference python inference_mac.py your_video.mp4 ``` ### Example Code ```python import torch from transformers import Qwen2VLForConditionalGeneration, AutoProcessor from peft import PeftModel # Use MPS for Apple Silicon device = "mps" if torch.backends.mps.is_available() else "cpu" # Load model model = Qwen2VLForConditionalGeneration.from_pretrained( "Qwen/Qwen2-VL-2B-Instruct", torch_dtype=torch.float16 ).to(device) model = PeftModel.from_pretrained(model, "Raullen/petvlm") model.eval() processor = AutoProcessor.from_pretrained("Raullen/petvlm") ``` For complete Mac installation guide and optimized inference script, see: - [Mac Installation Guide](https://github.com/raullenchai/petvlm/blo...

Note24 oct.huggingface.co/Raullen/petvlm

Source context: 4 downloads · 0 likes · Pipeline video-text-to-text · Library peft · Repo Raullen/petvlm

Note7 aoûthuggingface.co/Raullen/petvlm

Source context: 6 downloads · 0 likes · Pipeline video-text-to-text · Library peft · Repo Raullen/petvlm