Saltar al contenido
ModelPublicadoExterno
Browse files

Petvlm

A video-language model fine-tuned to detect naughty vs nice pet behavior with a sarcastic pet detective persona.

Fuentehuggingface.co/Raullen/petvlm/tree/3fc2b40433672cd95215aa28fc8e3ae098541a6d
TipoOtherModelo baseQwen/Qwen2-VL-2B-InstructVersiónRevision 3fc2b40Bcalidad

Fuente del modelo

Tipo
Other
Tamaño del archivo
70,5 MB
Modelo base
Qwen/Qwen2-VL-2B-Instruct
Versión
vRevision 3fc2b40
Actualizado
24 oct 2026
Descargas
4
Fuente
Hugging Face

Descripción de la fuente

PetVLM v1.1 - Sherlock Pet

A video-language model fine-tuned to detect naughty vs nice pet behavior with a sarcastic pet detective persona.

Accuracy: 87.5% on held-out test set (7/8 correct)

Model Description

PetVLM analyzes pet videos and classifies behavior:

  • ALERT! - Naughty behavior (climbing furniture, scratching, stealing food)
  • All Clear - Good behavior (sleeping, eating properly, playing nicely)

Training Data

  • Dataset Size: 175 videos (cleaned and balanced)
  • Label Balance: 51% ALERT / 49% All Clear
  • Best Checkpoint: epoch 3.6
  • Training Time: ~13 minutes on RTX 4090

Intended Uses

  • Home pet monitoring
  • Pet behavior analysis
  • Entertainment (sarcastic detective persona)

Running on Mac (Apple Silicon)

PetVLM runs on Mac computers with Apple Silicon (M1/M2/M3) using Metal Performance Shaders (MPS).

Requirements

ComponentRequirement
macOS13.0 (Ventura) or later
Python3.10+
RAM8GB minimum (16GB recommended)

Memory Usage by Precision

PrecisionMemorySpeed (M2 Pro)
FP16 + MPS~6 GB3-5s/video
INT4 + MPS~2.5 GB5-8s/video
FP32 + CPU~10 GB30-60s/video

Quick Start

# Install dependencies
pip install torch torchvision torchaudio
pip install transformers peft accelerate
pip install qwen-vl-utils imageio[ffmpeg]

# Run inference
python inference_mac.py your_video.mp4

Example Code

import torch
from transformers import Qwen2VLForConditionalGeneration, AutoProcessor
from peft import PeftModel

# Use MPS for Apple Silicon
device = "mps" if torch.backends.mps.is_available() else "cpu"

# Load model
model = Qwen2VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2-VL-2B-Instruct",
    torch_dtype=torch.float16
).to(device)

model = PeftModel.from_pretrained(model, "Raullen/petvlm")
model.eval()

processor = AutoProcessor.from_pretrained("Raullen/petvlm")

For complete Mac installation guide and optimized inference script, see:

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 1
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 4
  • optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.05
  • num_epochs: 15.0

Training results

Training LossEpochStepValidation Loss
3.28590.6369252.8292
2.01431.2548501.9940
1.68291.8917751.7856
1.47742.50961001.7516
1.19083.12741251.6817
1.20463.76431501.6712
0.80124.38221751.7618
0.95955.02001.7216
0.51155.63692251.8212
0.46666.25482502.1777
0.40826.89172752.0293
0.22627.50963002.3396
0.17748.12743252.3459
0.15128.76433502.4745
0.07199.38223752.7719
0.074110.04002.7706
0.067910.63694252.8739
0.03611.25484503.0192
0.0311.89174753.0430
0.022612.50965003.1513
0.015513.12745253.1610
0.01513.76435503.1778
0.016914.38225753.2098
0.022315.06003.2078

Framework versions

  • PEFT 0.17.1
  • Transformers 4.57.1
  • Pytorch 2.10.0+cu128
  • Datasets 4.0.0
  • Tokenizers 0.22.2

Fuentes

1 fuente
Hugging FacePrincipalVersión 3fc2b40433672cd95215aa28fc8e3ae098541a6d
huggingface.co/Raullen/petvlm/tree/3fc2b40433672cd95215aa28fc8e3ae098541a6d

Verificado 24 oct

Download files

Artefactos del modelo

4 artefactos

adapter_model.safetensors

safetensors · 70,5 MB · SHA-256 6e6ca2140cc4…3003 · Hugging Face

Download file

checkpoint-150/adapter_model.safetensors

safetensors · 70,5 MB · SHA-256 6e6ca2140cc4…3003 · Hugging Face

Download file

checkpoint-550/adapter_model.safetensors

safetensors · 70,5 MB · SHA-256 c6ea1d743da9…5dc7 · Hugging Face

Download file

checkpoint-600/adapter_model.safetensors

safetensors · 70,5 MB · SHA-256 25f3841e9633…60ef · Hugging Face

Download file

Extractos de fuentes

3 extractos
Readme7 agohuggingface.co/Raullen/petvlm

--- library_name: peft license: mit base_model: Qwen/Qwen2-VL-2B-Instruct tags: - llama-factory - lora - transformers - video-language-model - pet-detection pipeline_tag: video-text-to-text model-index: - name: petvlm-v1.1 results: [] --- # PetVLM v1.1 - Sherlock Pet A video-language model fine-tuned to detect naughty vs nice pet behavior with a sarcastic pet detective persona. **Accuracy: 87.5%** on held-out test set (7/8 correct) ## Model Description PetVLM analyzes pet videos and classifies behavior: - **ALERT!** - Naughty behavior (climbing furniture, scratching, stealing food) - **All Clear** - Good behavior (sleeping, eating properly, playing nicely) ## Training Data - **Dataset Size**: 175 videos (cleaned and balanced) - **Label Balance**: 51% ALERT / 49% All Clear - **Best Checkpoint**: epoch 3.6 - **Training Time**: ~13 minutes on RTX 4090 ## Intended Uses - Home pet monitoring - Pet behavior analysis - Entertainment (sarcastic detective persona) ## Running on Mac (Apple Silicon) PetVLM runs on Mac computers with Apple Silicon (M1/M2/M3) using Metal Performance Shaders (MPS). ### Requirements | Component | Requirement | |-----------|-------------| | macOS | 13.0 (Ventura) or later | | Python | 3.10+ | | RAM | 8GB minimum (16GB recommended) | ### Memory Usage by Precision | Precision | Memory | Speed (M2 Pro) | |-----------|--------|----------------| | FP16 + MPS | ~6 GB | 3-5s/video | | INT4 + MPS | ~2.5 GB | 5-8s/video | | FP32 + CPU | ~10 GB | 30-60s/video | ### Quick Start ```bash # Install dependencies pip install torch torchvision torchaudio pip install transformers peft accelerate pip install qwen-vl-utils imageio[ffmpeg] # Run inference python inference_mac.py your_video.mp4 ``` ### Example Code ```python import torch from transformers import Qwen2VLForConditionalGeneration, AutoProcessor from peft import PeftModel # Use MPS for Apple Silicon device = "mps" if torch.backends.mps.is_available() else "cpu" # Load model model = Qwen2VLForConditionalGeneration.from_pretrained( "Qwen/Qwen2-VL-2B-Instruct", torch_dtype=torch.float16 ).to(device) model = PeftModel.from_pretrained(model, "Raullen/petvlm") model.eval() processor = AutoProcessor.from_pretrained("Raullen/petvlm") ``` For complete Mac installation guide and optimized inference script, see: - [Mac Installation Guide](https://github.com/raullenchai/petvlm/blo...

Note24 octhuggingface.co/Raullen/petvlm

Source context: 4 downloads · 0 likes · Pipeline video-text-to-text · Library peft · Repo Raullen/petvlm

Note7 agohuggingface.co/Raullen/petvlm

Source context: 6 downloads · 0 likes · Pipeline video-text-to-text · Library peft · Repo Raullen/petvlm