Ir para o conteúdo
ModelListadosExterno
Browse files

Petvlm

A video-language model fine-tuned to detect naughty vs nice pet behavior with a sarcastic pet detective persona.

Fontehuggingface.co/Raullen/petvlm/tree/3fc2b40433672cd95215aa28fc8e3ae098541a6d
TipoOtherModelo baseQwen/Qwen2-VL-2B-InstructVersãoRevision 3fc2b40Bnota

Fonte do modelo

Tipo
Other
Tamanho do arquivo
70,5 MB
Modelo base
Qwen/Qwen2-VL-2B-Instruct
Versão
vRevision 3fc2b40
Atualizado
24 de out. de 2026
Downloads
4
Fonte
Hugging Face

Descrição da fonte

PetVLM v1.1 - Sherlock Pet

A video-language model fine-tuned to detect naughty vs nice pet behavior with a sarcastic pet detective persona.

Accuracy: 87.5% on held-out test set (7/8 correct)

Model Description

PetVLM analyzes pet videos and classifies behavior:

  • ALERT! - Naughty behavior (climbing furniture, scratching, stealing food)
  • All Clear - Good behavior (sleeping, eating properly, playing nicely)

Training Data

  • Dataset Size: 175 videos (cleaned and balanced)
  • Label Balance: 51% ALERT / 49% All Clear
  • Best Checkpoint: epoch 3.6
  • Training Time: ~13 minutes on RTX 4090

Intended Uses

  • Home pet monitoring
  • Pet behavior analysis
  • Entertainment (sarcastic detective persona)

Running on Mac (Apple Silicon)

PetVLM runs on Mac computers with Apple Silicon (M1/M2/M3) using Metal Performance Shaders (MPS).

Requirements

ComponentRequirement
macOS13.0 (Ventura) or later
Python3.10+
RAM8GB minimum (16GB recommended)

Memory Usage by Precision

PrecisionMemorySpeed (M2 Pro)
FP16 + MPS~6 GB3-5s/video
INT4 + MPS~2.5 GB5-8s/video
FP32 + CPU~10 GB30-60s/video

Quick Start

# Install dependencies
pip install torch torchvision torchaudio
pip install transformers peft accelerate
pip install qwen-vl-utils imageio[ffmpeg]

# Run inference
python inference_mac.py your_video.mp4

Example Code

import torch
from transformers import Qwen2VLForConditionalGeneration, AutoProcessor
from peft import PeftModel

# Use MPS for Apple Silicon
device = "mps" if torch.backends.mps.is_available() else "cpu"

# Load model
model = Qwen2VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2-VL-2B-Instruct",
    torch_dtype=torch.float16
).to(device)

model = PeftModel.from_pretrained(model, "Raullen/petvlm")
model.eval()

processor = AutoProcessor.from_pretrained("Raullen/petvlm")

For complete Mac installation guide and optimized inference script, see:

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 1
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 4
  • optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.05
  • num_epochs: 15.0

Training results

Training LossEpochStepValidation Loss
3.28590.6369252.8292
2.01431.2548501.9940
1.68291.8917751.7856
1.47742.50961001.7516
1.19083.12741251.6817
1.20463.76431501.6712
0.80124.38221751.7618
0.95955.02001.7216
0.51155.63692251.8212
0.46666.25482502.1777
0.40826.89172752.0293
0.22627.50963002.3396
0.17748.12743252.3459
0.15128.76433502.4745
0.07199.38223752.7719
0.074110.04002.7706
0.067910.63694252.8739
0.03611.25484503.0192
0.0311.89174753.0430
0.022612.50965003.1513
0.015513.12745253.1610
0.01513.76435503.1778
0.016914.38225753.2098
0.022315.06003.2078

Framework versions

  • PEFT 0.17.1
  • Transformers 4.57.1
  • Pytorch 2.10.0+cu128
  • Datasets 4.0.0
  • Tokenizers 0.22.2

Fontes

1 fonte
Hugging FacePrimárioVersão 3fc2b40433672cd95215aa28fc8e3ae098541a6d
huggingface.co/Raullen/petvlm/tree/3fc2b40433672cd95215aa28fc8e3ae098541a6d

Verificado 24 de out.

Download files

Artefatos de modelo

4 artefatos

adapter_model.safetensors

safetensors · 70,5 MB · SHA-256 6e6ca2140cc4…3003 · Hugging Face

Download file

checkpoint-150/adapter_model.safetensors

safetensors · 70,5 MB · SHA-256 6e6ca2140cc4…3003 · Hugging Face

Download file

checkpoint-550/adapter_model.safetensors

safetensors · 70,5 MB · SHA-256 c6ea1d743da9…5dc7 · Hugging Face

Download file

checkpoint-600/adapter_model.safetensors

safetensors · 70,5 MB · SHA-256 25f3841e9633…60ef · Hugging Face

Download file

Trechos de fonte

3 trechos
Readme7 de ago.huggingface.co/Raullen/petvlm

--- library_name: peft license: mit base_model: Qwen/Qwen2-VL-2B-Instruct tags: - llama-factory - lora - transformers - video-language-model - pet-detection pipeline_tag: video-text-to-text model-index: - name: petvlm-v1.1 results: [] --- # PetVLM v1.1 - Sherlock Pet A video-language model fine-tuned to detect naughty vs nice pet behavior with a sarcastic pet detective persona. **Accuracy: 87.5%** on held-out test set (7/8 correct) ## Model Description PetVLM analyzes pet videos and classifies behavior: - **ALERT!** - Naughty behavior (climbing furniture, scratching, stealing food) - **All Clear** - Good behavior (sleeping, eating properly, playing nicely) ## Training Data - **Dataset Size**: 175 videos (cleaned and balanced) - **Label Balance**: 51% ALERT / 49% All Clear - **Best Checkpoint**: epoch 3.6 - **Training Time**: ~13 minutes on RTX 4090 ## Intended Uses - Home pet monitoring - Pet behavior analysis - Entertainment (sarcastic detective persona) ## Running on Mac (Apple Silicon) PetVLM runs on Mac computers with Apple Silicon (M1/M2/M3) using Metal Performance Shaders (MPS). ### Requirements | Component | Requirement | |-----------|-------------| | macOS | 13.0 (Ventura) or later | | Python | 3.10+ | | RAM | 8GB minimum (16GB recommended) | ### Memory Usage by Precision | Precision | Memory | Speed (M2 Pro) | |-----------|--------|----------------| | FP16 + MPS | ~6 GB | 3-5s/video | | INT4 + MPS | ~2.5 GB | 5-8s/video | | FP32 + CPU | ~10 GB | 30-60s/video | ### Quick Start ```bash # Install dependencies pip install torch torchvision torchaudio pip install transformers peft accelerate pip install qwen-vl-utils imageio[ffmpeg] # Run inference python inference_mac.py your_video.mp4 ``` ### Example Code ```python import torch from transformers import Qwen2VLForConditionalGeneration, AutoProcessor from peft import PeftModel # Use MPS for Apple Silicon device = "mps" if torch.backends.mps.is_available() else "cpu" # Load model model = Qwen2VLForConditionalGeneration.from_pretrained( "Qwen/Qwen2-VL-2B-Instruct", torch_dtype=torch.float16 ).to(device) model = PeftModel.from_pretrained(model, "Raullen/petvlm") model.eval() processor = AutoProcessor.from_pretrained("Raullen/petvlm") ``` For complete Mac installation guide and optimized inference script, see: - [Mac Installation Guide](https://github.com/raullenchai/petvlm/blo...

Note24 de out.huggingface.co/Raullen/petvlm

Source context: 4 downloads · 0 likes · Pipeline video-text-to-text · Library peft · Repo Raullen/petvlm

Note7 de ago.huggingface.co/Raullen/petvlm

Source context: 6 downloads · 0 likes · Pipeline video-text-to-text · Library peft · Repo Raullen/petvlm