Cattle Vision Framework

--- license: mit language: - en tags: - cattle - behavior-recognition - object-detection - video-classification - pytorch - rfdetr - videomae - animal-behavior datasets: - sakifkhan98/cattle-vision-data --- # Cattle Vision Framework — Model Weights Pretrained model weights for the [Cattle Vision Framework](https://github.com/sakifkhan98/cattle-vision-framework), an MS thesis project on multi-behavior recognition in dairy cattle from surveillance video. **Pipeline:** RF-DETR detection → SAM2 segmentation → OC-SORT tracking → VideoMAE behavior classification **Datasets:** CBVD-5 (indoor barn, 12 GB) and CVB (outdoor pasture, 15 GB) --- ## Available Weights | Filename | Size | Description | Val Metric | |----------|------|-------------|------------| | `rf-detr-medium.pth` | 387 MB | RF-DETR-Medium backbone (COCO pretrained, not fine-tuned) | — | | `rf-detr-seg-medium.pt` | 137 MB | RF-DETR-Seg-Medium fine-tuned on SAM2 pseudo-labels (Phase 3b) | — | | `rfdetr_combined_v1_best.pth` | 128 MB | RF-DETR cattle detector, trained on CBVD-5 + CVB combined | 70.4% mAP@50 | | `videomae_combined_v1.pt` | 330 MB | VideoMAE Config 5: trained on CBVD-5+CVB, eval on both | macro-F1=0.7537 | | `videomae_cvb_v1.pt` | 330 MB | VideoMAE Config 2: trained on CVB, eval on CVB | macro-F1=0.7607 | | `videomae_cbvd5_v1.pt` | 330 MB | VideoMAE Config 1: trained on CBVD-5, eval on CBVD-5 | macro-F1=0.3149 | | `videomae_cbvd5_to_cvb_v1.pt` | 330 MB | VideoMAE Config 3: trained on CBVD-5, eval on CVB (cross-domain) | macro-F1=0.1690 | | `videomae_cvb_to_cbvd5_v1.pt` | 330 MB | VideoMAE Config 4: trained on CVB, eval on CBVD-5 (cross-domain) | macro-F1=0.1789 | --- ## 7-Class Behavior Taxonomy | ID | Behavior | Datasets | |----|----------|---------| | 0 | Standing | CBVD-5, CVB | | 1 | Lying | CBVD-5, CVB | | 2 | Foraging/Grazing | CBVD-5, CVB | | 3 | Drinking | CBVD-5, CVB | | 4 | Ruminating | CBVD-5, CVB | | 5 | Grooming | CVB only | | 6 | Other | CVB only | Cross-dataset evaluation uses IDs 0–4 only. --- ## Behavior Classification Results (VideoMAE) | Config | Train | Eval | macro-F1 | F1-Standing | F1-Lying | F1-Foraging | F1-Drinking | F1-Ruminating | |--------|-------|------|----------|-------------|----------|-------------|-------------|---------------| | Config 1 | CBVD-5 | CBVD-5 | 0.3149 | 0.906 | 0.303 | 0.896 | 0.000 | 0.100 | | Config 2 | CVB | CVB | **0.7607...