Saltar al contenido
ModelPublicadoExterno
Descargar

TimesNet

This project presents a hybrid architecture that replaces the temporal attention mechanism in TimeSformer with RetNet, achieving:

Fuentehuggingface.co/sumit7488/TimesNet/tree/1293b6ceae3f816debfeac1cd8d6cb6c0c4b75cc
TipoOtherVersiónRevision 1293b6cBcalidad

Fuente del modelo

Tipo
Other
Tamaño del archivo
494 MB
Versión
vRevision 1293b6c
Actualizado
24 oct 2026
Descargas
10
Fuente
Hugging Face

Descripción de la fuente

🎬 TimeSformer + RetNet Hybrid for Efficient Video Action Recognition

This project presents a hybrid architecture that replaces the temporal attention mechanism in TimeSformer with RetNet, achieving:

  • ⚡ Faster training
  • 🧠 Lower memory usage
  • 🎯 Comparable or improved accuracy

🚀 Model Variants

We trained and evaluated 4 configurations:

ModelDataset
TimeSformer (Baseline)UCF101
TimeSformer (Baseline)HMDB51
TimeSformer + RetNet (Hybrid)UCF101
TimeSformer + RetNet (Hybrid)HMDB51

🧠 Proposed Architecture

🔹 Baseline

  • TimeSformer
  • Full spatio-temporal attention

🔹 Hybrid Model (Proposed)

  • Spatial Attention → TimeSformer
  • Temporal Modeling → RetNet

👉 RetNet replaces temporal self-attention to reduce complexity from:

  • Quadratic → Linear time

📊 Hybrid Model Training Results (UCF101)

EpochTrain LossTrain AccVal LossVal AccF1
14.52750.04584.15960.35420.3076
23.66470.40892.64960.75500.7214
32.42210.69951.53130.86230.8509
41.88740.78411.22900.89610.8918
51.72680.81041.15840.90750.9040
61.66150.81451.10880.91670.9142
71.60760.81911.09620.92020.9168
81.51000.82341.08650.92600.9233
91.47040.82321.08120.92600.9226

🏆 Best Performance (Hybrid Model)

  • Validation Accuracy: 92.60%
  • F1 Score: 0.9233
  • Achieved at Epoch 8

⚡ Efficiency Comparison

MetricTimeSformerHybrid (RetNet)
Peak GPU Memory~9.3–9.8 GB~7.2 GB ✅
Training SpeedSlowerFaster ✅
Temporal ComplexityO(n2)O(n) ✅

👉 ~25% memory reduction with comparable performance.

🔁 Training Strategy

Due to Kaggle’s 12-hour runtime limit, training was performed in stages:

  • Initial training
  • Save best checkpoint
  • Resume from .safetensors
  • Continue training

⚙️ Training Details

  • Mixed Precision Training (torch.cuda.amp)
  • Checkpoint-based training
  • Per-class evaluation reports
  • GPU: Kaggle environment

📦 Base Model

  • facebook/timesformer-base-finetuned-k400

🚀 Usage

pip install torch torchvision transformers

Fuentes

1 fuente
Hugging FacePrincipalVersión 1293b6ceae3f816debfeac1cd8d6cb6c0c4b75cc
huggingface.co/sumit7488/TimesNet/tree/1293b6ceae3f816debfeac1cd8d6cb6c0c4b75cc

Verificado 24 oct

Artefactos del modelo

1 artefacto

model.safetensors

safetensors · 494 MB · SHA-256 f6ab5b13bd6f…3aef · Hugging Face

Download file

Extractos de fuentes

3 extractos
Readme6 agohuggingface.co/sumit7488/TimesNet

--- license: mit tags: - video-classification - timesformer - retnet - action-recognition - ucf101 - hmdb51 - transformers - efficient-models datasets: - ucf101 - hmdb51 --- # 🎬 TimeSformer + RetNet Hybrid for Efficient Video Action Recognition This project presents a **hybrid architecture** that replaces the temporal attention mechanism in TimeSformer with **RetNet**, achieving: - ⚡ Faster training - 🧠 Lower memory usage - 🎯 Comparable or improved accuracy --- ## 🚀 Model Variants We trained and evaluated **4 configurations**: | Model | Dataset | |------|--------| | TimeSformer (Baseline) | UCF101 | | TimeSformer (Baseline) | HMDB51 | | **TimeSformer + RetNet (Hybrid)** | UCF101 | | **TimeSformer + RetNet (Hybrid)** | HMDB51 | --- ## 🧠 Proposed Architecture ### 🔹 Baseline - **TimeSformer** - Full spatio-temporal attention ### 🔹 Hybrid Model (Proposed) - Spatial Attention → TimeSformer - Temporal Modeling → **RetNet** 👉 RetNet replaces temporal self-attention to reduce complexity from: - **Quadratic → Linear time** --- ## 📊 Hybrid Model Training Results (UCF101) | Epoch | Train Loss | Train Acc | Val Loss | Val Acc | F1 | |------|------------|-----------|----------|---------|-----| | 1 | 4.5275 | 0.0458 | 4.1596 | 0.3542 | 0.3076 | | 2 | 3.6647 | 0.4089 | 2.6496 | 0.7550 | 0.7214 | | 3 | 2.4221 | 0.6995 | 1.5313 | 0.8623 | 0.8509 | | 4 | 1.8874 | 0.7841 | 1.2290 | 0.8961 | 0.8918 | | 5 | 1.7268 | 0.8104 | 1.1584 | 0.9075 | 0.9040 | | 6 | 1.6615 | 0.8145 | 1.1088 | 0.9167 | 0.9142 | | 7 | 1.6076 | 0.8191 | 1.0962 | 0.9202 | 0.9168 | | 8 | 1.5100 | 0.8234 | 1.0865 | **0.9260** | **0.9233** | | 9 | 1.4704 | 0.8232 | 1.0812 | 0.9260 | 0.9226 | --- ## 🏆 Best Performance (Hybrid Model) - **Validation Accuracy:** **92.60%** - **F1 Score:** 0.9233 - Achieved at Epoch 8 --- ## ⚡ Efficiency Comparison | Metric | TimeSformer | Hybrid (RetNet) | |-------|------------|----------------| | Peak GPU Memory | ~9.3–9.8 GB | **~7.2 GB** ✅ | | Training Speed | Slower | **Faster** ✅ | | Temporal Complexity | O(n²) | **O(n)** ✅ | 👉 **~25% memory reduction** with comparable performance. --- ## 🔁 Training Strategy Due to Kaggle’s **12-hour runtime limit**, training was performed in stages: - Initial training - Save best checkpoint - Resume from `.safetensors` - Continue training --- ## ⚙️ Training Details - Mixed Precision Training (`torch...

Note24 octhuggingface.co/sumit7488/TimesNet

Source context: 10 downloads · 0 likes · Pipeline video-classification · Library transformers · Repo sumit7488/TimesNet

Note6 agohuggingface.co/sumit7488/TimesNet

Source context: 5 downloads · 0 likes · Pipeline video-classification · Library transformers · Repo sumit7488/TimesNet