Loading
Du hast einen ComfyUI-Workflow? Erstelle direkt im Browser eine verständliche Workflow-Dokumentation.Workflow verständlich machen.Jetzt ausprobieren ↗
This model is a fine-tuned version of facebook/wav2vec2-base for multi-label emotion classification from audio.
Modellquelle
Quellenbeschreibung
This model is a fine-tuned version of facebook/wav2vec2-base for multi-label emotion classification from audio.
The model uses a Hydra architecture with multiple binary classification heads, one for each emotion. It supports ignore labels (-100) for handling uncertain or unlabeled data points.
Quellen
1 QuelleVerifiziert 13. Sept.
Modellartefakte
1 ArtefaktQuellenauszüge
2 Auszüge| Emotion | Accuracy | F1 Score | Valid Samples | Ignored Samples |
|---|---|---|---|---|
| angry | 0.8762 | 0.0000 | 4749 | 0 |
| disgust | 0.9872 | 0.0000 | 4749 | 0 |
| fear | 0.9821 | 0.0000 | 4749 | 0 |
| happy | 0.8793 | 0.0000 | 4749 | 0 |
| neutral | 0.6477 | 0.0000 | 4749 | 0 |
| depress | 0.9111 | 0.0000 | 4749 | 0 |
{
"model_id": "facebook/wav2vec2-base",
"emotion_labels": [
"angry",
"disgust",
"fear",
"happy",
"neutral",
"depress",
"surprise",
"frustrated",
"excited"
],
"num_classes": 9,
"ignore_label": -100,
"batch_size": 4,
"num_epochs": 6,
"learning_rate": 0.0001,
"warmup_steps": 100,
"gradient_accumulation_steps": 4,
"classification_threshold": 0.5,
"weight_decay": 0.01,
"max_grad_norm": 1.0,
"seed": 42
}
This model was trained with support for ignore labels (-100). During inference:
import torch
from transformers import Wav2Vec2Processor
# Load model and processor
model = # Load your model here
processor = Wav2Vec2Processor.from_pretrained('mangoesai/wav2vec2-emotion-classifier-V2')
# Process audio
inputs = processor(audio_array, sampling_rate=16000, return_tensors="pt", padding=True)
# Get predictions
with torch.no_grad():
outputs = model(inputs.input_values, inputs.attention_mask)
probabilities = torch.sigmoid(outputs)
predictions = (probabilities > 0.5).int()
If you use this model, please cite:
@misc{wav2vec2-emotion-classifier,
author = {Your Name},
title = {Wav2Vec2 Multi-Label Emotion Classifier},
year = {2024},
publisher = {HuggingFace},
url = {https://huggingface.co/mangoesai/wav2vec2-emotion-classifier-V2}
}
--- language: en license: apache-2.0 tags: - audio-classification - emotion-recognition - wav2vec2 - multi-label-classification datasets: - custom_emotion_dataset metrics: - accuracy - f1 model-index: - name: Wav2Vec2 Emotion Classifier with Ignore Label Support results: - task: type: audio-classification name: Multi-Label Emotion Classification dataset: name: custom_emotion_dataset type: custom_emotion_dataset metrics: - type: accuracy value: 0.0000 - type: f1 value: 0.0000 --- # Wav2Vec2 Multi-Label Emotion Classifier This model is a fine-tuned version of [facebook/wav2vec2-base](https://huggingface.co/facebook/wav2vec2-base) for multi-label emotion classification from audio. ## Model Description The model uses a Hydra architecture with multiple binary classification heads, one for each emotion. It supports ignore labels (-100) for handling uncertain or unlabeled data points. ### Supported Emotions - angry, disgust, fear, happy, neutral, depress, surprise, frustrated, excited ## Training Results ### Overall Performance - **Best Validation F1 Score**: 0.0000 - **Best Validation Accuracy**: 0.0000 - **Best Validation Loss**: inf - **Best Epoch**: 0 ### Per-Emotion Performance (at best epoch) | Emotion | Accuracy | F1 Score | Valid Samples | Ignored Samples | |---------|----------|----------|---------------|-----------------| | angry | 0.8762 | 0.0000 | 4749 | 0 | | disgust | 0.9872 | 0.0000 | 4749 | 0 | | fear | 0.9821 | 0.0000 | 4749 | 0 | | happy | 0.8793 | 0.0000 | 4749 | 0 | | neutral | 0.6477 | 0.0000 | 4749 | 0 | | depress | 0.9111 | 0.0000 | 4749 | 0 | | surprise | 0.9299 | 0.0000 | 4749 | 0 | | frustrated | 0.6949 | 0.0000 | 1996 | 2753 | | excited | 0.7996 | 0.0000 | 1996 | 2753 | ### Sample Statistics - **Total Training Samples**: 18994 - **Total Validation Samples**: 4749 - **Total Valid Labels**: 37235 - **Total Ignored Labels**: 5506 - **Ignore Rate**: 12.9% ## Training Configuration ```json { "model_id": "facebook/wav2vec2-base", "emotion_labels": [ "angry", "disgust", "fear", "happy", "neutral", "depress", "surprise", "frustrated", "excited" ], "num_classes": 9, "ignore_label": -100, "batch_size": 4, "num_epochs": 6, "learning_rate": 0.0001, "warmup_steps": 100, "gradient_accumulation_steps": 4, "classification_threshold": 0.5, "weight_decay": 0.01, "max_grad_norm": 1.0, "seed": 42 } ``` ## Us...
Source context: 11 downloads · 0 likes · Pipeline audio-classification · Repo on1onmangoes/wav2vec2-emotion-classifier-V2
| surprise | 0.9299 | 0.0000 | 4749 | 0 |
| frustrated | 0.6949 | 0.0000 | 1996 | 2753 |
| excited | 0.7996 | 0.0000 | 1996 | 2753 |