Whisper Medium Mizonal3 E2 Lus v2026.06
> Disclaimer / Notice: Details for these are in Peer Review and publications of the paper will be made available soon for more details.
Source du modèle
- Type
- Text model
- Taille du fichier
- 2,85 GB
- Modèle de base
- openai/whisper-medium
- Version
- vmodel
- Mis à jour
- 19 oct. 2026
- Téléchargements
- 5
- Source
- Hugging Face
Description de la source
Mizo Automatic Speech Recognition (ASR) Models v3.0
Disclaimer / Notice: Details for these are in Peer Review and publications of the paper will be made available soon for more details.
whisper-medium-mizonal3-E2-lus-v2026.06
This model is a fine-tuned version of openai/whisper-medium on the MiZonal v3.0 dataset. Note: ~1 hour of conversational speech was added to this dataset version.
It achieves the following results on the evaluation set:
- Wer: 21.7728
- Cer: 7.3593
- Real Time Factor: 0.0520
Quick Inference
import torch
import librosa
from transformers import WhisperProcessor, WhisperForConditionalGeneration
device = "cuda" if torch.cuda.is_available() else "cpu"
processor = WhisperProcessor.from_pretrained("andrewbawitlung/whisper-medium-mizonal3-E2-lus-v2026.06")
model = WhisperForConditionalGeneration.from_pretrained("andrewbawitlung/whisper-medium-mizonal3-E2-lus-v2026.06").to(device)
audio, sr = librosa.load("your_audio.wav", sr=16000)
input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to(device)
with torch.no_grad():
predicted_ids = model.generate(input_features, max_new_tokens=256)
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print(transcription)
Model description
Experiment Configurations
This repository is part of a series of experiments. The different configurations are:
- E1 (Baseline): Standard training configuration.
- E2 (Noise): Training with background noise augmentation.
- E3 (Speed): Training with speed perturbation augmentation.
- E4 (SpecAug): Training with SpecAugment (time and frequency masking).
- E5 (Combined): Training with a combination of all augmentations.
All Models in this Family
| Experiment | Hugging Face Repository |
|---|---|
| E1 (Baseline) | andrewbawitlung/whisper-medium-mizonal3-E1-lus-v2026.06 |
| E2 (Noise) | andrewbawitlung/whisper-medium-mizonal3-E2-lus-v2026.06 |
| E3 (Speed) | andrewbawitlung/whisper-medium-mizonal3-E3-lus-v2026.06 |
| E4 (SpecAug) | andrewbawitlung/whisper-medium-mizonal3-E4-lus-v2026.06 |
| E5 (Combined) | andrewbawitlung/whisper-medium-mizonal3-E5-lus-v2026.06 |
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: OptimizerNames.ADAMW_TORCH_FUSED
- lr_scheduler_type: SchedulerType.LINEAR
- num_epochs: 8
Training results
| step | epoch | train_loss | eval_loss | eval_wer | eval_cer | learning_rate | grad_norm |
|---|---|---|---|---|---|---|---|
| 250 | 0.46 | 0.6866 | 0.6110 | 34.26 | 12.72 | 1.49e-04 | 7.83 |
| 500 | 0.91 | 0.8029 | 0.8721 | 63.79 | 36.05 | 2.99e-04 | 20.60 |
| 750 | 1.37 | 0.6327 | 0.7277 | 36.49 | 17.06 | 2.81e-04 | 5.54 |
| 1000 | 1.82 | 0.4858 | 0.6910 | 34.48 | 14.43 | 2.62e-04 | 3.86 |
| 1250 | 2.28 | 0.3171 | 0.6293 | 30.86 | 12.28 | 2.42e-04 | 3.36 |
| 1500 | 2.73 | 0.2640 | 0.6324 | 30.76 | 11.94 | 2.23e-04 | 3.04 |
| 1750 | 3.19 | 0.1781 | 0.6204 | 35.20 | 15.72 | 2.04e-04 | 2.49 |
| 2000 | 3.64 | 0.1584 | 0.6050 | 28.89 | 11.35 | 1.84e-04 | 2.56 |
| 2250 | 4.10 | 0.0972 | 0.6102 | 28.05 | 11.92 | 1.65e-04 | 1.73 |
| 2500 | 4.55 | 0.0894 | 0.5966 | 25.34 | 9.41 | 1.46e-04 | 1.68 |
| 2750 | 5.01 | 0.0659 | 0.6184 | 24.96 | 9.42 | 1.27e-04 | 0.96 |
| 3000 | 5.46 | 0.0524 | 0.6331 | 25.60 | 10.95 | 1.07e-04 | 1.43 |
| 3250 | 5.92 | 0.0345 | 0.6023 | 23.26 | 8.42 | 8.81e-05 | 0.72 |
| 3500 | 6.38 | 0.0192 | 0.5784 | 24.83 | 10.80 | 6.88e-05 | 0.25 |
| 3750 | 6.83 | 0.0097 | 0.6011 | 21.91 | 8.17 | 4.96e-05 | 0.13 |
| 4000 | 7.29 | 0.0038 | 0.5964 | 20.29 | 7.09 | 3.03e-05 | 0.12 |
| 4250 | 7.74 | 0.0020 | 0.6115 | 19.98 | 6.81 | 1.10e-05 | 0.01 |
Sources
1 source
Sources
1 sourceVérifié 19 oct.
Artefacts du modèle
1 artefact
Artefacts du modèle
1 artefactExtraits de sources
3 extraits
Extraits de sources
3 extraits--- language: - lus license: apache-2.0 pipeline_tag: automatic-speech-recognition base_model: openai/whisper-medium tags: - generated_from_trainer datasets: - andrewbawitlung/MiZonal-v3.0 metrics: - wer - cer model-index: - name: whisper-medium-mizonal3-E2-lus-v2026.06 results: - task: name: Automatic Speech Recognition type: automatic-speech-recognition dataset: name: MiZonal v3.0 type: andrewbawitlung/MiZonal-v3.0 config: default split: test metrics: - name: Wer type: wer value: 21.7728 - name: Cer type: cer value: 7.3593 - name: Real Time Factor type: rtf value: 0.0520 ---  > **Disclaimer / Notice:** Details for these are in Peer Review and publications of the paper will be made available soon for more details. <!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. --> # whisper-medium-mizonal3-E2-lus-v2026.06 This model is a fine-tuned version of [openai/whisper-medium](https://huggingface.co/openai/whisper-medium) on the **MiZonal v3.0** dataset. Note: ~1 hour of conversational speech was added to this dataset version. It achieves the following results on the evaluation set: - Wer: 21.7728 - Cer: 7.3593 - Real Time Factor: 0.0520 ## Quick Inference ```python import torch import librosa from transformers import WhisperProcessor, WhisperForConditionalGeneration device = "cuda" if torch.cuda.is_available() else "cpu" processor = WhisperProcessor.from_pretrained("andrewbawitlung/whisper-medium-mizonal3-E2-lus-v2026.06") model = WhisperForConditionalGeneration.from_pretrained("andrewbawitlung/whisper-medium-mizonal3-E2-lus-v2026.06").to(device) audio, sr = librosa.load("your_audio.wav", sr=16000) input_features = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to(device) with torch.no_grad(): predicted_ids = model.generate(input_features, max_new_tokens=256) transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0] print(transcription) ``` ## Model description ### Experiment Configurations This repository is part of a series of experiments. The different configurations are: - **E1 (Baseline):** Standard training configuration. - **E2 (Noise):** Training with background noise augme...
Source context: 5 downloads · 0 likes · Pipeline automatic-speech-recognition · Repo andrewbawitlung/whisper-medium-mizonal3-E2-lus-v2026.06
Source context: 49 downloads · 0 likes · Pipeline automatic-speech-recognition · Repo andrewbawitlung/whisper-medium-mizonal3-E2-lus-v2026.06
