--- language: - mfe license: apache-2.0 library_name: transformers tags: - whisper - automatic-speech-recognition - speech - kreol-morisien - mauritian-creole - fine-tuned base_model: openai/whisper-large-v3 pipeline_tag: automatic-speech-recognition model-index: - name: whisper-largev3-km-indomain20-withac results: - task: type: automatic-speech-recognition name: Speech Recognition dataset: name: Kreol Morisien In-Domain (20h) type: custom metrics: - type: wer value: 10.09 name: WER --- # Whisper Large-v3 — Kreol Morisien (In-Domain Fine-Tuned, With AC, 20h) This model is a fine-tuned version of [openai/whisper-large-v3](https://huggingface.co/openai/whisper-large-v3) on 20 hours of in-domain Kreol Morisien (Mauritian Creole) audio data, with accent conditioning. ## Model Details | Parameter | Value | |---|---| | Base Model | `openai/whisper-large-v3` | | Parameters | ~1.55B | | Language | Kreol Morisien (mfe) | | Task | Transcription | | Best WER | **10.09%** | ## Training Data - **Domain:** In-domain Kreol Morisien speech - **Total Duration:** ~20 hours - **Split:** 90% train / 10% validation - **Format:** 16kHz mono WAV - **Preprocessing:** Text normalization (lowercased, whitespace-collapsed, smart quotes replaced) ## Training Configuration | Hyperparameter | Value | |---|---| | Epochs | 7 | | Batch Size (per device) | 8 | | Gradient Accumulation Steps | 4 | | Effective Batch Size | 32 | | Learning Rate | 3e-5 | | LR Scheduler | Cosine | | Warmup Steps | 200 | | Weight Decay | 0.1 | | Precision | bf16 | | Generation Beams (eval) | 1 (greedy) | | Best Checkpoint | Step 742 | ## Training Results | Step | Training Loss | Validation Loss | WER | |---|---|---|---| | 100 | 1.7207 | 0.3548 | 0.2322 | | 200 | 0.8420 | 0.2530 | 0.1429 | | 300 | 0.4116 | 0.2448 | 0.1275 | | 400 | 0.1855 | 0.2396 | 0.1173 | | 500 | 0.0825 | 0.2450 | 0.1091 | | 600 | 0.0322 | 0.2556 | 0.1033 | | 700 | 0.0083 | 0.2599 | 0.1013 | | 742 | 0.0083 | 0.2605 | 0.1009 | ## Usage ```python from transformers import WhisperProcessor, WhisperForConditionalGeneration import soundfile as sf processor = WhisperProcessor.from_pretrained("Shagufta/whisper-largev3-km-indomain20-withac") model = WhisperForConditionalGeneration.from_pretrained("Shagufta/whisper-largev3-km-indomain20-withac") speech, sr = sf.read("audio.wav", dtype="float32") input_features = processor.feature_extractor( speech, sampling_rate=16000, return_tensors="pt" ).input_features predicted_ids = model.generate( input_features, max_length=256, num_beams=5, ) transcription = processor.tokenizer.batch_decode(predicted_ids, skip_special_tokens=True)[0] print(transcription) ``` ## Model Config ```python model.config.forced_decoder_ids = None model.config.suppress_tokens = [] ``` ## Limitations - Trained on 20 hours of in-domain data — may not generalize well to out-of-domain Kreol Morisien speech. - Kreol Morisien is not an officially supported Whisper language, so no language token is set. - Signs of overfitting observed (training loss near 0 while validation loss plateaus). Best checkpoint selected by WER. ## Framework - **Transformers** (Seq2SeqTrainer) - **Evaluate** (WER metric)
