Whisper Small Cv11
A plain openai/whisper-small (unmodified architecture) fine-tuned on 11 languages of the CC0 Whispered corpus (32k clips/language, 350 h). No custom code — works in transformers, faster-whisper, CTranslate2,...
Modellquelle
- Art
- Text model
- Dateigröße
- 461 MB
- Basismodell
- openai/whisper-small
- Version
- vmodel
- Aktualisiert
- 27. Sept. 2026
- Downloads
- 6
- Quelle
- Hugging Face
Quellenbeschreibung
Whisper-Small-CV11 — 11-language fine-tune, standard architecture, loads anywhere
A plain openai/whisper-small (unmodified architecture) fine-tuned on 11 languages of the
CC0 Whispered corpus
(32k clips/language, ~350 h). No custom code — works in transformers, faster-whisper, CTranslate2,
whisper.cpp conversions, anywhere a Whisper checkpoint works.
It is also the scientific control for the WhisperMLA release (trained identically to the MLA variants, minus the conversion) — which makes its numbers directly comparable to theirs.
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
model = AutoModelForSpeechSeq2Seq.from_pretrained("burakaydinofficial/whisper-small-cv11") # no trust_remote_code needed
processor = AutoProcessor.from_pretrained("burakaydinofficial/whisper-small-cv11")
Results (CommonVoice-17 test, n=1500/lang, greedy, Whisper normalization + Arabic folding)
| Lang | stock whisper-small | this model | metric |
|---|---|---|---|
| en | 11.7 | 12.2 | WER % |
| de | 15.4 | 15.0 | WER % |
| es | 12.1 | 9.7 | WER % |
| fr | 22.9 | 19.2 | WER % |
| ru | 17.3 | 13.6 | WER % |
| tr | 25.2 | 20.9 | WER % |
| cy | 74.2 | 60.7 | WER % |
| ar | 48.4 | 35.6 | WER % |
| th | 19.7 | 11.3 | CER % |
| zh | 34.4 | 14.9 | CER % |
| ka | 178.0 | 86.2 | WER % (unusable in both — model-class floor) |
Better than stock on 10 of 11 languages (−0.5 on en). Encoder frozen during fine-tuning (the multilingual encoder is preserved); 15,000 steps, warmup+cosine, dev-checkpoint-selected; bf16.
Limitations
- Coverage = these 11 languages; fine-tuning erodes Whisper's other ~88 (unseen scripts most — measured in the accompanying study). Use stock Whisper for languages far from this set.
- Georgian (ka) reported for transparency; not usable at the whisper-small tier.
- Read-speech domain (CommonVoice + FLEURS-validated); not benchmarked on far-field/telephony.
Acoustic conditions of the evaluation
Evaluated on crowdsourced consumer-microphone recordings with real environmental noise — traffic, room reverb, variable devices — CommonVoice's native conditions, not studio audio. The numbers above already include that heterogeneity. Not yet benchmarked: far-field, telephony (8 kHz), overlapping speech.
(Note: this is the unconverted control — it has no MHA→MLA conversion, so "compression cost" does not apply to it. The SNR-robustness-of-compression result lives on the MLA cards, where it is measured as MLA-vs-this-control.)
Quellen
1 Quelle
Quellen
1 QuelleVerifiziert 30. Aug.
Modellartefakte
1 Artefakt
Modellartefakte
1 ArtefaktQuellenauszüge
2 Auszüge
Quellenauszüge
2 Auszüge--- license: apache-2.0 base_model: openai/whisper-small datasets: - burakaydinofficial/Whispered language: [en, de, es, fr, ru, tr, cy, ar, ka, th, zh] pipeline_tag: automatic-speech-recognition library_name: transformers tags: [whisper, multilingual, speech-recognition, fine-tuned, control] --- # Whisper-Small-CV11 — 11-language fine-tune, standard architecture, loads anywhere A **plain `openai/whisper-small`** (unmodified architecture) fine-tuned on 11 languages of the CC0 [Whispered](https://huggingface.co/datasets/burakaydinofficial/Whispered) corpus (32k clips/language, ~350 h). No custom code — works in transformers, faster-whisper, CTranslate2, whisper.cpp conversions, anywhere a Whisper checkpoint works. It is also the **scientific control** for the WhisperMLA release (trained identically to the MLA variants, minus the conversion) — which makes its numbers directly comparable to theirs. ```python from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor model = AutoModelForSpeechSeq2Seq.from_pretrained("burakaydinofficial/whisper-small-cv11") # no trust_remote_code needed processor = AutoProcessor.from_pretrained("burakaydinofficial/whisper-small-cv11") ``` ## Results (CommonVoice-17 test, n=1500/lang, greedy, Whisper normalization + Arabic folding) | Lang | stock whisper-small | **this model** | metric | |---|---|---|---| | en | 11.7 | 12.2 | WER % | | de | 15.4 | **15.0** | WER % | | es | 12.1 | **9.7** | WER % | | fr | 22.9 | **19.2** | WER % | | ru | 17.3 | **13.6** | WER % | | tr | 25.2 | **20.9** | WER % | | cy | 74.2 | **60.7** | WER % | | ar | 48.4 | **35.6** | WER % | | th | 19.7 | **11.3** | CER % | | zh | 34.4 | **14.9** | CER % | | ka | 178.0 | 86.2 | WER % (unusable in both — model-class floor) | Better than stock on 10 of 11 languages (−0.5 on en). Encoder frozen during fine-tuning (the multilingual encoder is preserved); 15,000 steps, warmup+cosine, dev-checkpoint-selected; bf16. ## Limitations - Coverage = these 11 languages; fine-tuning erodes Whisper's other ~88 (unseen scripts most — measured in the accompanying study). Use stock Whisper for languages far from this set. - Georgian (ka) reported for transparency; not usable at the whisper-small tier. - Read-speech domain (CommonVoice + FLEURS-validated); not benchmarked on far-field/telephony. ## Acoustic conditions of the eval...
Source context: 6 downloads · 0 likes · Pipeline automatic-speech-recognition · Library transformers · Repo burakaydinofficial/whisper-small-cv11
