East Africa Omniasr 300m

East Africa OmniASR 300M

This model grew out of my entry in the AfriVoices East Africa ASR Hackathon. As a Kalenjin developer who had previously worked on translation and TTS, I used the challenge as the push to explore speech recognition for languages I care about. I also wanted to try a strong CTC system instead of following the usual Whisper route, so I built this project around Meta's official omniASR_CTC_300M model from the Omnilingual ASR project.

The result is a 325M-parameter CTC model for Swahili, Kikuyu, Dholuo, Somali, Kalenjin, and Maasai. I continued improving it after the hackathon, with the goal of turning the competition checkpoint into a model that can be tested and used outside Kaggle.

Training data

The supervised mixture combines AfriVoices Kenya data for Kikuyu, Dholuo, Somali, Kalenjin, and Maasai with AfriVoices Swahili. It contains both scripted and natural unscripted speech, including regional accents, long-form speech, code-switching, and varied recording conditions.

The training sampler balanced the six languages at roughly 477–500 weighted hours each, approximately 2,900 sampler-weighted hours in total. Audio was standardized to mono 16 kHz and transcripts were normalized consistently. Known text-encoding inconsistencies were repaired before the final training run. The published checkpoint is from step 10,000.

The repository contains the step-10,000 fairseq2 checkpoint, FP32 and dynamic INT8 ONNX exports, the v1 tokenizer, and language-specific KenLM binaries.

Results

Development benchmark: 2,354 held-out AfriVoices clips across all six languages. KenLM uses beam 64 and the checked-in decoder_config.json parameters.

Evaluation	Greedy WER	Greedy CER	KenLM WER	KenLM CER
Swahili	16.00%	3.96%	9.23%	2.90%
Kikuyu	42.66%	8.78%	34.52%	8.90%
Dholuo	47.26%	9.92%	37.28%	8.98%
Somali	55.03%	21.11%	39.97%	20.40%
Kalenjin	70.81%	17.28%	61.43%	19.68%
Maasai	65.89%	19.01%	56.11%	18.30%
Corpus/micro	53.06%	14.91%	42.60%	14.87%
Language macro	49.61%	13.34%	39.76%	13.19%

The final decoder also obtained 38.956% WER on the hackathon's hidden late-submission leaderboard. That number is reported separately because the hidden set has no public per-language or CER breakdown.

WER and CER are error rates: lower is better. Decoder parameters were selected on development data, so the development KenLM scores are validation results, not an untouched test estimate.

CPU usage (greedy CTC)

from pathlib import Path

import numpy as np
import sherpa_onnx
import soundfile as sf
from huggingface_hub import snapshot_download

root = Path(snapshot_download(
    "mutaician/east-africa-omniasr-300m",
    allow_patterns=["onnx/model.int8.onnx", "onnx/tokens.txt"],
))
recognizer = sherpa_onnx.OfflineRecognizer.from_omnilingual_asr_ctc(
    model=str(root / "onnx/model.int8.onnx"),
    tokens=str(root / "onnx/tokens.txt"),
    num_threads=2,
    provider="cpu",
)
audio, sample_rate = sf.read("audio.wav", dtype="float32")
if audio.ndim == 2:
    audio = audio.mean(axis=1)
stream = recognizer.create_stream()
stream.accept_waveform(sample_rate, np.asarray(audio, dtype=np.float32))
recognizer.decode_stream(stream)
print(stream.result.text)

Use mono 16 kHz audio for reproducible behavior. The KenLM binaries and exact alpha/beta values are under kenlm/ and decoder_config.json; the companion Hugging Face Space will expose greedy, balanced, and full beam-64 modes.

Repository layout

• onnx/model.int8.onnx: recommended free-CPU deployment model.
• onnx/model.onnx: FP32 reference export.
• onnx/tokens.txt: complete CTC vocabulary.
• fairseq2/checkpoint/model/: original step-10,000 checkpoint shards.
• fairseq2/config.yaml: training configuration.
• tokenizer/omniASR_tokenizer.model: OmniASR v1 tokenizer.
• kenlm/: six binary 5-gram language models and vocabularies.
• normalization.json: audio, chunking, and text normalization contract.
• decoder_config.json: language routing and final KenLM parameters.
• evaluation.json: aggregate and per-language WER/CER metrics.

Limitations

Kalenjin and Maasai remain the weakest languages in the development benchmark. KenLM improves WER substantially but can slightly worsen CER for some languages. The model does not perform automatic language identification; pass the language explicitly when using language-model decoding. Accuracy on new recording conditions can differ from these development and leaderboard scores.

Attribution

Fine-tuned from Meta's official omniASR_CTC_300M model and v1 tokenizer from Omnilingual ASR, released under Apache 2.0. Training speech comes from the AfriVoices East Africa hackathon data; see the AfriVoices-KE paper for more information on the Kenyan-language collection.