Tiny Audio Embedded

Tiny Audio A speech recognition model trained in 24 hours on a single GPU for ~$12. Built with —a minimal, hackable ASR framework. ## Quick Start ## Usage Examples ### Basic Transcription ### Batch Processing ### Word-Level Timestamps ### Streaming Inference ### Using with torch directly ### GPU Inference ### Half Precision ## Architecture Only the projector is trained (~12M params). The encoder and decoder remain frozen, leveraging their pretrained knowledge. | Component | Model | Parameters | Status | |-----------|-------|------------|--------| | Audio Encoder | GLM-ASR-Nano-2512 | ~600M | Frozen | | Projector | 2-layer MLP | ~12M | Trained | | Language Model | Qwen3-0.6B | ~600M | Frozen | ### How It Works 1. : GLM-ASR converts 16kHz audio into frame-level embeddings (768-dim) 2. : A 2-layer MLP with frame stacking bridges the audio and text embedding spaces 3. : Qwen3 generates text autoregressively, conditioned on the projected audio The projector reduces sequence length via frame stacking: ## Model Specifications | Specification | Value | |---------------|-------| | Input | Audio (16kHz mono) | | Output | Text transcription | | Max Audio Length | ~30 seconds (limited by encoder) | | Vocabulary | Qwen3 tokenizer | | Languages | English only | | Generation | Greedy decoding (num_beams=1, do_sample=False) | ## Training Details | | | |---|---| | | LoquaciousSet (25,000 hours) | | | Single NVIDIA A40 | | | ~24 hours | | | ~$12 | | | AdamW | | | 1e-4 | | | 4 | | | 50,000 | ## Limitations - : Not trained on other languages - : Expects 16kHz audio (other rates resampled automatically) - : Best for clips under 30 seconds - : May degrade on: - Heavily accented speech - Noisy or low-quality audio - Domain-specific terminology - Overlapping speakers - : Output is lowercase without punctuation by default ## Requirements Optional for streaming: ## Files | File | Description | |------|-------------| | | Model configuration | | | Projector weights (~48MB) | | | Audio preprocessing config | | | Tokenizer | | | Tokenizer config | | | Special tokens | Note: Only the projector weights are stored. The encoder (GLM-ASR) and decoder (Qwen3) are loaded from their respective HuggingFace repos. ## Citation If you use this model, please cite: ## Links - - Train your own model - - Learn ASR from scratch - - Try it in your browser ## Acknowledgments - for the audio encoder - for the language model - for training data ## License MIT