Cohere Transcribe 03 2026 ONNX

--- license: apache-2.0 language: - ar - de - el - en - es - fr - it - ja - ko - nl - pl - pt - vi - zh pipeline_tag: automatic-speech-recognition tags: - audio - speech-recognition - transcription - onnx - directml base_model: - CohereLabs/cohere-transcribe-03-2026 --- # Cohere Transcribe 03-2026 — DirectML-safe, fully-static ONNX ONNX export of [CohereLabs/cohere-transcribe-03-2026](https://huggingface.co/CohereLabs/cohere-transcribe-03-2026) restructured to run **correctly and fast on the ONNX Runtime DirectML EP** (and every other EP). Weight bytes are identical to [onnx-community/cohere-transcribe-03-2026-ONNX](https://huggingface.co/onnx-community/cohere-transcribe-03-2026-ONNX); the encoder proto additionally embeds the cross-attention projection weights hoisted out of the decoder (see below). ## Why the stock export can't run well on DirectML - The fused decoder attention (`com.microsoft.MultiHeadAttention` cross-attn + `GroupQueryAttention` self-attn) **crashes / mis-computes** on the DML EP (ORT ≤ 1.24, unfixed). - Even after decomposing to plain ops, the DML EP **re-fuses its graph on every autoregressive shape change** (~34 ms/token fixed), so naïve GPU decode loses to CPU. ## What changed (three passes, `tools/onnx/cohere_decompose_attention.py`) 1. **Attention decomposed** to Reshape/Transpose/MatMul/Softmax/… with an explicit causal mask; the two cross-attn `If` nodes flattened away. 2. **Loop-invariant cross-KV hoisted into the encoder** — the cross-attention K/V (which depend only on the audio) are computed once per utterance in `encoder_model*.onnx` (extra `cross_attn.*` outputs) instead of on every decoded token. 3. **Fully static decode** — the self-KV cache is a fixed-length buffer updated by a masked write, the cross-KV are padded to a fixed length with a `cross_bias` mask, and every per-step shape is constant. The DML EP now compiles the decoder **once** and reuses it, so per-token cost is **constant ~4 ms regardless of audio length**, entirely on the GPU. Two decoders are shipped per precision: - `decoder_model_merged*.onnx` — **fully static** (fixed self- + cross-KV). Fastest on DirectML (metadata `winstt_static_kv`). - `decoder_model_merged*_dyn.onnx` — **growing self-KV** (cross still fixed). Faster on the CPU EP, which doesn't benefit from static shapes. WinSTT loads this one when the decoder is CPU-bound. Bo...