Moss Transcribe Diarize Zhtw GGUF
Same architecture, very different deployment envelope. This project fine-tunes (not prunes) OpenMOSS-Team/MOSS-Transcribe-Diarize — Whisper-medium encoder + Qwen3-0.6B decoder, 0.9B params, single-pass...
Model source
- Kind
- Other
- File size
- 13.6 MB–3.39 GB
- Base model
- OpenMOSS-Team/MOSS-Transcribe-Diarize
- Version
- vCampplus
- Updated
- Sep 14, 2026
- Source
- Hugging Face
Source description
What's different vs. the original MOSS-Transcribe-Diarize?
Same architecture, very different deployment envelope. This project fine-tunes
(not prunes) OpenMOSS-Team/MOSS-Transcribe-Diarize
— Whisper-medium encoder + Qwen3-0.6B decoder, 0.9B params, single-pass
[start][Sxx]text[end] output, Apache-2.0 — and re-engineers everything around it:
| Original MOSS-TD 0.9B | This project (v7 lineage) | |
|---|---|---|
| Architecture / license | Whisper-medium enc + Qwen3-0.6B dec · Apache-2.0 | identical (fine-tuned, nothing pruned) |
| Language & script | general Mandarin, Simplified-leaning output | Traditional Chinese (Taiwan) enforced end-to-end + zh-TW/EN code-switch; conservative ITN |
| Domain | general speech | real meetings: ~3 000 synthetic zh-TW meetings (TTS) + 55.8 h 立法院 IVOD with fused whisperX×pyannote labels (speaker-purity filtered) |
| Held-out zh-TW meeting MER | 0.395* | ~0.18 (*script-normalized; part of the base gap is Simplified↔Traditional) |
| Long-meeting diarization (123 min) | DER 0.74 | DER 0.195 · consistency 0.905 (cross-window CAM++ linking v2: tag-pooled units + cannot-link clustering — 2 h meeting 24→10 speakers) |
| Diarization under fine-tuning | n/a (base) | defended: 8× speaker-tag CE + KL-anchor to base at [Sxx] positions — FT rounds no longer collapse speakers |
| Code-switch regression (ASCEND) | reference | v5 0.417 → v6 0.285 → v6.1 0.267 (fp16); v7 q4-GGUF 0.317 vs v6.1-q4 0.352 |
| Long-audio decoding | full attention, KV grows O(audio) | streaming fine-tune: bounded 45 s audio-KV window (monotonic eviction) — flat memory, ~20 % faster decode, DER parity via linking |
| Decode robustness | — | engine guards: tick-stall loop breaker, advancing-clock cycle detector, slow-cycle collapse, speech-aware premature-EOS suppression, eviction-sized KV + capacity guard, per-window watchdog + recovery |
| Quantization | bf16 release | diarization-defended q4 QAT → q4_K_M GGUF (707 MB) · int8 ONNX (ternary/q3 and encoder-ternary measured & rejected) |
| Runtimes | HF transformers (GPU) | + in-browser WASM (multithread/iOS/WebGPU), native CPU C++ (RapidSpeech.cpp), Jetson Nano (CUDA 10.2/sm_53), sherpa-onnx, ONNX web |
| Live demos | — | WASM (fully local) · native C++ Space |
MOSS-Transcribe-Diarize zh-TW — GGUF for RapidSpeech.cpp
GGUF conversions of Luigi/moss-transcribe-diarize-zhtw
for the RapidSpeech.cpp
ggml runtime — pure C++ / WebAssembly, no Python, on-device.
Current release: moss-td-zhtw-v7-q4_k_m.gguf (deployed on both live demos).
Single-pass Traditional-Chinese / English meeting transcription with speaker
diarization + timestamps, output as [start][Sxx]text[end] segments.
Files
| File | Quant | Size | Notes |
|---|---|---|---|
moss-td-zhtw-v7-q4_k_m.gguf | Q4_K_M | ~707 MB | Current (deployed). CE-only int4 QAT of v6.1 — q4-robust weights that fix a dense-speech repetition loop occurring only at q4 (f16 was clean). q4-GGUF ASCEND all 0.317 (beats v6.1-q4's 0.352); keeps v6.1's sentence-cadence markers + diarization. Engine-default 45 s audio-KV eviction. |
moss-td-zhtw-v61-q4_k_m.gguf | Q4_K_M | ~707 MB | v6.1 marker-density FT (sentence-cadence markers). Superseded by v7 (q4-fragile: can repetition-loop on long dense windows). |
moss-td-zhtw-v6stream-q4_k_m.gguf | Q4_K_M | ~707 MB | v6-stream: bounded 45 s audio-KV window FT; ASCEND all 0.285. |
moss-td-zhtw-v5kl-qat-q4_k_m.gguf | Q4_K_M | ~707 MB | v5-kl diarization-defended QAT (use with RS_AUDIO_KV_WINDOW=0 — v5-era models drift under eviction). |
moss-td-zhtw-v5kl-q4_k_m.gguf | Q4_K_M | ~668 MB | v5-kl plain conversion. Preserves the v5 diarization fix but knife-edge on long dense windows — prefer the QAT build. |
moss-td-zhtw-v5kl-q4_k_m_embq8.gguf | Q4_K_M + q8_0 embed | ~705 MB | For all-GPU decode on Jetson Nano gen1 (old-ggml CUDA has no q6_K get_rows kernel). |
Architecture in the GGUF: MossTD = Whisper-medium audio encoder (80-mel conv1d
stem, 24 layers) + a 4-frame time-merge VQ adaptor + Qwen3-0.6B decoder, with
audio embeddings spliced into the decoder token stream.
Usage (RapidSpeech.cpp)
Requires a RapidSpeech.cpp build with the MossTD arch. The model consumes an
HF-exact Whisper log-mel; the C++ WhisperMelExtractor (direct-DFT, n_fft=400)
matches it, or the browser/WASM path can compute the mel in JS and feed it via
rs_wasm_moss_transcribe_mel.
# native CPU offline test harness
moss-td-test moss-td-zhtw-v7-q4_k_m.gguf meeting.wav
[3.60][S01]好,請主任秘書報告出席人數。[7.80][8.59][S02]報告委員會出席委員11人,已足法定人數。[11.00]
The model was trained/served with the instruction prompt:
请将音频转写为文本,每一段需以起始时间戳和说话人编号([S01]、[S02]、[S03]...)开头,正文为对应的语音内容,并在段末标注结束时间戳,以清晰标明该段语音范围。
Provenance
Converted with scripts/convert_moss_td_to_gguf.py (f16 GGUF) then quantized to
Q4_K_M with rs-quantize. Verified bit-exact against the HuggingFace model's
audio embeddings and matching diarized transcription on held-out Taiwan
Legislative Yuan (IVOD) meeting audio.
Acknowledgements
- Fine-tuned model: Luigi/moss-transcribe-diarize-zhtw
- Base model: OpenMOSS-Team/MOSS-Transcribe-Diarize (Apache-2.0)
- Runtime: RapidSpeech.cpp (ggml)
Sources
1 source
Sources
1 sourceVerified Aug 7
Model artifacts
4 artifacts
Model artifacts
4 artifactsSource excerpts
2 excerpts
Source excerpts
2 excerpts--- license: apache-2.0 base_model: Luigi/moss-transcribe-diarize-zhtw language: - zh - en tags: - automatic-speech-recognition - speaker-diarization - timestamps - zh-tw - traditional-chinese - ggml - gguf - rapidspeech - wasm - on-device pipeline_tag: automatic-speech-recognition --- ## What's different vs. the original MOSS-Transcribe-Diarize? Same architecture, very different deployment envelope. This project **fine-tunes** (not prunes) [OpenMOSS-Team/MOSS-Transcribe-Diarize](https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize) — Whisper-medium encoder + Qwen3-0.6B decoder, 0.9B params, single-pass `[start][Sxx]text[end]` output, Apache-2.0 — and re-engineers everything around it: | | Original MOSS-TD 0.9B | This project (v7 lineage) | |---|---|---| | **Architecture / license** | Whisper-medium enc + Qwen3-0.6B dec · Apache-2.0 | identical (fine-tuned, nothing pruned) | | **Language & script** | general Mandarin, Simplified-leaning output | **Traditional Chinese (Taiwan)** enforced end-to-end + zh-TW/EN code-switch; conservative ITN | | **Domain** | general speech | real meetings: ~3 000 synthetic zh-TW meetings (TTS) + 55.8 h 立法院 IVOD with fused whisperX×pyannote labels (speaker-purity filtered) | | **Held-out zh-TW meeting MER** | 0.395* | **~0.18** (*script-normalized; part of the base gap is Simplified↔Traditional) | | **Long-meeting diarization (123 min)** | DER 0.74 | **DER 0.195 · consistency 0.905** (cross-window CAM++ linking v2: tag-pooled units + cannot-link clustering — 2 h meeting 24→10 speakers) | | **Diarization under fine-tuning** | n/a (base) | defended: 8× speaker-tag CE + KL-anchor to base at `[Sxx]` positions — FT rounds no longer collapse speakers | | **Code-switch regression (ASCEND)** | reference | v5 0.417 → v6 0.285 → v6.1 0.267 (fp16); **v7 q4-GGUF 0.317 vs v6.1-q4 0.352** | | **Long-audio decoding** | full attention, KV grows O(audio) | **streaming fine-tune: bounded 45 s audio-KV window** (monotonic eviction) — flat memory, ~20 % faster decode, DER parity via linking | | **Decode robustness** | — | engine guards: tick-stall loop breaker, advancing-clock cycle detector, slow-cycle collapse, speech-aware premature-EOS suppression, eviction-sized KV + capacity guard, per-window watchdog + recovery | | **Quantization** | bf16 release | diarization-defended **q4 QAT** → q4_K_M GGUF (707 MB) · int8 ONNX (ternary/q3 and...
Source context: 2212 downloads · 1 likes · Pipeline automatic-speech-recognition · Repo Luigi/moss-transcribe-diarize-zhtw-gguf
