ComfyUI-WhisperXX

Features

• whisper.cpp native — DLL + ctypes binding directly to C API, no Python wrapper
• All whisper_full_params — every parameter exposed as a node widget
• Two advance toggles — show_advance_cpp (core whisper params) + show_advance_ext (UVR/alignment/diarization detail)
• 7 output sockets — text, segments_json, srt, vtt, tsv, aud, json_result (no file I/O)
• Auto-download — DLLs downloaded from GitHub Releases on first use (no manual build needed)
• GPU acceleration — Vulkan, OpenCL, CUDA, Metal (auto-detected at runtime)
• UVR vocal separation — BSRoformer.cpp DLL (GGUF model, no temp files)
• VAD — cpp-annote (community-1 segmentation model, DLL + onnxruntime)
• Pre-filter — RMS-based energy filter to prevent hallucinations on silence
• Alignment — sherpa-onnx CTC (zipformer, word-level timestamps)
• Diarization — cpp-annote DLL (no Python dependencies)
• Cross-platform — Windows/MSVC, Linux/GCC, macOS/Clang (DLL/so/dylib)

Quick Start

cd ComfyUI/custom_nodes/
git clone --recurse-submodules https://github.com/obvirm/ComfyUI-WhisperCPP
cd ComfyUI-WhisperCPP
pip install -r requirements.txt

That's it! DLLs are automatically downloaded from GitHub Releases on first use. No build step required.

Build from Source (Optional)

If you want to build the DLLs yourself (e.g., for GPU-specific optimizations):

# whisper.cpp — auto-detect GPU backends
python build_whisper_cpp.py

# BSRoformer DLL
python build_bs_roformer.py

# cpp-annote DLL
python build_cpp_annote.py

Build Options

# whisper.cpp
python build_whisper_cpp.py --gpu vulkan   # Force Vulkan
python build_whisper_cpp.py --gpu cuda     # Force CUDA
python build_whisper_cpp.py --gpu cpu      # CPU only

# BSRoformer
python build_bs_roformer.py --gpu          # Auto-detect GPU
python build_bs_roformer.py                # CPU only

# cpp-annote
python build_cpp_annote.py --cuda          # CUDA ONNX Runtime
python build_cpp_annote.py --gpu           # Auto-detect GPU
python build_cpp_annote.py                 # CPU only

Modules

All modules follow the same pattern: C API header → shared library → ctypes binding.

Pipeline

Audio Input
  │
  ├─ [UVR] BSRoformer.dll → vocal separation (optional)
  │    16kHz → Resample(44100) → DLL → mono → Resample(16000)
  │
  ├─ [Pre-filter] RMS energy → speech detection (optional)
  │    Only transcribe sections with audio energy > threshold
  │
  ├─ [VAD] cpp-annote.dll → speech segmentation (optional)
  │
  ├─ whisper.dll → transcribe (per-segment if VAD/pre-filter on)
  │
  ├─ [Alignment] sherpa-onnx CTC → word timestamps (default ON)
  │
  └─ [Diarization] cpp-annote DLL → speaker labels (optional)
       │
       └─ 7 output sockets

Auto-Download

On first use, each module automatically downloads its DLLs from GitHub Releases:

1. _find_library() → DLL not found
2. gpu_detect.detect_gpu() → NVIDIA/Vulkan/OpenCL/CPU
3. auto_download.download_module("whisper", dir, has_gpu)
   → Download whisper.dll, ggml-base.dll, ggml-cpu.dll, ggml.dll
   → (GPU) also download ggml-vulkan.dll
4. _find_library() → found!

• Version is dynamic — reads from pyproject.toml (no manual updates)
• GPU-aware — downloads Vulkan/OpenCL DLLs when GPU is detected
• Skip existing — wo