ComfyUI-StableAudioX

ComfyUI-AudioX

A powerful audio generation extension for ComfyUI that integrates AudioX models a finetuned version of stable audio tools for high-quality audio synthesis from text and video inputs. Currently only working on min. 16gb Vram - tested on a single 4090.

🎵 Features

• Text to Audio: Generate high-quality audio from text descriptions with enhanced conditioning
• Text to Music: Create musical compositions with style, tempo, and mood controls
• Video to Audio: Extract and generate audio from video content with advanced conditioning
• Enhanced Conditioning: Separate CFG scales, conditioning weights, negative prompting, and prompt enhancement
• Professional Audio Processing: Volume control with LUFS normalization, limiting, and precise gain staging
• Video Processing: Mute videos and combine with generated audio

🚀 Installation

1. System Dependencies (Required)

Install these system dependencies first:

Windows:

# Install ffmpeg (required for video processing)
# Download from: https://ffmpeg.org/download.html
# Or use chocolatey: choco install ffmpeg

# Install Microsoft Visual C++ Build Tools (if not already installed)
# Download from: https://visualstudio.microsoft.com/visual-cpp-build-tools/

Linux/Ubuntu:

sudo apt update
sudo apt install ffmpeg libsndfile1-dev build-essential

macOS:

brew install ffmpeg libsndfile

2. Clone Repository and Install Python Dependencies

cd ComfyUI/custom_nodes
git clone https://github.com/lum3on/ComfyUI-StableAudioX.git
cd ComfyUI-StableAudioX

# Install Python dependencies
pip install -r requirements.txt

# Optional: Run dependency checker to verify installation
python install_dependencies.py

Model Setup – AudioX

1. Model File: Download from Hugging Face - model.ckpt
2. Config File: Download from Hugging Face - config.json
3. Place both files in: ComfyUI/models/diffusion_models/

rename model.ckpt file to AudioX.ckpt

Alternative Download via Hugging Face CLI

# Install huggingface-hub if not already installed
pip install huggingface-hub

# Download AudioX model files
huggingface-cli download HKUSTAudio/AudioX model.ckpt --local-dir ComfyUI/models/diffusion_models/
huggingface-cli download HKUSTAudio/AudioX config.json --local-dir ComfyUI/models/diffusion_models/

Model Directory Structure:

ComfyUI/models/diffusion_models/
├── model.safetensors         # AudioX model
└── model_config.json         # Model configuration file

System Requirements

• VRAM: 6GB+ recommended for optimal performance
• RAM: 16GB+ recommended
• Storage: ~5GB for model files
• GPU: CUDA-compatible GPU recommended (CPU supported but slower)

📋 Available Nodes

Core Generation Nodes

• AudioX Model Loader: Load AudioX models with device configuration and auto-detect config files
• AudioX Text to Audio: Basic text-to-audio generation with automatic prompt enhancement
• AudioX Text to Music: Basic text-to-music generation with automatic prompt enhancement
• AudioX Video to Audio: Basic video-to-audio generation with automatic prompt enhancement
• AudioX Video to Music: Generate musical soundtracks for videos

Enhanced Generation Nodes ⭐

• AudioX Enhanced Text to Audio: Advanced text-to-audio with negative prompting, templates, style modifiers, and conditioning modes
• AudioX Enhanced Text to Music: Advanced music generation with style, tempo, mood controls, and musical enhancement
• AudioX Enhanced Video to Audio: Advanced video-to-audio with separate CFG scales, conditioning weights, and enhanced prompting

Processing & Utility Nodes

• AudioX Audio Processor: Pr