Voxcpm2 Finetuned 100hr
This is a fine-tuned version of the VoxCPM2 model, optimized for Bengali text-to-speech generation. It was trained on a 100-hour Bengali speech dataset.
Modellquelle
- Art
- Text model
- Dateigröße
- 359 MB–4,27 GB
- Version
- vRevision 293fd76
- Aktualisiert
- 31. Okt. 2026
- Downloads
- 11
- Quelle
- Hugging Face
Quellenbeschreibung
VoxCPM2 Fine-Tuned (Bengali - 100 Hours)
This is a fine-tuned version of the VoxCPM2 model, optimized for Bengali text-to-speech generation. It was trained on a 100-hour Bengali speech dataset.
Fine-tuning Details & Optimizations
Training a massive 3-billion parameter multimodal architecture normally requires multiple A100s. To squeeze the entire fine-tuning pipeline onto a single 48GB GPU (NVIDIA RTX A6000), the following advanced optimizations were utilized:
- bfloat16 Native Execution: Model weights and gradients were cast natively to
bfloat16, bypassing standard PyTorch 32-bit overhead and cutting memory usage in half. - 8-bit AdamW (BitsAndBytes): The optimizer states were heavily quantized, reducing the optimizer VRAM footprint from ~24 GB down to ~6 GB without sacrificing convergence stability.
- Expandable Segments: PyTorch's
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:Truewas enabled, which prevented debilitating VRAM fragmentation crashes during dynamic context length generation and gradient accumulation. - Memory-Optimized DataLoader: Arrow-format datasets were pre-cached, bypassing redundant mapping/casting during initialization and radically improving dataset streaming speed.
- Strict Memory Clamping:
batch_size: 1paired withgrad_accum_steps: 32(effective batch size 32) and cappedmax_batch_tokens: 3000to prevent unpredictable sequence lengths from OOMing the 48GB limit during backward passes.
Final Training Metrics:
- Final Training Diff Loss: 0.9512
- Final Validation Diff Loss: 0.9539
(Note: The checkpoint with the lowest validation loss was actually step 500 (val loss: 0.8210), but this repository contains the final converged weights at step 1122).
Inference Ready Code
You can use this model instantly by loading it with the official voxcpm repository:
import sys
import torch
import soundfile as sf
import os
# Ensure you have cloned the VoxCPM repo and added it to your python path
sys.path.insert(0, "/path/to/VoxCPM/src")
import voxcpm
# 1. Provide the path to these downloaded HF weights
model_path = "./voxcpm2-finetuned-100hr"
# 2. Initialize the model
model = voxcpm.VoxCPM(voxcpm_model_path=model_path)
# 3. Enter your target text
text = "ধান ফুরাল, পান ফুরাল, খাজনার উপায় কী? আর কটা দিন সবুর করো, রসুন বুনেছি।"
print(f"Generating audio for text: '{text}'")
# 4. Generate
wav = model.generate(
text=text,
cfg_value=2.0,
inference_timesteps=10,
normalize=True,
denoise=False
)
# 5. Save the output
output_path = "output.wav"
sample_rate = model.tts_model.sample_rate
sf.write(output_path, wav, sample_rate)
print(f"✅ Success! Audio saved to {output_path}")
Quellen
1 Quelle
Quellen
1 QuelleVerifiziert 31. Okt.
Download files
Repository files
File inventory only; completeness is unverified.
Modellartefakte
2 Artefakte
Modellartefakte
2 Artefaktemodel.safetensors
safetensors · BF16 (unverified) · 4,27 GB · SHA-256 111d682f33df…9550 · Hugging Face
Download fileQuellenauszüge
3 Auszüge
Quellenauszüge
3 Auszüge--- language: - bn tags: - text-to-speech - TTS - voxcpm - bengali - bfloat16 pipeline_tag: text-to-speech --- # VoxCPM2 Fine-Tuned (Bengali - 100 Hours) This is a fine-tuned version of the [VoxCPM2](https://huggingface.co/openbmb/VoxCPM2) model, optimized for Bengali text-to-speech generation. It was trained on a 100-hour Bengali speech dataset. ## Fine-tuning Details & Optimizations Training a massive 3-billion parameter multimodal architecture normally requires multiple A100s. To squeeze the entire fine-tuning pipeline onto a single 48GB GPU (NVIDIA RTX A6000), the following advanced optimizations were utilized: * **bfloat16 Native Execution:** Model weights and gradients were cast natively to `bfloat16`, bypassing standard PyTorch 32-bit overhead and cutting memory usage in half. * **8-bit AdamW (BitsAndBytes):** The optimizer states were heavily quantized, reducing the optimizer VRAM footprint from ~24 GB down to ~6 GB without sacrificing convergence stability. * **Expandable Segments:** PyTorch's `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True` was enabled, which prevented debilitating VRAM fragmentation crashes during dynamic context length generation and gradient accumulation. * **Memory-Optimized DataLoader:** Arrow-format datasets were pre-cached, bypassing redundant mapping/casting during initialization and radically improving dataset streaming speed. * **Strict Memory Clamping:** `batch_size: 1` paired with `grad_accum_steps: 32` (effective batch size 32) and capped `max_batch_tokens: 3000` to prevent unpredictable sequence lengths from OOMing the 48GB limit during backward passes. ### Final Training Metrics: * **Final Training Diff Loss:** 0.9512 * **Final Validation Diff Loss:** 0.9539 *(Note: The checkpoint with the lowest validation loss was actually step 500 (`val loss: 0.8210`), but this repository contains the final converged weights at step 1122).* ## Inference Ready Code You can use this model instantly by loading it with the official `voxcpm` repository: ```python import sys import torch import soundfile as sf import os # Ensure you have cloned the VoxCPM repo and added it to your python path sys.path.insert(0, "/path/to/VoxCPM/src") import voxcpm # 1. Provide the path to these downloaded HF weights model_path = "./voxcpm2-finetuned-100hr" # 2. Initialize the model model = voxcpm.VoxCPM(voxcpm_model_path=model_path) #...
Source context: 11 downloads · 0 likes · Pipeline text-to-speech · Library voxcpm · Repo actorcritic/voxcpm2-finetuned-100hr
Source context: 16 downloads · 0 likes · Pipeline text-to-speech · Library voxcpm · Repo actorcritic/voxcpm2-finetuned-100hr
