Rawnet2

Improved version available: caa-speech-detection-asvspoof2019/rawnet2-plateau — rawnet2_plateau (plateau LR scheduler) achieves 16.79% eval EER (vs 15.09% here, note: this is a harder run; plateau scheduling significantly improves dev EER to 0.0090%) and 0.5065 tandem min t-DCF.

RawNet2 — AI-Generated Speech Detector

Binary bonafide-vs-spoof classifier operating directly on raw audio waveforms, trained on ASVspoof 2019 LA.

Model Details

Local version: rawnet2_weighted

Architecture: RawNet2 (Tak et al., ICASSP 2021)

Component	Config
Sinc filterbank front-end	128 filters, kernel length 129, linear frequency scale, fixed (non-learnable)
Residual stack (stage 1)	2 × ResBlock, 128 channels
Residual stack (stage 2)	4 × ResBlock, 512 channels
GRU	hidden size 1024
Embedding dim	1024
Classifier	Linear(1024 → 2)

Each residual block uses pre-activation BN → LeakyReLU(0.3) → Conv → BN → LeakyReLU → Conv, MaxPool(3), and Filter-wise Feature Map Scaling (FMS, Jung et al. 2020).

Input: Raw audio waveform at 16 kHz, padded or truncated to 64 000 samples (4 s), shape (B, T).

Parameters: ~1.1 M
Checkpoint size: 147 MB (best.pt = 154 189 314 bytes, saved at epoch 27, early-stopped at epoch 37 with patience 10)

Loss: Cross-entropy with class weights [8.837, 1.0] (spoof : bonafide), reflecting the ~8.84:1 imbalance in ASVspoof 2019 LA training data (22 800 spoof / 2 580 bonafide).

Training Data

Dataset: ASVspoof 2019 Logical Access (LA)

Split	Utterances	Attacks
Train	~25 000	A01–A06 (known)
Dev	~25 000	A01–A06 (known)
Eval	~71 000	A07–A19 (unseen)

No data augmentation was applied in this baseline run.

Evaluation Results

Baseline to beat: EER 8.09% (LFCC + GMM).

Split	EER	tandem min t-DCF	In-the-Wild EER
Dev (baseline → improved)	0.0398% → 0.0090%	—	—
Eval	15.09%	—	—
Eval (improved: rawnet2_plateau)	16.79%	0.5065	38.57%

Note on t-DCF scale: t-DCF values use the normalized [0, 1] convention. Tandem t-DCF uses ASV scores following the official ASVspoof 2019 formula.

The low dev EER reflects overfitting to the known A01–A06 attack pool shared between train and dev. The eval EER of 15.09% on unseen attacks A07–A19 shows that this configuration does not generalise well to novel spoofing methods. For production use, consider lcnn_v7_cqt (3.26% eval EER) from the same project.

Usage

Download the checkpoint:

huggingface-cli download caa-speech-detection-asvspoof2019/rawnet2

Load into the model:

import torch
from src.models.rawnet2.model import RawNet2Model

config = {
    "sinc_filters": 128,
    "sinc_filter_length": 129,
    "sample_rate": 16000,
    "sinc_scale": "linear",
    "learnable_sinc": False,
    "first_block_channels": 128,
    "second_block_channels": 512,
    "num_first_blocks": 2,
    "num_second_blocks": 4,
    "gru_hidden": 1024,
    "embedding_dim": 1024,
    "class_weights": [8.837, 1.0],
}

model = RawNet2Model(config)
state = torch.load("best.pt", map_location="cpu")
model.load_state_dict(state)
model.eval()

# waveform: (B, T) float32 tensor at 16 kHz, T = 64000
logits = model({"waveform": waveform})["logits"]
probs  = torch.softmax(logits, dim=-1)  # [:, 0] = spoof, [:, 1] = bonafide

The model returns {"logits": Tensor[B, 2]}. Class 0 = spoof, class 1 = bonafide.

Intended Use & Limitations

• Trained and validated on the Logical Access (LA) scenario of ASVspoof 2019. Not evaluated on Physical Access or other corpora.
• The sinc filterbank is fixed (non-learnable), matching the Tak et al. 2021 anti-spoofing recipe.
• Generalization to unseen attacks is limited: eval EER is 15.09% on A07–A19, substantially above the LFCC+GMM baseline. Features learned from A01–A06 do not transfer well to the neural-vocoder attacks in the eval set.
• Not intended for deployment without further validation on domain-matched data.

Citation

ASVspoof 2019 dataset:

@inproceedings{wang2020asvspoof,
  title     = {{ASVspoof} 2019: A large-scale public database of synthesized, converted and replayed speech},
  author    = {Wang, Xin and others},
  booktitle = {Computer Speech \& Language},
  volume    = {64},
  year      = {2020}
}

RawNet2:

@inproceedings{tak2021rawnet2,
  title     = {End-to-End Anti-Spoofing with {RawNet2}},
  author    = {Tak, Hemlata and Patino, Jose and Todisco, Massimiliano and Nautsch, Andreas and Evans, Nicholas and Larcher, Anthony},
  booktitle = {ICASSP},
  year      = {2021}
}

Feature Map Scaling (FMS):

@inproceedings{jung2020rawnet,
  title     = {Improved {RawNet} with Feature Map Scaling for Text-Independent Speaker Verification using Raw Waveforms},
  author    = {Jung, Jee-weon and Kim, Seung-bin and Shim, Hye-jin and Kim, Ju-ho and Yu, Ha-eun and Chung, Joon Son},
  booktitle = {Interspeech},
  year      = {2020}
}