Phonoq 2.0 Multilingual
Framewise phonological feature recognition for multilingual speech.
Modellquelle
- Art
- Other
- Dateigröße
- 1,22–1,41 GB
- Version
- vRevision 2141fbb
- Aktualisiert
- 25. Okt. 2026
- Downloads
- 50
- Quelle
- Hugging Face
Quellenbeschreibung
PhonoQ 2.0 Multilingual
Framewise phonological feature recognition for multilingual speech.
This model returns phonological probabilities for manner, vowel height, vowel backness, place, and voicing, plus a hard conditional 22-feature representation per frame. It is a modernized successor to the original PhonoQ system: https://github.com/TAriasVergara/PhonoQ
Usage
pip install torch "transformers>=4.35,<6" soundfile safetensors
import soundfile as sf
import torch
from transformers import AutoFeatureExtractor, AutoModel
model_id = "abnerh/phonoq-2.0-multilingual"
audio, sr = sf.read("example.wav")
if audio.ndim > 1:
audio = audio.mean(axis=1)
processor = AutoFeatureExtractor.from_pretrained(model_id)
inputs = processor(audio, sampling_rate=sr, return_tensors="pt", padding=True)
model = AutoModel.from_pretrained(model_id, trust_remote_code=True)
model.eval()
with torch.no_grad():
out = model(**inputs)
print(out.features.shape) # [1, T, 22]
print(out.manner_probabilities.shape) # [1, T, 9]
print(out.vowel_height.shape) # [1, T, 3]
print(out.vowel_backness.shape) # [1, T, 3]
print(out.place_probabilities.shape) # [1, T, 5]
print(out.voice_probabilities.shape) # [1, T, 2]
Outputs
features: hard conditional 22-dimensional features,[B, T, 22]manner_probabilities:[B, T, 9]vowel_height:[B, T, 3]vowel_backness:[B, T, 3]place_probabilities:[B, T, 5]voice_probabilities:[B, T, 2]attention_mask: valid encoder frames,[B, T]feature_names: names for the 22 feature dimensions
Feature order:
silence, stop, nasal, rhotic, fricative, affricate, approximant, lateral, vowel,
high, mid, low, front, central, back,
labial, alveolar, velar, palatal, postalveolar,
voiceless, voiced
Viewing Probabilities
The following snippet prints only the non-silence region.
manner_labels = [
"silence", "stop", "nasal", "rhotic", "fricative", "affricate",
"approximant", "lateral", "vowel",
]
manner = out.manner_probabilities[0]
mask = out.attention_mask[0].bool()
manner = manner[mask]
best_manner = manner.argmax(dim=-1)
non_silence = (best_manner != 0).nonzero(as_tuple=True)[0]
if len(non_silence) == 0:
print("No non-silence frames found.")
else:
start = int(non_silence[0])
end = int(non_silence[-1]) + 1
print(f"Non-silence frame range: {start}-{end - 1}")
print()
for frame_idx in range(start, end):
probs = manner[frame_idx]
best = int(probs.argmax())
print(f"{frame_idx:03d} {manner_labels[best]:10s} {float(probs[best]):.3f}")
CLI
This repository includes best.ckpt for PhonoQ CLI compatibility:
pip install git+https://github.com/abnerLing/PhonoQ-2.0.git
phonoq predict example.wav \
--model abnerh/phonoq-2.0-multilingual \
--outdir outputs \
--pretty
Notes
This model uses custom Transformers code and must be loaded with
trust_remote_code=True.
The multilingual checkpoint is intended for use across English, Spanish, German, and Czech speech.
Quellen
1 Quelle
Quellen
1 QuelleVerifiziert 25. Okt.
Download files
Modellartefakte
2 Artefakte
Modellartefakte
2 ArtefakteQuellenauszüge
3 Auszüge
Quellenauszüge
3 Auszüge--- license: mit library_name: transformers tags: - audio - speech - phonology - phonological-features - wav2vec2 - multilingual - phonoq language: - en - es - de - cs pipeline_tag: audio-classification --- # PhonoQ 2.0 Multilingual Framewise phonological feature recognition for multilingual speech. This model returns phonological probabilities for manner, vowel height, vowel backness, place, and voicing, plus a hard conditional 22-feature representation per frame. It is a modernized successor to the original PhonoQ system: https://github.com/TAriasVergara/PhonoQ ## Usage ```bash pip install torch transformers soundfile safetensors ``` ```python import soundfile as sf import torch from transformers import AutoFeatureExtractor, AutoModel model_id = "abnerh/phonoq-2.0-multilingual" audio, sr = sf.read("example.wav") if audio.ndim > 1: audio = audio.mean(axis=1) processor = AutoFeatureExtractor.from_pretrained(model_id) inputs = processor(audio, sampling_rate=sr, return_tensors="pt", padding=True) model = AutoModel.from_pretrained(model_id, trust_remote_code=True) model.eval() with torch.no_grad(): out = model(**inputs) print(out.features.shape) # [1, T, 22] print(out.manner_probabilities.shape) # [1, T, 9] print(out.vowel_height.shape) # [1, T, 3] print(out.vowel_backness.shape) # [1, T, 3] print(out.place_probabilities.shape) # [1, T, 5] print(out.voice_probabilities.shape) # [1, T, 2] ``` ## Outputs - `features`: hard conditional 22-dimensional features, `[B, T, 22]` - `manner_probabilities`: `[B, T, 9]` - `vowel_height`: `[B, T, 3]` - `vowel_backness`: `[B, T, 3]` - `place_probabilities`: `[B, T, 5]` - `voice_probabilities`: `[B, T, 2]` - `attention_mask`: valid encoder frames, `[B, T]` - `feature_names`: names for the 22 feature dimensions Feature order: ```text silence, stop, nasal, rhotic, fricative, affricate, approximant, lateral, vowel, high, mid, low, front, central, back, labial, alveolar, velar, palatal, postalveolar, voiceless, voiced ``` ## Viewing Probabilities The following snippet prints only the non-silence region. ```python manner_labels = [ "silence", "stop", "nasal", "rhotic", "fricative", "affricate", "approximant", "lateral", "vowel", ] manner = out.manner_probabilities[0] mask = out.attention_mask[0].bool() manner = manner[mask] best_manner = manner.argmax(dim=-1) non_silence = (best_manner != 0).nonzero(as_tu...
--- license: mit library_name: transformers tags: - audio - speech - phonology - phonological-features - wav2vec2 - multilingual - phonoq language: - en - es - de - cs pipeline_tag: audio-classification --- # PhonoQ 2.0 Multilingual Framewise phonological feature recognition for multilingual speech. This model returns phonological probabilities for manner, vowel height, vowel backness, place, and voicing, plus a hard conditional 22-feature representation per frame. It is a modernized successor to the original PhonoQ system: https://github.com/TAriasVergara/PhonoQ ## Usage ```bash pip install torch "transformers>=4.35,<6" soundfile safetensors ``` ```python import soundfile as sf import torch from transformers import AutoFeatureExtractor, AutoModel model_id = "abnerh/phonoq-2.0-multilingual" audio, sr = sf.read("example.wav") if audio.ndim > 1: audio = audio.mean(axis=1) processor = AutoFeatureExtractor.from_pretrained(model_id) inputs = processor(audio, sampling_rate=sr, return_tensors="pt", padding=True) model = AutoModel.from_pretrained(model_id, trust_remote_code=True) model.eval() with torch.no_grad(): out = model(**inputs) print(out.features.shape) # [1, T, 22] print(out.manner_probabilities.shape) # [1, T, 9] print(out.vowel_height.shape) # [1, T, 3] print(out.vowel_backness.shape) # [1, T, 3] print(out.place_probabilities.shape) # [1, T, 5] print(out.voice_probabilities.shape) # [1, T, 2] ``` ## Outputs - `features`: hard conditional 22-dimensional features, `[B, T, 22]` - `manner_probabilities`: `[B, T, 9]` - `vowel_height`: `[B, T, 3]` - `vowel_backness`: `[B, T, 3]` - `place_probabilities`: `[B, T, 5]` - `voice_probabilities`: `[B, T, 2]` - `attention_mask`: valid encoder frames, `[B, T]` - `feature_names`: names for the 22 feature dimensions Feature order: ```text silence, stop, nasal, rhotic, fricative, affricate, approximant, lateral, vowel, high, mid, low, front, central, back, labial, alveolar, velar, palatal, postalveolar, voiceless, voiced ``` ## Viewing Probabilities The following snippet prints only the non-silence region. ```python manner_labels = [ "silence", "stop", "nasal", "rhotic", "fricative", "affricate", "approximant", "lateral", "vowel", ] manner = out.manner_probabilities[0] mask = out.attention_mask[0].bool() manner = manner[mask] best_manner = manner.argmax(dim=-1) non_silence = (best_manner != 0).no...
Source context: 50 downloads · 2 likes · Pipeline audio-classification · Library transformers · Repo abnerh/phonoq-2.0-multilingual
