Phonoq 2.0 Multilingual

--- license: mit library_name: transformers tags: - audio - speech - phonology - phonological-features - wav2vec2 - multilingual - phonoq language: - en - es - de - cs pipeline_tag: audio-classification --- # PhonoQ 2.0 Multilingual Framewise phonological feature recognition for multilingual speech. This model returns phonological probabilities for manner, vowel height, vowel backness, place, and voicing, plus a hard conditional 22-feature representation per frame. It is a modernized successor to the original PhonoQ system: https://github.com/TAriasVergara/PhonoQ ## Usage ```bash pip install torch transformers soundfile safetensors ``` ```python import soundfile as sf import torch from transformers import AutoFeatureExtractor, AutoModel model_id = "abnerh/phonoq-2.0-multilingual" audio, sr = sf.read("example.wav") if audio.ndim > 1: audio = audio.mean(axis=1) processor = AutoFeatureExtractor.from_pretrained(model_id) inputs = processor(audio, sampling_rate=sr, return_tensors="pt", padding=True) model = AutoModel.from_pretrained(model_id, trust_remote_code=True) model.eval() with torch.no_grad(): out = model(**inputs) print(out.features.shape) # [1, T, 22] print(out.manner_probabilities.shape) # [1, T, 9] print(out.vowel_height.shape) # [1, T, 3] print(out.vowel_backness.shape) # [1, T, 3] print(out.place_probabilities.shape) # [1, T, 5] print(out.voice_probabilities.shape) # [1, T, 2] ``` ## Outputs - `features`: hard conditional 22-dimensional features, `[B, T, 22]` - `manner_probabilities`: `[B, T, 9]` - `vowel_height`: `[B, T, 3]` - `vowel_backness`: `[B, T, 3]` - `place_probabilities`: `[B, T, 5]` - `voice_probabilities`: `[B, T, 2]` - `attention_mask`: valid encoder frames, `[B, T]` - `feature_names`: names for the 22 feature dimensions Feature order: ```text silence, stop, nasal, rhotic, fricative, affricate, approximant, lateral, vowel, high, mid, low, front, central, back, labial, alveolar, velar, palatal, postalveolar, voiceless, voiced ``` ## Viewing Probabilities The following snippet prints only the non-silence region. ```python manner_labels = [ "silence", "stop", "nasal", "rhotic", "fricative", "affricate", "approximant", "lateral", "vowel", ] manner = out.manner_probabilities[0] mask = out.attention_mask[0].bool() manner = manner[mask] best_manner = manner.argmax(dim=-1) non_silence = (best_manner != 0).nonzero(as_tu...

--- license: mit library_name: transformers tags: - audio - speech - phonology - phonological-features - wav2vec2 - multilingual - phonoq language: - en - es - de - cs pipeline_tag: audio-classification --- # PhonoQ 2.0 Multilingual Framewise phonological feature recognition for multilingual speech. This model returns phonological probabilities for manner, vowel height, vowel backness, place, and voicing, plus a hard conditional 22-feature representation per frame. It is a modernized successor to the original PhonoQ system: https://github.com/TAriasVergara/PhonoQ ## Usage ```bash pip install torch "transformers>=4.35,<6" soundfile safetensors ``` ```python import soundfile as sf import torch from transformers import AutoFeatureExtractor, AutoModel model_id = "abnerh/phonoq-2.0-multilingual" audio, sr = sf.read("example.wav") if audio.ndim > 1: audio = audio.mean(axis=1) processor = AutoFeatureExtractor.from_pretrained(model_id) inputs = processor(audio, sampling_rate=sr, return_tensors="pt", padding=True) model = AutoModel.from_pretrained(model_id, trust_remote_code=True) model.eval() with torch.no_grad(): out = model(**inputs) print(out.features.shape) # [1, T, 22] print(out.manner_probabilities.shape) # [1, T, 9] print(out.vowel_height.shape) # [1, T, 3] print(out.vowel_backness.shape) # [1, T, 3] print(out.place_probabilities.shape) # [1, T, 5] print(out.voice_probabilities.shape) # [1, T, 2] ``` ## Outputs - `features`: hard conditional 22-dimensional features, `[B, T, 22]` - `manner_probabilities`: `[B, T, 9]` - `vowel_height`: `[B, T, 3]` - `vowel_backness`: `[B, T, 3]` - `place_probabilities`: `[B, T, 5]` - `voice_probabilities`: `[B, T, 2]` - `attention_mask`: valid encoder frames, `[B, T]` - `feature_names`: names for the 22 feature dimensions Feature order: ```text silence, stop, nasal, rhotic, fricative, affricate, approximant, lateral, vowel, high, mid, low, front, central, back, labial, alveolar, velar, palatal, postalveolar, voiceless, voiced ``` ## Viewing Probabilities The following snippet prints only the non-silence region. ```python manner_labels = [ "silence", "stop", "nasal", "rhotic", "fricative", "affricate", "approximant", "lateral", "vowel", ] manner = out.manner_probabilities[0] mask = out.attention_mask[0].bool() manner = manner[mask] best_manner = manner.argmax(dim=-1) non_silence = (best_manner != 0).no...

Source context: 50 downloads · 2 likes · Pipeline audio-classification · Library transformers · Repo abnerh/phonoq-2.0-multilingual