Loading
Have a ComfyUI workflow? Get a readable Workflow Doc in your browser.Make a workflow readable.Try it now ↗
AMALIA-SFT-FALA is a Portuguese ASR checkpoint based on the SLAM-ASR architecture. It connects a pre-trained speech encoder to a large language model through a learned projection module, allowing the LLM to generate...
Model source
Source description
AMALIA-SFT-FALA is a Portuguese ASR checkpoint based on the SLAM-ASR architecture. It connects a pre-trained speech encoder to a large language model through a learned projection module, allowing the LLM to generate text transcriptions from speech representations.
The model follows the SLAM-ASR design:
audio → speech encoder → projector → LLM → transcription
The speech encoder extracts acoustic representations from the input audio. These representations are mapped by a trainable projector into the embedding space expected by the language model. The LLM then autoregressively generates the transcription.
Sources
1 sourceVerified Sep 22
Model artifacts
1 artifactSource excerpts
2 excerptsAMALIA-speech-encoder, also available on Hugging FaceAMALIA-9B-1225-SFTThis model is intended for research and experimentation on European Portuguese ASR, especially in settings where speech representations are connected to an LLM instead of using a standalone encoder-decoder ASR model.
Example use cases include:
AMALIA-FALA models are split into two parts:
The model weights are not cloned manually by the user. They are downloaded automatically from Hugging Face the first time the model is loaded.
Install the lightweight AMALIA-FALA loader directly from Hugging Face:
pip install git+https://huggingface.co/amalia-llm/amalia-fala-loader
from amalia_fala import pipeline
asr = pipeline(
"amalia-llm/AMALIA-SFT-FALA",
device="cuda:0",
)
result = asr("/path/to/audio.wav")
print(result["text"])
The checkpoint is downloaded automatically from:
amalia-llm/AMALIA-SFT-FALA
and cached locally by Hugging Face. Future runs reuse the cached checkpoint.
If you need access to the underlying model and tokenizer:
from amalia_fala import load_model
model, tokenizer, model_config, dataset_config = load_model(
"amalia-llm/AMALIA-SFT-FALA",
device="cuda:0",
)
To use the AMALIA-SFT-FALA model, change the repository name:
from amalia_fala import pipeline
asr = pipeline(
"amalia-llm/AMALIA-DPO-FALA",
device="cuda:0",
)
result = asr("/path/to/audio.wav")
print(result["text"])
The model was developed in the context of European Portuguese ASR experiments using FalAR and CAMÕES ASR data.
FalAR contains more than 5000 hours of parliamentary speech from the Assambleia da República.
CAMÕES consists of a curated collection of up to 14 sub-corpora containing different domains and speech styles, totaling approximately 425 hours of speech.
The encoder AMALIA-speech-encoder was trained separately and reused as the speech encoder in this SLAM-ASR setup.
This checkpoint is intended as a research artifact. Performance may vary depending on audio quality, speaker domain, recording conditions, and transcription style. The model may be less reliable on noisy audio, long-form speech, code-switching, or domains that differ from the training data.
This model builds on the SLAM-ASR idea of connecting a speech encoder to a large language model for automatic speech recognition.
--- license: mit datasets: - inesc-id/FalAR - inesc-id/camoes_asr language: - pt base_model: - amalia-llm/AMALIA-9B-1225-SFT - amalia-llm/AMALIA-speech-encoder pipeline_tag: automatic-speech-recognition --- # Model card AMALIA-SFT-FALA is a Portuguese ASR checkpoint based on the [SLAM-ASR architecture](https://arxiv.org/abs/2402.08846). It connects a pre-trained speech encoder to a large language model through a learned projection module, allowing the LLM to generate text transcriptions from speech representations. ## Architecture The model follows the SLAM-ASR design: ```text audio → speech encoder → projector → LLM → transcription ``` The speech encoder extracts acoustic representations from the input audio. These representations are mapped by a trainable projector into the embedding space expected by the language model. The LLM then autoregressively generates the transcription. ## Components * **Base architecture:** SLAM-ASR * **Speech encoder:** `AMALIA-speech-encoder`, also available on Hugging Face * **Language model:** `AMALIA-9B-1225-SFT` * **Task:** Automatic Speech Recognition * **Language:** Portuguese, with focus on European Portuguese ## Intended Use This model is intended for research and experimentation on European Portuguese ASR, especially in settings where speech representations are connected to an LLM instead of using a standalone encoder-decoder ASR model. Example use cases include: * transcribing Portuguese speech; * evaluating LLM-augmented ASR systems; * comparing SLAM-style ASR with conventional ASR models; * research on European Portuguese speech processing. ## AMALIA-FALA Usage AMALIA-FALA models are split into two parts: 1. **Model repository**: contains the model weights and example assets. 2. **Loader repository**: contains the lightweight Python code used to load and run the model. The model weights are **not cloned manually** by the user. They are downloaded automatically from Hugging Face the first time the model is loaded. --- ### 1. Install the loader package Install the lightweight AMALIA-FALA loader directly from Hugging Face: ```bash pip install git+https://huggingface.co/amalia-llm/amalia-fala-loader ``` --- ### 2. Run inference on your own audio file ```python from amalia_fala import pipeline asr = pipeline( "amalia-llm/AMALIA-SFT-FALA", device="cuda:0", ) result = asr("/path/to/audio.wav")...
Source context: 0 downloads · 2 likes · Pipeline automatic-speech-recognition · Repo amalia-llm/AMALIA-SFT-FALA