Loading
Have a ComfyUI workflow? Get a readable Workflow Doc in your browser.Make a workflow readable.Try it now ↗
This is the Stage B checkpoint for FINCH, a bioacoustic species identification framework that fuses audio classification with spatiotemporal priors from eBird abundance data. A frozen NatureLM-audio encoder is paired...
Model source
Source description
This is the Stage B checkpoint for FINCH, a bioacoustic species identification framework that fuses audio classification with spatiotemporal priors from eBird abundance data. A frozen NatureLM-audio encoder is paired with a learned gating network that adaptively weights audio evidence against space-time context.
Sources
1 sourceVerified Sep 19
Model artifacts
1 artifactSource excerpts
2 excerptsThe Stage B model computes fused logits as:
final_logits = audio_logits / T + w(a, x, t) * log(prior + eps)
where w(a, x, t) is a small gating MLP conditioned on audio confidence, prior confidence, location, and time-of-year. T and eps are learned scalars.
Only the trainable parameters (classifier head, gating network, temperature, epsilon) are stored here (~3 MB). The frozen NatureLM-audio encoder (~8B params) is downloaded separately from EarthSpeciesProject/NatureLM-audio at load time.
Requires Python 3.10+, uv, and a HuggingFace account with access to Meta Llama 3.1 8B Instruct (required by NatureLM-audio).
# Clone the repository
git clone https://github.com/leharris3/birdnoise.git
cd birdnoise
# Install dependencies
uv sync
cd NatureLM-audio && uv pip install -r requirements.txt && cd ..
# Log in to HuggingFace
huggingface-cli login
from Models import HFStageBModel
# Download weights from the Hub
model = HFStageBModel.from_pretrained("leharris3/FINCH")
w(a, x, t) + learned T, eps (warm-started from Stage A)@article{ovanger2026adaptive,
title = {Adaptive Evidence Weighting for Audio-Spatiotemporal Fusion},
author = {Oscar Ovanger and Levi Harris and Timothy H. Keitt},
journal = {arXiv preprint arXiv:2602.03817},
year = {2026},
url = {https://arxiv.org/abs/2602.03817}
}
--- library_name: finch pipeline_tag: audio-classification tags: - model_hub_mixin - pytorch_model_hub_mixin - bioacoustics - bird-species license: apache-2.0 language: - en datasets: - birdclef-2021 --- # FINCH: Adaptive Evidence Weighting for Audio-Spatiotemporal Fusion This is the Stage B checkpoint for **FINCH**, a bioacoustic species identification framework that fuses audio classification with spatiotemporal priors from [eBird](https://ebird.org/) abundance data. A frozen [NatureLM-audio](https://huggingface.co/EarthSpeciesProject/NatureLM-audio) encoder is paired with a learned gating network that adaptively weights audio evidence against space-time context. - **Paper:** [arXiv:2602.03817](https://arxiv.org/abs/2602.03817) - **Code:** [github.com/leharris3/birdnoise](https://github.com/leharris3/birdnoise) ## Model description The Stage B model computes fused logits as: ``` final_logits = audio_logits / T + w(a, x, t) * log(prior + eps) ``` where `w(a, x, t)` is a small gating MLP conditioned on audio confidence, prior confidence, location, and time-of-year. `T` and `eps` are learned scalars. Only the trainable parameters (classifier head, gating network, temperature, epsilon) are stored here (\~3 MB). The frozen NatureLM-audio encoder (\~8B params) is downloaded separately from [EarthSpeciesProject/NatureLM-audio](https://huggingface.co/EarthSpeciesProject/NatureLM-audio) at load time. ## How to use Requires Python 3.10+, [uv](https://docs.astral.sh/uv/), and a HuggingFace account with access to [Meta Llama 3.1 8B Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) (required by NatureLM-audio). ```bash # Clone the repository git clone https://github.com/leharris3/birdnoise.git cd birdnoise # Install dependencies uv sync cd NatureLM-audio && uv pip install -r requirements.txt && cd .. # Log in to HuggingFace huggingface-cli login ``` ```python from Models import HFStageBModel # Download weights from the Hub model = HFStageBModel.from_pretrained("leharris3/FINCH") ``` ## Training details - **Dataset:** BirdCLEF 2021 (184 species) - **Encoder:** NatureLM-audio (frozen) - **Stage A:** Linear probe + scalar fusion weight (30 epochs) - **Stage B:** Gating network `w(a, x, t)` + learned T, eps (warm-started from Stage A) - **Best val accuracy:** 82.6% ## Citation ```bibtex @article{ovanger2026adaptive, title = {Ada...
Source context: 8 downloads · 0 likes · Pipeline audio-classification · Library finch · Repo leharris3/FINCH