FINCH

--- library_name: finch pipeline_tag: audio-classification tags: - model_hub_mixin - pytorch_model_hub_mixin - bioacoustics - bird-species license: apache-2.0 language: - en datasets: - birdclef-2021 --- # FINCH: Adaptive Evidence Weighting for Audio-Spatiotemporal Fusion This is the Stage B checkpoint for **FINCH**, a bioacoustic species identification framework that fuses audio classification with spatiotemporal priors from [eBird](https://ebird.org/) abundance data. A frozen [NatureLM-audio](https://huggingface.co/EarthSpeciesProject/NatureLM-audio) encoder is paired with a learned gating network that adaptively weights audio evidence against space-time context. - **Paper:** [arXiv:2602.03817](https://arxiv.org/abs/2602.03817) - **Code:** [github.com/leharris3/birdnoise](https://github.com/leharris3/birdnoise) ## Model description The Stage B model computes fused logits as: ``` final_logits = audio_logits / T + w(a, x, t) * log(prior + eps) ``` where `w(a, x, t)` is a small gating MLP conditioned on audio confidence, prior confidence, location, and time-of-year. `T` and `eps` are learned scalars. Only the trainable parameters (classifier head, gating network, temperature, epsilon) are stored here (\~3 MB). The frozen NatureLM-audio encoder (\~8B params) is downloaded separately from [EarthSpeciesProject/NatureLM-audio](https://huggingface.co/EarthSpeciesProject/NatureLM-audio) at load time. ## How to use Requires Python 3.10+, [uv](https://docs.astral.sh/uv/), and a HuggingFace account with access to [Meta Llama 3.1 8B Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) (required by NatureLM-audio). ```bash # Clone the repository git clone https://github.com/leharris3/birdnoise.git cd birdnoise # Install dependencies uv sync cd NatureLM-audio && uv pip install -r requirements.txt && cd .. # Log in to HuggingFace huggingface-cli login ``` ```python from Models import HFStageBModel # Download weights from the Hub model = HFStageBModel.from_pretrained("leharris3/FINCH") ``` ## Training details - **Dataset:** BirdCLEF 2021 (184 species) - **Encoder:** NatureLM-audio (frozen) - **Stage A:** Linear probe + scalar fusion weight (30 epochs) - **Stage B:** Gating network `w(a, x, t)` + learned T, eps (warm-started from Stage A) - **Best val accuracy:** 82.6% ## Citation ```bibtex @article{ovanger2026adaptive, title = {Ada...