Satnogs Signal Classifier

--- license: mit library_name: transformers pipeline_tag: image-classification tags: - satnogs - radio - signal-detection - waterfall datasets: - ryroeu/satnogs-signal-waterfalls --- # satnogs-signal-classifier ResNet-18 fine-tuned to classify SatNOGS **waterfall** spectrograms as **signal vs no-signal** (narrowband FSK/GFSK cubesat telemetry). A read-only **triage aid** — it suggests, it does not auto-vet. Inputs are **cropped to the spectrogram region** (colorbar + axes removed), which re-centers the signal. ## Held-out test metrics Held-out test set (436 observations) combining two axes held out of training: an **entirely unseen satellite** (FrontierSat) plus **held-out ground stations** whose noise/RFI fingerprint is kept out of training: | Metric | Model | Classical baseline | |---|---|---| | ROC-AUC | **0.827** | 0.570 | | PR-AUC | 0.829 | 0.557 | | precision@10 | **1.000** | 0.600 | Cross-satellite generalization (held-out **FrontierSat**, 240 obs, never trained on): ROC-AUC **0.772**. By mode: GFSK 0.93, FSK 0.92, FSK AX.100 Mode 5 0.79. ## Labels `0 = without-signal`, `1 = with-signal`; predict P(with-signal) = softmax index 1. Trained on **gold human `waterfall_status` vettings** — never the decode-based observation `status`. ## Training data Dataset: [ryroeu/satnogs-signal-waterfalls](https://huggingface.co/datasets/ryroeu/satnogs-signal-waterfalls) — 4 train satellites (OTP-2, CUBEBEL-2, AEPEX, CatSat), held-out satellite FrontierSat; ~1,189 gold waterfalls, cropped to the spectrogram. ## Limits & caveats - **Sampling bias:** gold labels skew toward clearer passes than the unvetted firehose; real-world performance on marginal/faint observations will be lower than these numbers. - **Test-set holdout is two-axis, not intersectional:** the unseen-satellite slice is held out by satellite, so some of its passes come from stations that also appear in training — that slice measures cross-satellite generalization, not fully station-unseen performance. - **Narrow family:** trained on narrowband FSK/GFSK telemetry; generalization beyond it is unverified. - **Read-only triage aid**, not an auto-vetter. precision@10 = 1.0 means the *top of the ranked queue* is reliable. ## Usage ```python from transformers import pipeline clf = pipeline("image-classification", model="ryroeu/satnogs-signal-classifier") clf("waterfall.png") ```