--- license: apache-2.0 base_model: google/gemma-4-31B-it library_name: peft tags: - gemma-4 - lora - peft - vision-language - drone-captioning - anti-uav --- # Gemma 4 31B-it — Anti-UAV Scene Captioner (LoRA) A LoRA adapter for **google/gemma-4-31B-it** trained to describe still frames from anti-UAV surveillance camera feeds — drone presence, position in frame, sky conditions, and visible scene structure. Trained as the captioner stage of a chained drone-pipeline: **YOLO detector → ByteTrack → Gemma 4 captioner (this).** ## Training | | | |---|---| | base | google/gemma-4-31B-it | | method | LoRA (4-bit nf4) — Google cookbook recipe (eager attn, bf16 quant storage) | | LoRA r / α | 16 / 16, target_modules="all-linear" | | training data | 658 (frame, caption) pairs from Anti-UAV-RGBT, captions produced by Qwen2.5-VL-7B teacher | | epochs / steps | 2 / 166 | | effective batch | 8 (1 × grad-accum 8) | | LR | 2e-4 constant, max_grad_norm 0.3 | | eval loss | 0.179 (down from 0.241 first eval) | | eval token accuracy | 93.4% | | hardware | 3× NVIDIA RTX 3090 (model parallelism via balanced device_map, ~8GB/GPU) | ## Use ```python from peft import PeftModel from transformers import AutoProcessor, AutoModelForImageTextToText, BitsAndBytesConfig import torch bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_quant_storage=torch.bfloat16) base = AutoModelForImageTextToText.from_pretrained( "google/gemma-4-31B-it", quantization_config=bnb, attn_implementation="eager", dtype=torch.bfloat16, device_map="auto", ) model = PeftModel.from_pretrained(base, "sapoepsilon/gemma4-31b-drone-captioner") processor = AutoProcessor.from_pretrained("google/gemma-4-31B-it") ``` ## Caveats - Captions are derived from a **VLM teacher** (Qwen2.5-VL-7B), not human labels — supervision is noisy and inherits the teacher's biases - Trained on a narrow distribution: anti-UAV surveillance reticle/HUD imagery (Anti-UAV-RGBT). Out-of-distribution frames may degrade - Style is fairly templated ("The image shows a drone presence ...") which is intentional for downstream parsing but may sound formulaic ## License Adapter weights: Apache 2.0. Base model retains its original Google Gemma license.
