Gemma 4 E2B It Oceanguard Marine Debris

Gemma 4 E2B — OceanGuard Marine Debris LoRA Adapter

A LoRA fine-tune of google/gemma-4-E2B-it that adapts the model to open-vocabulary detection of anthropogenic marine debris on consumer mobile hardware. The adapter is the training artefact behind the OceanGuard AI Android application — a fully offline marine-debris intelligence toolkit submitted to the Kaggle Gemma 4 Good Hackathon (Global Resilience track, Unsloth $10 K bonus) and built on top of two prior peer-reviewed publications on underwater debris detection (Sánchez-Ferrer et al., PRL 2023; IbPRIA 2022).

Headline result. On the held-out evaluation split (n = 200 stratified samples, real_test_holdout.jsonl) the LoRA fine-tune reaches [email protected] = 0.3256, against 0.1067 for the unmodified base model (Δ = +0.2189, a +205 % relative improvement). The winner is exp12_vision_lora, which unfreezes the SigLIP2 vision encoder via LoRA on top of the language-only adapter. Canonical metrics are tracked in finetune/experiments/results/best.json and results.md of the source repository.

Files in This Repository

This repository ships the LoRA adapter artefacts. The LiteRT-LM runtime build (.litertlm) is in progress — see §Deployment for the current status:

Artefact	Purpose	Consumed by
adapter_model.safetensors + adapter_config.json	LoRA adapter (≈120 MB, r=16, α=32, both language + vision towers) on top of google/gemma-4-E2B-it	PEFT / Unsloth pipelines, the reproducibility notebook
gemma-4-E2B-it-oceanguard-Q4_K_M.gguf	Merged model quantized Q4_K_M (≈3.4 GB, 5.88 BPW; large 262k-token vocab drives size); llama.cpp compatible	OceanGuard Android app via LlamaTextEngine + OCEANGUARD_FT tier, or any llama.cpp client
gemma-4-E2B-it-oceanguard.litertlm (pending)	LiteRT-LM runtime build of the merged base + adapter (≈2.6 GB)	The OceanGuard Android app, via the LiteRTTextEngine FINETUNED slot

The adapter is fully reproducible and usable today via PEFT / Unsloth. The GGUF Q4_K_M build is available for immediate use with llama.cpp and the OceanGuard Android app. The .litertlm export depends on Gemma 4 support landing in MediaPipe's tasks.python.genai.converter or in ai-edge-torch. See §Deployment.

Model Description

OceanGuard AI runs Gemma 4 E2B fully on-device on a Samsung Galaxy S22 Ultra (Exynos 2200, CPU + Vulkan/Xclipse 920) for open-vocabulary deep analysis of marine debris over a 50-class taxonomy that collapses down to 11 canonical ecological-impact families (BOTTLE, CAN, FISHING_NET, GLOVE, MASK, METAL_DEBRIS, PLASTIC_DEBRIS, TIRE, FABRIC_DEBRIS, GLASS_DEBRIS, OTHER).

This adapter aligns Gemma 4 E2B with the exact structured output contract that the Android application consumes — a JSON array of {box_2d, label, material} objects — and reduces the rate at which detections fall through to the generic PLASTIC_DEBRIS catch-all. The same LoRA also improves material attribution for the downstream ecological-impact lookup that drives the app's health-score and report-generation pipelines.

The adapter is distributed as a standard PEFT LoRA (≈120 MB safetensors, r=16, α=32) attached to a bf16 base model. Compared to a language-only adapter, the published model additionally adapts the SigLIP2 vision encoder via the same low-rank decomposition — this is what unlocks the dramatic gain on texture-heavy classes (Fishing_Net, Glove) reported below.

Intended Use

In-scope use cases.

• Mobile and edge marine-debris detection for citizen science, NGO field operations, and coastal monitoring campaigns.
• Research baselines for open-vocabulary detection over specialised environmental taxonomies.
• A grounding component for tool-augmented reporting pipelines (see the OceanGuard single-conversation agentic tool-calling architecture).

Out-of-scope use cases.

• Autonomous decision-making (cleanup dispatch, regulatory enforcement, environmental fines). This model is a triage and reporting aid; final decisions must remain with a qualified human.
• Real-time safety-critical detection (collision avoidance, navigation). End-to-end latency on the reference device is on the order of seconds per image.
• General-purpose object detection outside the marine-debris domain — the adapter narrows the visual prior towards marine debris and may degrade on unrelated tasks.

Training Data

The training set is a permissive-license merge of three open marine-debris datasets, including the CleanSea corpus introduced by the model author (IbPRIA 2022), plus a synthetic-renders set that expands the rare-class footprint:

Source	License	Annotations	Used in
CleanSea	Research-friendly (per dataset card)	COCO bbox, 8-class	All experiments
Ocean_garbage	Research-friendly (per dataset card)	COCO bbox, 8-class	All experiments
Neural_Ocean	Research-friendly (per dataset card)	COCO bbox, 8-class	All experiments
DenSea synthetic renders	Apache 2.0	COCO bbox, 8-class	exp01-exp09, exp11, exp12

The ablation grid was run over twelve dataset compositions that vary the ratio of synthetic-to-real images and the LoRA target modules. The published adapter corresponds to experiment exp12_vision_lora, which uses the largest available training mix (n = 24 446 samples: 10 809 synthetic + 13 637 real) combined with the vision-encoder LoRA that the language-only experiments leave untouched. The held-out evaluation set is a separate 200-sample stratified split from real_test_holdout.jsonl never seen during training. The 8-class taxonomy (Bottle, Can, Fishing_Net, Glove, Mask, Metal_Debris, Plastic_Debris, Tire) is extended at inference time to a 50-class fine-grained vocabulary that is mapped back to the 11 canonical families through a deterministic lookup table; see Gemma4VisionDetector.LABEL_TO_TYPE in the Android source for the full mapping.

Geographic coverage. The three source datasets were collected predominantly in coastal waters around Japan and the wider north-western Pacific. As a result the training distribution is biased towards the debris taxonomy, water clarity, lighting and sediment patterns of that region. This is reflected in the §Limitations and §Bias sections below; users surveying outside Japan-Pacific waters should expect a measurable domain shift and validate the adapter on a local hold-out before operational use.

Re-distributors of the merged dataset must comply with the original dataset licenses.

Training Procedure

The published adapter is the winner of a 12-experiment LoRA ablation grid. All runs used Unsloth FastVisionModel 2026.5.2 on a single workstation GPU, in bf16.

Configuration (best: exp12_vision_lora)

Hyper-parameter	Value
Base model	unsloth/gemma-4-E2B-it
LoRA r / α / dropout	16 / 32 / 0
Trainable parameters	≈ 60 M (≈ 1.2 % of the 5.15 B base)
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Vision LoRA	true — same target modules applied to the SigLIP2 vision encoder and the language tower
Language LoRA	true
Optimiser	adamw_8bit (Unsloth fused), lr = 1e-4 cosine schedule, weight_decay = 0.01, warmup_ratio = 0.1
Batch size	1 per device × 32 gradient accumulation = effective 32
Sequence length / image token budget	1 024 / 64
Precision	bf16
Planned epochs	3 (2 292 steps)
Effective steps trained	1 150 (≈ 50 % of plan)
Seed	42
Loss	causal-LM over the box_2d+label+material JSON tokens

Why training stopped at step 1 150

The vision-encoder LoRA roughly doubles the trainable parameter count compared to a language-only adapter, which raises the peak Windows commit charge (RAM + pagefile) for the training process beyond the 112 GB ceiling configured on the training workstation (96 GB physical RAM + 16 GB pagefile). After two CUDA / OS-level crashes during the original 3-epoch run, the experiment was restarted with a multi-resume strategy (200-step chunks, fresh Python process each chunk to drain the PIL-image cache held by the eager dataset loader). The first chunk produced checkpoint-1150 — and an intermediate evaluation showed that the model had already overtaken the previous language-only winner (exp10_real_full, [email protected] = 0.3253). The orchestrator therefore promoted checkpoint-1150 to the canonical artefact and stopped further chunks. Full 3-epoch training on this configuration would require a larger pagefile or a lazy-loading data collator — both noted in the project roadmap.

Convergence

Loss decreased monotonically from 14.78 at step 10 to 0.4991 at step 1 150; the last 20 logging windows had a standard deviation of ≈ 0.013 and a minimum loss of 0.4991 at step 1 150, with no late-stage divergence. The language-only baseline (exp11_real_synth_full, identical data) reached 0.5224 at the same step — the vision LoRA delivered a sustained ≈ 0.022 absolute (≈ 4 %) loss advantage from step 250 onwards.

Hardware and wall-clock

• 1× NVIDIA RTX 5090 (32 GiB GDDR7), Windows 11, CUDA 12.8, PyTorch 2.7
• Training wall-clock: ≈ 5 h 30 min end-to-end (initial run to step 1 097 + recovery cycle to step 1 150 + final eval)
• Deployment target: Samsung Galaxy S22 Ultra (Exynos 2200) via LiteRT-LM 0.11.0

Software

unsloth 2026.5.2 · transformers 5.5.0 · peft · trl · accelerate · bitsandbytes · torch 2.7.0+cu128.

The complete reproducible pipeline (prepare_datasets → generate_configs → run_experiments → eval_grid → collect_results → auto_promote_best) is published in the source repository under OceanguardAI-App/finetune/experiments/. The multi-resume orchestrator (finetune/multi_resume_exp12.py) and the WMI-hang workaround (finetune/wmi_bypass.py) used to recover the published checkpoint are also included.

Evaluation

All runs are evaluated on the held-out evaluation split of the merged dataset, with the same JSON contract and post-processing as the production Android app (temperature = 0.0, max_new_tokens = 256). The canonical metrics in finetune/experiments/results/best.json and results.md are reproduced verbatim in the tables below.

Aggregate

Metric	Base Gemma 4 E2B	OceanGuard LoRA (exp12_vision_lora)
[email protected]	0.1067	0.3256 (+0.2189)
Predictions emitted	168	298
JSON validity	0.995	0.885
Mean inference latency (RTX 5090)	1.88 s	7.19 s

Per-class [email protected]

| Class | Base | LoRA (vision_lora) | Δ |...