Gemma 4 E4B It Merged

--- base_model: google/gemma-4-E4B-it library_name: transformers pipeline_tag: image-text-to-text license: gemma tags: - gemma4 - lora - merged - endpoints-template --- # gemma-4-E4B-it-merged `google/gemma-4-E4B-it` with a LoRA adapter (q/k/v/o/gate/up/down on the language model, r=16, alpha=32) merged into the base weights. bf16. ## Inference Endpoints Ships a custom `handler.py` (Inference Toolkit) because the Gemma 4 arch is too new for the default container. `requirements.txt` pins `transformers>=5.12.1`. Deploy on a GPU instance with **>=24 GB** VRAM (model is ~16 GB bf16). Request body: ```json {"inputs": "What is a LoRA adapter?", "parameters": {"max_new_tokens": 256}} ``` ## Local use ```python import torch from transformers import AutoModelForImageTextToText, AutoProcessor m = AutoModelForImageTextToText.from_pretrained( "dxv2k/gemma-4-E4B-it-merged", dtype=torch.bfloat16, device_map="auto") p = AutoProcessor.from_pretrained("dxv2k/gemma-4-E4B-it-merged") msgs = [{"role": "user", "content": [{"type": "text", "text": "Hello"}]}] enc = p.apply_chat_template(msgs, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(m.device) out = m.generate(**enc, max_new_tokens=128) print(p.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=True)) ```