Gemma 4 E4B It Sdnq Dynamic 8bit
This is an int8 quantized version of google/gemma-4-E4B-it using SDNQ (SD.Next Quantization) with the dynamic option, Hadamard Rotation and quantized embeddings.
Fonte do modelo
- Tipo
- Text model
- Tamanho do arquivo
- 8,06 GB
- Modelo base
- google/gemma-4-E4B-it
- Versão
- vmodel
- Atualizado
- 20 de out. de 2026
- Downloads
- 16
- Fonte
- Hugging Face
Descrição da fonte
Gemma 4 E4B it SDNQ Dynamic INT8
This is an int8 quantized version of google/gemma-4-E4B-it using SDNQ (SD.Next Quantization) with the dynamic option, Hadamard Rotation and quantized embeddings.
Note: You need SDNQ v0.2.1 or superior (the quantized embeddings + Hadamard path requires the nn.Embedding fix that landed in 0.2.1).
Usage
Load it with transformers like the original model. You need to import sdnq first so it registers into the transformers quantizer registry — without it, loading fails with a size-mismatch error:
import sdnq # registers the SDNQ quantizer into transformers
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
model_id = "OzzyGT/gemma_4_E4B_it_sdnq_dynamic_8bit"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype=torch.bfloat16, device_map="cuda")
messages = [{"role": "user", "content": [{"type": "text", "text": "Explain quantization in one sentence."}]}]
inputs = processor.apply_chat_template(
messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt"
).to(model.device)
out = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Fontes
1 fonte
Fontes
1 fonteVerificado 20 de out.
Artefatos de modelo
1 artefato
Artefatos de modelo
1 artefatoTrechos de fonte
3 trechos
Trechos de fonte
3 trechos--- library_name: transformers license: apache-2.0 license_link: https://ai.google.dev/gemma/docs/gemma_4_license base_model: - google/gemma-4-E4B-it base_model_relation: quantized tags: - sdnq - gemma pipeline_tag: any-to-any --- # Gemma 4 E4B it SDNQ Dynamic INT8 This is an int8 quantized version of [google/gemma-4-E4B-it](https://huggingface.co/google/gemma-4-E4B-it) using [SDNQ](https://github.com/Disty0/sdnq) (SD.Next Quantization) with the dynamic option, Hadamard Rotation and quantized embeddings. Note: You need SDNQ v0.2.1 or superior (the quantized embeddings + Hadamard path requires the `nn.Embedding` fix that landed in 0.2.1). ## Usage Load it with `transformers` like the original model. You need to `import sdnq` first so it registers into the transformers quantizer registry — without it, loading fails with a size-mismatch error: ```python import sdnq # registers the SDNQ quantizer into transformers import torch from transformers import AutoModelForImageTextToText, AutoProcessor model_id = "OzzyGT/gemma_4_E4B_it_sdnq_dynamic_8bit" processor = AutoProcessor.from_pretrained(model_id) model = AutoModelForImageTextToText.from_pretrained(model_id, dtype=torch.bfloat16, device_map="cuda") messages = [{"role": "user", "content": [{"type": "text", "text": "Explain quantization in one sentence."}]}] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt" ).to(model.device) out = model.generate(**inputs, max_new_tokens=256) print(processor.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)) ```
Source context: 16 downloads · 0 likes · Pipeline any-to-any · Library transformers · Repo OzzyGT/gemma_4_E4B_it_sdnq_dynamic_8bit
Source context: 68 downloads · 0 likes · Pipeline any-to-any · Library transformers · Repo OzzyGT/gemma_4_E4B_it_sdnq_dynamic_8bit
