Tinman Gemma4 Companion GGUF

Usage

llama.cpp

./llama-cli -m gemma4-companion-Q4_K_M.gguf -cnv \
  -p "You are a warm, caring companion..."

LM Studio / GPT4All

Load the GGUF file directly from the UI.

System Prompts

See main model card for companion, roleplay, and assistant mode system prompts.

gemma4-companion-Q8_0.gguf

gguf · 7.43 GB · SHA-256 2241a21e6b7c…e25f · Hugging Face

--- license: apache-2.0 base_model: - TinmanLabSL/gemma4-companion-merged tags: - gguf - gemma4 - companion - roleplay - uncensored - quantized pipeline_tag: any-to-any --- # Gemma 4 E4B Companion — GGUF Quantizations GGUF quantized versions of [TinmanLabSL/gemma4-companion-merged](https://huggingface.co/TinmanLabSL/gemma4-companion-merged). See the [main model card](https://huggingface.co/TinmanLabSL/gemma4-companion-merged) for full details on training, system prompts, and deployment. ## Available Quantizations | File | Quant | Size | Quality | Use Case | |------|-------|------|---------|----------| | `gemma4-companion-bf16.gguf` | BF16 | 14 GB | Lossless | Reference / re-quantization | | `gemma4-companion-Q8_0.gguf` | Q8_0 | 7.5 GB | Near-lossless | Desktop with ample RAM | | `gemma4-companion-Q4_K_M.gguf` | Q4_K_M | 5.0 GB | Good | **Recommended** for most users | ## Usage ### llama.cpp ```bash ./llama-cli -m gemma4-companion-Q4_K_M.gguf -cnv \ -p "You are a warm, caring companion..." ``` ### LM Studio / GPT4All Load the GGUF file directly from the UI. ## System Prompts See [main model card](https://huggingface.co/TinmanLabSL/gemma4-companion-merged#-mode-system--system-prompt-toggle) for companion, roleplay, and assistant mode system prompts.