Sarashina2.2 3b Sft v4 Dpo GGUF

--- license: mit language: - ja base_model: sbintuitions/sarashina2.2-3b-instruct-v0.1 tags: - gguf - llama.cpp - japanese - DPO - ai-assistant --- # HinoMoto-3B sft_v4 + DPO (GGUF) DPO-tuned variant of HinoMoto-3B sft_v4 (Sarashina2.2-3B-instruct + 4-axis SFT + DPO refinement). ## Variants | Variant | BPW | Size | |---|---:|---:| | Q3_K_M | 3.91 | ~1.6 GB | | Q4_K_M | 4.92 | ~2.0 GB | | Q5_K_M | 5.72 | ~2.3 GB | | Q6_K | 6.60 | ~2.6 GB | | fp16 (raw GGUF) | 16.00 | ~6.7 GB | ## DPO training - **base**: sft_v4 (4-axis SFT + 11% NLI replay) - **preference data**: 108 pairs from HinoMoto-Bench-ja ideal_response_example (chosen) vs sft_v4 generation (rejected) - **method**: TRL DPO, beta=0.1, 100 steps, LoRA r=16 - **purpose**: align model toward "silence" (短く適切な応答) and family-tone consistency ## Quick start ```bash # Q3_K_M (fastest, 1.6 GB) huggingface-cli download FiShota/sarashina2.2-3b-sft-v4-dpo-gguf sft_v4_dpo.Q3_K_M.gguf --local-dir . ~/projects/llama.cpp/build/bin/llama-server -m sft_v4_dpo.Q3_K_M.gguf -c 2048 -ngl 99 --port 8092 ``` ## License MIT (inherits from Sarashina2.2-3B-instruct-v0.1, MIT licensed by SB Intuitions). ## Lineage base → sbintuitions/sarashina2.2-3b-instruct-v0.1 sft → FiShota/sarashina2.2-3b-sft-v3-Q4_K_M-gguf (4-axis SFT v3) sft → FiShota/sarashina2.2-3b-sft-v4-gguf (sft_v3 + 11% JNLI replay) **dpo → THIS** (sft_v4 + DPO 108 pairs) ## See also - HinoMoto-Bench-ja: https://github.com/FIshota/hinomoto-bench-ja (家族会話 4 軸評価) - HinoMoto note: https://note.com/fishota - 100M from-scratch sister: https://huggingface.co/FiShota/hinomoto-100m-v15-wsd-zloss-ema