ModelÜberprüftExtern

Qwen3.5 4B MoLE

8 LoRA experts trained on Qwen3.5-4B base for the MoLE Research project.

Quellehuggingface.co/hotdogs/Qwen3.5-4B-MoLE/tree/bfb2eca42f90f8ba90116a813ef938934bc159d5

ArtLoRABasismodellhuihui-ai/Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliteratedVersionVerleger@hotdogsCBewertung

Modellquelle

Art: LoRA
Dateigröße: 162 MB
Basismodell: huihui-ai/Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliterated
Version: vadapter_model
Aktualisiert: 2. Okt. 2026
Verleger: @hotdogs
Downloads: 282
Parameter: 4B
Quelle: Hugging Face

Quellenbeschreibung

Qwen3.5-4B MoLE (Mixture of LoRA Experts)

8 LoRA experts trained on Qwen3.5-4B base for the MoLE Research project.

Experts

Expert	Dataset	eval_loss	Status
`codex`	50K coding	0.497	✅ PASSED
`logos`	50K reasoning	0.409	✅ PASSED
`scientia`	smoltalk 20K	0.865	✅ PASSED
`dialogos`	ultrachat 30K	1.256	✅ PASSED
`poiesis`	gpt4all-j 50K	1.645	usable
`psyche`	hh-rlhf 50K	1.766	usable
`aegis`	dolphin 10K	1.504	usable
`polyglot`	opus-100 20K	1.521	usable

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = "huihui-ai/Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliterated"
model = AutoModelForCausalLM.from_pretrained(base)
model = PeftModel.from_pretrained(model, "hotdogs/Qwen3.5-4B-MoLE/experts/codex")

Training

Framework: Unsloth QLoRA (4-bit)
LoRA: r=32, alpha=64
Max steps: 300 per expert
Quality gate: eval_loss < 1.5
Hardware: NVIDIA RTX 4060 Ti 16GB

Quellen

1 Quelle

Hugging FacePrimärVersion bfb2eca42f90f8ba90116a813ef938934bc159d5

huggingface.co/hotdogs/Qwen3.5-4B-MoLE/tree/bfb2eca42f90f8ba90116a813ef938934bc159d5

Verifiziert 2. Okt.

Modellartefakte

1 Artefakt

experts/aegis/adapter_model.safetensors

safetensors · 162 MB · SHA-256 a9053a7d397f…2e4f · Hugging Face

Herunterladen

Quellenauszüge

3 Auszüge

Readme11. Julihuggingface.co/hotdogs/Qwen3.5-4B-MoLE

--- license: apache-2.0 language: - en - th - multilingual tags: - qwen3 - lora - mole - mixture-of-lora-experts - unsloth base_model: huihui-ai/Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliterated --- # Qwen3.5-4B MoLE (Mixture of LoRA Experts) 8 LoRA experts trained on Qwen3.5-4B base for the MoLE Research project. ## Experts | Expert | Dataset | eval_loss | Status | |--------|---------|-----------|--------| | `codex` | 50K coding | 0.497 | ✅ PASSED | | `logos` | 50K reasoning | 0.409 | ✅ PASSED | | `scientia` | smoltalk 20K | 0.865 | ✅ PASSED | | `dialogos` | ultrachat 30K | 1.256 | ✅ PASSED | | `poiesis` | gpt4all-j 50K | 1.645 | usable | | `psyche` | hh-rlhf 50K | 1.766 | usable | | `aegis` | dolphin 10K | 1.504 | usable | | `polyglot` | opus-100 20K | 1.521 | usable | ## Usage ```python from peft import PeftModel from transformers import AutoModelForCausalLM base = "huihui-ai/Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliterated" model = AutoModelForCausalLM.from_pretrained(base) model = PeftModel.from_pretrained(model, "hotdogs/Qwen3.5-4B-MoLE/experts/codex") ``` ## Training - Framework: Unsloth QLoRA (4-bit) - LoRA: r=32, alpha=64 - Max steps: 300 per expert - Quality gate: eval_loss < 1.5 - Hardware: NVIDIA RTX 4060 Ti 16GB

Note2. Okt.huggingface.co/hotdogs/Qwen3.5-4B-MoLE

Source context: 282 downloads · 0 likes · Repo hotdogs/Qwen3.5-4B-MoLE

Note17. Julihuggingface.co/hotdogs/Qwen3.5-4B-MoLE

Source context: 324 downloads · 0 likes · Repo hotdogs/Qwen3.5-4B-MoLE