ModelVerificadoExterno

Qwen3.5 4B MoLE

8 LoRA experts trained on Qwen3.5-4B base for the MoLE Research project.

Fuentehuggingface.co/hotdogs/Qwen3.5-4B-MoLE/tree/bfb2eca42f90f8ba90116a813ef938934bc159d5

TipoLoRAModelo basehuihui-ai/Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliteratedVersiónPublicador@hotdogsCcalidad

Fuente del modelo

Tipo: LoRA
Tamaño del archivo: 162 MB
Modelo base: huihui-ai/Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliterated
Versión: vadapter_model
Actualizado: 2 oct 2026
Publicador: @hotdogs
Descargas: 282
Parámetros: 4B
Fuente: Hugging Face

Descripción de la fuente

Qwen3.5-4B MoLE (Mixture of LoRA Experts)

8 LoRA experts trained on Qwen3.5-4B base for the MoLE Research project.

Experts

Expert	Dataset	eval_loss	Status
`codex`	50K coding	0.497	✅ PASSED
`logos`	50K reasoning	0.409	✅ PASSED
`scientia`	smoltalk 20K	0.865	✅ PASSED
`dialogos`	ultrachat 30K	1.256	✅ PASSED
`poiesis`	gpt4all-j 50K	1.645	usable
`psyche`	hh-rlhf 50K	1.766	usable
`aegis`	dolphin 10K	1.504	usable
`polyglot`	opus-100 20K	1.521	usable

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = "huihui-ai/Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliterated"
model = AutoModelForCausalLM.from_pretrained(base)
model = PeftModel.from_pretrained(model, "hotdogs/Qwen3.5-4B-MoLE/experts/codex")

Training

Framework: Unsloth QLoRA (4-bit)
LoRA: r=32, alpha=64
Max steps: 300 per expert
Quality gate: eval_loss < 1.5
Hardware: NVIDIA RTX 4060 Ti 16GB

Fuentes

1 fuente

Hugging FacePrincipalVersión bfb2eca42f90f8ba90116a813ef938934bc159d5

huggingface.co/hotdogs/Qwen3.5-4B-MoLE/tree/bfb2eca42f90f8ba90116a813ef938934bc159d5

Verificado 2 oct

Artefactos del modelo

1 artefacto

experts/aegis/adapter_model.safetensors

safetensors · 162 MB · SHA-256 a9053a7d397f…2e4f · Hugging Face

Descargar

Extractos de fuentes

3 extractos

Readme11 julhuggingface.co/hotdogs/Qwen3.5-4B-MoLE

--- license: apache-2.0 language: - en - th - multilingual tags: - qwen3 - lora - mole - mixture-of-lora-experts - unsloth base_model: huihui-ai/Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliterated --- # Qwen3.5-4B MoLE (Mixture of LoRA Experts) 8 LoRA experts trained on Qwen3.5-4B base for the MoLE Research project. ## Experts | Expert | Dataset | eval_loss | Status | |--------|---------|-----------|--------| | `codex` | 50K coding | 0.497 | ✅ PASSED | | `logos` | 50K reasoning | 0.409 | ✅ PASSED | | `scientia` | smoltalk 20K | 0.865 | ✅ PASSED | | `dialogos` | ultrachat 30K | 1.256 | ✅ PASSED | | `poiesis` | gpt4all-j 50K | 1.645 | usable | | `psyche` | hh-rlhf 50K | 1.766 | usable | | `aegis` | dolphin 10K | 1.504 | usable | | `polyglot` | opus-100 20K | 1.521 | usable | ## Usage ```python from peft import PeftModel from transformers import AutoModelForCausalLM base = "huihui-ai/Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliterated" model = AutoModelForCausalLM.from_pretrained(base) model = PeftModel.from_pretrained(model, "hotdogs/Qwen3.5-4B-MoLE/experts/codex") ``` ## Training - Framework: Unsloth QLoRA (4-bit) - LoRA: r=32, alpha=64 - Max steps: 300 per expert - Quality gate: eval_loss < 1.5 - Hardware: NVIDIA RTX 4060 Ti 16GB

Note2 octhuggingface.co/hotdogs/Qwen3.5-4B-MoLE

Source context: 282 downloads · 0 likes · Repo hotdogs/Qwen3.5-4B-MoLE

Note17 julhuggingface.co/hotdogs/Qwen3.5-4B-MoLE

Source context: 324 downloads · 0 likes · Repo hotdogs/Qwen3.5-4B-MoLE