ModelVerificadoExterno

Qwen3.5 4B MoLE

8 LoRA experts trained on Qwen3.5-4B base for the MoLE Research project.

Fontehuggingface.co/hotdogs/Qwen3.5-4B-MoLE/tree/bfb2eca42f90f8ba90116a813ef938934bc159d5

TipoLoRAModelo basehuihui-ai/Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliteratedVersãoPublicador@hotdogsCnota

Fonte do modelo

Tipo: LoRA
Tamanho do arquivo: 162 MB
Modelo base: huihui-ai/Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliterated
Versão: vadapter_model
Atualizado: 2 de out. de 2026
Publicador: @hotdogs
Downloads: 282
Parâmetros: 4B
Fonte: Hugging Face

Descrição da fonte

Qwen3.5-4B MoLE (Mixture of LoRA Experts)

8 LoRA experts trained on Qwen3.5-4B base for the MoLE Research project.

Experts

Expert	Dataset	eval_loss	Status
`codex`	50K coding	0.497	✅ PASSED
`logos`	50K reasoning	0.409	✅ PASSED
`scientia`	smoltalk 20K	0.865	✅ PASSED
`dialogos`	ultrachat 30K	1.256	✅ PASSED
`poiesis`	gpt4all-j 50K	1.645	usable
`psyche`	hh-rlhf 50K	1.766	usable
`aegis`	dolphin 10K	1.504	usable
`polyglot`	opus-100 20K	1.521	usable

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = "huihui-ai/Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliterated"
model = AutoModelForCausalLM.from_pretrained(base)
model = PeftModel.from_pretrained(model, "hotdogs/Qwen3.5-4B-MoLE/experts/codex")

Training

Framework: Unsloth QLoRA (4-bit)
LoRA: r=32, alpha=64
Max steps: 300 per expert
Quality gate: eval_loss < 1.5
Hardware: NVIDIA RTX 4060 Ti 16GB

Fontes

1 fonte

Hugging FacePrimárioVersão bfb2eca42f90f8ba90116a813ef938934bc159d5

huggingface.co/hotdogs/Qwen3.5-4B-MoLE/tree/bfb2eca42f90f8ba90116a813ef938934bc159d5

Verificado 2 de out.

Artefatos de modelo

1 artefato

experts/aegis/adapter_model.safetensors

safetensors · 162 MB · SHA-256 a9053a7d397f…2e4f · Hugging Face

Baixar

Trechos de fonte

3 trechos

Readme11 de jul.huggingface.co/hotdogs/Qwen3.5-4B-MoLE

--- license: apache-2.0 language: - en - th - multilingual tags: - qwen3 - lora - mole - mixture-of-lora-experts - unsloth base_model: huihui-ai/Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliterated --- # Qwen3.5-4B MoLE (Mixture of LoRA Experts) 8 LoRA experts trained on Qwen3.5-4B base for the MoLE Research project. ## Experts | Expert | Dataset | eval_loss | Status | |--------|---------|-----------|--------| | `codex` | 50K coding | 0.497 | ✅ PASSED | | `logos` | 50K reasoning | 0.409 | ✅ PASSED | | `scientia` | smoltalk 20K | 0.865 | ✅ PASSED | | `dialogos` | ultrachat 30K | 1.256 | ✅ PASSED | | `poiesis` | gpt4all-j 50K | 1.645 | usable | | `psyche` | hh-rlhf 50K | 1.766 | usable | | `aegis` | dolphin 10K | 1.504 | usable | | `polyglot` | opus-100 20K | 1.521 | usable | ## Usage ```python from peft import PeftModel from transformers import AutoModelForCausalLM base = "huihui-ai/Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliterated" model = AutoModelForCausalLM.from_pretrained(base) model = PeftModel.from_pretrained(model, "hotdogs/Qwen3.5-4B-MoLE/experts/codex") ``` ## Training - Framework: Unsloth QLoRA (4-bit) - LoRA: r=32, alpha=64 - Max steps: 300 per expert - Quality gate: eval_loss < 1.5 - Hardware: NVIDIA RTX 4060 Ti 16GB

Note2 de out.huggingface.co/hotdogs/Qwen3.5-4B-MoLE

Source context: 282 downloads · 0 likes · Repo hotdogs/Qwen3.5-4B-MoLE

Note17 de jul.huggingface.co/hotdogs/Qwen3.5-4B-MoLE

Source context: 324 downloads · 0 likes · Repo hotdogs/Qwen3.5-4B-MoLE