Glm5.1 Tiny

--- pipeline_tag: text-generation --- # 缩放模型 > 由 model-scaler v0.1.0 生成于 2026-04-21 07:04:45 UTC ## 来源信息 - **源模型**: /models/ckpts/ZhipuAI/GLM-5.1 ## 架构参数对比 | 参数 | 原始值 | 缩放值 | |------|--------|--------| | num_hidden_layers | 78 | 4 | | hidden_size | 6144 | 2048 | | num_attention_heads | 64 | 32 | | intermediate_size | 12288 | 4096 | | n_routed_experts | 256 | 16 | | n_shared_experts | 1 | 1 | | num_experts_per_tok | 8 | 2 | | moe_intermediate_size | 2048 | 1024 | | first_k_dense_replace | 3 | 1 | | index_head_dim | 128 | 128 | | index_n_heads | 32 | 4 | | index_topk | 2048 | 128 | | q_lora_rank | 2048 | 512 | | kv_lora_rank | 512 | 512 | | qk_head_dim | 256 | 192 | | qk_nope_head_dim | 192 | 128 | | qk_rope_head_dim | 64 | 64 | | v_head_dim | 256 | 128 | ## 使用方法 ```python from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained( "./path/to/scaled-model", trust_remote_code=True, ) tokenizer = AutoTokenizer.from_pretrained( "./path/to/scaled-model", trust_remote_code=True, ) ```