--- library_name: pytorch tags: - quantization - gptq - awq - admm - llm --- # Quantization bundles INT-quantized weight bundles for several backbone LLMs, produced by [Quant_ADMM](https://github.com/) (private). Each leaf subdir is one config. Each config directory contains: - `int_bundle.pt` — packed integer weights (per-group `S, Z` + packed int4/3 q) - `aux_state.pt` — non-quantized state (RMSNorm weights, biases, embed, lm_head). **Critical for AWQ-style methods** that fold per-channel rescaling into surrounding modules. - `run_args.json` — exact command-line config (bitwidth, method, ADMM hyperparams, etc.) ## Layout ``` QuantADMM-bundles/ / _ bit_g _calib _ / int_bundle.pt aux_state.pt run_args.json ``` ## Contents ### `Llama-2-13b-hf/` | subdir | method | wbits | groupsize | calib | extra | |---|---|---|---|---|---| | `llama-2-13b-hf_admm_cd_4bit_g128_calibc4_noqep_actorder` | `admm_cd` | 4 | 128 | c4 | lam=0.1, iters=200 | | `llama-2-13b-hf_admm_cd_4bit_g128_calibc4_qep_damp1.0_corr0.5_actorder` | `admm_cd` | 4 | 128 | c4 | lam=0.1, iters=200 | | `llama-2-13b-hf_awq_4bit_g128_calibc4_noqep` | `awq` | 4 | 128 | c4 | | | `llama-2-13b-hf_babai_4bit_g128_calibc4_noqep_K5_actorder` | `babai` | 4 | 128 | c4 | | | `llama-2-13b-hf_gptaq_4bit_g128_calibc4_noqep_actorder` | `gptaq` | 4 | 128 | c4 | | | `llama-2-13b-hf_gptq_4bit_g128_calibc4_noqep_actorder` | `gptq` | 4 | 128 | c4 | | | `llama-2-13b-hf_gptq_4bit_g128_calibc4_qep_damp1.0_corr0.5_actorder` | `gptq` | 4 | 128 | c4 | | | `llama-2-13b-hf_quarot_gptq_4bit_g128_calibc4_noqep_actorder` | `quarot_gptq` | 4 | 128 | c4 | | | `llama-2-13b-hf_spinquantR_w4_external` | `spinquant_rotation_train` | 4 | ? | ? | | | `llama-2-13b-hf_spinquant_gptq_4bit_g128_calibc4_noqep_actorder` | `spinquant_gptq` | 4 | 128 | c4 | | ### `Llama-2-7b-hf/` | subdir | method | wbits | groupsize | calib | extra | |---|---|---|---|---|---| | `llama-2-7b-hf_admm_cd_4bit_g128_calibc4_noqep_actorder` | `admm_cd` | 4 | 128 | c4 | lam=0.1, iters=200 | | `llama-2-7b-hf_admm_cd_4bit_g128_calibc4_qep_damp1.0_corr0.5_actorder` | `admm_cd` | 4 | 128 | c4 | lam=0.1, iters=200 | | `llama-2-7b-hf_awq_4bit_g128_calibc4_noqep` | `awq` | 4 | 128 | c4 | | | `llama-2-7b-hf_babai_4bit_g128_calibc4_noqep_K5_actorder` | `babai` | 4 | 128 | c4 | | | `llama-2-7b-hf_gptaq_4bit_g128_calibc4_noqep_actorder` | `gptaq` | 4 | 128 | c4 | | | `llama-2-7b-hf_gptq_4bit_g128_calibc4_noqep_actorder` | `gptq` | 4 | 128 | c4 | | | `llama-2-7b-hf_gptq_4bit_g128_calibc4_qep_damp1.0_corr0.5_actorder` | `gptq` | 4 | 128 | c4 | | | `llama-2-7b-hf_quarot_gptq_4bit_g128_calibc4_noqep_actorder` | `quarot_gptq` | 4 | 128 | c4 | | | `llama-2-7b-hf_spinquant_gptq_4bit_g128_calibc4_noqep_actorder` | `spinquant_gptq` | 4 | 128 | c4 | | ### `Meta-Llama-3-8B/` | subdir | method | wbits | groupsize | calib | extra | |---|---|---|---|---|---| | `meta-llama-3-8b_admm_cd_4bit_g128_calibc4_noqep_actorder` | `admm_cd` | 4 | 128 | c4 | lam=0.1, iters=200 | | `meta-llama-3-8b_admm_cd_4bit_g128_calibc4_qep_damp1.0_corr0.5_actorder` | `admm_cd` | 4 | 128 | c4 | lam=0.1, iters=200 | | `meta-llama-3-8b_awq_4bit_g128_calibc4_noqep` | `awq` | 4 | 128 | c4 | | | `meta-llama-3-8b_babai_4bit_g128_calibc4_noqep_K5_actorder` | `babai` | 4 | 128 | c4 | | | `meta-llama-3-8b_gptaq_4bit_g128_calibc4_noqep_actorder` | `gptaq` | 4 | 128 | c4 | | | `meta-llama-3-8b_gptq_4bit_g128_calibc4_noqep_actorder` | `gptq` | 4 | 128 | c4 | | | `meta-llama-3-8b_gptq_4bit_g128_calibc4_qep_damp1.0_corr0.5_actorder` | `gptq` | 4 | 128 | c4 | | | `meta-llama-3-8b_quarot_gptq_4bit_g128_calibc4_noqep_actorder` | `quarot_gptq` | 4 | 128 | c4 | | | `meta-llama-3-8b_spinquant_gptq_4bit_g128_calibc4_noqep_actorder` | `spinquant_gptq` | 4 | 128 | c4 | | ### `Qwen3-0.6B/` | subdir | method | wbits | groupsize | calib | extra | |---|---|---|---|---|---| | `qwen3-0.6b_admm_cd_4bit_g128_calibc4_noqep_actorder` | `admm_cd` | 4 | 128 | c4 | lam=0.1, iters=200 | | `qwen3-0.6b_admm_cd_4bit_g128_calibc4_qep_damp1.0_corr0.5_actorder` | `admm_cd` | 4 | 128 | c4 | lam=0.1, iters=200 | | `qwen3-0.6b_awq_4bit_g128_calibc4_noqep` | `awq` | 4 | 128 | c4 | | | `qwen3-0.6b_babai_4bit_g128_calibc4_noqep_K0_actorder` | `babai` | 4 | 128 | c4 | | | `qwen3-0.6b_babai_4bit_g128_calibc4_noqep_K5_actorder` | `babai` | 4 | 128 | c4 | | | `qwen3-0.6b_gptaq_4bit_g128_calibc4_noqep_actorder` | `gptaq` | 4 | 128 | c4 | | | `qwen3-0.6b_gptq_4bit_g128_calibc4_noqep_actorder` | `gptq` | 4 | 128 | c4 | | | `qwen3-0.6b_gptq_4bit_g128_calibc4_qep_damp1.0_corr0.5_actorder` | `gptq` | 4 | 128 | c4 | | | `qwen3-0.6b_quarot_gptq_4bit_g128_calibc4_noqep_actorder` | `quarot_gptq` | 4 | 128 | c4 | | | `qwen3-0.6b_spinquantR_w4_external` | `spinquant_rotation_train` | 4 | ? | ? | | | `qwen3-0.6b_spinquant_gptq_4bit_g128_calibc4_noqep_actorder` | `spinquant_gptq` | 4 | 128 | c4 | | ### `Qwen3-4B/` | subdir | method | wbits | groupsize | calib | extra | |---|---|---|---|---|---| | `qwen3-4b_admm_cd_4bit_g128_calibc4_noqep_actorder` | `admm_cd` | 4 | 128 | c4 | lam=0.1, iters=200 | | `qwen3-4b_admm_cd_4bit_g128_calibc4_qep_damp1.0_corr0.5_actorder` | `admm_cd` | 4 | 128 | c4 | lam=0.1, iters=200 | | `qwen3-4b_awq_4bit_g128_calibc4_noqep` | `awq` | 4 | 128 | c4 | | | `qwen3-4b_babai_4bit_g128_calibc4_noqep_K5_actorder` | `babai` | 4 | 128 | c4 | | | `qwen3-4b_gptaq_4bit_g128_calibc4_noqep_actorder` | `gptaq` | 4 | 128 | c4 | | | `qwen3-4b_gptq_4bit_g128_calibc4_noqep_actorder` | `gptq` | 4 | 128 | c4 | | | `qwen3-4b_gptq_4bit_g128_calibc4_qep_damp1.0_corr0.5_actorder` | `gptq` | 4 | 128 | c4 | | | `qwen3-4b_quarot_gptq_4bit_g128_calibc4_noqep_actorder` | `quarot_gptq` | 4 | 128 | c4 | | | `qwen3-4b_spinquantR_w4_external` | `spinquant_rotation_train` | 4 | ? | ? | | | `qwen3-4b_spinquant_gptq_4bit_g128_calibc4_noqep_actorder` | `spinquant_gptq` | 4 | 128 | c4 | | ### `Qwen3-8B/` | subdir | method | wbits | groupsize | calib | extra | |---|---|---|---|---|---| | `qwen3-8b_admm_cd_4bit_g128_calibc4_noqep_actorder` | `admm_cd` | 4 | 128 | c4 | lam=0.1, iters=200 | | `qwen3-8b_admm_cd_4bit_g128_calibc4_qep_damp1.0_corr0.5_actorder` | `admm_cd` | 4 | 128 | c4 | lam=0.1, iters=200 | | `qwen3-8b_awq_4bit_g128_calibc4_noqep` | `awq` | 4 | 128 | c4 | | | `qwen3-8b_babai_4bit_g128_calibc4_noqep_K5_actorder` | `babai` | 4 | 128 | c4 | | | `qwen3-8b_gptaq_4bit_g128_calibc4_noqep_actorder` | `gptaq` | 4 | 128 | c4 | | | `qwen3-8b_gptq_4bit_g128_calibc4_noqep_actorder` | `gptq` | 4 | 128 | c4 | | | `qwen3-8b_gptq_4bit_g128_calibc4_qep_damp1.0_corr0.5_actorder` | `gptq` | 4 | 128 | c4 | | | `qwen3-8b_quarot_gptq_4bit_g128_calibc4_noqep_actorder` | `quarot_gptq` | 4 | 128 | c4 | | | `qwen3-8b_spinquantR_w4_external` | `spinquant_rotation_train` | 4 | ? | ? | | | `qwen3-8b_spinquant_gptq_4bit_g128_calibc4_noqep_actorder` | `spinquant_gptq` | 4 | 128 | c4 | | ## How to load (in our repo) ```bash python main.py {base_model} c4 admm_chol \ --wbits 4 --groupsize 128 \ --save-model '' \ --load-model / / /int_bundle.pt ``` `main.py` automatically picks up the sibling `aux_state.pt` from the same directory. ## Method legend - `gptq` — sequential per-column rounding with Hessian-aware error compensation - `admm_chol` — math-clean ADMM for per-column integer least squares (our method) - `awq` — activation-aware per-channel rescaling + RTN fake-quantize - `awq_gptq` — AWQ rescaling, then GPTQ as inner quantizer - `awq_admm_chol` — AWQ rescaling, then ADMM-chol as inner quantizer (our method) - `quip` — QUIP (random-rotation + LDLQ) at no-group setting
