Hermes 4 14B NVFP4
NVFP4-quantized version of NousResearch/Hermes-4-14B, an instruction-following and function-calling model built on Qwen/Qwen3-14B. It is not a newly trained model. This is an unofficial community quantization;...
Model source
- Kind
- Text model
- File size
- Not recorded yet
- Base model
- NousResearch/Hermes-4-14B
- Version
- vRevision 293c25d
- Updated
- Oct 3, 2026
- Publisher
- @tonythethompson
- Downloads
- 110
- Parameters
- 14B
- Tasks
- Text generation
- Source
- Hugging Face
Source description
Hermes-4-14B — NVFP4
Overview
NVFP4-quantized version of NousResearch/Hermes-4-14B, an instruction-following and function-calling model built on Qwen/Qwen3-14B. It is not a newly trained model. This is an unofficial community quantization; NousResearch is the original model author.
NVFP4 is NVIDIA's 4-bit floating-point quantization format. This variant targets high-throughput inference on NVIDIA hardware with NVFP4 acceleration. For details of the NVFP4 format and its hardware support, consult NVIDIA's TensorRT-LLM documentation.
Source
| Field | Value |
|---|---|
| Upstream model | NousResearch/Hermes-4-14B |
| Upstream source revision | d6ce765c8b83f847357b98254be079afa0c6ca76 |
| Export tool/script | NVIDIA TensorRT Model Optimizer (modelopt) 0.45.0 |
| Quantization recipe | NVFP4 (NVIDIA 4-bit floating point) safetensors |
| Base model (weights) | Qwen/Qwen3-14B |
Files
| File | Size | Description |
|---|---|---|
model-00001-of-00002.safetensors | ~8.99 GB | NVFP4 transformer weights: U8-packed 4-bit + FP8 (E4M3) group scales + FP32 global/input scales + BF16 norms |
model-00002-of-00002.safetensors | ~1.56 GB | BF16 embeddings / output head (excluded from NVFP4) |
model.safetensors.index.json | — | Weight map / shard index |
config.json | — | Model + quantization config |
hf_quant_config.json | — | modelopt NVFP4 quantization config |
generation_config.json | — | Generation defaults |
chat_template.jinja | — | Chat template |
tokenizer.json + tokenizer_config.json | — | Tokenizer |
.quant_summary.txt | — | Per-module quantization summary log |
Intended Use
A general instruction-following and function-calling LLM, quantized to NVFP4 for high-throughput inference on NVIDIA hardware. The NVFP4 variant targets NVIDIA GPUs with native FP4 tensor-core support; on other GPUs, NVFP4 weights may require emulation or upcast, which reduces the throughput benefit (verify with your TensorRT-LLM version).
Runtime Notes
- Library: Hugging Face Transformers (or TensorRT-LLM for NVFP4 acceleration).
- NVFP4 native acceleration requires NVIDIA GPU support; validate on the target hardware and TensorRT-LLM version before production use.
- Context length: inherited from Qwen3-14B / Hermes-4-14B; consult the upstream model card.
Precision and Packaging
Export tooling, precision, and quantization are recorded in the Source table above. This packaging mirror does not publish independent parity benchmarks; validate on your target execution provider before production use.
Limitations
- NVFP4 native acceleration is hardware-specific; not all NVIDIA GPUs support native FP4 tensor-core operations.
- Quantization introduces approximation error relative to the FP16/BF16 baseline.
- No repository-specific quality benchmark is documented here.
- English-dominant model; multilingual quality may vary.
License
Apache 2.0 — inherited from
NousResearch/Hermes-4-14B. This packaging repo adds no new license terms.
Sources
1 source
Sources
1 sourceVerified Oct 3
Source excerpts
3 excerpts
Source excerpts
3 excerpts--- license: apache-2.0 base_model: NousResearch/Hermes-4-14B base_model_relation: quantized pipeline_tag: text-generation library_name: transformers language: - en tags: - transformers - nvfp4 - quantized - text-generation --- # Hermes-4-14B — NVFP4 ## Overview NVFP4-quantized version of [NousResearch/Hermes-4-14B](https://huggingface.co/NousResearch/Hermes-4-14B), an instruction-following and function-calling model built on [Qwen/Qwen3-14B](https://huggingface.co/Qwen/Qwen3-14B). It is not a newly trained model. This is an unofficial community quantization; NousResearch is the original model author. NVFP4 is NVIDIA's 4-bit floating-point quantization format. This variant targets high-throughput inference on NVIDIA hardware with NVFP4 acceleration. For details of the NVFP4 format and its hardware support, consult NVIDIA's TensorRT-LLM documentation. ## Source | Field | Value | |---|---| | Upstream model | [NousResearch/Hermes-4-14B](https://huggingface.co/NousResearch/Hermes-4-14B) | | Upstream source revision | `d6ce765c8b83f847357b98254be079afa0c6ca76` | | Export tool/script | NVIDIA TensorRT Model Optimizer (modelopt) 0.45.0 | | Quantization recipe | NVFP4 (NVIDIA 4-bit floating point) safetensors | | Base model (weights) | [Qwen/Qwen3-14B](https://huggingface.co/Qwen/Qwen3-14B) | ## Files | File | Size | Description | |---|---|---| | `model-00001-of-00002.safetensors` | ~8.99 GB | NVFP4 transformer weights: U8-packed 4-bit + FP8 (E4M3) group scales + FP32 global/input scales + BF16 norms | | `model-00002-of-00002.safetensors` | ~1.56 GB | BF16 embeddings / output head (excluded from NVFP4) | | `model.safetensors.index.json` | — | Weight map / shard index | | `config.json` | — | Model + quantization config | | `hf_quant_config.json` | — | modelopt NVFP4 quantization config | | `generation_config.json` | — | Generation defaults | | `chat_template.jinja` | — | Chat template | | `tokenizer.json` + `tokenizer_config.json` | — | Tokenizer | | `.quant_summary.txt` | — | Per-module quantization summary log | ## Intended Use A general instruction-following and function-calling LLM, quantized to NVFP4 for high-throughput inference on NVIDIA hardware. The NVFP4 variant targets NVIDIA GPUs with native FP4 tensor-core support; on other GPUs, NVFP4 weights may require emulation or upcast, which reduces the throughput...
Source context: 110 downloads · 1 likes · Pipeline text-generation · Library transformers · Repo tonythethompson/Hermes-4-14B-NVFP4
Source context: 33 downloads · 0 likes · Pipeline text-generation · Library transformers · Repo tonythethompson/Hermes-4-14B-NVFP4
