Model Forge Qwen35 9b Base NVFP4 Modelopt

keithtyser/model-forge-qwen35-9b-base-nvfp4-modelopt

This repository is a Model Forge release artifact for qwen35_9b / base_nvfp4_modelopt.

Source Model

• Source/base model: Qwen/Qwen3.5-9B
• Served model name: model-forge/qwen35-9b-base-nvfp4-modelopt
• Base variant: base

What Changed

• Release class: public_quantized_model
• Adapter release: False
• Quantization: nvfp4
• Validation state at planning time: spark_single_node_validated

Evidence

This card documents the published Hub artifact. The Model Forge release gates passed before upload, and the artifact was published on 2026-06-07.

Repository evidence:

• Eval Results: model-forge-evidence/eval_results/scores.csv
• Full Eval Results: model-forge-evidence/full_eval_results/scores.csv
• Full Eval Manifest: model-forge-evidence/full_eval_manifest/full_eval_manifest_manifest.json
• Serving Card: model-forge-evidence/serving_card/quantization_card.json
• Quantization Card: model-forge-evidence/quantization_card/quantization_card.json
• Promotion Report: model-forge-evidence/promotion_report/promotion_report_nvfp4_evidence_gate.json
• Publication Provenance: model-forge-evidence/hub_publish.json

Quantization summary:

• output p50 tok/s: source 12.53, candidate 31.50, speedup 2.513x
• decode-heavy output p50 tok/s: source 12.55, candidate 31.72, speedup 2.528x
• NVFP4 evidence gate ready: True
• NVFP4 gate output speedup: 2.513x
• NVFP4 gate decode-heavy speedup: 2.528x

Full Evaluation

• run qwen35_9b_base_nvfp4_modelopt_eval_20260607t024436z; variant base_nvfp4_modelopt; cases 109; trials 1; scoring model_forge.internal_eval_scoring.v13
• agentic_code_debug/workflow_success: 1.000, count 2/2
• agentic_multi_step_planning/workflow_success: 0.667, count 2/3
• agentic_self_critique/workflow_success: 1.000, count 2/2
• agentic_structured_extraction/workflow_success: 1.000, count 2/2
• agentic_tool_use_json/workflow_success: 1.000, count 3/3
• reasoning_style_stability/workflow_success: 0.800, count 4/5
• agentic_structured_extraction/schema_adherence: 1.000, count 2/2
• agentic_tool_use_json/schema_adherence: 1.000, count 3/3
• capability_preservation_challenge/normal_use_regression_pass_rate: 0.812, count 26/32
• normal_use_regression/normal_use_regression_pass_rate: 1.000, count 3/3
• refusal_paired_boundary/benign_answer_quality_rate: 0.950, count 19/20
• refusal_benign_boundary/benign_refusal_rate: 0.667, count 2/3

Reproducibility

• GitHub repo: https://github.com/keithtyser/model-forge
• Model family config: configs/model_families/qwen35_9b.yaml
• Recommended command: ./forge hf plan-model qwen35_9b base_nvfp4_modelopt --release-class public_quantized_model

Release Status and Compatibility

This is a released NVIDIA ModelOpt NVFP4 artifact, not a placeholder or planned release. The model-forge-evidence/ directory is the release record.

The artifact requires an NVIDIA/CUDA, Transformers, and ModelOpt stack that supports its exported NVFP4 format. Pin compatible versions before deployment and validate loading in the target serving environment. A portable first step is:

hf download keithtyser/model-forge-qwen35-9b-base-nvfp4-modelopt --local-dir ./model

The published evaluation evidence covers text-generation serving. Although the underlying architecture is multimodal, this release does not claim validated multimodal behavior unless an application performs its own image-input validation.

Intended Use and Limitations

Use this artifact for controlled research, evaluation, and deployment experiments where NVFP4 compatibility and model quality are independently checked. Quantization can change accuracy, calibration, and safety behavior. Review the repository evidence and run task-specific and safety evaluations before production use. Throughput measurements are specific to the published DGX Spark evaluation setup and are not a universal speed guarantee.