--- license: other license_name: nvidia-open-model-license license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/ language: - en library_name: mlx pipeline_tag: image-text-to-text tags: - nemotron - nemotron-h - mamba - mamba2 - ssm - mixture-of-experts - multimodal - vision - audio - video - speech - omni - reasoning - mlx - jang - JANGTQ2 - apple-silicon base_model: nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 ---  # Nemotron-3-Nano-Omni-30B-A3B-Reasoning · JANGTQ2 **12.6 GB** · **~85 tok/s** decode on M4 Max · 30B / 3B-active hybrid Mamba-2 + Attention + MoE · **native MLX, zero PyTorch in the hot path** Full multimodal (text + image + audio + video) port of NVIDIA's [Nemotron-3-Nano-Omni-30B-A3B-Reasoning](https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16) to Apple MLX, all four modalities running natively on Metal: | Modality | Native MLX time | vs PyTorch hybrid | |---|---:|---:| | Text | ... + answer chat.turn("Solve: 17 + 28 = ?", enable_thinking=True) # Reasoning OFF: faster, more direct chat.turn("Solve: 17 + 28 = ?", enable_thinking=False) ``` ### Text-only fast path (mlx_lm or load_jangtq) For chat-only use cases, skip the multimodal load and use the LLM directly: ```python from mlx_lm import load, generate model, tokenizer = load("JANGQ-AI/Nemotron-3-Nano-Omni-30B-A3B-JANGTQ2") prompt = tokenizer.apply_chat_template( [{"role": "user", "content": "Capital of France?"}], tokenize=False, add_generation_prompt=True, ) print(generate(model, tokenizer, prompt=prompt, max_tokens=20)) # "Paris." ``` (For JANGTQ4 / JANGTQ2: replace `mlx_lm.load` with `from jang_tools.load_jangtq import load_jangtq_model`. Vision/sound weights are silently dropped on the text-only path.) ## Architecture (52 hybrid layers) ``` hybrid_override_pattern (52 chars): "MEMEM*EMEMEM*EMEMEM*EMEMEM*EMEMEM*EMEMEMEM*EMEMEMEME" 23 × M = Mamba-2 SSM (state-space, O(1) cache per token) 23 × E = MoE (128 routed × 6) (ReLU2 activation, no gate_proj) 6 × * = Attention (GQA 32q / 2kv heads, NO RoPE, head_dim=128) Multimodal towers (fp16, native MLX): vision_model → RADIO ViT (NVIDIA C-RADIOv2-H) mlp1 → LayerNorm + Linear + GELU + Linear → llm_hidden sound_encoder → ParakeetEncoder (24 Conformer layers) sound_projection → RMSNorm + Linear + SquaredReLU + Linear → llm_hidden Cache (multi-turn): M layers → MambaCache (size=2: conv state + ssm state) O(1)/token * layers → KVCacheSimple O(L)/token E layers → stateless Native context: 262 144 tokens (no RoPE extrapolation needed) ``` ## Special tokens | Token | ID | Purpose | |---|---:|---| | ` ` | 18 | Image / video patch placeholder (video reuses ) | | ` ` | 27 | Audio frame placeholder | | ` ` ... ` ` | — | Image/video region wrapper | | ` ` ... ` ` | — | Audio region wrapper | | ` ` | 11 | EOS for chat | ## Sampling guidance | Mode | temperature | top_p | When | |---|---|---|---| | Greedy | 0.0 | — | Deterministic; reasoning-correct | | **Recommended** | 0.6 | 0.95 | DeepSeek-style sampler, balanced | | Avoid | 1.0 | 1.0 | At 2-bit (JANGTQ2): flat logit + quant noise → garbage tokens | ## Swift / vMLX support Native Swift port is in [`vmlx-swift-lm`](https://github.com/vmlx/vmlx-swift-lm) under `Libraries/MLXVLM/Models/NemotronHOmni/`. The full multimodal pipeline (NemotronHOmni wrapper + RADIOVision + Parakeet + Projectors + image/audio/video preprocessors) compiles cleanly. Shared video utilities in `Libraries/MLXVLM/VLMVideoUtils.swift` are reused by Qwen 2/2.5/3/3.5/3.6 VL and Kimi VL for cross-VLM compatibility. ```swift import MLXVLM let frames = try await vlmExtractFramesUniform(url: videoURL, targetFrames: 32) let pixels = vlmStackFramesIntoChannels(frames, imageSize: 512, temporalPatchDim: 2) // → MLXArray (n_groups, T*3=6, 512, 512) for RADIO video_embedder ``` ## License NVIDIA Open Model License — see the [base model](https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16) for full terms. Quantization, conversion, native MLX port, and runtime by [Jinho Jang](https://huggingface.co/JANGQ-AI) (eric@jangq.ai). --- 🦖 [**Osaurus**](https://github.com/dinoki-ai/osaurus) is the open-source MLX inference server for Apple Silicon. 🌀 [**JANG**](https://huggingface.co/JANGQ-AI) is the quantization + runtime stack powering this bundle.
