Ir para o conteúdo
ModelVerificadoExterno
Fonte aberta

Zenyx v2 Instruct

Zenyx-v2 Instruct is the supervised fine-tuned instruction-following variant of the Zenyx v2 base model. It is trained with a strict assistant-only masking setup and a ChatML-style conversation template for dialogue,...

Fontehuggingface.co/Arko007/zenyx-v2-instruct
Tipotext-generationModelo baseArko007/zenyx-v2-basePublicador@Arko007Cnota

Fonte do modelo

Tipo
text-generation
Tamanho do arquivo
162–487 MB
Modelo base
Arko007/zenyx-v2-base
Atualizado
14 de out. de 2026
Publicador
@Arko007
Tarefas
Geração de texto
Fonte
Hugging Face

Descrição da fonte

Zenyx-v2 Instruct

Zenyx-v2 Instruct is the supervised fine-tuned instruction-following variant of the Zenyx v2 base model. It is trained with a strict assistant-only masking setup and a ChatML-style conversation template for dialogue, instruction following, code, reasoning, and tool-oriented training data. The current release is still in active training, so evaluation metrics, benchmark scores, and final qualitative results will be added later.

Model summary

  • Base model: Arko007/zenyx-v2-base
  • Tokenizer: Arko007/zenyx-v2-tokenizer
  • Model family: Zenyx v2
  • Training stage: Supervised fine-tuning
  • Context length: 8,192 tokens
  • Architecture: Recurrent Transformer-style block with MLAAttention, ConvSwiGLU, RMSNorm, YaRN RoPE scaling, and tied input/output embeddings
  • Framework: JAX + Flax + Optax
  • Target hardware: TPU v5e-8

Intended use

This checkpoint is intended for:

  • chat and instruction following
  • reasoning and math-heavy responses
  • code generation and code repair
  • tool-oriented conversational workflows
  • long-context generation within the supported sequence length

Training data

The SFT pipeline uses a large mixed instruction corpus assembled from multiple public datasets, including reasoning, math, code, tool use, safety, and conversation sources. The training script currently registers 57 dataset streams, with examples such as NVIDIA Nemotron SFT blends, OpenMath/MathInstruct-style corpora, code-reasoning datasets, agentic/tool-calling data, and conversational instruction sets. Data is normalized into a unified message format and converted to ChatML turns before packing into fixed-length blocks. fileciteturn0file0

Architecture

Zenyx-v2 Instruct uses a compact but high-capacity recurrent block design:

  • token embeddings tied to the final logits layer
  • MLA-style attention with separate query and key/value latent projections
  • grouped key/value heads
  • depthwise causal convolution inside the feed-forward path via ConvSwiGLU
  • RMSNorm throughout the stack
  • YaRN-scaled rotary positional embeddings for extended context stability
  • repeated application of a small set of unique blocks across multiple recurrences

The script instantiates the model with 576 hidden width, 9 attention heads, 3 KV heads, 8 unique blocks, 4 recurrences, and 3 auxiliary MTP heads. fileciteturn0file0

Training setup

The SFT recipe in the script uses:

  • global batch size: 128 sequences
  • per-core batch size: 1
  • gradient accumulation: 16
  • learning rate: 5e-5
  • minimum LR: 1e-6
  • AdamW optimizer with beta1 0.9, beta2 0.95, epsilon 1e-8, and weight decay 0.01
  • warmup: 500 steps
  • cosine schedule horizon: 50,000 steps
  • evaluation every 500 steps
  • checkpointing every 500 steps

The loss is computed with assistant-only masking, so user and system tokens are excluded from optimization targets. fileciteturn0file0

Training status

This model is still in training. Final metrics, validation curves, benchmark numbers, and comparative evals will be added once training completes.

How to use

Use the tokenizer’s ChatML template and the model’s expected special tokens:

messages = [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Explain backpropagation simply."},
]

For best results, format prompts as conversation turns rather than plain text completions.

Limitations

  • Final benchmark quality is not yet known because training is ongoing.
  • Performance may vary across math, code, safety, and tool-use tasks depending on the data mixture and training progress.
  • The model was trained with a long-context setup, but practical results on very long prompts should still be validated after training completes.

Safety

This model may generate incorrect, incomplete, or overly confident responses. Treat outputs as assistance, not ground truth. Apply application-level filtering and human review where needed.

Citation

If you use this checkpoint in a publication, project, or demo, cite the Zenyx-v2 Instruct repository and include the final model card once metrics are available.

Status note

Metrics, evaluation tables, and sample generations will be appended after the current training run finishes.

Fontes

1 fonte
Hugging FacePrimário
huggingface.co/Arko007/zenyx-v2-instruct

Verificado 17 de set.

Artefatos de modelo

64 artefatos

params/step_0000500.msgpack

msgpack · 162 MB · SHA-256 9eb588690959…4137 · Hugging Face

params/step_0001000.msgpack

msgpack · 162 MB · SHA-256 59fbf95e49b0…a740 · Hugging Face

params/step_0001500.msgpack

msgpack · 162 MB · SHA-256 e7d5550b7ca0…1099 · Hugging Face

params/step_0002000.msgpack

msgpack · 162 MB · SHA-256 3a7ae5fce15d…87d7 · Hugging Face

params/step_0002500.msgpack

msgpack · 162 MB · SHA-256 9b867a671526…8e76 · Hugging Face

params/step_0003000.msgpack

msgpack · 162 MB · SHA-256 2b5dff59c945…057b · Hugging Face

params/step_0003500.msgpack

msgpack · 162 MB · SHA-256 a25c6d220807…8e37 · Hugging Face

params/step_0004000.msgpack

msgpack · 162 MB · SHA-256 df6a09f55706…9eb4 · Hugging Face

params/step_0004500.msgpack

msgpack · 162 MB · SHA-256 6c19a3908246…c4eb · Hugging Face

params/step_0005000.msgpack

msgpack · 162 MB · SHA-256 e1625e0b0129…46a9 · Hugging Face

params/step_0005500.msgpack

msgpack · 162 MB · SHA-256 d16731aefb4b…2d86 · Hugging Face

params/step_0006000.msgpack

msgpack · 162 MB · SHA-256 8550ee5879de…aa0d · Hugging Face

params/step_0006500.msgpack

msgpack · 162 MB · SHA-256 797937f79528…d3a9 · Hugging Face

params/step_0007000.msgpack

msgpack · 162 MB · SHA-256 fb18774c8842…532f · Hugging Face

params/step_0007500.msgpack

msgpack · 162 MB · SHA-256 eb10cdb183fc…12ad · Hugging Face

params/step_0008000.msgpack

msgpack · 162 MB · SHA-256 aca531de5851…5b26 · Hugging Face

params/step_0008500.msgpack

msgpack · 162 MB · SHA-256 1a9757373f10…8836 · Hugging Face

params/step_0009000.msgpack

msgpack · 162 MB · SHA-256 9cded5cdd1e5…f4db · Hugging Face

params/step_0009500.msgpack

msgpack · 162 MB · SHA-256 55c43edb9670…2c53 · Hugging Face

params/step_0010000.msgpack

msgpack · 162 MB · SHA-256 c462bc5a5f7e…6d1f · Hugging Face

params/step_0010500.msgpack

msgpack · 162 MB · SHA-256 f1472b51b54a…e51d · Hugging Face

params/step_0011000.msgpack

msgpack · 162 MB · SHA-256 1859d8542483…435b · Hugging Face

params/step_0011500.msgpack

msgpack · 162 MB · SHA-256 c1a9f0d27d48…bad8 · Hugging Face

params/step_0012000.msgpack

msgpack · 162 MB · SHA-256 28ad606e4a45…e7ff · Hugging Face

params/step_0012500.msgpack

msgpack · 162 MB · SHA-256 ff3770062832…3d55 · Hugging Face

params/step_0013000.msgpack

msgpack · 162 MB · SHA-256 77ea45ac8f25…5a81 · Hugging Face

params/step_0013500.msgpack

msgpack · 162 MB · SHA-256 96f30950d484…50ba · Hugging Face

params/step_0014000.msgpack

msgpack · 162 MB · SHA-256 3064eaff6bdc…9a2e · Hugging Face

params/step_0014500.msgpack

msgpack · 162 MB · SHA-256 d64630cb0fb0…5b07 · Hugging Face

params/step_0015000.msgpack

msgpack · 162 MB · SHA-256 24b4d1a25019…da70 · Hugging Face

params/step_0015500.msgpack

msgpack · 162 MB · SHA-256 7f9d6d6c1aaf…27d9 · Hugging Face

params/step_0016000.msgpack

msgpack · 162 MB · SHA-256 686b1c811c4a…b44c · Hugging Face

params/step_0016500.msgpack

msgpack · 162 MB · SHA-256 e58655aee1fc…0bf9 · Hugging Face

params/step_0017000.msgpack

msgpack · 162 MB · SHA-256 fa9cf3e3c8b8…ecc6 · Hugging Face

params/step_0017500.msgpack

msgpack · 162 MB · SHA-256 451fd84e2efc…c088 · Hugging Face

params/step_0018000.msgpack

msgpack · 162 MB · SHA-256 a7e41debcdb8…e8b3 · Hugging Face

params/step_0018500.msgpack

msgpack · 162 MB · SHA-256 4f8025cd9669…22dd · Hugging Face

params/step_0019000.msgpack

msgpack · 162 MB · SHA-256 ac0d36cff0a8…1394 · Hugging Face

params/step_0019500.msgpack

msgpack · 162 MB · SHA-256 3fb18a4880d9…9b52 · Hugging Face

params/step_0020000.msgpack

msgpack · 162 MB · SHA-256 2d957709852b…b1b5 · Hugging Face

params/step_0020500.msgpack

msgpack · 162 MB · SHA-256 3fbc99747f74…dbc6 · Hugging Face

params/step_0021000.msgpack

msgpack · 162 MB · SHA-256 3ec86fca13ea…86bd · Hugging Face

params/step_0021500.msgpack

msgpack · 162 MB · SHA-256 8085b7d49afe…b77c · Hugging Face

params/step_0022000.msgpack

msgpack · 162 MB · SHA-256 7c275cb214ba…7908 · Hugging Face

params/step_0022500.msgpack

msgpack · 162 MB · SHA-256 2d766dc7d716…48d6 · Hugging Face

params/step_0023000.msgpack

msgpack · 162 MB · SHA-256 6bbe94cb5b37…e1f8 · Hugging Face

params/step_0023500.msgpack

msgpack · 162 MB · SHA-256 e04afc5744c4…32e3 · Hugging Face

params/step_0024000.msgpack

msgpack · 162 MB · SHA-256 3ea909ae9174…0b73 · Hugging Face

params/step_0024500.msgpack

msgpack · 162 MB · SHA-256 f9960207af55…64ae · Hugging Face

params/step_0025000.msgpack

msgpack · 162 MB · SHA-256 252eb8cf31c5…6fe1 · Hugging Face

params/step_0025500.msgpack

msgpack · 162 MB · SHA-256 5f2113b2509f…fe69 · Hugging Face

params/step_0026000.msgpack

msgpack · 162 MB · SHA-256 60b86788ef96…4150 · Hugging Face

params/step_0026500.msgpack

msgpack · 162 MB · SHA-256 993056b65058…fc04 · Hugging Face

params/step_0027000.msgpack

msgpack · 162 MB · SHA-256 6a4f038ff797…595d · Hugging Face

params/step_0027500.msgpack

msgpack · 162 MB · SHA-256 882aea53c2aa…aab3 · Hugging Face

params/step_0028000.msgpack

msgpack · 162 MB · SHA-256 db945b7ccf18…8614 · Hugging Face

params/step_0028500.msgpack

msgpack · 162 MB · SHA-256 641958085506…ad70 · Hugging Face

params/step_0029000.msgpack

msgpack · 162 MB · SHA-256 592697005302…fb4a · Hugging Face

state/step_0000500.msgpack

msgpack · 487 MB · SHA-256 4fbed4883e44…ddfe · Hugging Face

state/step_0001000.msgpack

msgpack · 487 MB · SHA-256 488de9bf828b…94b6 · Hugging Face

state/step_0001500.msgpack

msgpack · 487 MB · SHA-256 d7b27b8f2755…d5e0 · Hugging Face

state/step_0002000.msgpack

msgpack · 487 MB · SHA-256 fdfceba465ab…1833 · Hugging Face

state/step_0002500.msgpack

msgpack · 487 MB · SHA-256 fb87c9ecb24b…df2c · Hugging Face

state/step_0003000.msgpack

msgpack · 487 MB · SHA-256 393a27ac73cc…91b3 · Hugging Face

Trechos de fonte

3 trechos
Readme11 de jul.huggingface.co/Arko007/zenyx-v2-instruct

--- language: [en] license: apache-2.0 library_name: flax pipeline_tag: text-generation tags: - zenyx - instruct - sft - jax - flax - tpu - chatml - long-context - yarn - multi-query-attention - recurrent-transformer - tool-use base_model: Arko007/zenyx-v2-base model-index: - name: Zenyx-v2 Instruct results: [] --- # Zenyx-v2 Instruct Zenyx-v2 Instruct is the supervised fine-tuned instruction-following variant of the Zenyx v2 base model. It is trained with a strict assistant-only masking setup and a ChatML-style conversation template for dialogue, instruction following, code, reasoning, and tool-oriented training data. The current release is still in active training, so evaluation metrics, benchmark scores, and final qualitative results will be added later. ## Model summary - **Base model:** `Arko007/zenyx-v2-base` - **Tokenizer:** `Arko007/zenyx-v2-tokenizer` - **Model family:** Zenyx v2 - **Training stage:** Supervised fine-tuning - **Context length:** 8,192 tokens - **Architecture:** Recurrent Transformer-style block with MLAAttention, ConvSwiGLU, RMSNorm, YaRN RoPE scaling, and tied input/output embeddings - **Framework:** JAX + Flax + Optax - **Target hardware:** TPU v5e-8 ## Intended use This checkpoint is intended for: - chat and instruction following - reasoning and math-heavy responses - code generation and code repair - tool-oriented conversational workflows - long-context generation within the supported sequence length ## Training data The SFT pipeline uses a large mixed instruction corpus assembled from multiple public datasets, including reasoning, math, code, tool use, safety, and conversation sources. The training script currently registers 57 dataset streams, with examples such as NVIDIA Nemotron SFT blends, OpenMath/MathInstruct-style corpora, code-reasoning datasets, agentic/tool-calling data, and conversational instruction sets. Data is normalized into a unified message format and converted to ChatML turns before packing into fixed-length blocks. fileciteturn0file0 ## Architecture Zenyx-v2 Instruct uses a compact but high-capacity recurrent block design: - token embeddings tied to the final logits layer - MLA-style attention with separate query and key/value latent projections - grouped key/value heads - depthwise causal convolution inside the feed-forward path via ConvSwiGLU - RMSNorm throughout the stack -...

Note11 de jul.huggingface.co/Arko007/zenyx-v2-instruct

Source context: 0 downloads · 0 likes · Pipeline text-generation · Library flax · Repo Arko007/zenyx-v2-instruct

Note21 de abr.huggingface.co/api/models/Arko007/zenyx-v2-instruct

Source context: 0 downloads · 0 likes · Repo Arko007/zenyx-v2-instruct