Moirai 2 GGUF

Convert

Downloads the model from HuggingFace and writes a GGUF file:

# F16 (recommended)
./target/release/moirai-2-rs convert --model Salesforce/moirai-2.0-R-small --dtype f16 --output gguf/moirai2-f16.gguf

# Q8_0 (smallest)
./target/release/moirai-2-rs convert --dtype q8 --output gguf/moirai2-q8.gguf

# F32 (full precision)
./target/release/moirai-2-rs convert --dtype f32 --output gguf/moirai2-f32.gguf

To convert all dtypes at once:

./scripts/convert_all.sh

HuggingFace token (optional for public models):

HF_TOKEN=hf_... ./scripts/convert_all.sh

Inspect tensors

Print all tensor names and shapes from a .safetensors checkpoint:

./target/release/moirai-2-rs inspect-tensors models/model.safetensors

Infer

Run forecasting from stdin JSON:

echo '{"context": [1.0, 1.2, 1.5, 1.3, 1.8, 2.0, 1.9, 2.1], "horizon": 96}' \
  | ./target/release/moirai-2-rs infer --gguf gguf/moirai2-f16.gguf

Output is JSON in an OpenAI-compatible forecast format:

{
  "id": "forecast-000001932b7a1234",
  "object": "forecast",
  "created": 1749686400,
  "model": "moirai-2",
  "choices": [{
    "index": 0,
    "forecast": {
      "point": [2.1, 2.3, 2.5, "..."],
      "quantiles": {}
    },
    "finish_reason": "stop"
  }],
  "usage": {"context_length": 8, "forecast_length": 96}
}

Batch inference — pass multiple series as a nested array to get one Choice per series:

echo '{"context": [[1.0, 1.2, 1.5], [2.0, 2.2, 2.5]], "horizon": 96}' \
  | ./target/release/moirai-2-rs infer --gguf gguf/moirai2-f16.gguf

Multivariate inference — pass a 3D context array [batch][variate][time] to get a variates array in each choice. Each variate is processed independently (channel-independent):

echo '{
  "context": [
    [[1.0, 1.2, 1.5, 1.3, 1.8, 2.0, 1.9, 2.1],
     [0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2]]
  ],
  "horizon": 96
}' \
  | ./target/release/moirai-2-rs infer --gguf gguf/moirai2-f16.gguf

{
  "choices": [{
    "index": 0,
    "forecast": {
      "variates": [
        {"point": [2.1, 2.3, "..."], "quantiles": {}},
        {"point": [1.3, 1.4, "..."], "quantiles": {}}
      ]
    },
    "finish_reason": "stop"
  }]
}

Python bindings

Install with maturin inside a virtual environment:

python -m venv .venv && source .venv/bin/activate
pip install maturin
maturin develop --features python

import moirai_2_rs

model = moirai_2_rs.Moirai2("gguf/moirai2-f16.gguf")

result = model.forecast([1.0, 1.2, 1.5, 1.3, 1.8, 2.0], horizon=96)
point  = result["choices"][0]["forecast"]["point"]

# Batch — one Choice per series
result = model.forecast([[1.0, 1.2, 1.5], [2.0, 2.2, 2.5]], horizon=96)

forecast returns a Python dict in the same OpenAI-compatible format as the CLI.

Architecture notes

Moirai-2.0-R-small is a compact causal decoder:

• Backbone: Causal (decoder-only) transformer — unlike Moirai-1 which uses a bidirectional encoder; d_model=384, 6 layers, 6 heads
• Attention: Binary attention bias encoding variate identity (BinaryAttentionBias); partial interleaved RoPE applied to the first 32 of 64 head dimensions
• Decode loop: Predicts 4 patches (64 steps) per forward pass; runs multiple passes for longer horizons
• Normalization: PackedStdScaler with Bessel correction (ddof=1), min_scale=1e-5

moirai-2-rs/gguf/moirai2-q8.gguf

gguf · 11.5 MB · SHA-256 9b80a8eb04a8…afec · Hugging Face