Companion Engine — v1.0

A local LLM writes the answer and stages it as a shot; MiniMax H3 renders the shot with native synced audio. Nothing leaves your machine — no API keys, no cloud calls.

How it works

You type a question.

Ollama sends it to a local vision model along with the avatar image, so the avatar can see what itself looks like.

The LLM replies in JSON: a scene description of itself speaking, plus how many seconds it needs.

The scene description becomes the MiniMax H3 prompt; the duration is converted to a valid frame count.

MiniMax H3 renders the avatar speaking, with the voice cloned from your audio sample.

The conversation is remembered between runs, so you can keep talking to it.

What you get

Everything lives inside one Companion Engine subgraph. On the canvas you only touch:

Avatar The face. Doubles as for MiniMax and as the image the LLM sees. Voice sample The voice to clone ( ). A few seconds of clean speech is enough. Ollama server URL and model. Defaults to a local Ollama at 127.0.0.1:11434 . Your question The only field you change between runs. Resolution Output size, with a size reference note on the canvas. Four loaders Diffusion model, text encoder, video VAE, audio VAE.

Three outputs: the video, the LLM's raw JSON, and the exact prompt MiniMax received — the last two make debugging a bad take trivial.

Be aware that the videos take time to generate, my companion is in the outer rim of the solar system and its answers take some time to reach.

Requirements

ComfyUI , recent enough for MiniMaxH3ReferenceToVideo and subgraph support.

comfyui-ollama — the only custom node pack needed. Available in the ComfyUI Manager.

Ollama running with a vision-capable model pulled. The avatar has to be able to see its own picture, so a text-only model will hallucinate its appearance.

Every other node is ComfyUI core. No KJNodes, no Easy-Use, no attention patches, no cache nodes, no upscalers. That is deliberate: it should load for everyone and be easy to extend with whatever speed tricks your hardware likes. I'd suggest you apply as many optimisations as you can, but as these change rapidly I decided not to include them.

You need to be able to run Ollama and Minimax H3, so the VRAM+RAM requirements might be steep. I am running with 16GB VRAM + 64 RAM and each answer takes ten minutes or so to generate for me.

Models

From Comfy-Org/MiniMax-H3 :

ComfyUI/models/ ├── diffusion_models/ │ └── minimax_h3_ref2va_pruned_int8_convrot.safetensors ├── text_encoders/ │ └── qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors └── vae/ ├── minimax_h3_video_vae_fp16.safetensors └── minimax_h3_audio_vae_fp32.safetensors

Note this is the ref2va checkpoint, not the fl2va one used by the t2v/i2v templates.

Setup

Install comfyui-ollama and download the four model files above.

Start Ollama and pull a vision model.

Load the workflow. In the Ollama server node, hit refresh and pick your model — the field ships empty on purpose.

Load your avatar image and a voice sample.

Type a question and queue it.

Notes and tips

Memory : keep_context is on inside the subgraph. The avatar remembers previous turns for as long as ComfyUI stays open. The context window is not very big.

The LLM decides the length. It reports how many seconds it needs and a Math Expression converts that to frames. MiniMax H3 only accepts lengths where length % 17 == 5 — 5 s is 124 frames, 10 s is 243, 15 s is 362. The system prompt caps it at 10 s; raise it if you want longer takes.

Structured output, not regex. Ollama runs in JSON mode and the keys are pulled with Extract Text from JSON . Parsing free text here is a trap: the duration line ends up inside the video prompt and MiniMax tries to render it.

Bad take? Check the Answer (raw JSON) output first. Most misfires are the LLM drifting from the format, and that is fixable by editing the system prompt inside the subgraph.

Prompt wording matters a lot with ref2va. Keep the and tags and be explicit about which reference drives what.

Upgrade path : if your comfyui-ollama is recent, the Ollama Chat node handles history with proper roles and has a reset_session toggle. Drop-in improvement over Ollama Generate for this use case.

Credits

Built on the official ComfyUI MiniMax H3 ref2va template. LLM nodes by stavsap .