llamacpp-comfyui-autocaptioner

🦙 ComfyUI-LlamaCpp-Caption

ComfyUI custom node for captioning images using a local llama.cpp server with vision/multimodal support.

Features

• 🦙 LlamaCpp Caption — Send an image to your llama.cpp server and get back a clean caption
• Auto CoT removal — Strips <think>, <reasoning>, and <thought> blocks from reasoning model outputs
• JoyCaption compatible — Designed to slot into the ImageBatchPath → Caption → CaptionSaver workflow
• Batch support — When connected to JoyCaption's Image Batch Path (which outputs lists), ComfyUI auto-iterates and processes each image individually
• OpenAI-compatible — Works with any llama.cpp server running a vision model (Qwen-VL, Gemma 3, LLaVA, etc.)

Requirements

• A llama.cpp server running with a vision-capable model:

  # Example with a single GGUF that includes the projector:
  ./llama-server -m your-vision-model.gguf
  
  # Or with a separate projector:
  ./llama-server -m your-model.gguf --mmproj your-mmproj.gguf
  
  # Or from Hugging Face:
  ./llama-server -hf ggml-org/gemma-3-4b-it-GGUF
  

Installation

Option 1: Symlink (recommended for development)

# From your ComfyUI directory:
ln -s /path/to/llama_api_comfy custom_nodes/llama_api_comfy

Option 2: Copy

cp -r /path/to/llama_api_comfy custom_nodes/llama_api_comfy

Then install dependencies:

pip install requests Pillow numpy

Note: torch is already provided by ComfyUI's environment.

Node: 🦙 LlamaCpp Caption

Input	Type	Default	Description
image	IMAGE	(required)	The image to caption
prompt	STRING	"Describe this image in detail."	The prompt to send with the image
server_url	STRING	http://localhost:8080	URL of the llama.cpp server
temperature	FLOAT	0.7	Sampling temperature (0.0–2.0)
max_tokens	INT	1024	Maximum tokens in the response
system_prompt	STRING	(optional)	System prompt for the model
image_path	STRING	(optional, from ImageBatchPath)	Passthrough for CaptionSaver

Output	Type	Description
caption	STRING	The generated caption (CoT stripped)
image_path	STRING	Passthrough of input image_path

Usage with JoyCaption Nodes

This node is designed to work with ComfyUI-JoyCaption:

┌──────────────────┐       ┌─────────────────────┐       ┌───────────────┐
│ Image Batch Path │       │ 🦙 LlamaCpp Caption │       │ Caption Saver │
│                  │       │                     │       │               │
│        IMAGE ●───┼──────▶┼─● image             │       │               │
│   IMAGE_PATH ●───┼──────▶┼─● image_path        │       │               │
│                  │       │                     │       │               │
│                  │       │     caption ●───────┼──────▶┼─● string      │
│                  │       │  image_path ●───────┼──────▶┼─● image_path  │
└──────────────────┘       └─────────────────────┘       └───────────────┘

1. Image Batch Path loads images from a folder (outputs lists)
2. 🦙 LlamaCpp Caption receives each image + its file path, captions it, and passes both through
3. Caption Saver saves each caption as a .txt file matching the original image filename

ComfyUI auto-iterates over the list outputs from Image Batch Path, so every image in the batch gets captioned automatically.

License

MIT