comfyui-llamacpp-client

ComfyUI Llama.cpp Client Node

A comprehensive ComfyUI custom node that provides complete client functionality for llama-server from llama.cpp. This node acts as a bridge between ComfyUI workflows and llama-server instances, supporting every single parameter and endpoint that llama-server offers.

License Python ComfyUI llama.cpp

🎯 What This Repository Does

This repository provides a single, powerful ComfyUI node that can communicate with any llama-server instance and utilize all of its capabilities within ComfyUI workflows. Instead of being limited to basic text generation, you get access to:

• 8 Complete API Endpoints: Every llama-server endpoint with full parameter support
• 100+ Parameters: Every single parameter that llama-server accepts
• Advanced AI Features: Function calling, multimodal processing, structured output, and more
• Production Ready: Error handling, authentication, caching, and performance optimization
• Plug & Play: Easy integration into existing ComfyUI workflows

🚀 Key Capabilities

Complete API Coverage

Endpoint	Purpose	What You Can Do
/completion	Text generation	Stories, articles, creative writing, Q&A
/v1/chat/completions	Chat conversations	Multi-turn conversations, roleplay, assistants
/v1/embeddings	Text embeddings	Semantic search, clustering, similarity analysis
/tokenize	Text analysis	Token counting, text preprocessing
/detokenize	Token conversion	Debug tokenization, convert tokens back
/apply-template	Chat formatting	Test chat templates, format conversations
/infill	Code completion	Fill-in-the-middle coding, code assistance
/v1/rerank	Document ranking	Search relevance, document sorting

Advanced Sampling Methods

• DRY Sampling: Eliminates repetition with smart sequence detection
• XTC Sampling: Cross-token coherence for better consistency
• Mirostat: Perplexity-based sampling for controlled creativity
• Dynamic Temperature: Adaptive temperature that changes during generation
• Custom Sampler Chains: Define your own sampling pipeline

Structured Output & Constraints

• JSON Schema: Force valid JSON output with custom schemas
• BNF Grammar: Use formal grammars to constrain generation
• Logit Bias: Fine-tune token probabilities for specific words/concepts
• Stop Sequences: Precise control over when generation stops

Multimodal & Function Calling

• Vision Models: Process images with base64 encoding
• Function Calling: Let models call tools and functions
• Tool Integration: OpenAI-compatible function definitions
• Image References: Embed images directly in prompts

Performance & Production Features

• KV Cache Management: Reuse computations between requests
• Slot Management: Handle concurrent requests efficiently
• LoRA Adapters: Dynamic model adaptation per request
• Streaming: Real-time token generation
• Authentication: API key support for secure deployments

📦 Installation

Prerequisites

• ComfyUI installed and running
• llama-server (from llama.cpp) running somewhere accessible
• Python 3.7+ with pip

Install the Node

1. Clone into ComfyUI custom nodes:

   cd /path/to/ComfyUI/custom_nodes
   git clone https://github.com/fidecastro/comfyui-llamacpp-client.git
   

2. Install dependencies:

   cd comfyui-llamacpp-client
   pip install -r requirements.txt
   

3. Restart ComfyUI - The node will appear in AI/LlamaCpp category

Start Your llama-server

For basic functionality:

./llama-s