ComfyUI_LocalLLMNodes

ComfyUI_LocalLLMNodes

A custom node pack for ComfyUI to run Large Language Models (LLMs) locally for flux kontext dev prompt generation within ComfyUI workflows.

Purpose: This pack is designed to simplify the creation of detailed, high-quality prompts for advanced image generation models like Flux Kontext Dev. It achieves this by using a locally running LLM (Large Language Model) to process:

1. An English image description generated by nodes like Florence-2 or Janus-Pro.
2. A simple, user-provided instruction (e.g., "Make it look like it's being used in a luxury spa"). The local LLM combines these inputs according to a selected preset to generate a single, complex prompt specifically tailored for Flux Kontext Dev.

Important Note: If the "Set Local GGUF LLM Service Connector 🐑" node does not appear in your node library after installation, ensure you have installed the llama-cpp-python library. This library is required for the GGUF connector node to function and may need to be installed manually with GPU support flags for optimal performance (see Installation).

This pack provides nodes to connect to and utilize local LLMs (like Llama, Phi, Gemma, Hermes in Hugging Face PyTorch format, or GGUF models) without needing external API calls. It integrates with prompt generation workflows, such as those using Florence-2, to simplify creating prompts for models like Flux Kontext Dev.

Key Features & Included Nodes

• Local LLM Execution (Hugging Face & GGUF): Run powerful LLMs directly on your machine (CPU or GPU) using standard Hugging Face models or efficient GGUF models.
• Set Local LLM Service Connector Node (HuggingFace): Select and configure your local Hugging Face format LLM model.
• Set Local GGUF LLM Service Connector Node: Select and configure your local GGUF format LLM model file. Includes dropdown for device selection (CPU/GPU) and n_gpu_layers slider.
• Local Kontext Prompt Generator Node: (Key Feature) Generates detailed image prompts by combining image descriptions and simple user instructions using a connected local LLM.
• User Preset Management: Add and remove custom prompt generation presets using dedicated nodes (AddUserLocalKontextPreset, RemoveUserLocalKontextPreset).
• VRAM Optimization Ready: Includes commented code examples for integrating quantization (4-bit/8-bit bitsandbytes for Hugging Face, or n_gpu_layers for GGUF) to reduce memory footprint.
• Simplified User Experience: Use simple requests (e.g., "make it look like it's being used in a luxury spa") which are translated into complex, Flux-ready prompts.

Installation

1. Navigate to your ComfyUI installation directory.
2. Go to the custom_nodes folder.
3. Clone this repository:

   git clone https://github.com/your_username/ComfyUI_LocalLLMNodes.git
   

4. Install Dependencies: You can install dependencies using pip and the provided scripts or requirements.txt.
   • Option 1: Using Installation Scripts (Recommended for GPU Support):
     • Navigate to the ComfyUI_LocalLLMNodes directory:

       cd ComfyUI_LocalLLMNodes
       

     • Linux/macOS:

       chmod +x install_deps.sh
       ./install_deps.sh
       

     • Windows (Command Prompt):

       install_deps.bat
       

     • Windows (PowerShell):

       Set-ExecutionPolicy RemoteSigned -Scope CurrentUser # If needed
       .\install_deps.ps1
       

     • These scripts install core dependencies and llama-cpp-python with CUDA support.
   • Option 2: Standard pip install:

     cd ComfyUI_LocalLLMNodes
     pip install -r requirements.txt
     

     *Note: