Zum Inhalt springen
WorkflowAufgelistetExtern
Quelle öffnen

Mugen Jina IP Adapter

368M parameters Trained on Mugen w/Jina with Jina-clip-v2 + Jina-clip-v2 adapter as the text encoder. However, it also works on Mugen with the original CLIP-G + CLIP-L text encoders. The outputs are slightly...

Quellehuggingface.co/TheRemixer/Mugen-Jina-IP-Adapter/tree/340646dbf85254538e49a26437cf60cb9e09199e
AufgabeText zu BildVersionRelease 340646dbf85254538e49a26437cf60cb9e09199e — Revision 340646dGeschätzte VRAMNot yet characterizedDBewertung

Laufzeitprofil

Zugriff
Externer Arbeitsablauf
Status
Aufgelistet
Version
Revision 340646d
Quelle
Hugging Face
Aktualisiert
18. Okt. 2026

Quellenbeschreibung

IP-Adapter using Jina-clip-v2's vision tower, and to be used on Mugen

Quick Notes:

  • 368M parameters
  • Trained on Mugen w/Jina with Jina-clip-v2 + Jina-clip-v2 adapter as the text encoder.
    • However, it also works on Mugen with the original CLIP-G + CLIP-L text encoders. The outputs are slightly different.
  • Trained for 5 epochs with 208,000 unbatched steps in total, at a base resolution 1024x1024.
    • Started showing promising results from the 2nd epoch. I have included the previous epochs in this repo if you are interested in testing them.

Installation:

Usage:

  • See the workflow included in this repo on how to use the IP-adapter.
  • I mainly tested the IP-adapter as a style transfer, however it can do other things too.
  • Try different weighting if the default of 1.0 is too much. The default of 1.0 isn't recommened in all cases.

Credits:

What you need

Hardware and download-size estimates are not available yet.

Anforderungen

Für diese Workflow-Variante sind keine verifizierten Anforderungen verfügbar. Ohne Nachweis auf Dateiebene wird nichts als gefunden markiert.

KI-generierter Kommentar

KI-generierte Erklärung auf Grundlage von Quellen- und Konfigurationsdetails. Vorschläge sind klar gekennzeichnet.

Zweck

  • Mugen Jina IP Adapter is a ComfyUI workflow for reference-guided image generation: it combines positive and negative prompts, a reference image, and the `MugenJinaUNET.safetensors` checkpoint with a Jina IP-Adapter, then saves the generated image.

Fluss

  • It encodes the prompts with Jina text encoders, applies the Jina IP-Adapter to the checkpoint using the reference image, samples and decodes an image, then saves it.

Eingaben und Ausgaben

  • The starter workflow loads `workflow_input_image.jpg`, has separate positive and negative prompt fields, and sets the initial latent image size to 832 × 1216.

  • Saved images use a date-based folder and the `mugen_` filename prefix.

Einrichtung

  • Install the custom ComfyUI nodes from `lRemixl/ComfyUI_JinaCLIP_SDXL_IP_Adapter`; place `Mugen-Jina-IP-Adapter.safetensors` in `ComfyUI/models/ip_adapter/` and Jina-clip-v2 files in a new folder under `ComfyUI/models/LLM/`.

  • For the Jina setup, the guide also lists `MugenJinaUNET.safetensors` and `jina-clip-v2-adapter.safetensors`; it offers Mugen as an alternative.

Abhängigkeiten

  • The workflow requires the ComfyUI node packs `comfyui-kjnodes` 1.3.7 and `RES4LYF`, pinned to `7750bf7800b6ad9d670308a09989fc0c04c40cec`.

  • The workflow references `diffusion_pytorch_model.safetensors`, `jina-clip-v2-adapter.safetensors`, `Mugen-Jina-IP-Adapter.safetensors`, and `MugenJinaUNET.safetensors`.

Konfiguration

  • Try changing the adapter weight from 1.0 if its influence is too strong; the publisher notes that 1.0 may not suit every case.

    Vorschlag · nicht geprüft

  • To compare variations, try different padding modes and compare `euler` with `euler_ancestral`; the workflow note says these choices change outputs, especially with the IP-Adapter.

    Vorschlag · nicht geprüft

Vorsichtsmaßnahmen

  • The workflow points to `anotherDrive\testing\Jina-Clip\MugenJinaUNET.safetensors`; check that this path matches your ComfyUI files.

    Vorschlag · nicht geprüft

  • The publisher says Mugen's original CLIP-G and CLIP-L text encoders produce slightly different outputs from Jina-clip-v2 and its adapter.

Muss das bearbeitet werden?

Melden Sie sich an, um eine Änderungsanfrage zu senden.

Quellen

1 Quelle
Hugging FacePrimär
huggingface.co/TheRemixer/Mugen-Jina-IP-Adapter

Verifiziert 18. Okt.

Quellenauszüge

3 Auszüge
Readme25. Sept.huggingface.co/TheRemixer/Mugen-Jina-IP-Adapter

--- license: apache-2.0 base_model: - CabalResearch/Mugen library_name: diffusers tags: - IP-adapter - SDXL datasets: - deepghs/danbooru2024-webp-4Mpixel pipeline_tag: image-text-to-image --- # IP-Adapter using [Jina-clip-v2's](https://huggingface.co/jinaai/jina-clip-v2) vision tower, and to be used on [Mugen](https://huggingface.co/CabalResearch/Mugen) ## Quick Notes: - 368M parameters - Trained on [Mugen w/Jina](https://huggingface.co/TheRemixer/jina-clip-v2-adapter/blob/main/MugenJinaUNET.safetensors) with [Jina-clip-v2](https://huggingface.co/jinaai/jina-clip-v2) + [Jina-clip-v2 adapter](https://huggingface.co/TheRemixer/jina-clip-v2-adapter) as the text encoder. - However, it also works on [Mugen](https://huggingface.co/CabalResearch/Mugen) with the original CLIP-G + CLIP-L text encoders. The outputs are slightly different. - Trained for 5 epochs with `208,000` unbatched steps in total, at a base resolution `1024x1024`. - Started showing promising results from the 2nd epoch. I have included the previous epochs in this repo if you are interested in testing them. --- ## Installation: - Install custom ComfyUI nodes from [here](https://github.com/lRemixl/ComfyUI_JinaCLIP_SDXL_IP_Adapter) - Download the [IP-adapter](https://huggingface.co/TheRemixer/Mugen-Jina-IP-Adapter/blob/main/Mugen-Jina-IP-Adapter.safetensors) in this repo and place it at the path `ComfyUI/models/ip_adapter/` - Download [jina-clip-v2](https://huggingface.co/jinaai/jina-clip-v2) and put files under a new folder in `ComfyUI/models/LLM/` - Download [Mugen Jina (Cross-attn only trained)](https://huggingface.co/TheRemixer/jina-clip-v2-adapter/blob/main/MugenJinaUNET.safetensors) and [Jina-clip-v2 adapter](https://huggingface.co/TheRemixer/jina-clip-v2-adapter/blob/main/jina-clip-v2-adapter.safetensors) - Alternatively you can just use [Mugen](https://huggingface.co/CabalResearch/Mugen) ## Usage: - See the workflow included in this repo on how to use the IP-adapter. - I mainly tested the IP-adapter as a style transfer, however it can do other things too. - Try different weighting if the default of `1.0` is too much. The default of `1.0` isn't recommened in all cases. --- ### Credits: - [tencent-ailab](https://github.com/tencent-ailab/IP-Adapter): IP-adapter - [DeepGHS](https://huggingface.co/deepghs): Used [danbooru2024-webp-4MPixel](https://huggingface.co/datasets/deepghs/danboo...

Note18. Okt.huggingface.co/TheRemixer/Mugen-Jina-IP-Adapter

Source context: 67 downloads · 1 likes · Pipeline image-text-to-image · Library diffusers · Repo TheRemixer/Mugen-Jina-IP-Adapter

Note25. Sept.huggingface.co/TheRemixer/Mugen-Jina-IP-Adapter

Source context: 54 downloads · 1 likes · Pipeline image-text-to-image · Library diffusers · Repo TheRemixer/Mugen-Jina-IP-Adapter