Comfyui_LLM_party

图片 bilibili · youtube · text tutorial · Cloud disk address · QQ group · discord · About us C‌‌​‎​‎‏​‍‎​‎​‎‏​‌‎​‎‍​‍‏​‍‌​‌‏omfyui_llm_party aims to develop a complete set of nodes for LLM workflow construction based on comfyui as the front end. It allows users to quickly and conveniently build their own LLM workflows and easily integrate them into their existing image workflows.

Effect display

Project Overview

ComfyUI LLM Party, from the most basic LLM multi-tool call, role setting to quickly build your own exclusive AI assistant, to the industry-specific word vector RAG and GraphRAG to localize the management of the industry knowledge base; from a single agent pipeline, to the construction of complex agent-agent radial interaction mode and ring interaction mode; from the access to their own social APP (QQ, Feishu, Discord) required by individual users, to the one-stop LLM + TTS + ComfyUI workflow required by streaming media workers; from the simple start of the first LLM application required by ordinary students, to the various parameter debugging interfaces commonly used by scientific researchers, model adaptation. All of this, you can find the answer in ComfyUI LLM Party.

Quick Start

0. If you have never used ComfyUI and encounter some dependency issues while installing the LLM party in ComfyUI, please click here to download the Windows portable package that includes the LLM party. Please note that this portable package contains only the party and manager plugins, and is exclusively compatible with the Windows operating system.(If you need to install LLM party into an existing comfyui, this step can be skipped.)
1. Drag the following workflows into your comfyui, then use comfyui-Manager to install the missing nodes.

• Use API to call LLM: start_with_LLM_api
• Using aisuite to call LLM: start_with_aisuite
• Manage local LLM with ollama: start_with_Ollama
• Use local LLM in distributed format: start_with_LLM_local
• Use local LLM in GGUF format: start_with_LLM_GGUF
• Use local VLM in distributed format: start_with_VLM_local (Currently, support is extended for Llama-3.2-Vision/Qwen/Qwen2.5-VL/deepseek-ai/Janus-Pro.)
• Use local VLM in GGUF format: start_with_VLM_GGUF
• Utilize API calls to LLM for generating SD prompts and images: start_with_VLM_API_for_SD
• Employ ollama to call minicpm for generating SD prompts and images: start_with_ollama_minicpm_for_SD
• Utilize the local qwen-vl to generate SD prompts and images: start_with_qwen_vl_local_for_SD

2. If you are using API, fill in your base_url (it can be a relay API, make sure it ends with /v1/), for example: andapi_key` in the API LLM loader node.
3. If you are using ollama, turn on the is_ollama option in the API LLM loader node, no need to fill in base_url and api_key.
4. If you are using a local model, fill in your model path in the local model loader node, for example: E:\model\Llama-3.2-1B-Instruct. You can also fill in the Huggingface model repo id in the local model loader node, for example: lllyasviel/omost-llama-3-8b-4bits.
5. Due to the high usage threshold of this project, even if you choose the quick start, I hope you can patiently read through the project homepage.

Latest update

1. The LLM API node has now implemented a streaming output mode, which will display the text returned by the API in real-time on the console, allowing you to see the API's output live without waiting for the entire request to complete.
2. The LLM API node has added a reasoning_content output, which can automatically separate the reasoning and response of the R1 model.
3. A new branch named only_api has been added to the repository, containing only the API calling components. This is designed for users who require only API invocation. To use this branch, simply execute the command git clone -b only_api in the custom toolfolder ofcomfyui, and then follow the environment deployment instructions provided on the project's main page. Please note! It is essential to ensure that there are no other folders named comfyui_LLM_partywithin thecustom tool` folder.
4. The VLM local loader node now supports deepseek-ai/Janus-Pro, with an example workflow: Janus-Pro.
5. The VLM local loader node has already supported Qwen/Qwen2.5-VL-3B-Instruct, but you need to update the transformer to the latest version (pip install -U transformers), example workflow: qwen-vl
6. A brand new image hosting node has been added, currently supporting the image hosting services at (with the regional domain for China being ) and More image hosting services will be supported in the future. Sample workflow: Image Hosting
7. The imgbb image hosting service, which is compatible by default with the party, has been updated to the domain imgbb. The previous image hosting service was replaced due to its unfriendliness towards users in mainland China. I sincerely apologize, as it seems that the API service for the image hosting at has been discontinued. Therefore, the code has reverted to the original Thank you for your understanding. In the future, I will update a node that supports more image hosting services.
8. The MCP tool has been updated. You can modify the configuration in the 'mcp_config.json' file located in the party project folder to connect to your desired MCP server. You can find various MCP server configuration parameters that you may want to add here: modelcontextprotocol/servers. The default configuration for this project is the Everything server, which serves as a testing MCP server to verify its functionality. Reference workflow: start_with_MCP. Developer note: The MCP tool node can connect to the MCP server you have configured and convert the tools from the server into tools that can be directly used by LLMs. By configuring different local or cloud servers, you can experience all LLM tools available in the world.

User Guide

1. For the instructions for using the node, please refer to: how to use nodes
2. If there are any issues with the plugin or you have other questions, feel free to join the QQ group: 931057213 | discord：discord.
3. More workflows please refer to the workflow folder.

Vedio tutorial

loan list

Some of the nodes in this project have borrowed from the following projects. Thank you for your contributions to the open-source community!

1. pythongosssss/ComfyUI-Custom-Scripts
2. lllyasviel/Omost

Support:

Join the community

If there is a problem with the plugin or you have any other questions, please join our community.

1. discord:discord link
2. QQ group: 931057213
3. WeChat group: we_glm (enter the group after adding the small assistant WeChat)

Follow us

1. If you want to continue to pay attention to the latest features of this project, please follow the Bilibili account: 派酱
2. youtube@comfyui-LLM-party

Donation support

If my work has brought value to your day, consider fueling it with a coffee! Your support not only energizes the project but also warms the heart of the creator. ☕💖 Every cup makes a difference!

Star History

Star History Chart