Giter Club home page Giter Club logo

comfyui-clipseg's Introduction

Custom Nodes for ComfyUI: CLIPSeg and CombineSegMasks

This repository contains two custom nodes for ComfyUI that utilize the CLIPSeg model to generate masks for image inpainting tasks based on text prompts.

1. CLIPSeg

The CLIPSeg node generates a binary mask for a given input image and text prompt.

Inputs:

  • image: A torch.Tensor representing the input image.
  • text: A string representing the text prompt.
  • blur: A float value to control the amount of Gaussian blur applied to the mask.
  • threshold: A float value to control the threshold for creating the binary mask.
  • dilation_factor: A float value to control the dilation of the binary mask.

Outputs:

  • tensor_bw: A torch.Tensor representing the binary mask.
  • image_out_hm: A torch.Tensor representing the heatmap overlay on the input image.
  • image_out_bw: A torch.Tensor representing the binary mask overlay on the input image.

2. CombineSegMasks

The CombineSegMasks node combines two or optionally three masks into a single mask to improve masking of different areas.

Inputs:

  • image: A torch.Tensor representing the input image.
  • mask1: A torch.Tensor representing the first mask.
  • mask2: A torch.Tensor representing the second mask.
  • mask3 (optional): A torch.Tensor representing the third mask. Defaults to None.

Outputs:

  • combined_mask: A torch.Tensor representing the combined mask.
  • image_out_hm: A torch.Tensor representing the heatmap overlay of the combined mask on the input image.
  • image_out_bw: A torch.Tensor representing the binary mask overlay of the combined mask on the input image.

Installation

To use these custom nodes in your ComfyUI project, follow these steps:

  1. Clone this repository or download the source code.
  2. Put the clipseg.py file into your custom_nodes directory
  3. ???
  4. Profit

Usage

Below is an example for the intended workflow. The json file for the example can be found inside the 'workflow' directory

Requirements

  • PyTorch
  • CLIPSeg
  • OpenCV
  • numpy
  • matplotlib

Make sure that you have the required libraries installed to the venv of ComfyUI.

comfyui-clipseg's People

Contributors

biegert avatar time-river avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.