biegert / comfyui-clipseg Goto Github PK

ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
File "E:\DEV\ComfyUI_windows_portable\ComfyUI\execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "E:\DEV\ComfyUI_windows_portable\ComfyUI\execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "E:\DEV\ComfyUI_windows_portable\ComfyUI\execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "E:\DEV\ComfyUI_windows_portable\ComfyUI\custom_nodes\clipsegnode.py", line 216, in combine_masks
heatmap_resized = resize_image(heatmap, dimensions)
File "E:\DEV\ComfyUI_windows_portable\ComfyUI\custom_nodes\clipsegnode.py", line 44, in resize_image
return cv2.resize(image, dimensions, interpolation=cv2.INTER_LINEAR)
cv2.error: OpenCV(4.8.1) D:\a\opencv-python\opencv-python\opencv\modules\imgproc\src\resize.cpp:3699: error: (-215:Assertion failed) !dsize.empty() in function 'cv::hal::resize'

Loading 1 new model
100%|██████████| 24/24 [00:14<00:00, 1.63it/s]
pytorch_model.bin: 2%|▏ | 12.6M/603M [04:18<3:22:01, 48.7kB/s]
!!! Exception during processing !!!
Traceback (most recent call last):
File "H:\ComfyUI\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "H:\ComfyUI\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "H:\ComfyUI\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "H:\ComfyUI\ComfyUI\custom_nodes\clipseg.py", line 121, in segment_image
model = CLIPSegForImageSegmentation.from_pretrained("CIDAS/clipseg-rd64-refined")
File "H:\ComfyUI\python_embeded\lib\site-packages\transformers\modeling_utils.py", line 2222, in from_pretrained
resolved_archive_file = cached_file(
File "H:\ComfyUI\python_embeded\lib\site-packages\transformers\utils\hub.py", line 409, in cached_file
resolved_file = hf_hub_download(
File "H:\ComfyUI\python_embeded\lib\site-packages\huggingface_hub\utils_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "H:\ComfyUI\python_embeded\lib\site-packages\huggingface_hub\file_download.py", line 1457, in hf_hub_download
http_get(
File "H:\ComfyUI\python_embeded\lib\site-packages\huggingface_hub\file_download.py", line 552, in http_get
raise EnvironmentError(
OSError: Consistency check failed: file should be of size 603143713 but has size 12608704 (pytorch_model.bin).
We are sorry for the inconvenience. Please retry download and pass force_download=True, resume_download=False as argument.
If the issue persists, please let us know by opening an issue on https://github.com/huggingface/huggingface_hub.

i found the model in huggingface

https://huggingface.co/CIDAS/clipseg-rd64-refined/tree/main

but which directory should i put it in

I just managed to get Clipseg to work for still images, and it’s such a neat tool! I’m trying to set up a workflow to generate masks for video clips too.

First of all, I’m completely new to this so apologize if this is a plain silly idea for this particular custom node, but how would one go about to load a video instead of just a single image through the clipseg node?

When trying this, I get an error saying the "input array is receiving more dimensions than expected”:

Error occurred when executing CLIPSeg:

Too many dimensions: 4 > 3.

File "E:\COMFYUI\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\COMFYUI\ComfyUI\custom_nodes\clipseg.py", line 118, in segment_image
i = Image.fromarray(image_np, mode="RGB")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\snurrebassen\AppData\Local\Programs\Python\Python312\Lib\site-packages\PIL\Image.py", line 3103, in fromarray
raise ValueError(msg)

I was wondering if anyone has suggestions or workarounds for this issue.

tried downloading and installing latest CLIPSeg nodes and it fails on Comfyui startup with error Cannot import C:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-CLIPSeg-main module for custom nodes: [Errno 2] No such file or directory: 'C:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-CLIPSeg-main\init.py'.

How do I resolve this error?
thanks

Error occurred when executing CLIPSeg:

Failed to import transformers.models.vit.image_processing_vit because of the following error (look up to see its traceback):
A Message class can only inherit from Message

File "/content/drive/MyDrive/ComfyUI/execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/content/drive/MyDrive/ComfyUI/execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/content/drive/MyDrive/ComfyUI/execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/content/drive/MyDrive/ComfyUI/custom_nodes/clipseg.py", line 120, in segment_image
processor = CLIPSegProcessor.from_pretrained("CIDAS/clipseg-rd64-refined")
File "/usr/local/lib/python3.10/dist-packages/transformers/processing_utils.py", line 226, in from_pretrained
args = cls._get_arguments_from_pretrained(pretrained_model_name_or_path, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/processing_utils.py", line 268, in _get_arguments_from_pretrained
attribute_class = getattr(transformers_module, class_name)
File "/usr/local/lib/python3.10/dist-packages/transformers/utils/import_utils.py", line 1273, in getattr
value = getattr(module, name)
File "/usr/local/lib/python3.10/dist-packages/transformers/utils/import_utils.py", line 1272, in getattr
module = self._get_module(self._class_to_module[name])
File "/usr/local/lib/python3.10/dist-packages/transformers/utils/import_utils.py", line 1284, in _get_module
raise RuntimeError(

File "/Users/fredlefevre/AI/ComfyUI/custom_nodes/ComfyUI-CLIPSeg/init.py", line 16, in
from ...utils import OptionalDependencyNotAvailable, _LazyModule, is_torch_available
ImportError: attempted relative import beyond top-level package

Cannot import /Users/fredlefevre/AI/ComfyUI/custom_nodes/ComfyUI-CLIPSeg module for custom nodes: attempted relative import beyond top-level package

Thanks for any help

Exception during processing!!! HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /CIDAS/clipseg-rd64-refined/resolve/main/model.safetensors (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001F558555CC0>, 'Connection to huggingface.co timed out. (connect timeout=10)'))

Hi....this can't run in the latest Comfyui.
Could you update it ?
Thank you so much!

You should be creating your masks like ComfyUI creates them, so they're compatible with nodes that process them.

https://github.com/comfyanonymous/ComfyUI/blob/master/nodes.py#L990

They are inverted, for one.

how to solve this problem？？？？？

I'm reposting the issue from here:
ltdrdata/ComfyUI-Impact-Pack#116

I'm using Impact pack.
Very often (but not always), when I use FaceDetailer node with CLIPSegDetectorProvider - one of them spams ReadTimeoutError errors into console, with text like this:

Loads SAM model: E:\SD\ComfyUI-portable\ComfyUI\models\sams\sam_vit_b_01ec64.pth (device:Prefer GPU)
'(ReadTimeoutError("HTTPSConnectionPool(host='huggingface.co', port=443): Read timed out. (read timeout=10)"), '(Request ID: 621b36cd-5b11-4361-bbd3-c1f008d1a9e3)')' thrown while requesting HEAD https://huggingface.co/CIDAS/clipseg-rd64-refined/resolve/main/preprocessor_config.json
'(ReadTimeoutError("HTTPSConnectionPool(host='huggingface.co', port=443): Read timed out. (read timeout=10)"), '(Request ID: 50506107-44dc-4bf4-9fb7-48cf7ad9c640)')' thrown while requesting HEAD https://huggingface.co/CIDAS/clipseg-rd64-refined/resolve/main/tokenizer_config.json
'(ReadTimeoutError("HTTPSConnectionPool(host='huggingface.co', port=443): Read timed out. (read timeout=10)"), '(Request ID: c5467a5f-4ec8-4074-a2b4-00765779fe5e)')' thrown while requesting HEAD https://huggingface.co/CIDAS/clipseg-rd64-refined/resolve/main/config.json
# of Detected SEGS: 1
Detailer: segment upscale for ((419, 390)) | crop region (838, 780) x 1.8331748251748252 -> (1536, 1429)

The nodes author claims that they are just wrappers on top of CLIPSeg and the actual issue is here.

The error in question is not from FaceDetailer but rather from CLIPSeg. CLIPSeg has a part where it downloads the model through the network initially, so if there is no internet connection, this can become an issue.

I can't reproduce the issue on demand. It either happens or not, with no obvious cause. Sometimes my whole graph (heavily utilizing FaceDetailer + CLIPSeg) just works like a breeze. Sometimes, when I have a long sequence of such nodes, each of them spams the same set of errors. My best guess is I'm getting antispam-banned at huggingface after a while.

And looks like, the only thing CLIPSeg tries to retrieve is a couple of json files. Why don't just download them once?

Have you ever encountered this kind of problem? This kind of mosaic result occurs when cutting out pictures.

like: head
exclude: ear

(venv) D:\ComfyUI\custom_nodes\ComfyUI-CLIPSeg>pip install -r requirements.txt
Looking in indexes: https://mirrors.cloud.tencent.com/pypi/simple
Requirement already satisfied: matplotlib==3.7.1 in d:\comfyui\venv\lib\site-packages (from -r requirements.txt (line 2)) (3.7.1)
Requirement already satisfied: matplotlib-inline==0.1.6 in d:\comfyui\venv\lib\site-packages (from -r requirements.txt (line 3)) (0.1.6)
Collecting numpy==1.24.2 (from -r requirements.txt (line 4))
Using cached https://mirrors.cloud.tencent.com/pypi/packages/fa/df/53e8c0c8ccecf360b827a3d2b1b6060644c635c3149a9d6415a6fe4ccf44/numpy-1.24.2-cp310-cp310-win_amd64.whl (14.8 MB)
Requirement already satisfied: open-clip-torch==2.16.0 in d:\comfyui\venv\lib\site-packages (from -r requirements.txt (line 5)) (2.16.0)
Collecting opencv-python==4.7.0.72 (from -r requirements.txt (line 6))
Using cached https://mirrors.cloud.tencent.com/pypi/packages/36/98/fab8d982e2e2b57bdebcad64c7e5b5a14ac91c657cac509b9cf3fbea49d2/opencv_python-4.7.0.72-cp37-abi3-win_amd64.whl (38.2 MB)
Collecting Pillow==9.4.0 (from -r requirements.txt (line 7))
Using cached https://mirrors.cloud.tencent.com/pypi/packages/5e/7c/293136a5171800001be33c21a51daaca68fae954b543e2c015a6bb81a716/Pillow-9.4.0-cp310-cp310-win_amd64.whl (2.5 MB)
Requirement already satisfied: pytorch-lightning==2.0.0 in d:\comfyui\venv\lib\site-packages (from -r requirements.txt (line 8)) (2.0.0)
ERROR: Could not find a version that satisfies the requirement torch==2.2.0+cu121 (from versions: 1.11.0, 1.12.0, 1.12.1, 1.13.0, 1.13.1, 2.0.0, 2.0.1, 2.1.0, 2.1.1, 2.1.2, 2.2.0, 2.2.1)
ERROR: No matching distribution found for torch==2.2.0+cu121

Cannot import C:...\ComfyUI\custom_nodes\ComfyUI-CLIPSeg module for custom nodes: [Errno 2] No such file or directory: 'C:\...\SD-ComfyUI\ComfyUI\custom_nodes\ComfyUI-CLIPSeg\init.py'

!!! Exception during processing !!!
Traceback (most recent call last):
File "D:\Blender_ComfyUI\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Blender_ComfyUI\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Blender_ComfyUI\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Blender_ComfyUI\ComfyUI\custom_nodes\clipseg.py", line 154, in segment_image
heatmap_resized = resize_image(heatmap, dimensions)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\Blender_ComfyUI\ComfyUI\custom_nodes\clipseg.py", line 44, in resize_image
return cv2.resize(image, dimensions, interpolation=cv2.INTER_LINEAR)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^