I tried downloading a zip directly and cloning it directly to my notebook on Paperspace, but the script doesn't show up in the Scripts dropdown in either txt2img or img2img

Output the mask image along with the generated image on the Output UI, that way you can preview the mask, and use the generated mask with another script by simply dragging the outputted mask to the Input Mask area

Since SD faces get worse the smaller they get, it would be useful if some of the algorithms used for txt2mask could be used to automate the following process in one

1. txt2img eg photo of jmp909 person wearing a red hat etc
2. send the image to img2img
3. apply txt2mask on eg head or face
4. inpaint photo of jmp909 person at full resolution for just the masked part eg head or face (without the extra part of the prompt so would need a separate input as well for the simplified prompt)

Hope that makes sense

thanks

Okay so I was getting the Unpickling error and did the fix to delete the weights in clipseg and restarted but now I'm getting a new error.

Traceback (most recent call last):
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/ui.py", line 134, in f
res = list(func(*args, **kwargs))
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/webui.py", line 58, in f
res = func(*args, **kwargs)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/img2img.py", line 115, in img2img
processed = modules.scripts.scripts_img2img.run(p, *args)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/scripts.py", line 158, in run
processed = script.run(p, *script_args)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/scripts/txt2mask.py", line 175, in run
p.image_mask = get_mask().resize((p.init_images[0].width,p.init_images[0].height))
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/scripts/txt2mask.py", line 121, in get_mask
model.load_state_dict(torch.load(d64_file, map_location=torch.device('cuda')), strict=False);
File "/usr/local/lib/python3.7/dist-packages/torch/serialization.py", line 713, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/usr/local/lib/python3.7/dist-packages/torch/serialization.py", line 920, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
EOFError: Ran out of input

I don't know if this is possible or not but for example I want to change the cloth of multiple people across multiple images,or I want to change all the faces to certain someone's face with txt2image masking.Like deepfakes so to say. I may have sequence of images or random images doesn't matter.

So what I have in mind is as in the picture,I will mask out the faces,I will write a prompt and do a batch inpainting by utilizing the automatic masking of txt2mask.

I am not sure if you can do it alone or maybe automatic himself needs to be involved,but this is a great idea imo and maybe a new harder task or challenge for you :)

Please tell me it can be done.

In short what I am asking is a batch inpainting feature powered by clip masking with txt2mask.

this is an awesome script, but its readme overwrites AUTOMATICS

Hi there,thank you for fixing the full resolution problem.It works perfect.

I have changed the default delimeter from "|" to "," cos it makes more sense and easy to write.I would like to see it as a default.

Anyway,as title says it all how about negative masking like negative prompt.
For example I tried to mask chest and shoulders it worked but the blending of the neck area didn't go as good as I expected cos it was a very stylized picture.( I also tried many styles with prompt to blend better)
So I have added chest,shoulders,neck this time it included the whole face area. I used mask blur,mask precision and mask padding but yet I failed.

This gave me the idea to exclude with negative masking.As it is easy to mask out faces why not discard them from the total combined masked area? If this can be done we can achieve far more greater results with much precise control.

I am hoping that there is a way and I think it is possible.Because your delimeter adds masks together,it can subract from it.Ofc it requires another input field as negative prompt and better nagative prompts also shall support combining with a delimeter.

In addition to these a checkbox for inverting the mask can be a cool feature I believe.

I'm testing it, and I can't seem to get SD to replace the object masked. Perhaps that is just one of the current limitations of img2img. For example, trying to replace the astronaut riding a horse with riding an elephant, and I get strange output like this:

The mask prompt is "a horse" and the prompt is "an elephant." I can see the script is masking the horse well, but then the elephant is drawn all strange in the masked area. How do you get the astronaut riding an elephant? Again, this might not be a problem with txt2mask specifically, but with the img2img method. Should the top prompt contain a more general description of the entire scene, or only what is going in the masked area?

Images tell all.but In case yoyu need to full error as written.

img2img: Emma Watson face close up smiling
Error completing request
Arguments: ('Emma Watson face close up smiling', '', 'None', 'None', None, {'image': <PIL.Image.Image image mode=RGBA size=671x1000 at 0x19A9D5816F0>, 'mask': <PIL.PngImagePlugin.PngImageFile image mode=RGBA size=671x1000 at 0x19A9D582A40>}, None, 0, 72, 0, 13, 1, False, False, 1, 1, 1, 7.5, 0.4, -1.0, -1.0, 0, 0, 0, 512, 512, 0, 0, 64, True, 0, 8, '', '', '', '', 1, 50, 0, 4, 1, '

Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8

Tried leaves/leafs/leaf & got error.

Hello, this used to work fine but after a clean install of windows I get this error when I use batch img2img combined with TXT2Mask

*** Error completing request███████████████▎ | 9/27 [00:19<00:38, 2.17s/it]
*** Arguments: ('task(9v0vv1nqvnugvp9)', 5, 'lora:more_details:1 face of a person', '', [], None, None, None, None, None, None, None, 20, 16, 4, 0, 1, False, False, 1, 1, 7, 1.5, 0.4, -1.0, -1.0, 0, 0, 0, False, 0, 1280, 960, 1, 2, 0, 32, 0, 'C:\test', '', '', [], False, [], '', <gradio.routes.Request object at 0x000001EF6C8B1660>, 9, False, 'img2img', False, '', '', False, 'Euler a', False, 'v1-5-pruned-emaonly.safetensors [6ce0161689]', True, 0.5, True, 4, True, 32, True, False, 30, False, 6, False, 512, 512, '', False, 1, 'Both ▦', False, '', False, True, True, False, False, False, False, 100, 100, False, '', '', '', 'generateMasksTab', 4, 4, 2.5, 30, 1.03, 1, 1, 5, 0.5, 5, False, True, False, 20, '

\n
• CFG Scale should be 2 or lower.
\n

\n', True, True, '', '', True, 50, True, 1, 0, False, 4, 0.5, 'Linear', 'None', '

Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8

', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, '', '

Will upscale the image by the selected scale factor; use width and height sliders to set tile size

', 64, 0, 2, 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, 'face lips eyes forehead eyebrows ears cheeks chin head mouth tongue ', '', 100, 0, 0, True, '

If you like my work, please consider showing your support on Patreon. Thank you! ❤

') {}
Traceback (most recent call last):
File "C:\Users\David\stable-diffusion-webui\modules\call_queue.py", line 58, in f
res = list(func(*args, **kwargs))
File "C:\Users\David\stable-diffusion-webui\modules\call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "C:\Users\David\stable-diffusion-webui\modules\img2img.py", line 226, in img2img
process_batch(p, img2img_batch_input_dir, img2img_batch_output_dir, img2img_batch_inpaint_mask_dir, args, to_scale=selected_scale_tab == 1, scale_by=scale_by, use_png_info=img2img_batch_use_png_info, png_info_props=img2img_batch_png_info_props, png_info_dir=img2img_batch_png_info_dir)
File "C:\Users\David\stable-diffusion-webui\modules\img2img.py", line 118, in process_batch
infotext = proc.infotext(p, n)
File "C:\Users\David\stable-diffusion-webui\modules\processing.py", line 457, in infotext
return create_infotext(p, self.all_prompts, self.all_seeds, self.all_subseeds, comments=[], position_in_batch=index % self.batch_size, iteration=index // self.batch_size)
File "C:\Users\David\stable-diffusion-webui\modules\processing.py", line 624, in create_infotext
"Seed": p.all_seeds[0] if use_main_prompt else all_seeds[index],
IndexError: list index out of range

img2img: A robot's arm
Error completing request
Arguments: ("A robot's arm", '', 'None', 'None', <PIL.Image.Image image mode=RGB size=944x958 at 0x1FDA7385690>, {'image': <PIL.Image.Image image mode=RGBA size=944x958 at 0x1FDA7387310>, 'mask': <PIL.PngImagePlugin.PngImageFile image mode=RGBA size=944x958 at 0x1FDA7384250>}, None, 0, 20, 0, 4, 0, False, False, 1, 1, 1, 7, 0.75, -1.0, -1.0, 0, 0, 0, 512, 512, 0, 0, 64, False, 0, 8, '', '', '', '', 1, 50, 0, 4, 1, '

Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8

Any fixes known yet or am I missing something?

After the most recent update for SD2.0, I get this error while loading scripts.

I am using Mac M1 machine, probably non CUDA, and getting error when using txt2mask script:

RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.
Time taken: 1m 18.08s

When I try to use this script, this is all i get, and the prompt fails.

Or is this for the UI editor only?

I installed as you said, clone or download this repo and place the files in the base directory of Automatic's web UI and restarted the server, but the script is not showing the script's dropdown.

Searching for a fix for the "_pickle.UnpicklingError: invalid load key, 'v'." i wiped the 1KiB files only to have them replace with the same files. Doing a manual git pull of the clipseg repo resulted in the following error:
Error downloading object: weights/rd16-uni.pth (61545cd): Smudge error: Error downloading weights/rd16-uni.pth (61545cdb3a28f99d33d457c64a9721ade835a9dfbda604c459de6831c504167a): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.

Is there another host for these weights?

Hi, I get the following error :
0it [00:00, ?it/s]
Error completing request
Arguments: (1, 'photo of a man with beard and big eyes looking straight at camera, space stars and galaxies background', '', 'None', 'None', <PIL.Image.Image image mode=RGB size=1920x1786 at 0x22136802710>, {'image': <PIL.Image.Image image mode=RGBA size=512x512 at 0x22136802B00>, 'mask': <PIL.PngImagePlugin.PngImageFile image mode=RGBA size=512x512 at 0x22136801C00>}, None, None, 0, 100, 1, 4, 0, False, False, 1, 1, 12, 0, -1.0, -1.0, 0, 0, 0, False, 512, 512, 1, False, 32, 0, '', '', 9, 0, 0.1, 10, 7, 19.9, 0.1, 0.001, 'photo of a man with beard and eyes looking straight at camera, white wall background', '', -1, 50, 0, True, 4, 1, '

Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8

Will upscale the image to twice the dimensions; use width and height sliders to set tile size

thank you in advance.

I tried to get it to select "leaves, leaf" even tried "leafs" is there a way/place to visualize it's selection criteria before attempting to render?

``img2img: animal skulls, realistic
100%|███████████████████████████████████████| 335M/335M [00:07<00:00, 44.6MiB/s]
Error completing request
Arguments: ('animal skulls, realistic', '', 'None', 'None', <PIL.Image.Image image mode=RGB size=1024x1024 at 0x25CE318ADD0>, None, None, 0, 20, 0, 4, 0, False, False, 0, 1, 1, 7, 0.75, -1.0, -1.0, 0, 0, 0, 1024, 1024, 0, 0, 64, False, 0, 7, '', '', '', '', 1, 50, 0, 4, 1, 128, 4, 0, ['left', 'right', 'up', 'down'], None, False, 'leaves, leafs', 96, '

img2img: animal skulls, realistic
Error completing request
Arguments: ('animal skulls, realistic', '', 'None', 'None', <PIL.Image.Image image mode=RGB size=1024x1024 at 0x25CE31F3B80>, None, None, 0, 20, 0, 4, 0, False, False, 0, 1, 1, 7, 0.75, -1.0, -1.0, 0, 0, 0, 1024, 1024, 0, 0, 64, False, 0, 7, '', '', '', '', 1, 50, 0, 4, 1, 128, 4, 0, ['left', 'right', 'up', 'down'], None, False, 'leaf', 96, '<div class="gr-block gr-box relative w-full overflow-hidden border-solid border border-gray-200 gr-panel"><p>If you like my work, please consider showing your support on <strong><a href="https://patreon.com/thereforegames" target="_blank">Patreon</a></strong>. Thank you! &#10084;</p></div>', 1, '', 4, '', True) {}
Traceback (most recent call last):
  File "E:\stable-diffusion-webui\modules\ui.py", line 125, in f
    res = list(func(*args, **kwargs))
  File "E:\stable-diffusion-webui\webui.py", line 71, in f
    res = func(*args, **kwargs)
  File "E:\stable-diffusion-webui\modules\img2img.py", line 131, in img2img
    processed = modules.scripts.scripts_img2img.run(p, *args)
  File "E:\stable-diffusion-webui\modules\scripts.py", line 140, in run
    processed = script.run(p, *script_args)
  File "E:\stable-diffusion-webui\scripts\txt2mask.py", line 86, in run
    p.image_mask = get_mask()
  File "E:\stable-diffusion-webui\scripts\txt2mask.py", line 46, in get_mask
    model.load_state_dict(torch.load('./repositories/clipseg/weights/rd64-uni.pth', map_location=torch.device('cuda')), strict=False);
  File "E:\stable-diffusion-webui\venv\lib\site-packages\torch\serialization.py", line 713, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "E:\stable-diffusion-webui\venv\lib\site-packages\torch\serialization.py", line 920, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'v'.`

Is there a config to change the max accepted prompts? Thanks.

I tested this quite a bit and while I appreciate the work and the idea behind it, this is not useful for real world usage.
I've tested it on a variety of images, tried to mask hands, faces, flags, umbrellas, sheets on a table, arms or just people.
It did not work one single time regardless of "precision" settings.