thygate / stable-diffusion-webui-depthmap-script Goto Github PK

how cai i run in google colab?

i am getting this error when i start the Web UI

Error loading script: depthmap.py
Traceback (most recent call last):
File "D:\stable-diffusion-webui-master\stable-diffusion-webui\modules\scripts.py", line 195, in load_scripts
module = script_loading.load_module(scriptfile.path)
File "D:\stable-diffusion-webui-master\stable-diffusion-webui\modules\script_loading.py", line 13, in load_module
exec(compiled, module.dict)
File "D:\stable-diffusion-webui-master\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 35, in
from repositories.midas.midas.dpt_depth import DPTDepthModel
File "D:\stable-diffusion-webui-master\stable-diffusion-webui\repositories\midas\midas\dpt_depth.py", line 5, in
from .blocks import (
File "D:\stable-diffusion-webui-master\stable-diffusion-webui\repositories\midas\midas\blocks.py", line 21, in
from .backbones.next_vit import (
File "D:\stable-diffusion-webui-master\stable-diffusion-webui\repositories\midas\midas\backbones\next_vit.py", line 8, in
file = open("./externals/Next_ViT/classification/nextvit.py", "r")
FileNotFoundError: [Errno 2] No such file or directory: './externals/Next_ViT/classification/nextvit.py'

Hi there, thanks for your work on this. I was looking to create stereo images from the image + depth map and found this:
https://github.com/m5823779/stereo-image-generation

It's a bit out of date but I managed to get it working. I found the res101 depthmap output produces better results than the existing midas integration in that repo. I extracted the neccessary bits from stereo_generation_image.py and modified it to read the depth map output produced from your extension.

I've tested out the stereo image in an Oculus CV1 headset using Whirligig, images attached. If I use this frequently I'm happy to write a PR if you think it's a good addition to your plugin, otherwise in case you wanted to integrate it sooner, heres the necessary code to run it on the command line.

Cheers

import os
import cv2
import argparse
import numpy as np

MONITOR_W = 38.5


def write_depth(depth, bits=1, reverse=True):
    depth_min = depth.min()
    depth_max = depth.max()
    max_val = (2 ** (8 * bits)) - 1

    if depth_max - depth_min > np.finfo("float").eps:
        out = max_val * (depth - depth_min) / (depth_max - depth_min)
    else:
        out = 0
    if not reverse:
        out = max_val - out

    if bits == 2:
        depth_map = out.astype("uint16")
    else:
        depth_map = out.astype("uint8")

    return depth_map


def generate_stereo(left_img, depth, ipd):
    h, w, c = left_img.shape

    depth_min = depth.min()
    depth_max = depth.max()
    depth = (depth - depth_min) / (depth_max - depth_min)

    right = np.zeros_like(left_img)

    deviation_cm = ipd * 0.12
    deviation = deviation_cm * MONITOR_W * (w / 1920)

    print("\ndeviation:", deviation)

    for row in range(h):
        for col in range(w):
            col_r = col - int((1 - depth[row][col] ** 2) * deviation)
            # col_r = col - int((1 - depth[row][col]) * deviation)
            if col_r >= 0:
                right[row][col_r] = left_img[row][col]

    right_fix = np.array(right)
    gray = cv2.cvtColor(right_fix, cv2.COLOR_BGR2GRAY)
    rows, cols = np.where(gray == 0)
    for row, col in zip(rows, cols):
        for offset in range(1, int(deviation)):
            r_offset = col + offset
            l_offset = col - offset
            if r_offset < w and not np.all(right_fix[row][r_offset] == 0):
                right_fix[row][col] = right_fix[row][r_offset]
                break
            if l_offset >= 0 and not np.all(right_fix[row][l_offset] == 0):
                right_fix[row][col] = right_fix[row][l_offset]
                break

    return right_fix


def overlap(im1, im2):
    width1 = im1.shape[1]
    height1 = im1.shape[0]
    width2 = im2.shape[1]
    height2 = im2.shape[0]

    # final image
    composite = np.zeros((height2, width2, 3), np.uint8)

    # iterate through "left" image, filling in red values of final image
    for i in range(height1):
        for j in range(width1):
            try:
                composite[i, j, 2] = im1[i, j, 2]
            except IndexError:
                pass

    # iterate through "right" image, filling in blue/green values of final image
    for i in range(height2):
        for j in range(width2):
            try:
                composite[i, j, 1] = im2[i, j, 1]
                composite[i, j, 0] = im2[i, j, 0]
            except IndexError:
                pass

    return composite

def parse_args():
    parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
    parser.add_argument("image", metavar="IMAGE", type=str, help="Image")
    parser.add_argument("depth", metavar="DEPTH", type=str, help="16bit depth map")
    parser.add_argument("--output-dir", type=str, help="Output dir", default=".")
    parser.add_argument("--ipd", type=float, default=6.5, help="Interpupillary distance (cm)")
    return parser.parse_args()


def main():
    args = parse_args()
    ipd = args.ipd

    left_img = cv2.imread(args.image)
    depth_map = cv2.imread(args.depth, cv2.IMREAD_ANYDEPTH)

    depth_map = cv2.blur(depth_map, (3, 3))

    right_img = generate_stereo(left_img, depth_map, ipd)
    stereo = np.hstack([left_img, right_img])
    anaglyph = overlap(left_img, right_img)

    name, ext = os.path.splitext(os.path.basename(args.image))
    cv2.imwrite(os.path.join(args.output_dir, f"{name}_stereo.png"), stereo)
    # cv2.imwrite(os.path.join(args.output_dir, f"{name}_anaglyph.png"), anaglyph)


if __name__ == "__main__":
    main()

I just installed V0.2.4 and did not have any previous depth software on this machine. On the img2img tab, using batch img2img, with the BOOST checkbox selected, after hitting Generate I find the error below in console and the generate fails. I'm guessing that I need to install pix2pix but I'm not really sure.

DepthMap v0.2.4
device: cuda
Loading model weights from  ./models/midas/dpt_large-midas-2f21e586.pt
Downloading model weights to ./models/pix2pix/latest_net_G.pth
Error completing request
Arguments: (2, '', '', 'None', 'None', None, None, None, None, None, 0, 1, 0, 4, 0, 1, False, False, 1, 1, 1, 0.75, -1.0, -1.0, 0, 0, 0, False, 512, 512, 0, False, 32, 0, 'C:\\stable-diffusion-webui\\outputs\\txt2img-images\\travels\\00091', 'C:\\stable-diffusion-webui\\outputs\\txt2img-images\\travels\\00091\\depth2', 11, 0.9, 5, '0.0001', False, 'None', '', 0.1, False, '<ul>\n<li><code>CFG Scale</code> should be 2 or lower.</li>\n</ul>\n', True, True, '', '', True, 50, True, 1, 0, False, 4, 1, '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, False, '', '<p style="margin-bottom:0.75em">Will upscale the image to twice the dimensions; use width and height sliders to set tile size</p>', 64, 0, 1, '', 0, '', True, False, False, False, 4.0, '', 10.0, False, True, True, 30.0, True, False, False, 0, 0.0, 10.0, True, 30.0, True, 0, 0, 384, 384, True, False, True, True, True, False, True, 1) {}
Traceback (most recent call last):
  File "C:\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 190, in run
    download_file(pix2pixmodel_path,"https://sfu.ca/~yagiz/CVPR21/latest_net_G.pth")
  File "C:\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 84, in download_file
    with open(filename, 'wb') as fout:
FileNotFoundError: [Errno 2] No such file or directory: './models/pix2pix/latest_net_G.pth'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\stable-diffusion-webui\modules\call_queue.py", line 45, in f
    res = list(func(*args, **kwargs))
  File "C:\stable-diffusion-webui\modules\call_queue.py", line 28, in f
    res = func(*args, **kwargs)
  File "C:\stable-diffusion-webui\modules\img2img.py", line 146, in img2img
    process_batch(p, img2img_batch_input_dir, img2img_batch_output_dir, args)
  File "C:\stable-diffusion-webui\modules\img2img.py", line 46, in process_batch
    proc = modules.scripts.scripts_img2img.run(p, *args)
  File "C:\stable-diffusion-webui\modules\scripts.py", line 317, in run
    processed = script.run(p, *script_args)
  File "C:\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 455, in run
    del pix2pixmodel
UnboundLocalError: local variable 'pix2pixmodel' referenced before assignment

Say I have an existing image (I don't want to change at all through IMG2IMG) can I still use this?

Hello, I have installed the script as described but each time I run it I error out after image generation. Could anyone help me with this issue? Copy-pasted below. Thank you!

100%|██████████████████████████████████████████████████████████████████████████████████| 65/65 [03:03<00:00, 2.82s/it]
Total progress: 740it [29:42, 2.85s/it]
DepthMap v0.1.3
device: cuda
Loading midas model weights ..
./models/midas/dpt_hybrid-midas-501f0c75.pt
Computing depthmap(s) ..
Error completing request
Arguments: ('A trainer training many loud kittens in his shitbox apartment', '', 'None', 'None', 65, 4, True, False, 1, 1, 8.5, -1.0, -1.0, 0, 0, 0, False, 512, 832, False, 0.7, 0, 0, 1, 1, 384, 0, True, True, False, 1, False, False, False, None, '', 1, '', 0, '', True, False, False) {}
Traceback (most recent call last):
File "C:\Users\carte\stable-diffusion-webui\modules\ui.py", line 185, in f
res = list(func(*args, **kwargs))
File "C:\Users\carte\stable-diffusion-webui\webui.py", line 55, in f
res = func(*args, **kwargs)
File "C:\Users\carte\stable-diffusion-webui\modules\txt2img.py", line 45, in txt2img
processed = modules.scripts.scripts_txt2img.run(p, *args)
File "C:\Users\carte\stable-diffusion-webui\modules\scripts.py", line 282, in run
processed = script.run(p, *script_args)
File "C:\Users\carte\stable-diffusion-webui\scripts\depthmap.py", line 206, in run
out = np.zeros(depth.shape, dtype=depth.type)
AttributeError: 'numpy.ndarray' object has no attribute 'type'

This is very cool, thanks. I was making some 'fake ones' natively but, obv, this is better. Can you point me and others to any resources that might show one or a few distinct possibilities of what one can do with these generated files? Very curious after seeing some of your examples and just not sure where to start. Thank you!

My 2GB GPU is capable of running Stable Diffusion.
This script's GPU support also works just fine after generating an image.

After using the script, however, subsequent Stable Diffusion runs fail due to VRAM unavailability. Is the model persisting in VRAM after use? If so, can it be freed between uses?

Hello. I installed the script according to the instructions, but when I run webui, I get the following error and cannot use it. Is there anything I should try?

Error loading script: depthmap.py
Traceback (most recent call last):
  File "C:\stable-diffusion-webui\stable-diffusion-webui-master\modules\scripts.py", line 195, in load_scripts
    module = script_loading.load_module(scriptfile.path)
  File "C:\stable-diffusion-webui\stable-diffusion-webui-master\modules\script_loading.py", line 13, in load_module
    exec(compiled, module.__dict__)
  File "C:\stable-diffusion-webui\stable-diffusion-webui-master\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 33, in <module>
    from repositories.midas.midas.dpt_depth import DPTDepthModel
  File "C:\stable-diffusion-webui\stable-diffusion-webui-master\repositories\midas\midas\dpt_depth.py", line 5, in <module>
    from .blocks import (
  File "C:\stable-diffusion-webui\stable-diffusion-webui-master\repositories\midas\midas\blocks.py", line 21, in <module>
    from .backbones.next_vit import (
  File "C:\stable-diffusion-webui\stable-diffusion-webui-master\repositories\midas\midas\backbones\next_vit.py", line 8, in <module>
    file = open("./externals/Next_ViT/classification/nextvit.py", "r")
FileNotFoundError: [Errno 2] No such file or directory: './externals/Next_ViT/classification/nextvit.py'

hi there,
I did a lot of generation today and they worked perfecty, but now I am getting this error when generating an anaglyph red-cyan image from img2img in automatic1111 with this error:

AttributeError: 'numpy.ndarray' object has no attribute 'save'

img2img with side-by-side works, txt2img also still works (also with red/cyan).

did I miss something?
thanks in advance
best d
PS: thanks for this amazing tool!!!

I don't know your rationale for making this just a custom script, but in case you didn't know, there is a more convenient way for users to have new functionality added:

https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Developing-extensions

With this you can install the needed packages automatically and we could add your extension to the wiki which would let users install and use it right from UI.

Possibly a memory leaking issue, as a similar effect is seen here in this post for depth2mask

After a run using depthmap2mask the memory allocated during the run is not released, this is clearly visible in a resource monitor looking at the VRAM usage, and reproducible across two machines and several different attempts. Command line flags include --medvram, --full precision, and --no-halfs

I tried CPU and GPU, nothing worked. Tried to use an already generated image, set "crop and resize" and "denoising strength" to 0.

Full terminal output:

DepthMap v0.2.6 device: cpu Loading model weights from ./models/leres/res101.pth initialize network with normal loading the model from ./models/pix2pix\latest_net_G.pth Computing depthmap(s) .. Error completing request Arguments: (0, '', '', 'None', 'None', <PIL.Image.Image image mode=RGB size=512x704 at 0x29EC9671C90>, None, None, None, None, 0, 60, 0, 4, 0, 1, False, False, 1, 1, 1, 0, -1.0, -1.0, 0, 0, 0, False, 512, 768, 1, False, 32, 0, '', '', 9, '<ul>\n<li><code>CFG Scale</code> should be 2 or lower.</li>\n</ul>\n', True, True, '', '', True, 50, True, 1, 0, False, 4, 1, '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, False, False, '', '<p style="margin-bottom:0.75em">Will upscale the image by the selected scale factor; use width and height sliders to set tile size</p>', 64, 0, 2, 1, '', 0, '', True, False, False, 1, 4, 512, 768, False, False, True, True, True, False, True, 1) {} Traceback (most recent call last): File "J:\sd_06_10_2022\stable-diffusion-webui\modules\call_queue.py", line 45, in f res = list(func(*args, **kwargs)) File "J:\sd_06_10_2022\stable-diffusion-webui\modules\call_queue.py", line 28, in f res = func(*args, **kwargs) File "J:\sd_06_10_2022\stable-diffusion-webui\modules\img2img.py", line 150, in img2img processed = modules.scripts.scripts_img2img.run(p, *args) File "J:\sd_06_10_2022\stable-diffusion-webui\modules\scripts.py", line 328, in run processed = script.run(p, *script_args) File "J:\sd_06_10_2022\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 229, in run prediction = estimateboost(img, model, model_type, pix2pixmodel) File "J:\sd_06_10_2022\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 917, in estimateboost whole_image_optimal_size, patch_scale = calculateprocessingres(img, net_receptive_field_size, r_threshold_value, scale_threshold, whole_size_threshold) File "J:\sd_06_10_2022\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 517, in calculateprocessingres grad_region = cv2.dilate(grad_resized, kernel2, iterations=1) UnboundLocalError: local variable 'grad_resized' referenced before assignment

Hi, I am having an issue trying to generate the depth map inside webui. I am using anaconda prompt in a custom separate environment with stable-diffusion webui installed and working. The error happens after it loads in the midas model weights and try to start computing depthmap. Is there something I missed? Thanks in advance.

Traceback (most recent call last): File "D:\Super_Stable_Diffusion\stable-diffusion-webui\modules\ui.py", line 186, in f res = list(func(*args, **kwargs)) File "D:\Super_Stable_Diffusion\stable-diffusion-webui\webui.py", line 64, in f res = func(*args, **kwargs) File "D:\Super_Stable_Diffusion\stable-diffusion-webui\modules\txt2img.py", line 40, in txt2img processed = modules.scripts.scripts_txt2img.run(p, *args) File "D:\Super_Stable_Diffusion\stable-diffusion-webui\modules\scripts.py", line 159, in run processed = script.run(p, *script_args) File "D:\Super_Stable_Diffusion\stable-diffusion-webui\scripts\depthmap_2.py", line 175, in run info = create_infotext(p, p.all_prompts, p.all_seeds, p.all_subseeds, "", 0, 0) AttributeError: 'StableDiffusionProcessingTxt2Img' object has no attribute 'all_prompts'

Hello! Thank you for 0.3.1, I had quite a demo effect when I tried to show this cool plugin to my friend and it just refused to work because midas broke. It may be helpful to specify the origin of source files and maybe add their LICENSE files (just in case).

I use the script on a macbook pro with an apple m1max chip, 32 GB Ram. I updated the script to the latest version, tried different models and tried to compute it on GPU or CPU, I always end up with

"AttributeError: module 'torch._C' has no attribute '_cuda_setDevice'"

Does the script work on a mac? The Automatic1111 webui does work so far.

Is it normal for res101 to destroy all midas models? It's much faster while being more accurate

When I test it with its default settings it work showing side by side results.
the moment I untick "Combine into one image" it does not work.
Basicaly it work with showing combined results Vertical and Horizontal however it does not work making depth only.

below is the log:

(INFO) Result 0:
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  9.04it/s]
Total progress: 2it [00:34, 17.28s/it]
DepthMap v0.1.4
device: cuda
Loading midas model weights ..
./models/midas/dpt_large-midas-2f21e586.pt
Computing depthmap(s) ..
Error completing request
Arguments: (0, '', '', 'None', 'None', <PIL.Image.Image image mode=RGB size=512x512 at 0x278540D7880>, {'image': <PIL.Image.Image image mode=RGBA size=512x512 at 0x278540D7790>, 'mask': <PIL.PngImagePlugin.PngImageFile image mode=RGBA size=512x512 at 0x278540D79A0>}, None, None, 0, 20, 0, 4, 1, False, False, 1, 1, 7, 0, -1.0, -1.0, 0, 0, 0, False, 512, 512, 0, False, 32, 0, '', '', 9, 0.9, 5, '0.0001', False, 'None', '', 0.1, False, '<span style="margin-left:13px;" class="text-gray-500 text-[0.855rem] mb-2 block dark:text-gray-200 relative z-40">Unprompted</span>', False, '<div id="unprompted"><div id="ad" class=" gr-box border-solid border border-gray-200" style="border-radius:0 0 8px 8px"><a href="https://payhip.com/b/hdgNR" target="_blank"><img src="https://i.ibb.co/1MSpHL4/Fantasy-Card-Template2.png" style="float: left;width: 150px;margin-bottom:10px;"></a><h1 style="font-size: 20px;letter-spacing:0.015em;margin-top:10px;">NEW! <strong>Premium Fantasy Card Template</strong> is now available.</h1><p style="margin:1em 0;">Generate a wide variety of creatures and characters in the style of a fantasy card game. Perfect for heroes, animals, monsters, and even crazy hybrids.</p><a href="https://payhip.com/b/hdgNR" target=_blank><button class="gr-button gr-button-lg gr-button-secondary" title="View premium assets for Unprompted">Learn More ➜</button></a><hr style="margin:1em 0;clear:both;"><p style="max-width:80%"><em>Purchases help fund the continued development of Unprompted. Thank you for your support!</em> ❤</p></div><a id="toggle-ad" href="#" style="float:right;display: inline;position:absolute;right:20px;bottom:20px;">Show Ad</a></div>', 4.0, 1, 1, 0, 0, 0.0, 4.0, 0.1, 0.1, 1, True, False, False, 0, False, '', 1, False, 0, 1, False, False, False, '<p style="margin-bottom:0.75em">Render these video formats:</p>', '<p style="margin-bottom:0.75em">Animation Parameters</p>', '<p style="margin-bottom:0.75em">Prompt Template, applied to each keyframe below</p>', '<p style="margin-bottom:0.75em">Keyframe Format: <br>Time (s) | Desnoise | Zoom (/s) | X Shift (pix/s) | Y shift (pix/s) | Positive Prompts | Negative Prompts | Seed</p>', '10.0', '15', False, False, True, '1.0', '', '', '', 0.4, '0', '0', False, False, 0.1, 0.5, '<p style="margin-bottom:0.75em">Render these video formats:</p>', '<p style="margin-bottom:0.75em">Animation Parameters</p>', '<p style="margin-bottom:0.75em">Initial Parameters</p>', '<p style="margin-bottom:0.75em">Prompt Template, applied to each keyframe below</p>', '<p style="margin-bottom:0.75em">Props</p>', '<p style="margin-bottom:0.75em">Supported Keyframes:<br>time_s | prompt | positive_prompts | negative_prompts<br>time_s | transform | zoom | x_shift | y_shift | rotation<br>time_s | seed | new_seed_int<br>time_s | denoise | denoise_value<br>time_s | prop | prop_filename | x_pos | y_pos | scale | rotation<br>time_s | set_text | textblock_name | text_prompt | x | y | w | h | fore_color | back_color | font_name<br>time_s | clear_text | textblock_name<br>time_s | prop | prop_name | prop_filename | x pos | y pos | scale | rotation<br>time_s | set_stamp | stamp_name | stamp_filename | x pos | y pos | scale | rotation<br>time_s | clear_stamp | stamp_name<br>time_s | col_set<br>time_s | col_clear<br>time_s | model | Original1-4model, microworld_PublicPrompts, sd-v1-4, sd-v1-4-full-ema, sd-v1-5-inpainting, trinart2_step115000, v1-5-pruned, v1-5-pruned-emaonly, wd-v1-2-full-ema, wd-v1-3-full</p>', '10.0', '15', False, False, True, '1.0', '', '', '', 0.4, '0', '0', '0', False, False, 0.1, 0.5, '', True, False, '', '', 0, 384, 0, True, True, False, 0, '\n            <h3><strong>Combinations</strong></h3>\n            Choose a number of terms from a list, in this case we choose two artists\n            <code>{2$$artist1|artist2|artist3}</code>\n            If $$ is not provided, then 1$$ is assumed.\n            <br>\n            A range can be provided:\n            <code>{1-3$$artist1|artist2|artist3}</code>\n            In this case, a random number of artists between 1 and 3 is chosen.\n            <br/><br/>\n\n            <h3><strong>Wildcards</strong></h3>\n            <p>Available wildcards</p>\n            <ul>\n        <li>__adjective__</li><li>__artist__</li><li>__genre__</li><li>__site__</li><li>__style__</li></ul>\n            <br/>\n            <code>WILDCARD_DIR: scripts/wildcards</code><br/>\n            <small>You can add more wildcards by creating a text file with one term per line and name is mywildcards.txt. Place it in scripts/wildcards. <code>__mywildcards__</code> will then become available.</small>\n        ', None, '', 'outputs', 1550, False, False, '<ul>\n<li><code>CFG Scale</code> should be 2 or lower.</li>\n</ul>\n', True, True, '', '', True, 50, True, 1, 0, False, 256, 0, 1, 0, 0.25, 4, 1, '<p style="margin-bottom:0.75em">Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], '', 1, True, 100, False, False, False, '', 2, '<p style="margin-bottom:0.75em">Will upscale the image to twice the dimensions; use width and height sliders to set tile size</p>', 64, 0, False, 4.0, '', 10.0, False, False, True, 30.0, True, False, False, 10.0, True, 30.0, True, '', '', 100, 0, 0, True, '<div class="gr-block gr-box relative w-full overflow-hidden border-solid border border-gray-200 gr-panel"><p>If you like my work, please consider showing your support on <strong><a href="https://patreon.com/thereforegames" target="_blank">Patreon</a></strong>. Thank you! &#10084;</p></div>', 'Illustration', 'svg', True, True, False, 0.5, True, 16, True, 16, '', 24, 24, 0, 10, '00:00:00', '00:00:00', False, '', 1, 10, True, 1, False, 1, 0, 0, False, 1, '', 0, '', True, False, False, '<p style="font-weight:bold;margin-bottom:0.75em">Deforum v0.5-webui-beta</p>', '<p>This script is deprecated. Please use the full Deforum extension instead.<br>\nUpdate instructions:</p>', '<p>github.com/deforum-art/deforum-for-automatic1111-webui/blob/automatic1111-webui/README.md</p>', '<p>discord.gg/deforum</p>', '<div id="dynamic-prompting">\n    <h3><strong>Combinations</strong></h3>\n\n    Choose a number of terms from a list, in this case we choose two artists: \n    <code class="codeblock">{{2$artist1|artist2|artist3}}</code>\n\n    If $ is not provided, then 1$ is assumed.\n\n    A range can be provided:\n    <code class="codeblock">{{1-3$artist1|artist2|artist3}}</code>\n\n    In this case, a random number of artists between 1 and 3 is chosen.\n\n    <br/><br/>\n\n    <h3><strong>Wildcards</strong></h3>\n    \n\n    <br/>\n    If the groups wont drop down click <strong onclick="check_collapsibles()" style="cursor: pointer">here</strong> to fix the issue.\n\n    <br/><br/>\n\n    <code class="codeblock">WILDCARD_DIR: D:\\stable-diffusion-webui\\extensions\\sd-dynamic-prompts\\wildcards</code><br/>\n    <small onload="check_collapsibles()">You can add more wildcards by creating a text file with one term per line and name is mywildcards.txt. Place it in D:\\stable-diffusion-webui\\extensions\\sd-dynamic-prompts\\wildcards. <code class="codeblock">__&#60;folder&#62;/mywildcards__</code> will then become available.</small>\n</div>\n\n', False, 1, False, False, 1.0, 2.0, 'a painting in', 'style', 'picture frame, portrait photo', None) {}
Traceback (most recent call last):
  File "D:\\stable-diffusion-webui\venv\lib\site-packages\PIL\JpegImagePlugin.py", line 630, in _save
    rawmode = RAWMODE[im.mode]
KeyError: 'I;16'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "D:\stable-diffusion-webui\modules\ui.py", line 185, in f
    res = list(func(*args, **kwargs))
  File "D:\stable-diffusion-webui\webui.py", line 54, in f
    res = func(*args, **kwargs)
  File "D:\stable-diffusion-webui\modules\img2img.py", line 137, in img2img
    processed = modules.scripts.scripts_img2img.run(p, *args)
  File "D:\stable-diffusion-webui\modules\scripts.py", line 296, in run
    processed = script.run(p, *script_args)
  File "D:\stable-diffusion-webui\scripts\depthmap.py", line 227, in run
    images.save_image(Image.fromarray(img_output), p.outpath_samples, "", processed.seed, p.prompt, opts.samples_format, info=info, p=p, suffix="_depth")
  File "D:\stable-diffusion-webui\modules\images.py", line 520, in save_image
    image.save(fullfn, quality=opts.jpeg_quality)
  File "D:\stable-diffusion-webui\venv\lib\site-packages\PIL\Image.py", line 2320, in save
    save_handler(self, fp, filename)
  File "D:\stable-diffusion-webui\venv\lib\site-packages\PIL\JpegImagePlugin.py", line 632, in _save
    raise OSError(f"cannot write mode {im.mode} as JPEG") from e
OSError: cannot write mode I;16 as JPEG

Found this GUI, but the toolkit (links do not work).

Any ideas how to get this running?

https://github.com/ilharp/depth-estimate-gui

'numpy.ndarray' object has no attribute 'save'

Hi.

I have this error when using 'dpt_hybrid' model on 'GPU':

"view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead."

other models are ok and CPU is fine.

Wow, this is awesome!

Is there a way to use it in "Extras" on already existing images?
Thank you!

I installed the script on colab, but on reboot I got this error on version 0.3.4:

Error loading script: depthmap.py
Traceback (most recent call last):
File "/content/stable-diffusion-webui/modules/scripts.py", line 195, in load_scripts
module = script_loading.load_module(scriptfile.path)
File "/content/stable-diffusion-webui/modules/script_loading.py", line 13, in load_module
exec(compiled, module.dict)
File "/content/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/scripts/depthmap.py", line 40, in
from midas.dpt_depth import DPTDepthModel
File "/content/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/scripts/midas/dpt_depth.py", line 5, in
from .blocks import (
File "/content/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/scripts/midas/blocks.py", line 4, in
from .backbones.beit import (
File "/content/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/scripts/midas/backbones/beit.py", line 9, in
from timm.models.beit import gen_relative_position_index
ModuleNotFoundError: No module named 'timm.models.beit'

I put these two lines in install.py, but it didn't help:
if not launch.is_installed("timm"):
launch.run_pip("install timm", "timm requirement for depthmap script")

How can I use the script on colab?

I thought I made a mistake in my scripts but then I tried with yours and had the same problem. Now I don't see any difference in between your code and the one in midas repository about loading the hybrid model so I'm kind of clueless about why it does not work (I have not tried directly with the midas repo).

Traceback (most recent call last):
  File "/content/stable-diffusion-webui/modules/ui.py", line 185, in f
    res = list(func(*args, **kwargs))
  File "/content/stable-diffusion-webui/webui.py", line 57, in f
    res = func(*args, **kwargs)
  File "/content/stable-diffusion-webui/modules/img2img.py", line 137, in img2img
    processed = modules.scripts.scripts_img2img.run(p, *args)
  File "/content/stable-diffusion-webui/modules/scripts.py", line 317, in run
    processed = script.run(p, *script_args)
  File "/content/stable-diffusion-webui/scripts/depthmap.py", line 188, in run
    prediction = model.forward(sample)
  File "/content/stable-diffusion-webui/repositories/midas/midas/dpt_depth.py", line 108, in forward
    return super().forward(x).squeeze(dim=1)
  File "/content/stable-diffusion-webui/repositories/midas/midas/dpt_depth.py", line 71, in forward
    layer_1, layer_2, layer_3, layer_4 = forward_vit(self.pretrained, x)
  File "/content/stable-diffusion-webui/repositories/midas/midas/vit.py", line 59, in forward_vit
    glob = pretrained.model.forward_flex(x)
  File "/content/stable-diffusion-webui/repositories/midas/midas/vit.py", line 127, in forward_flex
    x = self.patch_embed.backbone(x)
  File "/usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/timm/models/resnetv2.py", line 409, in forward
    x = self.forward_features(x)
  File "/usr/local/lib/python3.8/site-packages/timm/models/resnetv2.py", line 403, in forward_features
    x = self.stem(x)
  File "/usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/torch/nn/modules/container.py", line 139, in forward
    input = module(input)
  File "/usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/timm/models/layers/std_conv.py", line 70, in forward
    self.weight.view(1, self.out_channels, -1), None, None,
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

Would you happen to be less in the darkness than me?

I get this error while launching automatic1111 webui and the script doesn't load:

Error loading script: depthmap.py
Traceback (most recent call last):
File "D:\Super stable diffusion 2.0\stable-diffusion-webui\modules\scripts.py", line 164, in load_scripts
module = script_loading.load_module(scriptfile.path)
File "D:\Super stable diffusion 2.0\stable-diffusion-webui\modules\script_loading.py", line 13, in load_module
exec(compiled, module.dict)
File "D:\Super stable diffusion 2.0\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 8, in
from modules.call_queue import wrap_gradio_gpu_call, wrap_queued_call, wrap_gradio_call
ModuleNotFoundError: No module named 'modules.call_queue'

Can you help me out?

Actual behaviour
For instance with the prompt building, with a batch count of 2, you'll get output names like:
00675-1600648804-building.png
00676-1600648805-building.png

00677-1600648804-building_depth.png
00678-1600648804-building_depth.png

Expected behaviour
So the 1600648804 in the second depth image name should be incremented by 1. This can be a bit a frustration when creating similar depth maps because the numbering is the same. This also makes it so the depth image won't give the correct information to png_info.

Some of the observations I've made
processed.seed that is used during saving seems to be only the initial seed. There should have some sort of n_iter or batch count index added to it. It would be nice to do something like processed.seed + processed.n_iter ; however, n_iter isn't attached to processed.

Possible solution
Changing processed.seed to processed.all_seeds[count-1] does seem to work. I haven't tested it for every situation.

*background_removed.png and *foreground_mask.png is not saved to subdirectory.

EDIT referring to "Save images to a subdirectory" and "Directory name pattern" in Settings.

Hello! I am getting this error running the script on Auto11111's webui. I just found this script this morning, so i haven't run it successfully yet. I was able to clone the midas repo stuff as instructed.

Error loading script: depthmap.py
Traceback (most recent call last):
  File "C:\Users\Linds\stable-diffusion-webui\modules\scripts.py", line 169, in load_scripts
    compiled = compile(text, scriptfile.path, 'exec')
  File "C:\Users\Linds\stable-diffusion-webui\scripts\depthmap.py", line 70
    <title>stable-diffusion-webui-depthmap-script/depthmap.py at main · thygate/stable-diffusion-webui-depthmap-script</title>
                                                                      ^
SyntaxError: invalid character '·' (U+00B7)

Have I bunged something up?

venv "E:\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Commit hash: 98947d173e3f1667eba29c904f681047dea9de90
Installing requirements for Web UI

loading Dreambooth reqs from E:\stable-diffusion-webui\extensions\sd_smartprocess\requirements.txt
Checking Smart Crop requirements.

Launching Web UI with arguments:
Error loading script: depthmap.py
Traceback (most recent call last):
File "E:\stable-diffusion-webui\modules\scripts.py", line 164, in load_scripts
module = script_loading.load_module(scriptfile.path)
File "E:\stable-diffusion-webui\modules\script_loading.py", line 11, in load_module
compiled = compile(text, path, 'exec')
File "E:\stable-diffusion-webui\scripts\depthmap.py", line 72
<title>stable-diffusion-webui-depthmap-script/depthmap.py at main · thygate/stable-diffusion-webui-depthmap-script · GitHub</title>
^
SyntaxError: invalid character '·' (U+00B7)

LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla' with 512 in_channels
Loading weights [19810fe6] from E:\stable-diffusion-webui\models\Stable-diffusion\berry_mix.ckpt
Applying cross attention optimization (Doggettx).
Model loaded.
Loaded a total of 0 textual inversion embeddings.
Embeddings:
Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch().

Midas and script in proper places

It's either fragmenting the VRAM or not de-allocating the model when finished, it does not seem to reuse the already loaded depth-map model. I can use the dream-booth extension to clear all the memory and gets things started again, maybe you could do something like that. It happens on both img2img and txt2img.
Computing depthmap(s) .. Total progress: 100%|████████████████████████████| 1/1 [00:09<00:00, 9.13s/it] INFO:dynamic_prompting.py:Prompt matrix will create 1 images in a total of 1 bat ches. 100%|████████████████████████████████████████████| 1/1 [00:14<00:00, 14.40s/it] Total progress: 0%| | 0/1 [00:00<?, ?it/s] DepthMap v0.1.8 device: cuda Loading midas model weights .. ./models/midas/dpt_large-midas-2f21e586.pt Computing depthmap(s) .. Total progress: 100%|████████████████████████████| 1/1 [00:08<00:00, 8.93s/it] INFO:dynamic_prompting.py:Prompt matrix will create 1 images in a total of 1 bat ches. Error completing request Arguments: (0, '', '', 'None', 'None', <PIL.Image.Image image mode=RGB size=512x 640 at 0x264A3795CF0>, None, None, None, 0, 20, 0, 4, 1, False, False, 1, 1, 7, 0, -1.0, -1.0, 0, 0, 0, False, 512, 512, 0, False, 32, 0, '', '', 1, '<div class ="dynamic-prompting">\n <h3><strong>Combinations</strong></h3>\n\n Choose a number of terms from a list, in this case we choose two artists: \n <code c lass="codeblock">{2$$artist1|artist2|artist3}</code><br/>\n\n If $$ is not pr ovided, then 1$$ is assumed.<br/><br/>\n\n If the chosen number of terms is g reater than the available terms, then some terms will be duplicated, otherwise c hosen terms will be unique. This is useful in the case of wildcards, e.g.\n < code class="codeblock">{2$$__artist__}</code> is equivalent to <code class="code block">{2$$__artist__|__artist__}</code><br/><br/>\n\n A range can be provide d:\n <code class="codeblock">{1-3$$artist1|artist2|artist3}</code><br/>\n In this case, a random number of artists between 1 and 3 is chosen.<br/><br/>\n\ n Wildcards can be used and the joiner can also be specified:\n <code clas s="codeblock">{{1-$$and$$__adjective__}}</code><br/>\n\n Here, a random numbe r between 1 and 3 words from adjective.txt will be chosen and joined together wi th the word \'and\' instead of the default comma.\n\n <br/><br/>\n\n <h3>< strong>Wildcards</strong></h3>\n \n\n <br/>\n If the groups wont drop d own click <strong onclick="check_collapsibles()" style="cursor: pointer">here</s trong> to fix the issue.\n\n <br/><br/>\n\n <code class="codeblock">WILDCA RD_DIR: T:\\stable-diffusion-webui4\\extensions\\sd-dynamic-prompts\\wildcards</ code><br/>\n <small onload="check_collapsibles()">You can add more wildcards by creating a text file with one term per line and name is mywildcards.txt. Plac e it in T:\\stable-diffusion-webui4\\extensions\\sd-dynamic-prompts\\wildcards. <code class="codeblock">__&#60;folder&#62;/mywildcards__</code> will then become available.</small>\n</div>\n\n', True, False, 1, False, False, False, 100, 0.7, False, False, False, False, False, False, 0.9, 5, '0.0001', False, 'None', '', 0.1, False, 0, 0, 384, 384, False, False, True, True, True, 1, '<ul>\n<li><code> CFG Scale</code> should be 2 or lower.</li>\n</ul>\n', True, True, '', '', True, 50, True, 1, 0, False, 4, 1, '<p style="margin-bottom:0.75em">Recommended setti ngs: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8</p>', 128 , 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up' , 'down'], False, False, False, '', '<p style="margin-bottom:0.75em">Will upscal e the image to twice the dimensions; use width and height sliders to set tile si ze</p>', 64, 0, 1, '', 0, '', True, False, False, '<p style="font-weight:bold;ma rgin-bottom:0.75em">Deforum v0.5-webui-beta</p>', '<p>This script is deprecated. Please use the full Deforum extension instead.<br>\nUpdate instructions:</p>', '<p>github.com/deforum-art/deforum-for-automatic1111-webui/blob/automatic1111-we bui/README.md</p>', '<p>discord.gg/deforum</p>', '{inspiration}', None) {} Traceback (most recent call last): File "T:\stable-diffusion-webui4\modules\ui.py", line 185, in f res = list(func(*args, **kwargs)) File "T:\stable-diffusion-webui4\webui.py", line 57, in f res = func(*args, **kwargs) File "T:\stable-diffusion-webui4\modules\img2img.py", line 137, in img2img processed = modules.scripts.scripts_img2img.run(p, *args) File "T:\stable-diffusion-webui4\modules\scripts.py", line 317, in run processed = script.run(p, *script_args) File "T:\stable-diffusion-webui4\scripts\depthmap.py", line 63, in run processed = processing.process_images(p) File "T:\stable-diffusion-webui4\modules\processing.py", line 430, in process_ images res = process_images_inner(p) File "T:\stable-diffusion-webui4\modules\processing.py", line 496, in process_ images_inner p.init(p.all_prompts, p.all_seeds, p.all_subseeds) File "T:\stable-diffusion-webui4\modules\processing.py", line 841, in init self.init_latent = self.sd_model.get_first_stage_encoding(self.sd_model.enco de_first_stage(image)) File "T:\stable-diffusion-webui4\venv\lib\site-packages\torch\autograd\grad_mo de.py", line 27, in decorate_context return func(*args, **kwargs) File "T:\stable-diffusion-webui4\repositories\stable-diffusion\ldm\models\diff usion\ddpm.py", line 863, in encode_first_stage return self.first_stage_model.encode(x) File "T:\stable-diffusion-webui4\repositories\stable-diffusion\ldm\models\auto encoder.py", line 325, in encode h = self.encoder(x) File "T:\stable-diffusion-webui4\venv\lib\site-packages\torch\nn\modules\modul e.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "T:\stable-diffusion-webui4\repositories\stable-diffusion\ldm\modules\dif fusionmodules\model.py", line 442, in forward h = self.down[i_level].block[i_block](hs[-1], temb) File "T:\stable-diffusion-webui4\venv\lib\site-packages\torch\nn\modules\modul e.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "T:\stable-diffusion-webui4\repositories\stable-diffusion\ldm\modules\dif fusionmodules\model.py", line 130, in forward h = self.norm2(h) File "T:\stable-diffusion-webui4\venv\lib\site-packages\torch\nn\modules\modul e.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "T:\stable-diffusion-webui4\venv\lib\site-packages\torch\nn\modules\norma lization.py", line 272, in forward return F.group_norm( File "T:\stable-diffusion-webui4\venv\lib\site-packages\torch\nn\functional.py ", line 2516, in group_norm return torch.group_norm(input, num_groups, weight, bias, eps, torch.backends .cudnn.enabled) RuntimeError: CUDA out of memory. Tried to allocate 128.00 MiB (GPU 0; 8.00 GiB total capacity; 6.87 GiB already allocated; 0 bytes free; 7.16 GiB reserved in t otal by PyTorch) If reserved memory is >> allocated memory try setting max_split _size_mb to avoid fragmentation. See documentation for Memory Management and PY TORCH_CUDA_ALLOC_CONF

any solutions?

DepthMap v0.3.4
device: cuda
Loading model weights from ./models/leres/res101.pth
Computing depthmap(s) ..
0%| | 0/1 [00:00<?, ?it/s]

100%|████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 2.31it/s]
Done.
Running 3D Photo Inpainting ..
Downloading https://filebox.ece.vt.edu/~jbhuang/project/3DPhoto/model/edge-model.pth to ./models/3dphoto/edge_model.pth
All done.
Error completing request
Arguments: (0, <PIL.Image.Image image mode=RGB size=512x512 at 0x247F706ADD0>, None, '', '', 0, 0, 512, 512, True, False, False, True, True, False, True, 1, False, False, 2.5, 4, 0, False, 0, 1, True, 0, 2) {}
Traceback (most recent call last):
File "Q:\test\sd_new\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 506, in run_3dphoto
download_file(edgemodel_path,"https://filebox.ece.vt.edu/~jbhuang/project/3DPhoto/model/edge-model.pth")
File "Q:\test\sd_new\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 1225, in download_file
torch.hub.download_url_to_file(url, filename)
File "Q:\test\sd_new\venv\lib\site-packages\torch\hub.py", line 607, in download_url_to_file
f = tempfile.NamedTemporaryFile(delete=False, dir=dst_dir)
File "C:\ProgramData\Anaconda3\envs\stable\lib\tempfile.py", line 559, in NamedTemporaryFile
file = _io.open(dir, mode, buffering=buffering,
File "C:\ProgramData\Anaconda3\envs\stable\lib\tempfile.py", line 556, in opener
fd, name = _mkstemp_inner(dir, prefix, suffix, flags, output_type)
File "C:\ProgramData\Anaconda3\envs\stable\lib\tempfile.py", line 256, in _mkstemp_inner
fd = _os.open(file, flags, 0o600)
FileNotFoundError: [Errno 2] No such file or directory: 'Q:\test\sd_new\models\3dphoto\tmp4xy807xx'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "Q:\test\sd_new\modules\call_queue.py", line 45, in f
res = list(func(*args, **kwargs))
File "Q:\test\sd_new\modules\call_queue.py", line 28, in f
res = func(*args, **kwargs)
File "Q:\test\sd_new\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 1047, in run_generate
outputs, mesh_fi = run_depthmap(None, outpath, imageArr, imageNameArr, compute_device, model_type, net_width, net_height, match_size, invert_depth, boost, save_depth, show_depth, show_heat, combine_output, combine_output_axis, gen_stereo, gen_anaglyph, stereo_divergence, stereo_fill, stereo_balance, clipdepth, clipthreshold_far, clipthreshold_near, inpaint, fnExt, vid_ssaa)
File "Q:\test\sd_new\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 468, in run_depthmap
mesh_fi = run_3dphoto(device, inpaint_imgs, inpaint_depths, inputnames, outpath, fnExt, vid_ssaa)
File "Q:\test\sd_new\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 626, in run_3dphoto
del rgb_model
UnboundLocalError: local variable 'rgb_model' referenced before assignment

Traceback (most recent call last):
File "Q:\test\sd_new\venv\lib\site-packages\gradio\routes.py", line 321, in run_predict
output = await app.blocks.process_api(
File "Q:\test\sd_new\venv\lib\site-packages\gradio\blocks.py", line 1016, in process_api
data = self.postprocess_data(fn_index, result["prediction"], state)
File "Q:\test\sd_new\venv\lib\site-packages\gradio\blocks.py", line 945, in postprocess_data
if predictions[i] is components._Keywords.FINISHED_ITERATING:
IndexError: tuple index out of range

In version 0.3.4, I checked the Generate 3D inpainted mesh and demo videos. (Sloooow) option at the bottom. After generating the depthmap I got errors, and then I noticed that the first model could not be downloaded because the 3DPhoto folder did not exist in the models directory. After creating this manually, the generation was running.
This should be fixed in the script, so that if there is no such folder, it is created before the 3Dphoto models is downloaded.

I am getting this error when trying to launch this extension:

Error loading script: depthmap.py Traceback (most recent call last): File "A:\Stable Diffusion\stable-diffusion-webui\modules\scripts.py", line 184, in load_scripts File "A:\Stable Diffusion\stable-diffusion-webui\modules\script_loading.py", line 13, in load_module exec(compiled, module.__dict__) File "A:\Stable Diffusion\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 32, in <module> from repositories.midas.midas.dpt_depth import DPTDepthModel ModuleNotFoundError: No module named 'repositories.midas'

When I installed the extension, it successfully put both midas and BoostingMonocularDepth into the /repositories folder, so it's not like midas is missing entirely. However, I am running this entire thing on a drive that is apparently not formatted to be able to track ownership. So I had some issues when I was first installling the webui where I ended up using the command "git config --global --add safe.directory '*'" to avoid having to manually set ownership of every folder. I'm wondering if that's connected. But I haven't run into any other issues after using that command until now.

I just did a git pull and updated all extensions. Not as good a reproduction as a clean install for sure, but here's steps:
Click on Settings
Click Apply Settings (I don't think you even have to change anything)

Expected: Settings will save
Actual: Settings says Error in red below the apply button and in console is the following:

Traceback (most recent call last):
File "C:\stable-diffusion\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 321, in run_predict
output = await app.blocks.process_api(
File "C:\stable-diffusion\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1015, in process_api
result = await self.call_function(fn_index, inputs, iterator, request)
File "C:\stable-diffusion\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 856, in call_function
prediction = await anyio.to_thread.run_sync(
File "C:\stable-diffusion\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "C:\stable-diffusion\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "C:\stable-diffusion\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 867, in run
result = context.run(func, *args)
File "C:\stable-diffusion\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 1156, in
fn = lambda a, b: a if b > a else b,
TypeError: '>' not supported between instances of 'str' and 'bool'

Fix:
Change line 1156 of depthmap.py to:
fn = lambda a, b: a if b > str(a) else b,

The idea being to make the 'bool' a 'str' so the comparison won't fail. I would put this in as a pull request, except apparently no one else is having this problem? And I don't know python so I have no idea if my fix is even a good idea.

Hi,
First of all, awesome script! Love what you did!
Is it only me or does the script only add Depthmaps to the first half of alle images generated in a batch?

Right now, the resolution only makes a square image on output. Most images are not square, so would be nice to be able to do a vertical and horizontal image size separately.

After today's update, I am unable to start SD Web UI due to an issue with this Extension installation, possibly due to rembg, as per attached screenshot of the Windows console.
I am on Windows 10 Pro, with SD Web UI Commit hash: 97ff69eff338c6641f4abf430bf5ac112c1775e0

The extension was working fine yesterday. Please, have a look at these errors, thank you. I need to have all the extensions activated to properly translate the Web UI, Scripts and Extensions for the Italian Localization.

a bug was causing overflow when converting the depth to 8 bit, resulting in artifacts resembling dark spots in the depthmap output.

examples of artifact :

Generating inpainted mesh .. (go make some coffee) ..
WARNING:py.warnings:D:\ai\a1\extensions/stable-diffusion-webui-depthmap-script/scripts\inpaint\mesh_tools.py:174: RuntimeWarning: divide by zero encountered in divide
input_disp = 1./np.abs(input_depth)

Hi Team,

I used to be able to run DepthMap using v.0.7 but it stopped working. I reinstalled following the instructions and receive errors in the logs that prevent the depth maps from being generated.
Any help is appreciated, please let me know if I installed incorrectly but the instructions are very straightforward.

DepthMap Log:

device: cuda
Loading midas model weights ..
./models/midas/dpt_hybrid-midas-501f0c75.pt
Error verifying pickled file from ./models/midas/dpt_hybrid-midas-501f0c75.pt:
Traceback (most recent call last):
  File "C:\stable-diffusion-webui\modules\safe.py", line 83, in check_pt
    with zipfile.ZipFile(filename) as z:
  File "C:\Users\dusty\AppData\Local\Programs\Python\Python310\lib\zipfile.py", line 1267, in __init__
    self._RealGetContents()
  File "C:\Users\dusty\AppData\Local\Programs\Python\Python310\lib\zipfile.py", line 1334, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\stable-diffusion-webui\modules\safe.py", line 131, in load_with_extra
    check_pt(filename, extra_handler)
  File "C:\stable-diffusion-webui\modules\safe.py", line 98, in check_pt
    unpickler.load()
_pickle.UnpicklingError: persistent IDs in protocol 0 must be ASCII strings

-----> !!!! The file is most likely corrupted !!!! <-----
You can skip this check with --disable-safe-unpickle commandline argument, but that is not going to help you.


Error completing request
Arguments: ('hero sitting on the top of a rooftop overlooking gotham city, 8k, ultra details, HD', 'text, letters, words, signature, script, boring, dull, watermark, low quality, worst quality, jpeg artifacts', 'None', 'None', 45, 0, True, False, 6, 1, 7, -1.0, -1.0, 0, 0, 0, False, 576, 960, True, 0.75, 0, 0, 1, '<div class="dynamic-prompting">\n    <h3><strong>Combinations</strong></h3>\n\n    Choose a number of terms from a list, in this case we choose two artists: \n    <code class="codeblock">{2$$artist1|artist2|artist3}</code><br/>\n\n    If $$ is not provided, then 1$$ is assumed.<br/><br/>\n\n    If the chosen number of terms is greater than the available terms, then some terms will be duplicated, otherwise chosen terms will be unique. This is useful in the case of wildcards, e.g.\n    <code class="codeblock">{2$$__artist__}</code> is equivalent to <code class="codeblock">{2$$__artist__|__artist__}</code><br/><br/>\n\n    A range can be provided:\n    <code class="codeblock">{1-3$$artist1|artist2|artist3}</code><br/>\n    In this case, a random number of artists between 1 and 3 is chosen.<br/><br/>\n\n    Wildcards can be used and the joiner can also be specified:\n    <code class="codeblock">{{1-$$and$$__adjective__}}</code><br/>\n\n    Here, a random number between 1 and 3 words from adjective.txt will be chosen and joined together with the word \'and\' instead of the default comma.\n\n    <br/><br/>\n\n    <h3><strong>Wildcards</strong></h3>\n    \n\n    <br/>\n    If the groups wont drop down click <strong onclick="check_collapsibles()" style="cursor: pointer">here</strong> to fix the issue.\n\n    <br/><br/>\n\n    <code class="codeblock">WILDCARD_DIR: C:\\stable-diffusion-webui\\extensions\\sd-dynamic-prompts\\wildcards</code><br/>\n    <small onload="check_collapsibles()">You can add more wildcards by creating a text file with one term per line and name is mywildcards.txt. Place it in C:\\stable-diffusion-webui\\extensions\\sd-dynamic-prompts\\wildcards. <code class="codeblock">__&#60;folder&#62;/mywildcards__</code> will then become available.</small>\n</div>\n\n', True, False, 1, False, False, 100, 0.7, False, False, False, False, False, False, 0.9, 5, '0.0001', False, 'None', '', 0.1, False, 0, 1, 384, 384, True, True, True, True, False, 1, False, False, False, '', 1, '', 0, '', True, False, False, 1.0, 2.0, 'a painting in', 'style', 'picture frame, portrait photo', None) {}
Traceback (most recent call last):
  File "C:\stable-diffusion-webui\scripts\depthmap.py", line 103, in run
    model = DPTDepthModel(
  File "C:\stable-diffusion-webui\repositories\midas\midas\dpt_depth.py", line 105, in __init__
    self.load(path)
  File "C:\stable-diffusion-webui\repositories\midas\midas\base_model.py", line 13, in load
    if "optimizer" in parameters:
TypeError: argument of type 'NoneType' is not iterable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\stable-diffusion-webui\modules\ui.py", line 188, in f
    res = list(func(*args, **kwargs))
  File "C:\stable-diffusion-webui\webui.py", line 57, in f
    res = func(*args, **kwargs)
  File "C:\stable-diffusion-webui\modules\txt2img.py", line 46, in txt2img
    processed = modules.scripts.scripts_txt2img.run(p, *args)
  File "C:\stable-diffusion-webui\modules\scripts.py", line 290, in run
    processed = script.run(p, *script_args)
  File "C:\stable-diffusion-webui\scripts\depthmap.py", line 253, in run
    del model
UnboundLocalError: local variable 'model' referenced before assignment```

At L23 of the script, the path format separator "\" assumes the user is using Windows (which should also be escaped). The "/" separator works in both Windows and Linux (also Mac), which should probably be preferred.

If I want to see the images by crossing my eyes, instead of a 3D device (such as VR headset), the left and right images are in the wrong position.

It would be neat, and probably very easy for you, to have the option to flip the left and right images when the stereogram is saved.

After installing v0.1.1 the interface seems to be missing dropdowns for selecting models, Compute on and Combine axis in txt2img and img2img

hi, this is my error,, midas in repo, script in scripts

Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 12.54it/s]
DepthMap v0.1.0
device: cuda
Loading midas model weights ..
./models/midas/dpt_large-midas-2f21e586.pt
Error verifying pickled file from ./models/midas/dpt_large-midas-2f21e586.pt:
Traceback (most recent call last):
File "C:\Users\popec\Documents\GitHub\stable-diffusion-webui\modules\safe.py", line 76, in check_pt
with zipfile.ZipFile(filename) as z:
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2288.0_x64__qbz5n2kfra8p0\lib\zipfile.py", line 1267, in init
self._RealGetContents()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2288.0_x64__qbz5n2kfra8p0\lib\zipfile.py", line 1334, in _RealGetContents
raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\popec\Documents\GitHub\stable-diffusion-webui\modules\safe.py", line 97, in load
check_pt(filename)
File "C:\Users\popec\Documents\GitHub\stable-diffusion-webui\modules\safe.py", line 89, in check_pt
unpickler.load()
_pickle.UnpicklingError: persistent IDs in protocol 0 must be ASCII strings

-----> !!!! The file is most likely corrupted !!!! <-----
You can skip this check with --disable-safe-unpickle commandline argument, but that is not going to help you.

Error completing request
Arguments: ('city', '', 'None', 'None', 20, 0, False, False, 1, 1, 7, -1.0, -1.0, 0, 0, 0, False, 512, 512, False, 0.7, 0, 0, 1, 0.9, 5, '0.0001', False, 'None', '', 0.1, False, 0, 0, True, True, False, 1, False, False, False, '', 1, '', 0, '', True, True, False, 1.0, 2.0, 'a painting in', 'style', 'picture frame, portrait photo', None) {}
Traceback (most recent call last):
File "C:\Users\popec\Documents\GitHub\stable-diffusion-webui\modules\ui.py", line 185, in f
res = list(func(*args, **kwargs))
File "C:\Users\popec\Documents\GitHub\stable-diffusion-webui\webui.py", line 55, in f
res = func(*args, **kwargs)
File "C:\Users\popec\Documents\GitHub\stable-diffusion-webui\modules\txt2img.py", line 45, in txt2img
processed = modules.scripts.scripts_txt2img.run(p, *args)
File "C:\Users\popec\Documents\GitHub\stable-diffusion-webui\modules\scripts.py", line 297, in run
processed = script.run(p, *script_args)
File "C:\Users\popec\Documents\GitHub\stable-diffusion-webui\scripts\depthmap.py", line 83, in run
model = DPTDepthModel(
File "C:\Users\popec\Documents\GitHub\stable-diffusion-webui\repositories\midas\midas\dpt_depth.py", line 105, in init
self.load(path)
File "C:\Users\popec\Documents\GitHub\stable-diffusion-webui\repositories\midas\midas\base_model.py", line 13, in load
if "optimizer" in parameters:
TypeError: argument of type 'NoneType' is not iterable

thanks

JSONDecodeError: Expecting value: line 8 column 1 (char 7)

Getting this stack-trace:

DepthMap v0.2.7
device: cpu
Loading model weights from  ./models/leres/res101.pth
Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.
Error completing request
Arguments: (0, <PIL.Image.Image image mode=RGB size=512x512 at 0x1D02C2AAFB0>, None, '', '', 0, 4, 384, 384, False, False, True, True, True, False, True, 1) {}
Traceback (most recent call last):
  File "C:\Users\kenne\gitrepo\stable-diffusion-webui\modules\call_queue.py", line 45, in f
    res = list(func(*args, **kwargs))
  File "C:\Users\kenne\gitrepo\stable-diffusion-webui\modules\call_queue.py", line 28, in f
    res = func(*args, **kwargs)
  File "C:\Users\kenne\gitrepo\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 394, in run_generate
    outputs = run_depthmap(None, outpath, imageArr, imageNameArr, compute_device, model_type, net_width, net_height, match_size, invert_depth, boost, save_depth, show_depth, show_heat, combine_output, combine_output_axis)
  File "C:\Users\kenne\gitrepo\stable-diffusion-webui\extensions\stable-diffusion-webui-depthmap-script\scripts\depthmap.py", line 339, in run_depthmap
    return outimages
UnboundLocalError: local variable 'outimages' referenced before assignment

I attempted to simply move the reference, but that we are getting the error would suggest it never had a value when running before my change, and after my change, there is no error, but also other than telling me it executed in 0.32s, nothing.

First of, thanks for making this, it'll simplify the toolchain I worked out for this and keep things inside the WebUI.

I can't get it installed.

0. The installation directions could be amended to make more clear that the directories /repositories/midas will be made by the git clone command. I see now it's listed there, but I wasn't familiar with that name-as-create method of git cloning (I'm new to this, but I imagine others are too). Naively following the instructions had me making inside /scripts /repositories/midas/repositories/midas as a result of the git command, and then having to notice and fix the structure. I suggest changing to something like:
   "Clone the MiDaS repository /scripts with command
   git clone https://github.com/isl-org/MiDaS.git repositories/midas
   This will download the repo to /scripts inside the necessary subfolders."

1. I think the imports on depthmap.py lines 18-21 should be ".repositories.midas...." with the integral period. Otherwise I get this error:
   (webSD_2) C:\SD\stable-diffusion\stable-diffusion-webui>python C:\SD\stable-diffusion\stable-diffusion-webui\webui.py --precision full --no-half --medvram Error loading script: depthmap.py Traceback (most recent call last): File "C:\SD\stable-diffusion\stable-diffusion-webui\modules\scripts.py", line 155, in load_scripts exec(compiled, module.__dict__) File "C:\SD\stable-diffusion\stable-diffusion-webui\scripts\depthmap.py", line 18, in <module> from repositories.midas.midas.dpt_depth import DPTDepthModel ModuleNotFoundError: No module named 'repositories'

2. With that change made, I get this error:
   (webSD_2) C:\SD\stable-diffusion\stable-diffusion-webui>python C:\SD\stable-diffusion\stable-diffusion-webui\webui.py --precision full --no-half --medvram Error loading script: depthmap.py Traceback (most recent call last): File "C:\SD\stable-diffusion\stable-diffusion-webui\modules\scripts.py", line 155, in load_scripts exec(compiled, module.__dict__) File "C:\SD\stable-diffusion\stable-diffusion-webui\scripts\depthmap.py", line 18, in <module> from .repositories.midas.midas.dpt_depth import DPTDepthModel ModuleNotFoundError: No module named 'depthmap'
   So it still fails but it is possibly making the call into dpt_depth (though I'm not sure about that, no debugging print commands placed inside of dpt_depth print for me), perhaps the periods are not what are needed?

hello.
i used to work with depth map script for a month but in the past week after installing the script in web ui, depth map doesnt show up in the scripts menu of img2img or text2img tabs.
i know its installed already and its shown in installed tab and the extension folder is in the directory.
(and i didnt forget to restart the ui).
how can i use the script?
thanks for helping

Error loading script: depthmap.py
Traceback (most recent call last):
File "/stable-diffusion-webui/modules/scripts.py", line 184, in load_scripts
module = script_loading.load_module(scriptfile.path)
File "/stable-diffusion-webui/modules/script_loading.py", line 13, in load_module
exec(compiled, module.dict)
File "/stable-diffusion-webui/extensions/depthmap-script/scripts/depthmap.py", line 38, in
from lib.multi_depth_model_woauxi import RelDepthModel
ModuleNotFoundError: No module named 'lib'

Error since "ee06f97"

I've removed and reinstalled the plug-in, checked the midas/BoostingMonocularDepth repositories are in place.

Did the author added some library code without make the install steps?
Which are the install steps?
I can't find any "lib" folder, nor in the webui nor in this repository?