Question About Blurring of Overlapping Masks about multidiffusion HOT 4 CLOSED

Hi,
For the panorama generation, there are actually 8 overlapping crops.
Feel free to provide more information as to what you're trying and I'll try to help.

from multidiffusion.

Hm... I think your reply is a bit different from what I'm asking.
If you put many overlapping masks directly on top of each other, for example if there were 8 crops each being the entire latent space, would it still produce a non-blurry image?

The overlapping crops you refer to only overlap in some regions:

+----+---+----+
| A  |A&B| B  |
+----+---+----+

But what I'm asking about is if you have something like:

+----+---+----+
| A&B&C&D&... |
+----+---+----+

Will that region be blurred?
As you're minimizing a loss within a least squared sense, how can it maintain a sharp output?

I'm also curious whether you tried this on non-latent space diffusion models such as Deep-Floyd IF, and whether the same effect was observed?

from multidiffusion.

Note that the averaging is done during the generation process, essentially averaging incremental diffusion updates. Intuitively, the model converges to a path under which the distance of each denoising direction from the average becomes very small (i.e., all constraints are satisfied).

In case the different denoising suggestions pull to extremely different directions across the entire generation, the quality of the result should be affected accordingly. However, in the applications we considered, this assumption on the ability of the model to converge to a meaningful denoising trajectory empirically hold.

I tried to run a panorama generation on the Deep-Floyd model and the results were similar to SD, but this model requires heavy memory resources for using all cascaded model, which was limiting.

from multidiffusion.

alright, that makes sense, thank you!

I see, I've been experimenting only with SD, and if it does work with DF then maybe I should try that

from multidiffusion.