Comments (8)
i did some more testing with SDXL. from what i can see, it's probably not really worth using for SDXL at least from a performance standpoint. also the available blocks for SDXL seem to be
input: 4,5,7,8
output: 0,1,2,3,4,5
you can try using something like input 4,5
, output 3,4,5
however even at pretty high res it's only a small speed increase. testing with deep shrink default settings, generating to 2560x1792 i get 3.47IT/s without the attention patch and 3.11IT/s with it.
the higher resolution you use, more steps, and slower the sampler (i.e. samplers that call the model multiple times like dpmpp_2s_ancestral or dpmpp_sde) the more benefit you'll see. my test was with dpmpp_2s_ancestral.
it does seem like it helps with artifacts at high res, however SDXL in general doesn't seem to tolerate deep shrink very well.
example workflow (should have metadata unless github strips it):
from comfyui.
experimental implementation of the remaining parts: https://github.com/blepping/comfyui_jankhidiffusion
i strongly recommend reading the README since there are more than a few gotchas.
from comfyui.
i made a node with the MSW-MSA attention part: https://gist.github.com/blepping/02e389f660112097983684a8ea8093b1
it's a noticeable speed increase for high res generation and seems like it might also increase quality/reduce artifacts.
from comfyui.
i made a node with the MSW-MSA attention part: https://gist.github.com/blepping/02e389f660112097983684a8ea8093b1
it's a noticeable speed increase for high res generation and seems like it might also increase quality/reduce artifacts.
doesn't seem to do any difference with sdxl or is it me ? Are we supposed connect this after checkpoint and enter higher resolution numbers ? Or does it supposed to even make the default resolution faster ?
from comfyui.
i made a node with the MSW-MSA attention part: https://gist.github.com/blepping/02e389f660112097983684a8ea8093b1
it's a noticeable speed increase for high res generation and seems like it might also increase quality/reduce artifacts.doesn't seem to do any difference with sdxl or is it me ? Are we supposed connect this after checkpoint and enter higher resolution numbers ? Or does it supposed to even make the default resolution faster ?
Using HiDiffusion, you can prevent the phenomenon of having eight limbs appear even at high resolutions like 2048x2048.
from comfyui.
i made a node with the MSW-MSA attention part: https://gist.github.com/blepping/02e389f660112097983684a8ea8093b1
it's a noticeable speed increase for high res generation and seems like it might also increase quality/reduce artifacts.doesn't seem to do any difference with sdxl or is it me ? Are we supposed connect this after checkpoint and enter higher resolution numbers ? Or does it supposed to even make the default resolution faster ?
Using HiDiffusion, you can prevent the phenomenon of having eight limbs appear even at high resolutions like 2048x2048.
I know about that but this implementation doesn't seem to work at all, neither it speeds up at normal resolutions or high resolutions nor it seem to enhance the quality at any setting. (used both 2,3 - 3,4,5 and also default values (that are meant to be for sd1.5) both doesn't make a difference with sdxl.
from comfyui.
I know about that but this implementation doesn't seem to work at all, neither it speeds up at normal resolutions or high resolutions nor it seem to enhance the quality at any setting. (used both 2,3 - 3,4,5 and also default values (that are meant to be for sd1.5) both doesn't make a difference with sdxl.
my node only implements the MSW-MSA attention part, not the RAU-net part. i haven't done much testing with SDXL, i just checked that i still got reasonable results. with sd1.5 at least, the speed increase is about 30% at high resolution (i.e. 1536x1536) and it may help with artifacts but just the attention part doesn't let you generate natively at high resolution. you'll need to combine it with something like deep shrink.
the default sd1.5 settings aren't going to work properly with SDXL, SDXL doesn't even have an 11th block. hidiffusion's code has:
down_blocks.1.attentions.0.transformer_blocks.0
down_blocks.1.attentions.0.transformer_blocks.1
down_blocks.1.attentions.1.transformer_blocks.0
down_blocks.1.attentions.1.transformer_blocks.1
up_blocks.1.attentions.0.transformer_blocks.0
up_blocks.1.attentions.0.transformer_blocks.1
up_blocks.1.attentions.1.transformer_blocks.0
up_blocks.1.attentions.1.transformer_blocks.1
up_blocks.1.attentions.2.transformer_blocks.0
up_blocks.1.attentions.2.transformer_blocks.1
for sdxl, but that's the diffusers naming convention. i thought the conversion was first_number*3+second_number
but that might be wrong. (down blocks are input, up blocks are output)
from comfyui.
thank you! We can close this out
from comfyui.
Related Issues (20)
- OneTrainer loras trained with clip l and t5 keys are not loaded HOT 3
- Unable to fetch infor for lora/checkpoint HOT 3
- An error occurred while running to the InitFlux LoRATraining node,Repeatedly installing multiple times on the autodl platform, this error persists HOT 1
- [Bug]: Exception in IPAdapterUnifiedLoader HOT 1
- InitFluxLoRATraining datasets should not be an empty iterable
- FLUX CLIPTextEncode Erorr - NoneType' object has no attribute 'replace' HOT 2
- Torch not being compiled with CUDA enabled HOT 7
- UI Collision with mape helper HOT 1
- Save/Export Workflow dropdown arrow overlaps with the refresh-button HOT 2
- Integration of new Sampler: DC-Solver HOT 1
- ComfyUI really needs to add XSRF-protection, or make it VERY clear to its users thats its insecure to run (even on `localhost`) HOT 3
- Opening second ComfyUI window kills running one HOT 10
- Allocation on device error HOT 1
- OOM Allocation on device error on 4090 HOT 1
- After updating the latest version, I get an error,when I start running and can't open the page HOT 18
- New API, using polling task ID to obtain accurate progress HOT 1
- blank screen after update comfyui HOT 4
- CLIPTextEncode Allocation on device since update HOT 1
- compute_index_ranges_weights" not implemented for 'Half' HOT 3
- Flux errors since latest update HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from comfyui.