Performance issue starting at commit 08f92d5 adding 10 sec delay to start KSampler about comfyui HOT 3 CLOSED

I think this is because it's loading the actual transformer/unet when it reaches the ksampler. When it first looks like it's loading the checkpoint, it's probably just loading the t5 or clips for the prompt, depending on what model you're working with. Once it reaches the ksampler, if you have your task manager open, you'll see that your disk usage will indicate it's loading something.

I noticed this messing around with flux. Probably makes memory management much easier for bigger models, like ones that use t5 which takes up a lot of memory. That way, it can load the text encoders, do the encoding and then offload the models or purge them to make room for the model. I think there was also a PR that allows encoders to load directly to the vram now too, saving a potential copy step as well.

from comfyui.

Should be fixed now.

from comfyui.

I can confirm that this has been fixed in the latest commits, as I can no longer reproduce the issue.

got prompt
loaded completely 5201.311734390259 4897.0483474731445
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:16<00:00,  1.77it/s]
Prompt executed in 20.28 seconds
got prompt
loaded completely 5201.273587417602 4897.0483474731445
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:17<00:00,  1.76it/s]
Prompt executed in 20.22 seconds
got prompt
loaded completely 5229.418118667602 4897.0483474731445
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:17<00:00,  1.75it/s]
Prompt executed in 20.32 seconds

With a bonus that the total execution time is now 3-4 seconds faster. Thanks for all your incredible work.

from comfyui.