szhublox / ambw_comfyui Goto Github PK

So on the surface when you save SDXL checkpoints, you can continue to merge them in comfyui, and render images without problems. However if you load them up in kohya to fine-tune, or use them in Automatic1111's checkpoint merger tab, or use the stable-diffusion-model-toolkit you get tons of errors.

So here's the rabbit hole I went down and the dirty fix that seems to allow the model to work in everything. It would be great though if it would save structurally sound SDXL models somehow in order to remove the need for the workaround.

Test Case Workflow:

Kohya throws the following error:

The SD Toolkit Automatic1111 extension throws this. It thinks that it is a SD 1.5 VAE and want to prune most of the data:

In the advanced tab, it shows that it is actually missing SDXL classes as well as positional IDs:
advanced_tab.txt

When testing non-comfyui saved checkpoints it would show something like this:

Automatic1111 Checkpoint merger tab initially throws this error: AUTOMATIC1111/stable-diffusion-webui#10460

I then tried this comment:

Which then resulted in this error:

So that lead me to believe that it doesn't like the structure, as well as the metadata.

So I then trimmed the metadata with this script from a friend: https://gist.github.com/rvkwi/2df007fa3d6249474e18a7aa47d42507

Now I can use the checkpoint merger tab just fine. However the structure remains broken if I just resave it.

So the 2nd step of the workaround besides trimming the metadata was to merge with the base SDXL model again in position A with weighted_sum at 0.999999999999999999999. Example:

Now the toolkit correctly shows me this:

Any help getting this to work just in comfyui would be awesome, and I hope this little rabbit hole helps deduce the problem.

I just set up a new workflow and grabbed two models to merge, default settings, no images are generated, and this is the error it produces:

* Auto Merge Block Weighted 326:
  - Required input is missing: filename_prefix
Output will be ignored
invalid prompt: {'type': 'prompt_outputs_failed_validation', 'message': 'Prompt outputs failed validation', 'details': 'Required input is missing: filename_prefix', 'extra_info': {}}

Tried your Auto Merge node today and keeps telling me no module named clip
Error occurred when executing Auto Merge Block Weighted:

No module named 'clip'

File "D:\comfy\ComfyUI_windows_portable\ComfyUI\execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "D:\comfy\ComfyUI_windows_portable\ComfyUI\execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "D:\comfy\ComfyUI_windows_portable\ComfyUI\execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "D:\comfy\ComfyUI_windows_portable\ComfyUI\custom_nodes\ambw_comfyui_init_.py", line 117, in ambw
self.classifier = importlib.import_module(
File "importlib_init_.py", line 126, in import_module
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "D:\comfy\ComfyUI_windows_portable\ComfyUI\custom_nodes\ambw_comfyui\classifiers\laion.py", line 4, in
import clip