kohya-ss / sd-webui-additional-networks Goto Github PK

Hi i am getting the following error, what might be the issue ?

Error loading script: additional_networks.py
Traceback (most recent call last):
File "F:\Super SD\stable-diffusion-webui\modules\scripts.py", line 205, in load_scripts
module = script_loading.load_module(scriptfile.path)
File "F:\Super SD\stable-diffusion-webui\modules\script_loading.py", line 13, in load_module
exec(compiled, module.dict)
File "F:\Super SD\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\additional_networks.py", line 421, in
model = xy_grid.AxisOption(f"AddNet Model {i+1}", str, lambda p, x, xs, i=i: apply_model(p, x, xs, i), format_lora_model, confirm_models)
TypeError: AxisOption.new() missing 1 required positional argument: 'cost'

During training, los=nan forever.

using GTX 1660

I get this error at launch. Any ideas?

Launching Web UI with arguments: --xformers --api --gradio-img2img-tool color-sketch
Error loading script: additional_networks.py
Traceback (most recent call last):
  File "C:\ai\stable-diffusion-webui\modules\scripts.py", line 195, in load_scripts
    module = script_loading.load_module(scriptfile.path)
  File "C:\ai\stable-diffusion-webui\modules\script_loading.py", line 13, in load_module
    exec(compiled, module.__dict__)
  File "C:\ai\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\additional_networks.py", line 417, in <module>
    for script_class, path, basedir, script_module in scripts.scripts_data:
ValueError: not enough values to unpack (expected 4, got 3)

Stable Diffusion WebUI had Z type update on X/Y Plot, then it disappeared.

If you're using the Gradio UI from a different machine than the one hosting, the button to select the location of the LoRA will open a file selection dialog box on the host, not on the machine being used. From the user's perspective, it will look like the button isn't working. If you go to the host machine and select the LoRA, it will work, but it can't be set from a client unless they know the file path relative to the host machine and enter it manually.

Last downloaded assembly AUTOMATIC1111 January 11th. Installed additional-networks today, restarted web, restarted webui and still nothing happens.

that way we can load up loras natively or with this extension

Thank you for your work.
But I have a small request, is it possible to save an equivalent "_last.safetensors" when generating images?
For example, this model is trained in several batches of materials, and the proportionally fused model is pleasing in terms of image output.
If it could be done with just one model, that would be great.
This way, if I have a need, I can directly change the name and use this fused model instead of using several mixed models.
I'm sorry, if it's not possible, please closed issus, The built-in webui is not very user-friendly and not equivalent.

I would like to be able to add multiple extra paths to scan for LoRA models.

index 6bd87ad..6c579c9 100644
--- a/scripts/additional_networks.py
+++ b/scripts/additional_networks.py
@@ -26,9 +26,10 @@ def update_lora_models():
global lora_models
res = {}
paths = [lora_models_dir]

• extra_lora_path = shared.opts.data.get("additional_networks_extra_lora_path", None)
• if extra_lora_path and os.path.exists(extra_lora_path):
• paths.append(extra_lora_path)

• extra_lora_paths = shared.opts.data.get("additional_networks_extra_lora_path", None)
• for extra_lora_path in extra_lora_paths.split(","):

•  if extra_lora_paths and os.path.exists(extra_lora_path):
  

•    paths.append(extra_lora_path)
  

Maybe I don't understand something, and I will be very glad to help. I trained a neural network through Dreambooth using LoRA. I have a bunch of LoRA(.pt) files with different checkpoints. In the form of ckpt they work. But when I use them in additional networks, they don't work. They do not affect the image in any way, and even more so they do not add a trained character.

Hello! First of all, thank you for developing this useful extension and related training scripts.
I tried training and using the results to great success. Allowing all this in "average consumer" hardware is a tremendous feat.

Problem

Anyway, in a recent update, when the selection button was introduced, I found a strange issue. I get this error when clicking on the little folder icon next to the path:

I did install tkinter via pip in this virtual environment (Python 3.10.7 (main, Nov 24 2022, 19:45:47) [GCC 12.2.0]) , system is Ubuntu 22.10 64b, on a up-to-date webui clone (currently 874b975b) and sd-webui-additional-networks commit bec3d72.

The extension works normally when giving a LoRA path manually, as before, it's just the selector not functioning, everything else works perfectly, so it's a minor issue, but thought it'd be useful to bring it up.

Fix

To solve the issue I had to install the python3-tk package in my global system packages (I assume using pip outside the virtual env would work as well). After that the file selection dialog appears fine.

Proposed solution

Can a note be added to README noting that the tkinter package needs to be installed globally, as the one in the virtual environment will apparently not be used? Perhaps a browser-based dialog can be invoked instead, eliminating the dependency.

additional-networks option isn't visible no more on auto11 web UI after the new update,any fix?

So after closing #45 I wrote https://github.com/space-nuko/sd-webui-model-metadata-editor and it works pretty well. However I'm finding a few downsides from having the functionality separated from additional_networks:

1. No ability to pass a model from additional_networks into the metadata editor and vice-versa. I have to go to the dropdown to select the model a second time even though it's already selected in one place. The dropdown is super inconvenient with hundreds of models and it's double the work each time
2. The metadata editor tab makes the additional networks tab redundant, the additional networks tab is pretty empty currently
3. Model hashing has to be done separately for each extension since they have no knowledge of each others' hashing caches, eating up additional time doing the same exact work two times. It makes much more sense for them to share a single cache
4. Less adoption of the metadata system since many more people use additional_networks than they do model-metadata-editor. If nobody knows the metadata is even there then it doesn't really help anyone

I do know it's a lot of code, but I'm mulling whether or not the extra maintenance burden is worth the tighter integration with additional_networks. Perhaps this integration could be accomplished while keeping the separation, for example with the "pass to txt2img" feature that could be done by exposing the Gradio components in additional_networks so they can be manipulated with event handlers in model-metadata-editor. (But in that case I'm not sure about the other way around, passing the models in additional_networks into model-metadata-editor)

lora: https://civitai.com/models/7466

apparently made using the "latest kohya trainer"
as mentioned in https://civitai.com/models/7466?commentId=21696&modal=commentThread

  File "/usr/local/lib/python3.10/site-packages/gradio/routes.py", line 337, in run_predict
    output = await app.get_blocks().process_api(
  File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1015, in process_api
    result = await self.call_function(
  File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 833, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/stable-diffusion-webui/extensions/sd-webui-additional-networks/scripts/metadata_editor.py", line 249, in refresh_metadata
    count_max = max(top_tags.values())
ValueError: max() arg is an empty sequence

is the error in console

is only happening with vary new loras
older ones still load and edit metadata fine

Is it possible to put LORA weights to X/Y/Z plot in A1111? Both simple one and 2 that appears if "Separate UNet/Text Encoder weights" is selected. I'd like to have a way to make automatic generations with different weghts.

I encountered the same problem as #41, which states "LoRAs have no effects".

Background

I'm using SSDT to train LoRAs, the LoRA layers implementations are from loralib, which have weight scaling = (rank / alpha).

So if I want to use a alpha = 1, rank-16 LoRA produced by SSDT in AddNet, scale should be set to 1/16.

Some users found this additional scaling not convenient, I added an unscale weight option to scale weight by (alpha / rank) when converting SSDT checkpoint to AddNet format.

Investigation

I looked the state dict after unscaling.

All of the tensors have pretty small values, which can cause low numerical stability. ~20% of them have zero values, in this 20%, 15% in text encoder, 85% in UNet.

Experiment

In one LoRA (rank=16, alpha=1) I trained,

• All unscaled LoRAs have basically no effect.
• Not unscaled, scale = 0.0625 (1 / 16) worked as normal.

Conclusion and Solution

I suspect those zeros are products of underflow, which probably is the cause of #41.

Those underflows happens more often if rank is high.

At training time, add option: "alpha" to scale LoRA like loralib. Save alpha to LoRA metadata.

At inference time, add option: "scale weight" to scale LoRA weight by rank / alpha.

Backward Compatibility

Unfortunately, as you can imagine, almost all existing LoRAs have already underflowed.

If "scale weight" is enabled, for still using old LoRAs, if a LoRA have no alpha in metadata, do not scale.

Additional: NaNs

After AUTOMATIC1111/stable-diffusion-webui@9991967, when generating images, those underflowed LoRAs sometimes produces NaN errors.

Some users reported loss=NaN when using https://github.com/Linaqruf/kohya-trainer/ and https://github.com/Mikubill/naifu-diffusion/, especially at high rank. I suspect that's related to this issue.

Was getting the error after automatic1111's update so i used chatgpt to fix it ...

not sure how Github works with fixes to errors so i gonna put it here:

import os
import glob
import zipfile
import json
import stat
import sys
import inspect
from collections import OrderedDict

import torch

import modules.scripts as scripts
from modules import shared, script_callbacks
import gradio as gr

from modules.processing import Processed, process_images
from modules import sd_models
import modules.ui

from scripts import lora_compvis


MAX_MODEL_COUNT = 5
LORA_MODEL_EXTS = [".pt", ".ckpt", ".safetensors"]
lora_models = {}      # "My_Lora(abcd1234)" -> C:/path/to/model.safetensors
lora_model_names = {} # "my_lora" -> "My_Lora(abcd1234)"
lora_models_dir = os.path.join(scripts.basedir(), "models/lora")
os.makedirs(lora_models_dir, exist_ok=True)


def traverse_all_files(curr_path, model_list):
  f_list = [(os.path.join(curr_path, entry.name), entry.stat()) for entry in os.scandir(curr_path)]
  for f_info in f_list:
    fname, fstat = f_info
    if os.path.splitext(fname)[1] in LORA_MODEL_EXTS:
      model_list.append(f_info)
    elif stat.S_ISDIR(fstat.st_mode):
      model_list = traverse_all_files(fname, model_list)
  return model_list


def get_all_models(sort_by, filter_by, path):
  res = OrderedDict()
  fileinfos = traverse_all_files(path, [])
  filter_by = filter_by.strip(" ")
  if len(filter_by) != 0:
    fileinfos = [x for x in fileinfos if filter_by.lower() in os.path.basename(x[0]).lower()]
  if sort_by == "name":
    fileinfos = sorted(fileinfos, key=lambda x: os.path.basename(x[0]))
  elif sort_by == "date":
    fileinfos = sorted(fileinfos, key=lambda x: -x[1].st_mtime)
  elif sort_by == "path name":
    fileinfos = sorted(fileinfos)

  for finfo in fileinfos:
    filename = finfo[0]
    name = os.path.splitext(os.path.basename(filename))[0]
    # Prevent a hypothetical "None.pt" from being listed.
    if name != "None":
      res[name + f"({sd_models.model_hash(filename)})"] = filename

  return res


def find_closest_lora_model_name(search: str):
    if not search:
        return None
    if search in lora_models:
        return search
    search = search.lower()
    if search in lora_model_names:
        return lora_model_names.get(search)
    applicable = [name for name in lora_model_names.keys() if search in name.lower()]
    if not applicable:
        return None
    applicable = sorted(applicable, key=lambda name: len(name))
    return lora_model_names[applicable[0]]


def update_lora_models():
  global lora_models, lora_model_names
  res = OrderedDict()
  paths = [lora_models_dir]
  extra_lora_path = shared.opts.data.get("additional_networks_extra_lora_path", None)
  if extra_lora_path and os.path.exists(extra_lora_path):
    paths.append(extra_lora_path)
  for path in paths:
    sort_by = shared.opts.data.get("additional_networks_sort_models_by", "name")
    filter_by = shared.opts.data.get("additional_networks_model_name_filter", "")
    found = get_all_models(sort_by, filter_by, path)
    res = {**found, **res}
  lora_models = OrderedDict(**{"None": None}, **res)
  lora_model_names = {}
  for name_and_hash, filename in lora_models.items():
      if filename == None:
          continue
      name = os.path.splitext(os.path.basename(filename))[0].lower()
      lora_model_names[name] = name_and_hash


update_lora_models()


class Script(scripts.Script):
  def __init__(self) -> None:
    super().__init__()
    self.latest_params = [(None, None, None)] * MAX_MODEL_COUNT
    self.latest_networks = []
    self.latest_model_hash = ""

  def title(self):
    return "Additional networks for generating"

  def show(self, is_img2img):
    return scripts.AlwaysVisible

  def ui(self, is_img2img):
    # NOTE: Changing the contents of `ctrls` means the XY Grid support may need
    # to be updated, see end of file
    ctrls = []
    model_dropdowns = []
    self.infotext_fields = []
    with gr.Group():
      with gr.Accordion('Additional Networks', open=False):
        enabled = gr.Checkbox(label='Enable', value=False)
        ctrls.append(enabled)
        self.infotext_fields.append((enabled, "AddNet Enabled"))

        for i in range(MAX_MODEL_COUNT):
          with gr.Row():
            module = gr.Dropdown(["LoRA"], label=f"Network module {i+1}", value="LoRA")
            model = gr.Dropdown(list(lora_models.keys()),
                                label=f"Model {i+1}",
                                value="None")

            weight = gr.Slider(label=f"Weight {i+1}", value=1.0, minimum=-1.0, maximum=2.0, step=.05)
          ctrls.extend((module, model, weight))
          model_dropdowns.append(model)

          self.infotext_fields.extend([
              (module, f"AddNet Module {i+1}"),
              (model, f"AddNet Model {i+1}"),
              (weight, f"AddNet Weight {i+1}"),
          ])

        def refresh_all_models(*dropdowns):
          update_lora_models()
          updates = []
          for dd in dropdowns:
            if dd in lora_models:
              selected = dd
            else:
              selected = "None"
            update = gr.Dropdown.update(value=selected, choices=list(lora_models.keys()))
            updates.append(update)
          return updates

        refresh_models = gr.Button(value='Refresh models')
        refresh_models.click(refresh_all_models, inputs=model_dropdowns, outputs=model_dropdowns)
        ctrls.append(refresh_models)

    return ctrls

  def set_infotext_fields(self, p, params):
    for i, t in enumerate(params):
      module, model, weight = t
      if model is None or model == "None" or len(model) == 0 or weight == 0:
        continue
      p.extra_generation_params.update({
          "AddNet Enabled": True,
          f"AddNet Module {i+1}": module,
          f"AddNet Model {i+1}": model,
          f"AddNet Weight {i+1}": weight,
      })

  def process(self, p, *args):
    unet = p.sd_model.model.diffusion_model
    text_encoder = p.sd_model.cond_stage_model

    def restore_networks():
      if len(self.latest_networks) > 0:
        print("restoring last networks")
        for network, _ in self.latest_networks[::-1]:
          network.restore(text_encoder, unet)
        self.latest_networks.clear()

    if not args[0]:
      restore_networks()
      return

    params = []
    for i, ctrl in enumerate(args[1:]):
      if i % 3 == 0:
        param = [ctrl]
      else:
        param.append(ctrl)
        if i % 3 == 2:
          params.append(param)

    models_changed = (len(self.latest_networks) == 0)                   # no latest network (cleared by check-off)
    models_changed = models_changed or self.latest_model_hash != p.sd_model.sd_model_hash
    if not models_changed:
      for (l_module, l_model, l_weight), (module, model, weight) in zip(self.latest_params, params):
        if l_module != module or l_model != model or l_weight != weight:
          models_changed = True
          break

    if models_changed:
      restore_networks()
      self.latest_params = params
      self.latest_model_hash = p.sd_model.sd_model_hash

      for module, model, weight in self.latest_params:
        if model is None or model == "None" or len(model) == 0:
          continue
        if weight == 0:
          print(f"ignore because weight is 0: {model}")
          continue

        model_path = lora_models.get(model, None)
        if model_path is None:
          raise RuntimeError(f"model not found: {model}")

        if model_path.startswith("\"") and model_path.endswith("\""):             # trim '"' at start/end
          model_path = model_path[1:-1]
        if not os.path.exists(model_path):
          print(f"file not found: {model_path}")
          continue

        print(f"{module} weight: {weight}, model: {model}")
        if module == "LoRA":
          if os.path.splitext(model_path)[1] == '.safetensors':
            from safetensors.torch import load_file
            du_state_dict = load_file(model_path)
          else:
            du_state_dict = torch.load(model_path, map_location='cpu')

          network, info = lora_compvis.create_network_and_apply_compvis(du_state_dict, weight, text_encoder, unet)
          network.to(p.sd_model.device, dtype=p.sd_model.dtype)         # in medvram, device is different for u-net and sd_model, so use sd_model's

          print(f"LoRA model {model} loaded: {info}")
          self.latest_networks.append((network, model))
      if len(self.latest_networks) > 0:
        print("setting (or sd model) changed. new networks created.")

    self.set_infotext_fields(p, self.latest_params)


def read_lora_metadata(model_path, module):
  if model_path.startswith("\"") and model_path.endswith("\""):             # trim '"' at start/end
    model_path = model_path[1:-1]
  if not os.path.exists(model_path):
    return None

  metadata = None
  if module == "LoRA":
    if os.path.splitext(model_path)[1] == '.safetensors':
      from safetensors.torch import safe_open
      with safe_open(model_path, framework="pt") as f:
        metadata = f.metadata()

  return metadata


def on_ui_tabs():
  with gr.Blocks(analytics_enabled=False) as additional_networks_interface:
    with gr.Row().style(equal_height=False):
      with gr.Column(variant='panel'):
        with gr.Row():
          module = gr.Dropdown(["LoRA"], label=f"Network module (used throughout this tab)", value="LoRA", interactive=True)
          model = gr.Dropdown(list(lora_models.keys()), label=f"Model", value="None", interactive=True)
          modules.ui.create_refresh_button(model, update_lora_models, lambda: {"choices": list(lora_models.keys())}, "refresh_lora_models")

        with gr.Row():
            with gr.Column():
              gr.HTML(value="Get comma-separated list of models (for XY Grid)")
              model_dir = gr.Textbox("", label=f"Model directory", placeholder="Optional, uses selected model's directory if blank")
              model_sort_by = gr.Radio(label="Sort models by", choices=["name", "date", "path name"], value="name", type="value")
              get_list_button = gr.Button("Get List")
            with gr.Column():
              model_list = gr.Textbox(value="", label="Model list", placeholder="Model list will be output here")

      with gr.Column():
        metadata_view = gr.JSON(data="test", label="Network metadata")

    def update_metadata(module, model):
      if model == "None":
        return {}

      model_path = lora_models.get(model, None)
      if model_path is None:
        metadata_view.data = f"file not found: {model}"

      metadata = read_lora_metadata(model_path, module)

      if metadata is None:
        return "No metadata found."
      else:
        return metadata

    model.change(update_metadata, inputs=[module, model], outputs=[metadata_view])

    def output_model_list(module, model, model_dir, sort_by):
        if model_dir == "":
            # Get list of models with same folder as this one
            model_path = lora_models.get(model, None)
            if model_path is None:
                model_list.value = f"directory not found: {model_dir}"
            model_dir = os.path.dirname(model_path)

        if not os.path.isdir(model_dir):
            return f"directory not found: {model_dir}"

        found = get_all_models(sort_by, "", model_dir)
        return ", ".join(found.keys())

    get_list_button.click(output_model_list, inputs=[module, model, model_dir, model_sort_by], outputs=[model_list])

  return [(additional_networks_interface, "Additional Networks", "additional_networks")]


def update_script_args(p, value, arg_idx):
    for s in scripts.scripts_txt2img.alwayson_scripts:
        if isinstance(s, Script):
            args = list(p.script_args)
            # print(f"Changed arg {arg_idx} from {args[s.args_from + arg_idx - 1]} to {value}")
            args[s.args_from + arg_idx] = value
            p.script_args = tuple(args)
            break


def confirm_models(p, xs):
    for x in xs:
        if x in ["", "None"]:
            continue
        if not find_closest_lora_model_name(x):
            raise RuntimeError(f"Unknown LoRA model: {x}")


def apply_module(p, x, xs, i):
    update_script_args(p, True, 0)      # set Enabled to True
    update_script_args(p, x, 1 + 3 * i) # enabled, ({module}, model, weight), ...


def apply_model(p, x, xs, i):
    name = find_closest_lora_model_name(x)
    update_script_args(p, True, 0)
    update_script_args(p, name, 2 + 3 * i) # enabled, (module, {model}, weight), ...


def apply_weight(p, x, xs, i):
    update_script_args(p, True, 0)
    update_script_args(p, x, 3 + 3 * i) # enabled, (module, model, {weight), ...


LORA_METADATA_NAMES = {
    "ss_learning_rate": "Learning rate",
    "ss_text_encoder_lr": "Text encoder LR",
    "ss_unet_lr": "UNet LR",
    "ss_num_train_images": "# of training images",
    "ss_num_reg_images": "# of reg images",
    "ss_num_batches_per_epoch": "Batches per epoch",
    "ss_num_epochs": "Total epochs",
    "ss_batch_size_per_device": "Batch size/device",
    "ss_total_batch_size": "Total batch size",
    "ss_gradient_accumulation_steps": "Gradient accum. steps",
    "ss_max_train_steps": "Max train steps",
    "ss_lr_warmup_steps": "LR warmup steps",
    "ss_lr_scheduler": "LR scheduler",
    "ss_network_module": "Network module",
    "ss_network_dim": "Network dim",
    "ss_mixed_precision": "Mixed precision", 
    "ss_full_fp16": "Full FP16",
    "ss_v2": "V2",
    "ss_resolution": "Resolution",
    "ss_clip_skip": "Clip skip",
    "ss_max_token_length": "Max token length",
    "ss_color_aug": "Color aug",
    "ss_flip_aug": "Flip aug",
    "ss_random_crop": "Random crop",
    "ss_shuffle_caption": "Shuffle caption",
    "ss_cache_latents": "Cache latents",
    "ss_enable_bucket": "Enable bucket",
    "ss_min_bucket_reso": "Min bucket reso.",
    "ss_max_bucket_reso": "Max bucket reso.",
    "ss_seed": "Seed", 
    "ss_sd_model_name": "SD model name",
    "ss_vae_name": "VAE name"
}


def format_lora_model(p, opt, x):
    model = find_closest_lora_model_name(x)
    if model is None or model.lower() in ["", "none"]:
        return "None"

    value = xy_grid.format_value(p, opt, model)

    model_path = lora_models.get(model)
    metadata = read_lora_metadata(model_path, "LoRA")
    if not metadata:
        return value

    metadata_names = shared.opts.data.get("additional_networks_xy_grid_model_metadata", "").split(",")
    if not metadata_names:
        return value

    for name in metadata_names:
        name = name.strip()
        if name in metadata:
            formatted_name = LORA_METADATA_NAMES.get(name, name)
            value += f"\n{formatted_name}: {metadata[name]}, "

    return value.strip(" ").strip(",")


for scriptDataTuple in scripts.scripts_data:
    if os.path.basename(scriptDataTuple.path) == "xy_grid.py":
        xy_grid = scriptDataTuple.module
        for i in range(MAX_MODEL_COUNT):
           model = xy_grid.AxisOption(f"AddNet Model {i+1}", str, lambda p, x, xs, i=i: apply_model(p, x, xs, i), format_lora_model, confirm_models)
           weight = xy_grid.AxisOption(f"AddNet Weight {i+1}", float, lambda p, x, xs, i=i: apply_weight(p, x, xs, i), xy_grid.format_value_add_label, None)
           xy_grid.axis_options.extend([model, weight])


def on_ui_settings():
    section = ('additional_networks', "Additional Networks")
    shared.opts.add_option("additional_networks_extra_lora_path", shared.OptionInfo("", "Extra path to scan for LoRA models (e.g. training output directory)", section=section))
    shared.opts.add_option("additional_networks_sort_models_by", shared.OptionInfo("name", "Sort LoRA models by", gr.Radio, {"choices": ["name", "date", "path name"]}, section=section))
    shared.opts.add_option("additional_networks_model_name_filter", shared.OptionInfo("", "LoRA model name filter", section=section))
    shared.opts.add_option("additional_networks_xy_grid_model_metadata", shared.OptionInfo("", "Metadata to show in XY-Grid label for Model axes, comma-separated (example: \"ss_learning_rate, ss_num_epochs\")", section=section))


def on_infotext_pasted(infotext, params):
    for i in range(MAX_MODEL_COUNT):
        if f"AddNet Module {i+1}" not in params:
            params[f"AddNet Module {i+1}"] = "LoRA"
        if f"AddNet Model {i+1}" not in params:
            params[f"AddNet Model {i+1}"] = "None"
        if f"AddNet Weight {i+1}" not in params:
            params[f"AddNet Weight {i+1}"] = "0"


script_callbacks.on_ui_tabs(on_ui_tabs)
script_callbacks.on_ui_settings(on_ui_settings)
script_callbacks.on_infotext_pasted(on_infotext_pasted)
```

If a model file has a comma in the filename (like model [v1, overfit].safetensors), then the prompt it's used in won't be able to be reproduced, since the prompt infotext is delimited by commas.

Should sanitize/normalize the filename before saving it to the infotext.

I use google colab and I tried to load a lora model for first time and it gave me this error.

Total progress: 0it [00:00, ?it/s]LoRA weight: 1, model: hinataHyuugaLora_hinata(e3b0c442)
Error running process: /content/stable-diffusion-webui/extensions/sd-webui-additional-networks/scripts/additional_networks.py
Traceback (most recent call last):
File "/content/stable-diffusion-webui/modules/scripts.py", line 347, in process
script.process(p, *script_args)
File "/content/stable-diffusion-webui/extensions/sd-webui-additional-networks/scripts/additional_networks.py", line 234, in process
du_state_dict = load_file(model_path)
File "/usr/local/lib/python3.8/dist-packages/safetensors/torch.py", line 98, in load_file
with safe_open(filename, framework="pt", device=device) as f:
Exception: Error while deserializing header: HeaderTooSmall

Reloading extensions
Reloading custom scripts
Error loading script: additional_networks.py
Traceback (most recent call last):
File "C:\stable-diffusion-webui\modules\scripts.py", line 184, in load_scripts
module = script_loading.load_module(scriptfile.path)
File "C:\stable-diffusion-webui\modules\script_loading.py", line 13, in load_module
exec(compiled, module.dict)
File "C:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\additional_networks.py", line 11, in
from scripts import lora_compvis, model_util, metadata_editor, xyz_grid_support
File "C:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\model_util.py", line 11, in
from modules import shared, sd_models, hashes
ImportError: cannot import name 'hashes' from 'modules' (unknown location)

Error loading script: metadata_editor.py
Traceback (most recent call last):
File "C:\stable-diffusion-webui\modules\scripts.py", line 184, in load_scripts
module = script_loading.load_module(scriptfile.path)
File "C:\stable-diffusion-webui\modules\script_loading.py", line 13, in load_module
exec(compiled, module.dict)
File "C:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\metadata_editor.py", line 14, in
from modules.ui_components import ToolButton
ModuleNotFoundError: No module named 'modules.ui_components'

Error loading script: model_util.py
Traceback (most recent call last):
File "C:\stable-diffusion-webui\modules\scripts.py", line 184, in load_scripts
module = script_loading.load_module(scriptfile.path)
File "C:\stable-diffusion-webui\modules\script_loading.py", line 13, in load_module
exec(compiled, module.dict)
File "C:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\model_util.py", line 11, in
from modules import shared, sd_models, hashes
ImportError: cannot import name 'hashes' from 'modules' (unknown location)

Error loading script: xyz_grid_support.py
Traceback (most recent call last):
File "C:\stable-diffusion-webui\modules\scripts.py", line 184, in load_scripts
module = script_loading.load_module(scriptfile.path)
File "C:\stable-diffusion-webui\modules\script_loading.py", line 13, in load_module
exec(compiled, module.dict)
File "C:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\xyz_grid_support.py", line 5, in
from scripts import model_util, util
File "C:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\model_util.py", line 11, in
from modules import shared, sd_models, hashes
ImportError: cannot import name 'hashes' from 'modules' (unknown location)

The hashing currently uses webui's old sd_models.model_hash(), which is prone to collisions

Just the tensor part of the file should be hashed instead, and the length of the hash increased somewhat, maybe 10-12 characters. Hashing the whole file can cause inconsistencies if the metadata is updated later

To maintain backwards compatibility the new hashes for all LoRAs will have to be calculated somehow, possibly on startup. For future models sd-scripts should precalculate the hash. I tried two options:

1. Serializing the model first in memory, hashing the tensors region, adding the resulting hash to the metadata, saving the model with the hash included
2. Hashing the raw torch.Tensor structures one-by-one

First method means the model has to be serialized twice, second means it just has to be in the tensors format but it's much slower, and both methods give different hashes

I would prefer the first method for speed and eat the cost of double serializing when saving new models trained with sd-scripts, unless there's a better method I haven't found out yet

Sorry for asking here, there is no discussion thread. What does "Separate UNet/Text Encoder weights" do? It separates Lora weights to "UNet Weight" and "TEnc Weight". I haven't found any iformation or explanation on it.

Hallo,

i would like to have an Option to display relative path in the drop-down menu of LoRAs.
Just the way the Checkpoint-Selection currently works.

This would greatly help with organizing the LoRAs.
I mostly have each downloaded LoRA in different folder, together with png images that contain embedded promts, readme.txt files and etc.

\Clothes\Traditional Japanese dress\jpdress.safetensors
\Clothes\Negligee\epoch-000004.safetensors
...
\Styles\Katsuhiro Otomo\KatsuhiroEP5safetensors
\Styles\Kentaro Miura\KM.safetensors

Hey there! I think we know one another. ;)

I've got lora training support added to my extension, with the outputs being sent to /models/lora, and the unet/tenc weights being saved in separate files, where the unet is the "base name", and the tenc has "_txt" applied.

Would it be possible for you to add support to your extension so people can natively use the weights they generate?

Or, at the very least, we should talk and come up with a standardized solution that we can agree on to KISS for people.

Nice work on this, BTW.

Getting this error when trying to use a Lora model trained with the Lora training GUI: https://github.com/bmaltais/kohya_ss

When trying to use the lora model, Automatic1111 gives me:

and the model Meta data:

The Web Gui is updated, just installed the kohya_ss gui, so I know that's up to date.
All lora model's I've downloaded work perfectly fine, only issues with the one's I'm trying to train.

I can see the tabs, but the image results do not seem to change.

I changed the options, but the image did not change.

Did I do anything wrong?

After updating to the latest commit of the webui as well as updating the extension, console throws an error while loading the extension and it is not showing up in the webui:

[AddNet] Updating model hashes...
1%|█▏ | 1/69 [00:00<00:06, 9.84it/s]
Error loading script: additional_networks.py
Traceback (most recent call last):
File "F:\stable-diffusion-webui\modules\scripts.py", line 218, in load_scripts
script_module = script_loading.load_module(scriptfile.path)
File "F:\stable-diffusion-webui\modules\script_loading.py", line 13, in load_module
exec(compiled, module.dict)
File "F:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\additional_networks.py", line 394, in
update_lora_models()
File "F:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\additional_networks.py", line 379, in update_lora_models
res, res_legacy = get_all_models(paths, sort_by, filter_by)
File "F:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\additional_networks.py", line 294, in get_all_models
for res in p.imap_unordered(hash_model_file, fileinfos):
File "C:\Users\admin\AppData\Local\Programs\Python\Python310\lib\multiprocessing\pool.py", line 873, in next
raise value
File "C:\Users\admin\AppData\Local\Programs\Python\Python310\lib\multiprocessing\pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "F:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\additional_networks.py", line 273, in hash_model_file
metadata = safetensors_hack.read_metadata(filename)
File "F:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\safetensors_hack.py", line 22, in read_metadata
metadata = json.loads(metadata_bytes)
File "C:\Users\admin\AppData\Local\Programs\Python\Python310\lib\json_init_.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Users\admin\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\admin\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Hi,

After training with lora and wanting to use my .pt file I get the following message:

LoRA weight: 1, model: test_1000(61a77e3f)
Error running process: C:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\additional_networks.py
Traceback (most recent call last):
File "C:\stable-diffusion-webui\modules\scripts.py", line 357, in process
script.process(p, *script_args)
File "C:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\additional_networks.py", line 238, in process
network, info = lora_compvis.create_network_and_apply_compvis(du_state_dict, weight, text_encoder, unet)
File "C:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\lora_compvis.py", line 83, in create_network_and_apply_compvis
for key, value in du_state_dict.items():
AttributeError: 'list' object has no attribute 'items'

The commits:
Dreambooth revision: 9f4d931a319056c537d24669cb950d146d1537b0
SD-WebUI revision: 91c8d0dcfc9a5d46aec47fd3eb34c633c104b5bb
SD-WebUI-additional-networks: 4e383e2

Can you help me?

When webui added official support for LoRAs, it also included an experimental toggle to apply the LoRA to outputs instead of inputs. Relevant commit:
AUTOMATIC1111/stable-diffusion-webui@1bfec87

This seems to be useful for applying styles without drastically changing the composition, but having this option in the settings means that it applies to every LoRA loaded at once, which may be a problem if you have one LoRA which you do want to allow to change the composition, and another that you do not. Plus, LoRAs loaded by Additional Networks cannot currently be applied to outputs.

I propose adding a checkbox per LoRA model loaded that allows you to apply to outputs on an individual basis. I think this is in-line with the extra control that the Additional Networks extension gives over native LoRA support. If whoever takes this on is worried about GUI clutter, maybe it could be toggled with the same checkbox that separates the weights into UNet/TEnc weights (or maybe shrinking the horizontal space of the "Network module #" dropdown, which currently only has LoRAs as an option anyway).

I don't know if I did something wrong or it's normal but the lora with certain weights don't work.

Ex: When I put 2 of my loras I made with 0,9 and 0,5 weight the one with 0,9 don't work ( the console says: ignore because weight is 0 )
but if I change that one from 0,9 to 0,8 it works.

Images

Benn getting this error whenever I try to generate an image with a lora. Anyone know what this or how to fix it?

Error running process: D:\Downloads\Diffusion\TRYING AGAIN\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\additional_networks.py
Traceback (most recent call last):
File "D:\Downloads\Diffusion\TRYING AGAIN\stable-diffusion-webui\modules\scripts.py", line 347, in process
script.process(p, *script_args)
File "D:\Downloads\Diffusion\TRYING AGAIN\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\additional_networks.py", line 238, in process
network, info = lora_compvis.create_network_and_apply_compvis(du_state_dict, weight, text_encoder, unet)
File "D:\Downloads\Diffusion\TRYING AGAIN\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\lora_compvis.py", line 78, in create_network_and_apply_compvis
state_dict = network.apply_lora_modules(du_state_dict) # some weights are applied to text encoder
File "D:\Downloads\Diffusion\TRYING AGAIN\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\lora_compvis.py", line 246, in apply_lora_modules
state_dict = LoRANetworkCompvis.convert_state_dict_name_to_compvis(self.v2, du_state_dict)
File "D:\Downloads\Diffusion\TRYING AGAIN\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\lora_compvis.py", line 152, in convert_state_dict_name_to_compvis
compvis_name = LoRANetworkCompvis.convert_diffusers_name_to_compvis(v2, tokens[0])
File "D:\Downloads\Diffusion\TRYING AGAIN\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\lora_compvis.py", line 141, in convert_diffusers_name_to_compvis
assert cv_name is not None, f"conversion failed: {du_name}"
AssertionError: conversion failed: cond_stage_model

I am getting these errors

LoRA weight: 2, model: suzuka_san_kansei(aa86e195)
dimension: None, alpha: None, multiplier: 2
Error running process: /notebooks/stable-diffusion-webui/extensions/sd-webui-additional-networks/scripts/additional_networks.py
Traceback (most recent call last):
File "/notebooks/stable-diffusion-webui/modules/scripts.py", line 347, in process
script.process(p, *script_args)
File "/notebooks/stable-diffusion-webui/extensions/sd-webui-additional-networks/scripts/additional_networks.py", line 238, in process
network, info = lora_compvis.create_network_and_apply_compvis(du_state_dict, weight, text_encoder, unet)
File "/notebooks/stable-diffusion-webui/extensions/sd-webui-additional-networks/scripts/lora_compvis.py", line 96, in create_network_and_apply_compvis
network = LoRANetworkCompvis(text_encoder, unet, multiplier=multiplier, lora_dim=network_dim, alpha=network_alpha)
File "/notebooks/stable-diffusion-webui/extensions/sd-webui-additional-networks/scripts/lora_compvis.py", line 233, in init
self.text_encoder_loras, te_rep_modules = create_modules(LoRANetworkCompvis.LORA_PREFIX_TEXT_ENCODER,
File "/notebooks/stable-diffusion-webui/extensions/sd-webui-additional-networks/scripts/lora_compvis.py", line 214, in create_modules
lora = LoRAModule(lora_name, child_module, self.multiplier, self.lora_dim, self.alpha)
File "/notebooks/stable-diffusion-webui/extensions/sd-webui-additional-networks/scripts/lora_compvis.py", line 41, in init
self.lora_down = torch.nn.Linear(in_dim, lora_dim, bias=False)
File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/linear.py", line 96, in init
self.weight = Parameter(torch.empty((out_features, in_features), **factory_kwargs))
TypeError: empty() received an invalid combination of arguments - got (tuple, dtype=NoneType, device=NoneType), but expected one of:

• (tuple of ints size, *, tuple of names names, torch.memory_format memory_format, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
• (tuple of ints size, *, torch.memory_format memory_format, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)

I just installed this extension in A1111's Web UI and after restarting I was wondering why the "additional networks" tab isn't showing up.
Upon checking the commandline I found the following error:

Error loading script: additional_networks.py
Traceback (most recent call last):
  File "D:\stable-diffusion-2\modules\scripts.py", line 229, in load_scripts
    script_module = script_loading.load_module(scriptfile.path)
  File "D:\stable-diffusion-2\modules\script_loading.py", line 11, in load_module
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "D:\stable-diffusion-2\extensions\sd-webui-additional-networks\scripts\additional_networks.py", line 268, in <module>
    xyz_grid_support.initialize(Script)
  File "D:\stable-diffusion-2\extensions\sd-webui-additional-networks\scripts\xyz_grid_support.py", line 156, in initialize
    model = xy_grid.AxisOption(f"AddNet Model {i+1}", str, lambda p, x, xs, i=i: apply_model(p, x, xs, i), format_lora_model, confirm_models, cost=0.5, choices=lambda i=i: get_axis_model_choices(i))
TypeError: AxisOption.__new__() got an unexpected keyword argument 'cost'

This in on torch 1.13.1 on A1111 GUI version from February 5 (latest at time of writing).

I keep getting the following error which I believe is affecting other network models from loading

size = du_state_dict[list(du_state_dict.keys())[0]].size()          # if conv2d size is like [320,4,1,1]
AttributeError: 'dict' object has no attribute 'size'

The XY fix seems to be causing other "AlwaysVisible" extensions to be shown twice, and also run twice.

To reproduce the issue:

# extensions/test/scripts/test.py
import modules.scripts as scripts
import gradio as gr

class Script(scripts.Script):
  def title(self):
    return 'Test'

  def show(self, is_img2img):
    return scripts.AlwaysVisible

  def ui(self, is_img2img):
    test = gr.Checkbox(label='test', default=True, elem_id=self.elem_id('test'))

    return [test]

I followed installation instructions, everything appeared to install fine with it displayed in the extensions tab. But I don't see this panel anywhere on the UI.

The extensions I am currently using (if any of them conflicts idk)

If the setting When reading generation parameters from text into UI (from PNG info or pasted text), do not change the selected model/checkpoint. is enabled, this extension should not clear or replace the current LoRA model when loading an image from PNG Info tab. This will keep functionality in line with the rest of the webui.

Also the default strength value is 0 now? So I have to change it manually to 1 every time I load the webui? At least when loading from PNG Info tab it is.

It would be nice if you could leave the licence file as you put at https://github.com/kohya-ss/sd-scripts

Can you please help, I encountered an error like this when I was generating:

LoRA weight: 1, model: Mix2R4 (3a125b2e)
dimension: 1000, multiplier: 1
create LoRA for Text Encoder: 72 modules.
create LoRA for U-Net: 192 modules.
Error running process: C:\Stable Diffusion\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\additional_networks.py
Traceback (most recent call last):
  File "C:\Stable Diffusion\stable-diffusion-webui\modules\scripts.py", line 338, in process
    script.process(p, *script_args)
  File "C:\Stable Diffusion\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\additional_networks.py", line 177, in process
    network, info = lora_compvis.create_network_and_apply_compvis(du_state_dict, weight, text_encoder, unet)
  File "C:\Stable Diffusion\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\lora_compvis.py", line 78, in create_network_and_apply_compvis
    state_dict = network.apply_lora_modules(du_state_dict)              # some weights are applied to text encoder
  File "C:\Stable Diffusion\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\lora_compvis.py", line 246, in apply_lora_modules
    state_dict = LoRANetworkCompvis.convert_state_dict_name_to_compvis(self.v2, du_state_dict)
  File "C:\Stable Diffusion\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\lora_compvis.py", line 152, in convert_state_dict_name_to_compvis
    compvis_name = LoRANetworkCompvis.convert_diffusers_name_to_compvis(v2, tokens[0])
  File "C:\Stable Diffusion\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\lora_compvis.py", line 141, in convert_diffusers_name_to_compvis
    assert cv_name is not None, f"conversion failed: {du_name}"
AssertionError: conversion failed: betas

The result was the same as when I wasn't using lora.

Hi,
I was wondering if you plan to support LORA models created from other training code, such as from the Dreambooth extension?

I already trained some with that, so it would be nice to use them without training again.

Any chance of adding info about used additional networks in image metadata after rendering?
Like add this info:

• Network type (LoRa, etc.)
• Model
• Weight

Error running process: D:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\additional_networks.py
Traceback (most recent call last):
File "D:\stable-diffusion-webui\modules\scripts.py", line 347, in process
script.process(p, *script_args)
File "D:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\additional_networks.py", line 238, in process
network, info = lora_compvis.create_network_and_apply_compvis(du_state_dict, weight, text_encoder, unet)
File "D:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\lora_compvis.py", line 72, in create_network_and_apply_compvis
size = du_state_dict[list(du_state_dict.keys())[0]].size() # if conv2d size is like [320,4,1,1]
AttributeError: 'tuple' object has no attribute 'size'

getting distorted results

additional_networks.py", line 421, in
model = xy_grid.AxisOption(f"AddNet Model {i+1}", str, lambda p, x, xs, i=i: apply_model(p, x, xs, i), format_lora_model, confirm_models, 0.5)
TypeError: AxisOption.new() takes 6 positional arguments but 7 were given

C:\Users\shake\stable-diffusion-webui\venv\lib\site-packages\pkg_resources_init_.py:123: PkgResourcesDeprecationWarning: ansformers is an invalid version and will not be supported in a future release
warnings.warn(
Error loading script: additional_networks.py
Traceback (most recent call last):
File "C:\Users\shake\stable-diffusion-webui\modules\scripts.py", line 205, in load_scripts
module = script_loading.load_module(scriptfile.path)
File "C:\Users\shake\stable-diffusion-webui\modules\script_loading.py", line 13, in load_module
exec(compiled, module.dict)
File "C:\Users\shake\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\additional_networks.py", line 20, in
from modules import shared, sd_models, hashes
ImportError: cannot import name 'hashes' from 'modules' (unknown location)

Problem

I created some LoRA embeddings with the setting network dim = 1024. When I tested them on your extension, it detected them are having 768 network dim instead of 1024 dim, therefore it skipped almost all the layers in these embeddings due to the mismatch in tensor sizes. I checked the code and found out the bug is at line 73, lora_compvis.py. When I changed the value to 1024 manually, it worked. I guess that line can't handle any embedding which has more than 768 network dim.

Sample files

https://huggingface.co/hr16/Miwano-Rag-LoRA/
I trained these embeddings using kohya-LoRA-finetuner colab by Linaqruf

Hi,

Can't use BLIP, GIT or WD14 for captioning, having this message each time :

Some help will be nice !

Error loading script: additional_networks.py
Traceback (most recent call last):
  File "/content/stable-diffusion-webui/modules/scripts.py", line 229, in load_scripts
    script_module = script_loading.load_module(scriptfile.path)
  File "/content/stable-diffusion-webui/modules/script_loading.py", line 11, in load_module
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 848, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/content/stable-diffusion-webui/extensions/sd-webui-additional-networks/scripts/additional_networks.py", line 159, in <module>
    def split_path_list(path_list: str) -> list[str]:
TypeError: 'type' object is not subscriptable

(on actual version A1111)

I do not know what is the cause of this issue unfortuantely

I'm getting this error in the terminal when I start the web ui

Error loading script: additional_networks.py
Traceback (most recent call last):
File "/home/mouse/stable-diffusion-webui/modules/scripts.py", line 195, in load_scripts
module = script_loading.load_module(scriptfile.path)
File "/home/mouse/stable-diffusion-webui/modules/script_loading.py", line 13, in load_module
exec(compiled, module.dict)
File "/home/mouse/stable-diffusion-webui/extensions/sd-webui-additional-networks/scripts/additional_networks.py", line 419, in
xy_grid = scriptDataTuple.module
AttributeError: 'ScriptClassData' object has no attribute 'module'

Never mind: fixed by updating automatic1111

Not an issue, but is there any way to apply separate weights to LORA text encoder and UNET
For example:
α unet = 0.6
α text = 0.8