In version 24.0.4, the argument splitter is not good, because I get an error for this

Cannot use betas parameter for Lora about kohya_ss HOT 9 CLOSED

mykeehu commented on July 28, 2024

Cannot use betas parameter for Lora

from kohya_ss.

Comments (9)

bmaltais commented on July 28, 2024 2

Hummm... might be interesting co write a validator for things like this to raise an error when the syntax does not comply with what sd-scripts expect... or maybe just to remove the offending characters... I will think about it... should not be too difficult...

from kohya_ss.

5KilosOfCheese commented on July 28, 2024 1

Okay, but the parameter for betas looks like this, so I can't get around the comma: (0.9, 0.999) https://github.com/konstmish/prodigy So if I need the comma, what's the solution? I do not use toml.

"betas=(0.9,0.99)"
You can't have a space in the brackets.

from kohya_ss.

bmaltais commented on July 28, 2024

Make sure Thera are no comma between each group of parameters. Can you share the json config that cause the issue?

from kohya_ss.

mykeehu commented on July 28, 2024

Here is my config, I deleted personal paths. There's a comma in betas parameter. This was not a problem in previous versions:

{
  "LoRA_type": "Standard",
  "LyCORIS_preset": "full",
  "adaptive_noise_scale": 0,
  "additional_parameters": "",
  "block_alphas": "",
  "block_dims": "",
  "block_lr_zero_threshold": "",
  "bucket_no_upscale": true,
  "bucket_reso_steps": 64,
  "bypass_mode": false,
  "cache_latents": true,
  "cache_latents_to_disk": false,
  "caption_dropout_every_n_epochs": 0,
  "caption_dropout_rate": 0,
  "caption_extension": ".txt",
  "clip_skip": 1,
  "color_aug": false,
  "constrain": 0,
  "conv_alpha": 1,
  "conv_block_alphas": "",
  "conv_block_dims": "",
  "conv_dim": 1,
  "dataset_config": "",
  "debiased_estimation_loss": false,
  "decompose_both": false,
  "dim_from_weights": false,
  "dora_wd": false,
  "down_lr_weight": "",
  "enable_bucket": false,
  "epoch": 10,
  "extra_accelerate_launch_args": "",
  "factor": -1,
  "flip_aug": false,
  "fp8_base": false,
  "full_bf16": true,
  "full_fp16": false,
  "gpu_ids": "",
  "gradient_accumulation_steps": 1,
  "gradient_checkpointing": false,
  "huber_c": 0.1,
  "huber_schedule": "snr",
  "ip_noise_gamma": 0,
  "ip_noise_gamma_random_strength": false,
  "keep_tokens": 0,
  "learning_rate": 1,
  "log_tracker_config": "",
  "log_tracker_name": "",
  "lora_network_weights": "",
  "loss_type": "l2",
  "lr_scheduler": "cosine",
  "lr_scheduler_args": "",
  "lr_scheduler_num_cycles": "",
  "lr_scheduler_power": "",
  "lr_warmup": 0,
  "main_process_port": 0,
  "masked_loss": false,
  "max_bucket_reso": 2048,
  "max_data_loader_n_workers": "0",
  "max_grad_norm": 1,
  "max_resolution": "512,512",
  "max_timestep": 1000,
  "max_token_length": "225",
  "max_train_epochs": "",
  "max_train_steps": "",
  "mem_eff_attn": false,
  "mid_lr_weight": "",
  "min_bucket_reso": 512,
  "min_snr_gamma": 5,
  "min_timestep": 0,
  "mixed_precision": "bf16",
  "model_list": "custom",
  "module_dropout": 0,
  "multi_gpu": false,
  "multires_noise_discount": 0,
  "multires_noise_iterations": 0,
  "network_alpha": 64,
  "network_dim": 64,
  "network_dropout": 0,
  "noise_offset": 0,
  "noise_offset_random_strength": false,
  "noise_offset_type": "Original",
  "num_cpu_threads_per_process": 2,
  "num_machines": 1,
  "num_processes": 1,
  "optimizer": "Prodigy",
  "optimizer_args": "use_bias_correction=True weight_decay=0.5 d_coef=0.65 \"betas=(0.9, 0.901)\"",
  "output_name": "xxxxxx",
  "persistent_data_loader_workers": false,
  "pretrained_model_name_or_path": "M:/!SDModels/!Edzeni/realisticVisionV51_v51VAE_full.safetensors",
  "prior_loss_weight": 1,
  "random_crop": false,
  "rank_dropout": 0,
  "rank_dropout_scale": false,
  "rescaled": false,
  "resume": "",
  "sample_every_n_epochs": 1,
  "sample_every_n_steps": 0,
  "sample_prompts": "a xxxxxx woman in restaurant, upper body, red hair, realistic photo, many people in background, RAW photo, 8k uhd, dslr, soft lighting, high quality, film grain, Fujifilm XT3 --n lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry",
  "sample_sampler": "euler_a",
  "save_as_bool": false,
  "save_every_n_epochs": 1,
  "save_every_n_steps": 0,
  "save_last_n_steps": 0,
  "save_last_n_steps_state": 0,
  "save_model_as": "safetensors",
  "save_precision": "bf16",
  "save_state": false,
  "save_state_on_train_end": false,
  "scale_v_pred_loss_like_noise_pred": false,
  "scale_weight_norms": 1,
  "sdxl": false,
  "sdxl_cache_text_encoder_outputs": false,
  "sdxl_no_half_vae": true,
  "seed": "1217930809",
  "shuffle_caption": false,
  "stop_text_encoder_training": 0,
  "text_encoder_lr": 1,
  "train_batch_size": 1,
  "train_norm": false,
  "train_on_input": true,
  "training_comment": "",
  "unet_lr": 1,
  "unit": 1,
  "up_lr_weight": "",
  "use_cp": false,
  "use_scalar": false,
  "use_tucker": false,
  "use_wandb": false,
  "v2": false,
  "v_parameterization": false,
  "v_pred_like_loss": 0,
  "vae": "",
  "vae_batch_size": 0,
  "wandb_api_key": "",
  "wandb_run_name": "",
  "weighted_captions": false,
  "xformers": "xformers"
}

from kohya_ss.

bmaltais commented on July 28, 2024

Yeah, this new version if using the toml and it changes the rule for parameters... don't put " or , in there... This should fix the issue...

from kohya_ss.

mykeehu commented on July 28, 2024

Okay, but the parameter for betas looks like this, so I can't get around the comma: (0.9, 0.999)
https://github.com/konstmish/prodigy
So if I need the comma, what's the solution? I do not use toml.

from kohya_ss.

bmaltais commented on July 28, 2024

The dev branch code will now validate the lr scheduler and optimizer arguments and prevent starting the training if they do not comply with the needed format.

The empty fields for those two optional arguments show what format arguments should have.

from kohya_ss.

5KilosOfCheese commented on July 28, 2024

The dev branch code will now validate the arguments and prevent starting the training if they do not comply with the needed format.

This raises an interesting possibility. These systems have lots of arguments that can be leveraged for all sorts of purposes. This would probably be a big as, but would it be possible have a list and the correct formating. Like in a separate tab. This would help with user's troubleshooting efforts. Yes, the concept of having to documentation is always a big ask; but in general users should always know the correct means of interfacing with the system within the system.

I assume there is only limited and known amount and type of arguments which can be or are passed through?

from kohya_ss.

bmaltais commented on July 28, 2024

I wish kohya did. It would really be a document he should maintain.

from kohya_ss.

Cannot use betas parameter for Lora about kohya_ss HOT 9 CLOSED

Comments (9)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent