Giter Club home page Giter Club logo

Comments (2)

Light-7777 avatar Light-7777 commented on August 16, 2024

kohya_SS logs

23:15:25-499525 INFO     Command executed.                                      
/workspace/kohya_ss/venv/lib/python3.10/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
/workspace/kohya_ss/venv/lib/python3.10/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
2024-05-25 23:15:30.772890: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-05-25 23:15:30.772924: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-05-25 23:15:30.772942: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-05-25 23:15:30.777820: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/workspace/kohya_ss/venv/lib/python3.10/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
2024-05-25 23:15:32 INFO     Loading settings from            train_util.py:3744
                             /workspace/kohya_ss/lora2/model/                   
                             config_lora-20240525-231525.toml                   
                             ...                                                
                    INFO     /workspace/kohya_ss/lora2/model/ train_util.py:3763
                             config_lora-20240525-231525                        
2024-05-25 23:15:32 INFO     prepare tokenizers           sdxl_train_util.py:134
2024-05-25 23:15:33 INFO     update token length: 75      sdxl_train_util.py:159
                    INFO     Using DreamBooth method.       train_network.py:172
                    INFO     prepare images.                  train_util.py:1572
                    INFO     found directory                  train_util.py:1519
                             /workspace/kohya_ss/lora2/img/30                   
                             _yuuk1 woman contains 34 image                     
                             files                                              
                    INFO     1020 train images with           train_util.py:1613
                             repeating.                                         
                    INFO     0 reg images.                    train_util.py:1616
                    WARNING  no regularization images /       train_util.py:1621
                             正則化画像が見つかりませんでした                   
                    INFO     [Dataset 0]                      config_util.py:565
                               batch_size: 1                                    
                               resolution: (1024, 1024)                         
                               enable_bucket: True                              
                               network_multiplier: 1.0                          
                               min_bucket_reso: 256                             
                               max_bucket_reso: 2048                            
                               bucket_reso_steps: 64                            
                               bucket_no_upscale: True                          
                                                                                
                               [Subset 0 of Dataset 0]                          
                                 image_dir:                                     
                             "/workspace/kohya_ss/lora2/img/3                   
                             0_yuuk1 woman"                                     
                                 image_count: 34                                
                                 num_repeats: 30                                
                                 shuffle_caption: False                         
                                 keep_tokens: 0                                 
                                 keep_tokens_separator:                         
                                 secondary_separator: None                      
                                 enable_wildcard: False                         
                                 caption_dropout_rate: 0.0                      
                                 caption_dropout_every_n_epoc                   
                             hes: 0                                             
                                 caption_tag_dropout_rate:                      
                             0.0                                                
                                 caption_prefix: None                           
                                 caption_suffix: None                           
                                 color_aug: False                               
                                 flip_aug: False                                
                                 face_crop_aug_range: None                      
                                 random_crop: False                             
                                 token_warmup_min: 1,                           
                                 token_warmup_step: 0,                          
                                 is_reg: False                                  
                                 class_tokens: yuuk1 woman                      
                                 caption_extension: .txt                        
                                                                                
                                                                                
                    INFO     [Dataset 0]                      config_util.py:571
                    INFO     loading image sizes.              train_util.py:853
100%|██████████| 34/34 [00:00<00:00, 76423.55it/s]
                    INFO     make buckets                      train_util.py:859
                    WARNING  min_bucket_reso and               train_util.py:876
                             max_bucket_reso are ignored if                     
                             bucket_no_upscale is set, because                  
                             bucket reso is defined by image                    
                             size automatically /                               
                             bucket_no_upscaleが指定された場合                  
                             は、bucketの解像度は画像サイズか                   
                             ら自動計算されるため、min_bucket_                  
                             resoとmax_bucket_resoは無視されま                  
                             す                                                 
                    INFO     number of images (including       train_util.py:905
                             repeats) /                                         
                             各bucketの画像枚数(繰り返し回数                   
                             を含む)                                           
                    INFO     bucket 0: resolution (384, 704),  train_util.py:910
                             count: 60                                          
                    INFO     bucket 1: resolution (640, 896),  train_util.py:910
                             count: 30                                          
                    INFO     bucket 2: resolution (640, 960),  train_util.py:910
                             count: 30                                          
                    INFO     bucket 3: resolution (640, 1088), train_util.py:910
                             count: 120                                         
                    INFO     bucket 4: resolution (640, 1216), train_util.py:910
                             count: 30                                          
                    INFO     bucket 5: resolution (768, 1088), train_util.py:910
                             count: 30                                          
                    INFO     bucket 6: resolution (832, 704),  train_util.py:910
                             count: 30                                          
                    INFO     bucket 7: resolution (832, 1088), train_util.py:910
                             count: 30                                          
                    INFO     bucket 8: resolution (1024,       train_util.py:910
                             1024), count: 210                                  
                    INFO     bucket 9: resolution (1088, 832), train_util.py:910
                             count: 450                                         
                    INFO     mean ar error (without repeats):  train_util.py:915
                             0.024796466791225116                               
                    WARNING  clip_skip will be unexpected sdxl_train_util.py:343
                             /                                                  
                             SDXL学習ではclip_skipは動作                        
                             しません                                           
                    INFO     preparing accelerator          train_network.py:225
accelerator device: cuda
                    INFO     loading model for process 0/1 sdxl_train_util.py:30
                    INFO     load StableDiffusion          sdxl_train_util.py:70
                             checkpoint:                                        
                             /workspace/stable-diffusion-w                      
                             ebui/models/Stable-diffusion/                      
                             ponyDiffusionV6XL_v6StartWith                      
                             ThisOne.safetensors                                
                    INFO     building U-Net               sdxl_model_util.py:192
                    INFO     loading U-Net from           sdxl_model_util.py:196
                             checkpoint                                         
2024-05-25 23:15:35 INFO     U-Net: <All keys matched     sdxl_model_util.py:202
                             successfully>                                      
                    INFO     building text encoders       sdxl_model_util.py:205
                    INFO     loading text encoders from   sdxl_model_util.py:258
                             checkpoint                                         
                    INFO     text encoder 1: <All keys    sdxl_model_util.py:272
                             matched successfully>                              
Traceback (most recent call last):
  File "/workspace/kohya_ss/venv/bin/accelerate", line 8, in <module>
    sys.exit(main())
  File "/workspace/kohya_ss/venv/lib/python3.10/site-packages/accelerate/commands/accelerate_cli.py", line 47, in main
    args.func(args)
  File "/workspace/kohya_ss/venv/lib/python3.10/site-packages/accelerate/commands/launch.py", line 1017, in launch_command
    simple_launcher(args)
  File "/workspace/kohya_ss/venv/lib/python3.10/site-packages/accelerate/commands/launch.py", line 637, in simple_launcher
    raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/workspace/kohya_ss/venv/bin/python3', '/workspace/kohya_ss/sd-scripts/sdxl_train_network.py', '--config_file', '/workspace/kohya_ss/lora2/model/config_lora-20240525-231525.toml']' died with <Signals.SIGKILL: 9>.

from kohya_ss.

b-fission avatar b-fission commented on August 16, 2024

You are using these optimizer args scale_parameter=False relative_step=False warmup_init=False which are not applicable to AdamW8bit.

Try running it again without any of those options.

from kohya_ss.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.