Does dreambooth support multi-subjects training? about dreambooth-stable-diffusion HOT 12 CLOSED

There was a fork of Joe's than handled multi-concepts but it hasn't been touched in 6 months https://github.com/kanewallmann/Dreambooth-Stable-Diffusion

ShivamShrirao's Dreambooth (diffusers based) colab handles multi-concepts. https://github.com/ShivamShrirao/diffusers/examples/dreambooth

Thanks a lot~ I looked in ShivamShrirao's Dreambooth and found that it may not work well for multi-concepts: ShivamShrirao/diffusers#127
So the main question right now is: Are there reproducible successful multi-concepts training cases?

from dreambooth-stable-diffusion.

@qq110146 do you success in train multiple subject ?

from dreambooth-stable-diffusion.

depends on which dreambooth..... there are many repos and they're not all the same. you post this here and yet you link to huggingface diffusers dreambooth. the Joepenna repo is not diffusers based.

from dreambooth-stable-diffusion.

depends on which dreambooth..... there are many repos and they're not all the same. you post this here and yet you link to huggingface diffusers dreambooth. the Joepenna repo is not diffusers based.

yep. I'm just confused about... Does dreambooth support multi-subjects training? Or are there any successful repos of multi-subjects training？（E.g the Joepenna repo or the others?）

from dreambooth-stable-diffusion.

There was a fork of Joe's than handled multi-concepts but it hasn't been touched in 6 months
https://github.com/kanewallmann/Dreambooth-Stable-Diffusion

ShivamShrirao's Dreambooth (diffusers based) colab handles multi-concepts.
https://github.com/ShivamShrirao/diffusers/examples/dreambooth

from dreambooth-stable-diffusion.

You could take a look at the EveryDream Trainer. it's a bit different but it does mention multiple-concepts in its readme.

https://github.com/victorchall/EveryDream-trainer

from dreambooth-stable-diffusion.

Okay, thank you very much!

from dreambooth-stable-diffusion.

@qq110146 do you success in train multiple subject ?

Not yet. I trained 2 concepts with train_multi_subject_dreambooth.py but failed.

from dreambooth-stable-diffusion.

@qq110146 I understand that dreambooth mostly overwrites all weights related to the learning target instance during learning. The example ( train_multi_subject_dreambooth.py is crappy and there is no proof, so I don't know how it got approved.

from dreambooth-stable-diffusion.

any new related to training multi subject's ?

from dreambooth-stable-diffusion.

This repo has supported multi subjects for a bit, but it was never documented well.

I've updated the readme with instructions: https://github.com/JoePenna/Dreambooth-Stable-Diffusion#captions

from dreambooth-stable-diffusion.

I've been having good luck with multi subject using Kohya LoRA, i think it would probably work with training a full Dreambooth model

1. Putting training images of each subject in a different folder
2. Balance the folders using the repeats number. For instance, if you have 50 images of Bob, and 100 images of Alice, you want it to train on the Bob images twice as much, so you would have:

2_bob-example
1_alice-example

This will do two repeats on Bob for each epic, for a total of 200 steps.

3. Use a caption .txt file each training image with a sidecar, for each training image, using the subject name in the prompt, i.e.

bob-example, an image of a man smiling, [other tags go here]

or

alice-example, a woman looking at the camera, [other tags go here]

4. When using the LoRA/model, specify the model instance name (let's say this is examplepeople.ckpt), for example

examplepeople, bob-example, A man in outerspace <lora: examplepeople_v0.5:76>

This will show bob in outerspace, instead of alice, and looks really good, even without regularization images. I generally do people though. If you want to be more specific (for your dog), you can add categories to the folder name and regularization images. Like this:

2_bob-example man
1_alice-example woman
4_spot-example dog

then in the regularization/ folder:

1_man
1_woman
1_dog

With regularization images. Seems to work!

Sorry if this is too basic, but this works well for me so far.

from dreambooth-stable-diffusion.