ygtxr1997 / celebbasis Goto Github PK

B = PCA(Ck, p) = R^{mxp}， but the shape of C_bar is R^d, how to compute it?

I've been running it so far via Windows Subsystem for Linux 2, and the 01_start_train.sh bash script was crashing at the default 16 GB of RAM for my 32 GB Ubuntu install. I had to increase it to 26 GB (Windows 11 needs about 4 GB to operate) via the .wslconfig or it was getting killed due to lack of memory https://fizzylogic.nl/2023/01/05/how-to-configure-memory-limits-in-wsl2

I think there is an option to use the gpu to load the models directly, rather than copy them into RAM first, but I suppose this wouldn't be good for low VRAM GPUs (I have a 3090).

Do you have a google colab for this?

Thanks for this work.
What changes would be required to use SDXL instead of the 1.4 checkpoint?

Getting this error:
(celebbasis) user@amax:/data_heat/rjt_project/CelebBasis$ bash ./01_start_train.sh ./weights/sd-v1-4-full-ema.ckpt
Global seed set to 23
Running on GPUs 0,
Loading model from ./weights/sd-v1-4-full-ema.ckpt
Traceback (most recent call last):
File "main_id_embed.py", line 621, in
model = load_model_from_config(config, opt.actual_resume)
File "main_id_embed.py", line 32, in load_model_from_config
pl_sd = torch.load(ckpt, map_location="cpu")
File "/home/user/miniconda3/envs/celebbasis/lib/python3.8/site-packages/torch/serialization.py", line 600, in load
with _open_zipfile_reader(opened_file) as opened_zipfile:
File "/home/user/miniconda3/envs/celebbasis/lib/python3.8/site-packages/torch/serialization.py", line 242, in init
super(_open_zipfile_reader, self).init(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "main_id_embed.py", line 833, in
if trainer.global_rank == 0:
NameError: name 'trainer' is not defined

env:
I have configured the environment by following these steps:

I used the command conda env create -f environment.yaml to create the environment based on the specifications provided in the environment.yaml file.

Then, I activated the environment using the command conda activate celebbasis. By changing the environment name to "celebbasis," I ensured that I am working within this specific environment.

Regarding the dependencies mentioned:

The line # - -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers refers to the "taming-transformers" library. Due to network issues, I independently installed this dependency using the command pip install -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers.

Similarly, the line # - -e git+https://github.com/openai/CLIP.git@main#egg=clip refers to the "CLIP" library. You installed this dependency separately as well, using the command pip install -e git+https://github.com/openai/CLIP.git@main#egg=clip.

Next ,
About Face Alignment
I align the images in Img, like id0(1).jpg,...,id0(10).jpg. I think this is a minor issue.
Then, I run bash ./01_start_train.sh ./weights/sd-v1-4-full-ema.ckpt and meet the top error

Thanks

Very nice work and code repository!
I wonder if you can provide any source images and prompt for evaluation.
Besides, clear instruction for how to evaluate the customized models on the metrics of "Prompt" and "Detect" would be appreciated!
Thank you!

From my experience - does an amazing job.

I'm very confused, I see there are a lot of images in the /infer_images folder, including /reg or regularization images, and the stylegan3 examples under /ffhq.

However, the pickle file I made in step 00 (face alignment) doesn't generate or include reg images, so when it's unpickled it just contains my (single) aligned cropped photo (I used the pretty print package to display the unpickled dictionary)

and gives me an error message about missing regularization images.

the pickle_path seems to default to another hardcoded location from a fellow named gavin, and this path includes stylegan3 in the name, hence my confusion

Please advise!

sd-v1-4-full-ema.ckpt could not be found in the provided link

I needed to make a change in /evaluation/face_align/PIPNet/start_align.py on line 40 during the DataLoader object instance from num_workers=1 to num_workers=0, or else the tqdm was just hanging indefinitely, as seen in this forum post talking about num_workers>0 https://discuss.pytorch.org/t/dataloader-slower-with-num-workers-0/100970 (up to 50 times slower)

Perhaps it depends on system configuration, but this is what I needed to do to get it to run instantly, and not hang indefinitely, hope it helps others out!

on line 6 of /ldm/models/autoencoder.py there is an import for a module that doesn't exist in this repo

I had to git clone this repo https://github.com/CompVis/taming-transformers/tree/master

and copy over the /taming folder to the root directory so autoencoder.py would run successfully

Thanks for the really cool work, excited to try it.

Any timeline for code release?

Feel fee to close if you feel not relevant.

thank you for the amazing work you provide. I see that ' finetuning with multiple persons' on your TODO list .
when do you schedule to add it to your github repository?
@vinthony @ygtxr1997

using the 03_extract.sh to extract the embeddings, then fill the tensor of the embedding files into a webui style embedding file, and then good to go.

And from experiments, it seems 799 steps usually lead to a better result than that from fewer or more steps.

I found in the face_id.py file, there are classes that could specifically use the celeb-asian dataset, but the config files do not have any record of using those classes. Could I simply swap the default classes for those?

Where is the list of the 1.500 celebs?

Getting this error:

│ /home/xander/Projects/cog/GitHub_repos/CelebBasis/ldm/models/diffusion/ddpm.py:1087 in p_losses  │
│                                                                                                  │
│   1084 │   │   loss_simple = self.get_loss(model_output, target, mean=False).mean([1, 2, 3])     │
│   1085 │   │   loss_dict.update({f'{prefix}/loss_simple': loss_simple.mean()})                   │
│   1086 │   │                                                                                     │
│ ❱ 1087 │   │   logvar_t = self.logvar[t].to(self.device)                                         │
│   1088 │   │   loss = loss_simple / torch.exp(logvar_t) + logvar_t                               │
│   1089 │   │   # loss = loss_simple / torch.exp(self.logvar) + self.logvar                       │
│   1090 │   │   if self.learn_logvar:                                                             │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)

Changing
logvar_t = self.logvar[t].to(self.device)
to
logvar_t = self.logvar[t.cpu()].to(self.device)

Solved this, but that feels like the wrong place to fix this bug...

I got this error:

Epoch 0: 80%|▊| 800/1001 [06:35<01:39, 2.03it/s, loss=0.0671, v_num=0, train/loss_simple_step=0.0197, train/loss_vlb_step=7.03e-5, train/loss_step=0Epoch 0, global step 799: val/loss_simple_ema was not in top 1
Average Epoch time: 395.59 seconds
Average Peak memory 19447.18MiB
Epoch 0: 80%|▊| 801/1001 [06:35<01:38, 2.02it/s, loss=0.0671, v_num=0, train/loss_simple_step=0.0197, train/loss_vlb_step=7.03e-5, train/loss_step=0
Saving latest checkpoint...

Traceback (most recent call last):
File "main_id_embed.py", line 817, in
trainer.test(model, data)
File "/home/user/miniconda3/envs/celebbasis/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 911, in test
return self._call_and_handle_interrupt(self._test_impl, model, dataloaders, ckpt_path, verbose, datamodule)
File "/home/user/miniconda3/envs/celebbasis/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 685, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/home/user/miniconda3/envs/celebbasis/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 954, in _test_impl
results = self._run(model, ckpt_path=self.tested_ckpt_path)
File "/home/user/miniconda3/envs/celebbasis/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1128, in _run
verify_loop_configurations(self)
File "/home/user/miniconda3/envs/celebbasis/lib/python3.8/site-packages/pytorch_lightning/trainer/configuration_validator.py", line 42, in verify_loop_configurations
__verify_eval_loop_configuration(trainer, model, "test")
File "/home/user/miniconda3/envs/celebbasis/lib/python3.8/site-packages/pytorch_lightning/trainer/configuration_validator.py", line 186, in __verify_eval_loop_configuration
raise MisconfigurationException(f"No {loader_name}() method defined to run Trainer.{trainer_method}.")
pytorch_lightning.utilities.exceptions.MisconfigurationException: No test_dataloader() method defined to run Trainer.test.

env:
I have configured the environment by following these steps:

I used the command conda env create -f environment.yaml to create the environment based on the specifications provided in the environment.yaml file.

Then, I activated the environment using the command conda activate celebbasis. By changing the environment name to "celebbasis," I ensured that I am working within this specific environment.

Regarding the dependencies mentioned:

The line # - -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers refers to the "taming-transformers" library. Due to network issues, I independently installed this dependency using the command pip install -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers.

Similarly, the line # - -e git+https://github.com/openai/CLIP.git@main#egg=clip refers to the "CLIP" library. You installed this dependency separately as well, using the command pip install -e git+https://github.com/openai/CLIP.git@main#egg=clip.

Next ,
About Face Alignment
I align the images in Img, like id0(1).jpg,...,id0(10).jpg. I think this is a minor issue.
Then, I run bash ./01_start_train.sh ./weights/sd-v1-4-full-ema.ckpt and meet the top error

Thanks

I get the following error running bash 00_align_face.sh <...>/ori <...>/ffhq

(sd) wsl@MJDFW:~/repos/CelebBasis$ bash 00_align_face.sh /home/wsl/repos/CelebBasis/my_imgs/ori/ /home/wsl/repos/CelebBasis/my_imgs/ffhq/
running build_ext
skipping 'nms/cpu_nms.c' Cython extension (up-to-date)
Traceback (most recent call last):
  File "./evaluation/face_align/PIPNet/start_align.py", line 85, in <module>
    main(opt)
  File "./evaluation/face_align/PIPNet/start_align.py", line 35, in main
    folder_dataset = FolderAlignCrop(
  File "/home/wsl/repos/CelebBasis/evaluation/face_align/PIPNet/alignment/dataloader.py", line 40, in __init__
    self.net, self.detector = get_lmk_model()
  File "/home/wsl/repos/CelebBasis/evaluation/face_align/PIPNet/lib/tools.py", line 62, in get_lmk_model
    state_dict = torch.load(weight_file, map_location=device)
  File "/home/wsl/anaconda3/envs/sd/lib/python3.8/site-packages/torch/serialization.py", line 594, in load
    with _open_file_like(f, 'rb') as opened_file:
  File "/home/wsl/anaconda3/envs/sd/lib/python3.8/site-packages/torch/serialization.py", line 230, in _open_file_like
    return _open_file(name_or_buffer, mode)
  File "/home/wsl/anaconda3/envs/sd/lib/python3.8/site-packages/torch/serialization.py", line 211, in __init__
    super(_open_file, self).__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: '/gavin/code/FaceSwapping/inference/PIPNet/weights/epoch59.pth'

due to the hardcoded paths in the get_lmk_model() function in evaluation/face_align/PIPNet/lib/tools.py as seen here

As we can see here, there should be PIPNet weights (and epoch59.pth ?) which need to be downloaded (and instructions for doing so in the README.md)

Please advise, thanks!