shuhongchen / panic3d-anime-reconstruction Goto Github PK

In the shuhongchen/panic3d-anime-reconstruction:latest docker, the following error occurred.

# python3 -m _scripts.eval.generate && python3 -m _scripts.eval.measure
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/dl/panic3d/panic3d-anime-reconstruction/_scripts/eval/generate.py", line 38, in <module>
    rmline_model = rmline_wrapper.RMLineWrapper(('rmlineE_rmlineganA_n04', 199)).eval().to(device)
  File "/home/dl/panic3d/panic3d-anime-reconstruction/_train/img2img/util/rmline_wrapper.py", line 19, in __init__
    ckpt = userving.Checkpoint(*inferquery)
  File "/home/dl/panic3d/panic3d-anime-reconstruction/_util/serving_v1.py", line 17, in __init__
    self.im_ckpt = utraining.infer_module_checkpoint(run_name, ckpt)
  File "/home/dl/panic3d/panic3d-anime-reconstruction/_util/training_v1.py", line 191, in infer_module_checkpoint
    for fn in os.listdir(ir.run_dn_checkpoints)
AttributeError: 'NoneType' object has no attribute 'run_dn_checkpoints'

Please let me know how to fix it.

I am planning to use your code with my original dataset.
In the training script, there is a line
--resume_discrim=./_data/eg3d/networks/ffhq512-128.pkl, and within the train_loop, it mentions

    # Resume from existing pickle's discriminator. (shu added)
    if (resume_discrim_pkl is not None) and (rank == 0):
        print(f'Resuming discriminator from "{resume_discrim_pkl}"')
        with dnnlib.util.open_url(resume_discrim_pkl) as f:
            resume_data_discrim = legacy.load_network_pkl(f)
        for name, module in [('D', D), ]:
            misc.copy_params_and_buffers(resume_data_discrim[name], module, require_all=False)

suggesting it loads a pre-trained discriminator model.
Is this pickle file available to the public...? (or...... it would be greatly appreciated if you could make it available.)

thanks.

When I run "python3 -m _scripts.eval.generate && python3 -m _scripts.eval.measure", the following error occurred.

# python3 -m _scripts.eval.generate && python3 -m _scripts.eval.measure
Downloading: "https://download.pytorch.org/models/resnet50-0676ba61.pth" to /root/.cache/torch/hub/checkpoints/resnet50-0676ba61.pth
100%|█████████████████████████████████████████████████| 97.8M/97.8M [00:08<00:00, 11.7MB/s]
  0%|                                                               | 0/68 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/ishihara/Panic3d/panic3d-anime-reconstruction/_scripts/eval/generate.py", line 82, in <module>
    x = dk[bn]
  File "/home/ishihara/Panic3d/panic3d-anime-reconstruction/_databacks/lustrous_renders_v1.py", line 183, in __getitem__
    ret['image'] = I(f'{self.dn}/renders/{bn}.png')
  File "/home/ishihara/Panic3d/panic3d-anime-reconstruction/_util/twodee_v1.py", line 114, in __init__
    self.__init__(Image.open(self.data))
  File "/opt/conda/lib/python3.8/site-packages/PIL/Image.py", line 2953, in open
    fp = builtins.open(filename, "rb")
FileNotFoundError: [Errno 2] No such file or directory: './_data/lustrous/renders/daredemoE/fandom_align/genshin/aether/front.png'

Please let me know how to fix it.

It would be amazing! Nice job

im currently trying to install when i put the command on the terminal in the main directory project i got this error " root@LivingBig:/mnt/f/Avatars/vroid_renderer# docker pull make/docker_pull
Using default tag: latest
Error response from daemon: pull access denied for make/docker_pull, repository does not exist or may require 'docker login': denied: requested access to the resource is denied "

I'd like to accurately replicate talking head demo.
Please let me know the process and the script to use for talking head demo.

After make/docker_pull, I tried to run make/shell_docker.
But, the following error occurred.

$ make/shell_docker
non-network local connections being added to access control list

Please let me know how to fix it.

I don't undesrtand how to set up any repo because i don't know thing of programming stuff, can you explain better for someone who is dumb in this websside like me? please

python3 -m _scripts.eval.generate
Traceback (most recent call last):
File "/home/camib4a/anaconda3/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/camib4a/anaconda3/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/mnt/f/repos/panic3d-anime-reconstruction-master/_scripts/eval/generate.py", line 7, in
from _util.twodee_v1 import * ; import _util.twodee_v1 as u2d
File "/mnt/f/repos/panic3d-anime-reconstruction-master/_util/twodee_v1.py", line 28, in
_FN_ARIAL = f'{os.environ["PROJECT_DN"]}/_env/arial_monospaced_mt.ttf'
File "/home/camib4a/anaconda3/lib/python3.10/os.py", line 680, in getitem
raise KeyError(key) from None
KeyError: 'PROJECT_DN'

Thank you for your exceptional contribution! I would like to evaluate the illustration-to-Render module separately, but I am unable to locate the necessary training and testing data, specifically rmlineERA_train.pkl and rmlineERA_test.pkl. Although I have downloaded panic_data_models_merged.zip, panic_data_models_merged_data\lustrous\preprocessed\patch appears to be empty. Looking forward to your response.

Thanks for your great work.

I am not very familiar with docker. I would like to know whether I can run this project normally on Google Colab, since it is indicated that this project requires docker with a GPU. However, current Google Colab doesn't support docker.

Looking forward to your reply.

I don‘t know how to get the dorthoA, there is no script to generate it.

(I'm really sorry to keep asking you ...)

I'm encountering errors with these lines of code. Could you please explain how you created the features using ResNet?

just delete it !!!

Sorry for asking so many questions.
I get the following error
Traceback (most recent call last): File "/opt/conda/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/opt/conda/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/dl/prodata/panic3d-anime-reconstruction/_train/eg3dc/trainers/train_eclustrousC.py", line 542, in <module> main() # pylint: disable=no-value-for-parameter File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1128, in __call__ return self.main(*args, **kwargs) File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1053, in main rv = self.invoke(ctx) File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1395, in invoke return ctx.invoke(self.callback, **ctx.params) File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 754, in invoke return __callback(*args, **kwargs) File "/home/dl/prodata/panic3d-anime-reconstruction/_train/eg3dc/trainers/train_eclustrousC.py", line 537, in main launch_training(c=c, desc=desc, outdir=opts.outdir, dry_run=opts.dry_run) File "/home/dl/prodata/panic3d-anime-reconstruction/_train/eg3dc/trainers/train_eclustrousC.py", line 112, in launch_training subprocess_fn(rank=0, c=c, temp_dir=temp_dir) File "/home/dl/prodata/panic3d-anime-reconstruction/_train/eg3dc/trainers/train_eclustrousC.py", line 61, in subprocess_fn getattr(training_loop, c.training_loop_version)(rank=rank, **c) File "/home/dl/prodata/panic3d-anime-reconstruction/./_train/eg3dc/src/training/training_loop_v0.py", line 360, in training_loop loss.accumulate_gradients(phase=phase.name, real_img=real_img, real_c=real_c, real_cond=real_cond, gen_z=gen_z, gen_c=gen_c, gain=phase.interval, cur_nimg=cur_nimg) File "/home/dl/prodata/panic3d-anime-reconstruction/./_train/eg3dc/src/training/loss_orthocondA.py", line 483, in accumulate_gradients gen_img, _gen_ws = self.run_G(gen_z, gen_c, real_cond, swapping_prob=swapping_prob, neural_rendering_resolution=neural_rendering_resolution) File "/home/dl/prodata/panic3d-anime-reconstruction/./_train/eg3dc/src/training/loss_orthocondA.py", line 171, in run_G gen_output = self.G.f({ File "/home/dl/prodata/panic3d-anime-reconstruction/./_train/eg3dc/src/training/triplane.py", line 404, in f x['elevations'][i], KeyError: 'elevations'
Commenting out x['elevations'][i] causes x['azimuths'][0] to be a key error.
You also use 'elevation' in generate.py and other scripts, do you use what is defined in those scripts?

Also, my environment is as follows

CPU : Core™ i7-13700K
GPU : GeForce RTX 4070
Memory : 32GB
Drive : 2TB

We are currently adjusting batch size, etc., but is it still too hard to run with this computer performance? I would like to get your help in terms of views, parameter settings, etc.

or, is there some guide to test my own images? thanks!

Thanks for your work.

How to generate 3D reconstruction with my own single image ? Is there a script for that? thanks !

@ShuhongChen

After make/docker_pull, I tried to run make/shell_docker.
But, the following error occurred.

$ make/shell_docker
non-network local connections being added to access control list
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].

Please let me know how to fix it.

Hi there! I am trying to running script python3 -m _scripts.render_all_animerecon_daredemoE, however it failed with error information:

Same error also occured in rendering ortho, ortho_xyz, rgb60.
Is there any solution for this error? Thanks!

Thank you for your excellent work!
I would like to make a research based on yours, could you please publish the training script?

(. /_train/eg3dc/runs/ecrutileE_eclustrousC_n120/ecrutileE_eclustrousC_n120.sh was not found...)

Hi, this work really produces impressive result in the demo. I also try to build the repo and I have downloaded all the data from goole drive. But I found it too big to download all the necessary data listed in the repo. I wonder if I JUST want to run the inferece, then what data I should obtain?

P.S. now I run the _script.generate.py and it produces error like this:
No such file or directory: './_data/danbooru/_filters/intently_combatively_rules.json
Which dataset should I download to get this data?
Hope for your answer : )

Attempting to run the script:

./_train/eg3dc/runs/ecrutileE_eclustrousC_n120/ecrutileE_eclustrousC_n120.sh

Results in the following error:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/ericj/Source/panic3d-anime-reconstruction/_train/eg3dc/trainers/train_eclustrousC.py", line 572, in <module>
    main() # pylint: disable=no-value-for-parameter
  File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/ericj/Source/panic3d-anime-reconstruction/_train/eg3dc/trainers/train_eclustrousC.py", line 348, in main
    c.training_set_kwargs, dataset_name = init_dataset_kwargs(data_class=opts.data_class, data_subset=opts.data_subset)
  File "/home/ericj/Source/panic3d-anime-reconstruction/_train/eg3dc/trainers/train_eclustrousC.py", line 125, in init_dataset_kwargs
    dataset_obj = dnnlib.util.construct_class_by_name(**dataset_kwargs) # Subclass of training.dataset.Dataset.
  File "/home/ericj/Source/panic3d-anime-reconstruction/./_train/eg3dc/src/dnnlib/util.py", line 305, in construct_class_by_name
    return call_func_by_name(*args, func_name=class_name, **kwargs)
  File "/home/ericj/Source/panic3d-anime-reconstruction/./_train/eg3dc/src/dnnlib/util.py", line 298, in call_func_by_name
    func_obj = get_obj_by_name(func_name)
  File "/home/ericj/Source/panic3d-anime-reconstruction/./_train/eg3dc/src/dnnlib/util.py", line 291, in get_obj_by_name
    module, obj_name = get_module_from_obj_name(name)
  File "/home/ericj/Source/panic3d-anime-reconstruction/./_train/eg3dc/src/dnnlib/util.py", line 262, in get_module_from_obj_name
    importlib.import_module(module_name) # may raise ImportError
  File "/opt/conda/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/ericj/Source/panic3d-anime-reconstruction/_train/eg3dc/datasets/ecrutileE.py", line 27, in <module>
    from training.dataset import Dataset as DatasetEG3D
  File "/home/ericj/Source/panic3d-anime-reconstruction/./_train/eg3dc/src/training/dataset.py", line 247, in <module>
    import _databacks.lustrous_v0 as dkmodlust
ModuleNotFoundError: No module named '_databacks.lustrous_v0'

This is after I successfully run make/shell_docker. I presume there's an issue I'm having with python modules being recognized by python itself. Similarly, attempting to run the python command:

python3 -m _scripts.generate && python3 -m _scripts.measure

Generates a similar (albeit much smaller) error:

/opt/conda/bin/python3: No module named _scripts.generate

Presumably Python is unable to figure out how the modules are set up here. Here's my system information I think that would be relevant to this issue.

Python Version:  3.8.12
which python: /opt/conda/bin/python (from within the docker)
which python: /usr/bin/python3 (from within my main terminal)
OS: Ubuntu 22.04.2 LTS x86_64
CPU: AMD Ryzen 9 5900X
GPU: NVIDIA GeForce RTX 3070 Ti

When I run the 'python3 -m _scripts.generate && python3 -m _scripts.measure' command line, the following error occurred.

$ python3 -m _scripts.generate && python3 -m _scripts.measure
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/dl/panic3d/panic3d-anime-reconstruction/_scripts/generate.py", line 29, in <module>
    dk = dklustr.DatabackendMinna()
  File "/home/dl/panic3d/panic3d-anime-reconstruction/_databacks/lustrous_renders_v1.py", line 116, in __init__
    self.dn = f'{self.args.base.dn}/_data/lustrous'
AttributeError: 'dict' object has no attribute 'base'

Please tell me how to fix it.

The following error occurred.

$ python3 -m _scripts.eval.generate && python3 -m _scripts.eval.measure
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/dl/panic3d/panic3d-anime-reconstruction/_scripts/eval/generate.py", line 29, in <module>
    dk = dklustr.DatabackendMinna()
  File "/home/dl/panic3d/panic3d-anime-reconstruction/_databacks/lustrous_renders_v1.py", line 116, in __init__
    self.dn = f'{self.args.base.dn}/_data/lustrous'
AttributeError: 'dict' object has no attribute 'base'

Please let me know how to fix it.

Hello, Thank you for publishing this wonderful project.
I have a question about the answer to #25

I understood that the solution to #25 involves modifying generate.py to save the data from resnet_chonk into a pkl file, which can then be used for training. Consequently, I have altered generate.py and obtained the following data.

"{'resnet_chonk': tensor([[[[-3.1723e+00, -3.1356e+00, -3.0122e+00, ..., -3.0182e+00, -3.0302e+00, -3.0707e+00], [-3.1781e+00, -3.1504e+00, -2.9376e+00, ..., -2.8380e+00, -2.8835e+00, -3.1385e+00], [-3.1517e+00, -2.9036e+00, -2.5128e+00, ..., -1.6416e+00, -2.6505e+00, -3.0181e+00], ..., [ 1.1058e-01, 8.3331e-03, -6.3481e-01, ..., 1.1358e-01, -3.2545e-01, 2.4348e-02], [ 6.4464e-02, -8.6739e-02, -4.0153e-01, ..., 4.1071e-01, -2.2954e-01, -8.7458e-02], [ 1.4561e-01, -5.4080e-01, 3.4416e-02, ..., -1.7221e-01, 4.5234e-01, -4.2940e-01]]]], device='cuda:0')}

I tried to use the data but got the following error

Traceback (most recent call last): File "/opt/conda/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/opt/conda/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/dl/prodata/panic3d-anime-reconstruction/_train/eg3dc/trainers/train_eclustrousC.py", line 572, in <module> main() # pylint: disable=no-value-for-parameter File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1128, in __call__ return self.main(*args, **kwargs) File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1053, in main rv = self.invoke(ctx) File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 1395, in invoke return ctx.invoke(self.callback, **ctx.params) File "/opt/conda/lib/python3.8/site-packages/click/core.py", line 754, in invoke return __callback(*args, **kwargs) File "/home/dl/prodata/panic3d-anime-reconstruction/_train/eg3dc/trainers/train_eclustrousC.py", line 567, in main launch_training(c=c, desc=desc, outdir=opts.outdir, dry_run=opts.dry_run) File "/home/dl/prodata/panic3d-anime-reconstruction/_train/eg3dc/trainers/train_eclustrousC.py", line 112, in launch_training subprocess_fn(rank=0, c=c, temp_dir=temp_dir) File "/home/dl/prodata/panic3d-anime-reconstruction/_train/eg3dc/trainers/train_eclustrousC.py", line 61, in subprocess_fn getattr(training_loop, c.training_loop_version)(rank=rank, **c) File "/home/dl/prodata/panic3d-anime-reconstruction/./_train/eg3dc/src/training/training_loop_v0.py", line 284, in training_loop grid_size, images, labels = setup_snapshot_image_grid(training_set=training_set) File "/home/dl/prodata/panic3d-anime-reconstruction/./_train/eg3dc/src/training/training_loop_v0.py", line 74, in setup_snapshot_image_grid images, labels = zip(*[unpk(training_set[i]) for i in grid_indices]) File "/home/dl/prodata/panic3d-anime-reconstruction/./_train/eg3dc/src/training/training_loop_v0.py", line 74, in <listcomp> images, labels = zip(*[unpk(training_set[i]) for i in grid_indices]) File "/home/dl/prodata/panic3d-anime-reconstruction/_train/eg3dc/datasets/ecrutileE.py", line 49, in __getitem__ x = self.ds[idx % len(self.ds)] File "/home/dl/prodata/panic3d-anime-reconstruction/_train/eg3dc/datasets/ecrutileE.py", line 368, in __getitem__ 'resnet_chonk': torch.tensor(uutil.pload( RuntimeError: Could not infer dtype of dict
Can you please tell me the correct file contents of front.pkl?
Or if my understanding is wrong, please point it out to me.

After make/docker_pull, I tried to run make/shell_docker.
But, the following error occurred.

$ make/shell_docker
non-network local connections being added to access control list
docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/absolute/path/to/repo/_env/home/.bashrc" to rootfs at "/root/.bashrc": mount /absolute/path/to/repo/_env/home/.bashrc:/root/.bashrc (via /proc/self/fd/7), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type.
ERRO[0001] error waiting for container:

Please let me know how to fix it.

The following error occurred.

$ make/docker_pull
latest: Pulling from shuhongchen/panic3d-anime-reconstruction
Digest: sha256:952544f2869bea8b0d1479fe307b49f36638dcd49c8a6f43aa1f76cde39016fa
Status: Image is up to date for shuhongchen/panic3d-anime-reconstruction:latest
docker.io/shuhongchen/panic3d-anime-reconstruction:latest

$ make/shell_docker
non-network local connections being added to access control list
docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/absolute/path/to/repo/_env/home/.bashrc" to rootfs at "/root/.bashrc": mount /absolute/path/to/repo/_env/home/.bashrc:/root/.bashrc (via /proc/self/fd/7), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type.

Please let me know how to fix it.

I would like to try your code on a **new** data set and would like to know how you generated the json files of render_params. I understand that these json files will be used for training.
(I know that json files for original datasets are in the google drive that you have published. Thank you soooooo much)

I think that there are some traces of the json generation in the code (?)
https://github.com/ShuhongChen/vroid_renderer/blob/ea0891e422e3c9a236c5ecd1335c5ea504b575ea/_scripts/render_rgb.py#L324-L332

# uutil.pdump(self.camera.projection.matrix, mkfile('./temp/_cameras/intrinsic.pkl'))
# uutil.pdump(translation1 * rotation * translation0, mkfile('./temp/_cameras/extrinsic.pkl'))
# uutil.pdump({
#     'loc_origin': self.loc_origin,
#     'elev': elev,
#     'azim': azim,
#     'dist': dist,
# }, mkfile('./temp/_cameras/params.pkl'))
# exit(0)

Thanks for your work!

I wonder how to generate 'front.pkl' can be converted into a 3d model file that I can visualize in Blender.

=================

感谢你的工作！

我想知道如何可以把生成的front.pkl文件变成常用的3d文件，例如vrm，我希望在Blender中可视化这些模型

Hi, thank you for the great work!
I would like to apply this method to another image but "fandom_align_alignment.pkl" seems to be prepared for specific images.
So, it would be nice if I could catch up the details of this pkl file.
thank you in advance !

如题

A inference code for any anime portrait image

How can I make a 3D animation of an anime face from a 2D image of an arbitrary anime face?

I am using the pycharm and python 3.6 on my own computer, but I can't download genshen and only use virtualyoutuber dataset.
So the file structure is not right, and can you upload the dataset on baidu pandesk? thank you