ldhlwh / dfmgan Goto Github PK
View Code? Open in Web Editor NEW[AAAI 2023] Official implementation of "Few-Shot Defect Image Generation via Defect-Aware Feature Manipulation".
Home Page: https://arxiv.org/abs/2303.02389
License: Other
[AAAI 2023] Official implementation of "Few-Shot Defect Image Generation via Defect-Aware Feature Manipulation".
Home Page: https://arxiv.org/abs/2303.02389
License: Other
The readme instructs to use the pkl file generated in the pkls folder, but both the folder and the file cannot be found. Is it okay to use the network-snapshot-000400.pkl file from the runs folder instead?
Is it correct to train NG with the OK pkl file generated during the first training, and then generate defects using the pkl file obtained after training NG? However, the generated images during the generation process do not show any markings on the mask file.
您好,我的cuda环境是11.3,readme里面python及pytorch版本最高只支持到cuda11.1。请问有其他的环境可以使用吗?
您好,LdhIwh!!!首先感谢您将代码开源,我认为这对缺陷生成领域有着巨大贡献。同时,我有下面的一个问题,还望您指教。
在第一阶段训练好后,我尝试训练第二阶段。但是当训练到kimg=80.0时,程序意外停止(报错信息如下)。
numpy.core._exceptions.MemoryError: Unable to allocate 64.0 MiB for an array with shape (2048, 2048) and data type complex128
为了节约时间,当我再次尝试运行时,我更改了加载的预训练模型为network-snapshot-000080.pkl,执行命令为:
python train.py --data ./data/hazelnut_hole_mask.zip -outdir runs/hazelnut_hole \
--resume runs/hazelnut_hole/00000-hazelnut_hole_mask-auto1-kimg400-resumecustom/network-snapshot-000080.pkl \
--gpus 1 --kimg 400 --snap 10 --transfer res_block_match_dis
我想知道这是否会影响我的训练结果
I use PyTorch 2.3.1+cu118, 'conv2d_gradfix' and 'grid_sample_gradfix' operations are not supported, instead using basic 'torch.nn.functional.conv2d' and 'torch.nn.functional.grid_sample' operations. Also, RuntimeError has occurred that no function has been implemented to calculate the gradient for aten:grid_sampler_2d_backward.
Which PyTorch, Cuda version should I use to avoid errors and use that model well?
感谢您的工作!模型针对单个类型缺陷生成的效果不错,但是在我的数据集中,一张图片存在多种类型的缺陷,那么如果我需要生成包含多种缺陷的单个图像,那么我需要分别为每一类缺陷创建数据集,然后同时训练多个模型,每个模型生成一类缺陷然后再图片上叠加吗?这样操作过于复杂,而且生成的缺陷在下一步缺陷生成时会互相干扰吗?
我的GPU内存只有6G,我可以调整哪些参数减小模型的尺寸呢?
Describe the bug
A clear and concise description of what the bug is.
Traceback (most recent call last):
File "train.py", line 651, in
main() # pylint: disable=no-value-for-parameter
File "/home/vision-ai/.local/lib/python3.6/site-packages/click/core.py", line 1128, in call
return self.main(*args, **kwargs)
File "/home/vision-ai/.local/lib/python3.6/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/home/vision-ai/.local/lib/python3.6/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/vision-ai/.local/lib/python3.6/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/home/vision-ai/.local/lib/python3.6/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "train.py", line 644, in main
subprocess_fn(rank=0, args=args, temp_dir=temp_dir)
File "train.py", line 478, in subprocess_fn
training_loop.training_loop(rank=rank, **args)
File "/home/vision-ai/nas_data/hmjung/2023/02.multi_channel/DFMGAN/training/training_loop.py", line 146, in training_loop
training_set = dnnlib.util.construct_class_by_name(**training_set_kwargs) # subclass of training.dataset.Dataset
File "/home/vision-ai/nas_data/hmjung/2023/02.multi_channel/DFMGAN/dnnlib/util.py", line 296, in construct_class_by_name
return call_func_by_name(*args, func_name=class_name, **kwargs)
File "/home/vision-ai/nas_data/hmjung/2023/02.multi_channel/DFMGAN/dnnlib/util.py", line 291, in call_func_by_name
return func_obj(*args, **kwargs)
File "/home/vision-ai/nas_data/hmjung/2023/02.multi_channel/DFMGAN/training/dataset.py", line 178, in init
raw_shape = [len(self._image_fnames)] + list(self._load_raw_image(0).shape)
File "/home/vision-ai/nas_data/hmjung/2023/02.multi_channel/DFMGAN/training/dataset.py", line 217, in _load_raw_image
image = np.load(f)
File "/opt/conda/lib/python3.6/site-packages/numpy/lib/npyio.py", line 426, in load
fid.seek(-min(N, len(magic)), 1) # back-up
io.UnsupportedOperation: seek
To Reproduce
Steps to reproduce the behavior:
Desktop (please complete the following information):
Describe the Question
I am currently generating images based on my own dataset, and I want to generate individual snippets of the images. In other words, currently the model outputs an image of size 4x7. The description in the paper provides an option for a triplet like Figure 4 in the paper as well, but I want a single piece image. How can I do that?
训练时一直显示ImportError: No module named bias_act_plugin和upfirdn2d_plugin的警告,发现在使用torch.utils.cpp_extension.load的时候没有生成bias_act.o、bias_act.cuda.o、bias_act_plugin.pyd这些文件,请问这该怎么解决?
Hello. Thank you for sharing such a great piece of work.
I'm curious about the following training options # DFMGAN args:
ft = None,
transfer = None,
res_st = None,
uni_st = None,
mask_threshold = None,
lambda_match = None,
mode_seek = None,
lambda_ms = None,
no_round = None,
tanh_k = None,
tanh_mask = None,
dmatch_scale = None
Could you explain them briefly?
And would you recommend changing these options for better results?
I trained self-built dataset with your code. However, during the defect transfer phase, I found that it is unable to generate defects and the corresponding masks when using the original settings. I tried to adjust the values of lambda-ms and lambda-match many times, defects finally generated when I reduce the value of lambda-match to 0.2 even more smaller. That is to say, mask matching operation hinders defect generation in my dataset. I am so confused, can you give me some suggestions? The following is the failure case of the mask generation.
請問這可以在windows上運行嗎?
你這邊寫Stylegan2不支持windows,但stylegan2裡寫了支持windows
謝謝
in file dataset_tool.py line 73
img = np.array(PIL.Image.open(fname))
need to be convert to
img = np.array(PIL.Image.open(fname).convert('RGB'))
您好,
我这边想要复现论文的测试结果,但是使用Repo的代码测试的结果与论文的结果差别很大。您这边方便把你训练好的PreTrained Model上传供测试吗?
Hello,
I have been studying the DFMGAN paper and found it to be immensely beneficial for my research on surface defect recognition. The concepts of SDGAN and Defect-GAN presented in the paper have particularly piqued my interest.
I am curious about how the SDGAN and Defect-GAN models have been implemented. If possible, could you provide links to any official or unofficial code repositories for these models? Access to such resources would be invaluable for understanding and possibly extending your work.
Thank you for your contributions to the field,
Best regards!
你好,在生成的图片质量不差的情况下,为何我得到的kid指标为0.03几,想知道关于代码中计算kid的值是有什么讲究吗?
--我的超参数均设置为默认,图片为256*256,pretrain的kimg=3000,transfer的kimg=300
尊敬的作者您好!
如果想尝试其他数据集进行生成图像,例如carpet,如何制作carpet_good.pkl,carpet_hole.pkl
如果能告知具体步骤将不胜感激
training/training_loop.py", line 368, in training_loop
assert phase_real_img.ndim == 4 and phase_real_img.shape[1] == 4
AssertionError
/opt/conda/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 17 leaked semaphore objects to clean up at shutdown
warnings.warn('resource_tracker: There appear to be %d '
assert phase_real_img.ndim == 4 and phase_real_img.shape[1] == 4
When res_block_match_dis is enabled, the above error occurs.
Did you use any other augmentations while training for hazelnut hole defects?
Did you follow the method provided in the official GitHub exactly as it is for training hazelnut holes?
I resumed training for hole defects using the hazelnut good model you provided, but the performance seems different.
Describe the bug
I get the attached error and can't run pretrain.
I was able to execute up to pre-processing.
To Reproduce
Steps to reproduce the behavior:
cd DFMGAN
python train.py --data ./data/hazelnut_good.zip --outdir runs/hazelnut_good --gpus 1 --kimg 3000
/home/oshita-n/.pyenv/versions/anaconda3-2022.05/envs/dfmgan/lib/python3.8/site-packages/torch/nn/modules/module.py:1051: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at /opt/conda/conda-bld/pytorch_1623448234945/work/c10/core/TensorImpl.h:1156.)
return forward_call(*input, **kwargs)
Traceback (most recent call last):
File "train.py", line 644, in <module>
main() # pylint: disable=no-value-for-parameter
File "/home/oshita-n/.pyenv/versions/anaconda3-2022.05/envs/dfmgan/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/home/oshita-n/.pyenv/versions/anaconda3-2022.05/envs/dfmgan/lib/python3.8/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/oshita-n/.pyenv/versions/anaconda3-2022.05/envs/dfmgan/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/oshita-n/.pyenv/versions/anaconda3-2022.05/envs/dfmgan/lib/python3.8/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/oshita-n/.pyenv/versions/anaconda3-2022.05/envs/dfmgan/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "train.py", line 637, in main
subprocess_fn(rank=0, args=args, temp_dir=temp_dir)
File "train.py", line 472, in subprocess_fn
training_loop.training_loop(rank=rank, **args)
File "/home/oshita-n/github/DFMGAN/training/training_loop.py", line 532, in training_loop
result_dict = metric_main.calc_metric(metric=metric, G=snapshot_data['G_ema'],
File "/home/oshita-n/github/DFMGAN/metrics/metric_main.py", line 46, in calc_metric
results = _metric_dict[metric](opts)
File "/home/oshita-n/github/DFMGAN/metrics/metric_main.py", line 86, in fid50k_full
fid = frechet_inception_distance.compute_fid(opts, max_real=None, num_gen=50000)
File "/home/oshita-n/github/DFMGAN/metrics/frechet_inception_distance.py", line 29, in compute_fid
mu_gen, sigma_gen = metric_utils.compute_feature_stats_for_generator(
File "/home/oshita-n/github/DFMGAN/metrics/metric_utils.py", line 280, in compute_feature_stats_for_generator
features = detector(images, **detector_kwargs)
File "/home/oshita-n/.pyenv/versions/anaconda3-2022.05/envs/dfmgan/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
RuntimeError: MALFORMED INPUT: lanes dont match
Expected behavior
A clear and concise description of what you expected to happen.
Desktop (please complete the following information):
I would like to know what libraries and OS versions you can reliably run.
Thank you for sharing your code, the results of the experiment look great! However, I do have a few suggestions:
// 1. Regarding the issue of generating temporary files, it would be better to change it to "with tempfile.TemporaryDirectory(None, None, args.run_dir) as temp_dir:" to avoid any potential problems caused by default generation in /tmp.
// 2. Have you considered changing the code to a class-based approach? The current implementation of the training_loop function looks quite cumbersome.
// 3. When running the code, some .so files are generated in /raid/ubuntu/.cache/torch_extensions. If different versions of PyTorch are used, these files may become locked and cause the program to hang indefinitely.
In this case, simply deleting the temporary files should resolve the issue.
您好,Defect-GAN好像没有开源,您是自己复现的吗,能否提供一下这个方法的代码?
Describe the bug
I have pre-trained a dataset with defect masks and normal images.
The pre-training process was completed successfully, but during the transfer process, the "pkls" folder and pkl files corresponding to "--resume" were not generated. I manually created the folder and copied the last pkl file from the "runs" folder to that location.
Afterwards, during the transfer process, an error occurs after tick 10. It seems that the error is happening in the Dataloader, but I'm not exactly sure of the cause.
My PC has a GeForce RTX 4080 GPU and an i5-10400F CPU.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
I hope the transfer learning process proceeds smoothly after resolving the error.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
Additional context
Add any other context about the problem here.
注意到这行代码 c_dim=0 if self.transfer in ['res_block', 'res_block_match_dis', 'res_block_uni_dis']
想问一下,指定缺陷位置训练阶段,为什么不可以使用label同时作为生成条件?
根据您的代码,如果我要使用 --cond,正确方式应该是?
Hi,
I got error when in the stage2
RuntimeError: output with shape [1, 512, 1, 1] doesn't match the broadcast shape [3, 512, 1, 1]
Hello, im a researcher on data augmentation for defect images
I trained your model(DFMGAN) twice for two kind of custom data
First was showing very good result.
But First stage(normal image training) of second data shows some curious results.
The trained model generate rotated image from normal dataset images.
Then, i found the augmentation options on training_options.json
There is "rotate90 : 1" on the file.
Can you explain why DFMGAN generates rotated images?
预训练时报错,网上找不到相关的问题,可否给点提示
Traceback (most recent call last):
File "/home/zxy/exp/DFMGAN-main/train.py", line 645, in
main() # pylint: disable=no-value-for-parameter
File "/home/zxy/anaconda3/envs/dftgan/lib/python3.9/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/home/zxy/anaconda3/envs/dftgan/lib/python3.9/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/home/zxy/anaconda3/envs/dftgan/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/zxy/anaconda3/envs/dftgan/lib/python3.9/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/zxy/anaconda3/envs/dftgan/lib/python3.9/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/zxy/exp/DFMGAN-main/train.py", line 638, in main
subprocess_fn(rank=0, args=args, temp_dir=temp_dir)
File "/home/zxy/exp/DFMGAN-main/train.py", line 473, in subprocess_fn
training_loop.training_loop(rank=rank, **args)
File "/home/zxy/exp/DFMGAN-main/training/training_loop.py", line 532, in training_loop
result_dict = metric_main.calc_metric(metric=metric, G=snapshot_data['G_ema'],
File "/home/zxy/exp/DFMGAN-main/metrics/metric_main.py", line 46, in calc_metric
results = _metric_dictmetric
File "/home/zxy/exp/DFMGAN-main/metrics/metric_main.py", line 86, in fid50k_full
fid = frechet_inception_distance.compute_fid(opts, max_real=None, num_gen=50000)
File "/home/zxy/exp/DFMGAN-main/metrics/frechet_inception_distance.py", line 29, in compute_fid
mu_gen, sigma_gen = metric_utils.compute_feature_stats_for_generator(
File "/home/zxy/exp/DFMGAN-main/metrics/metric_utils.py", line 280, in compute_feature_stats_for_generator
features = detector(images, **detector_kwargs)
File "/home/zxy/anaconda3/envs/dftgan/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
RuntimeError: MALFORMED INPUT: lanes dont match
For the sake of time, I am wondering if the default setting for batch size, which I believe is 16, is the optimal for this model. I am wondering if a batch size of 16 is too large and will potentially have the learning rate set too high.
Describe the bug
When I run train.py (stage I), I get the following error: RuntimeError: derivative for aten::grid_sampler_2d_backward is not implemented
This happens in accumulate_gradients.
Thought it was related to nvidia (cuda) but now suspect internal torch issue.
Thoughts??? Time-sensitive issue 🙏🏼
Great work! But how can i find the supplementary material mentioned in the article?
When dealing with grayscale images in the grid class, how should the command in dataset_tool.py be configured?
The issue seems to be with this line:
img = np.concatenate((img, np.expand_dims(mask, axis=-1)), axis=2)
It appears to be causing the problem.
Hi, I am graduate student in korea.
I want to generate various defective industrial data images by utilizing your model,
so before handling our datas, I use MVTec Wood datasets (Defect : Scratch) to confirm and check the model's performances.
But, when I train model and inference with wood datasets, Its result is quite different with your paper's result.
Both Good and Mask images have not good quality than I expected. (Furthermore, It occurs Overfitting after 400 kimg in good images / Generated Defect images are not similar with MVTec dataset's original scratch things)
I want to know how to optimize the parameters ( that you used ) to develop image qualities.
What parameters did you handle? (It may be a little rude, but I'm most curious about it.)
Did you take other Augmentation tech except for ADA of StyleGAN2 - ADA ?
This is different from the point, when I generate a defective image in my dataset, in that model, both normal and defect images are generated and combined from Latent vector, but can I modify the model by inputting it into the normal image I have without generating the generated image and combining it with the generated defective image? (just generating defect images from latent vectors, but not for normal images. because, I think generated normal images of my industrial dataset will have poor quaility.)
Thanks for reading!
I appreciate for your model services.
您好,
我电脑只有12G显存,请问可以跑stylegan2-ada么,能够给点建议(尝试调整batchsize,但会带来一些bug),谢谢~
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.