lyndonzheng / tfill Goto Github PK

Hello
How are you?
Thanks for contributing to this project.
I am going to train a model on custom dataset without external mask data.
The custom dataset has ONLY 50K images without mask data.
So I set mask_type to [0,1,2] in base_options because there are external mask data.
Also I used n_iter=20000000 as you did.
Is these right?

Could you please tell me the params and flops of your model?(input=[B,3,256,256]) I'd like to use them for my comparison experiments in my paper. I'll be greatly appreciated if you could give me an early reply. Thank you!

When you train the model from scratch, is it normal that the img_g is completely noise? When continue training on the pretrained model, the img_g is close to the org_input but when training from scratch (I am currently on iter 200,000) the img_g is just blue or yellow.

Hello, thank you for releasing the code. I really appreciate your work. I am trying to train this model on a custom dataset. But I don't know how the data should be structured. Should I have two folders, one for the image and the other for the masks? What should be the input resolution for the images? What should be changed in "train.sh" to allow for training on a custom dataset?

Your paper is very good. I'm expecting for your code. Thanks!

Very great work, is there a c++ reasoning version?

   Hello
   Thank you for contributing to this great project!!
   I have trained a model on custom dataset with external mask data.
   There is my train options:

dataset_name="illustration"
python train.py \
--name ${dataset_name} \
--model tc \
--gpu_ids 0 \
--mask_type 3 \
--img_file ./datasets/${dataset_name}/source_imlist.txt \
--mask_file ./datasets/${dataset_name}/mask_imlist.txt \
--coarse_or_refine coarse \
--batch_size 16 \
--netD style \
--add_noise \
--attn_G \
--attn_D \
--netT original \
--n_encoders 12 \
--n_decoders 0 \
--iter_count 0 \
--n_iter 1000000 \
--save_latest_freq 20000 \
--save_iters_freq 200000 \
--continue_train

   When I tested with my trained model, the error copying a param with shape torch.Size([512, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([512, 512, 1, 1]).',) occur, and latest_net_D is unmatched is there something wrong about my train options? Waiting for your reply. Thank you.

requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8092): Max retries exceeded with url: /env/main (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000025B08842AC8>: Failed to establish a new connection: [WinError 10061] 由于目标计算机积极拒绝，无法连接。'))
Exception in user code:

[WinError 10061] 由于目标计算机积极拒绝，无法连接。
on_close() takes 1 positional argument but 3 were given
Visdom python client failed to establish socket to get messages from the server. This feature is optional and can be disabled by initializing Visdom with use_incoming_socket=False, which will prevent waiting for this request to timeout.

'--n_iter', type=int, default=20000000, help='# of iter with initial learning rate')
非常感谢您的工作，我在复现您的代码当中，使用了280000张celebahq图像在一张v100（32g）卡上训练，我直接采用的您给的默认2000w次迭代，但是迭代次数实在是太多了需要训练几十天，请问根据您的经验，我将n_iter修改成多大能得到较好的结果呢。

Hi, which celeba images did you use for test the model and comparison?
In this file you mentioned (--img_file ./examples/celeba/img/ \ --mask_file ./examples/celeba/mask/ \ ). How we can access or create test image for comparison?
@lyndonzheng

hi, how are you? Thank you for your exciting work.
But can you answer some questions I have, my program keeps getting errors:Command '['ninja', '-v']' returned non-zero exit status 1.andError building extension 'fused'.My window system, I guess the c++ compiler version problem, I tried VS22,19,17,but it didn't work, can you tell me your C++ compiler version,.
thank you.

Thank you for sharing your wonderful work. We would like to use your model as a baseline, but due to the difference in training settings, the direct application of the model parameters you provided to 256256 images would not yield particularly good results. So we would like to train a TFill for a 256256 image version, can you give us some advice on the training settings regarding the model?

I tried to load the provided pretrained weights on CelebA dataset to give predictions about the sample images from the folder examples/celeba. However, it says that the pretrained weights do not match with the network structures.

...
testing images = 2000                                                                           
model [TC] was created                                                                     
creating web directory ./results/celeba_styleD/test_latest                               
loading the model from ./checkpoints/celeba_styleD/celeba_styleD/latest_net_E.pth                                                 
Pretrained network E is unmatched                                        
loading the model from ./checkpoints/celeba_styleD/celeba_styleD/latest_net_G.pth                                 
Pretrained network G is unmatched                                                          
loading the model from ./checkpoints/celeba_styleD/celeba_styleD/latest_net_D.pth                                                      
Pretrained network D is unmatched                                           
loading the model from ./checkpoints/celeba_styleD/celeba_styleD/latest_net_T.pth                                                        
Pretrained network T is unmatched                                                         
---------- Networks initialized -------------    
...

Is this expected and would this affect the prediction?

In fact, the predictions I got are not reasonable, for example:

Thank you very much for your help.

I noticed that in the environment.yaml file the cuda version is 10, however I think I need cuda 11. I think pytorch 1.10 also supports cuda11, is it okay if I just change the cuda version in the yaml file to be cuda11?

Hi authors,

Thanks for your excellent work, well done!

I am wondering, did you retrain the SoTA models for comparison? Or you just use the released pre-train models and test with your own test images and makes for testing?

Many thanks for your help! Look forward to your reply!

Best wishes

Chr1s

how to train high resolution(1024) model
edit parameters--load_size' 5422 '--fine_size' 5122 --fixed_size' 256*2
and train images are in larger resolutions
that's ok?

Hi, Thank you very much for your great work!

I am testing your pretrained model on the example images you provided, but with the resolution of 256x256 (the original size is 512 but i want to test on 256). Here is my args:

python test.py --name celeba --img_file ./examples/celeba/img/ --mask_file ./examples/celeba/mask/ --results_dir ./results --model tc --coarse_or_refine refine --gpu_id 0 --no_shuffle --batch_size 1 --preprocess scale_shortside --mask_type 3 --load_size 256 --fine_size 256 --attn_G --add_noise

And here is the output results (all the images below are 256x256):

These are just 2 examples. I didnt modify any part of the code, I wonder why the refined results are worse than coarse results? Did I use the wrong args? I would be grateful if you can reply to my question, thanks in advance :)

When testing the model on CelebA-HQ dataset, we got exactly same outputs for multiple results. Is there anything wrong?

Hello，Thanks for your code!
But I have some questions ,
why i use the given pre-trained models to test on the test sets of CelebHQ, and i calculate the fid score is 5.03.
i use your code (def center_mask()) to generate the test sets with center masks .
could you please tell me what is the problem?

How to plot the information flow figure by inferring the Jacobian matrix between it to all pixels in the given masked input? Can you offer the code or the related links? Thank you.

Thanks for releasing the code.
During installing the code, I met a small problem with the base model. The notification is "In model.base_model.py, there should be a subclass of BaseModel with class name that matches base in lowercase.". Could you kindly tell me how to fix it? I really appreciate it.

Hello
How are you?
Thanks for contributing to this project.
I found that your pre-trained models were trained on 512x512 resolution data.
Can the pre-trained models on 512x512 resolution data be applied on higher resolution (ex: 2040x2040) images?

I am getting this error:

--------------Options--------------
activation: leakyrelu
add_noise: True
attn_D: False
attn_E: False
attn_G: True
batch_size: 1
checkpoints_dir: ./checkpoints
coarse_or_refine: refine
data_powers: 5
display_env: main
display_id: None
display_port: 8092
display_server: http://localhost
display_single_pane_ncols: 0
display_winsize: 256
down_layers: 4
dropout: 0.0
embed_dim: 512
embed_type: learned
epoch: latest
eval: False
fine_size: 512
fixed_size: 256
gpu_ids: 0
how_many: inf
img_file: ./examples/celeba/img/
img_nc: 3
init_gain: 0.02
init_type: kaiming
isTrain: False
kernel_E: 1
kernel_G: 3
kernel_T: 1
lipip_path: ./model/lpips/vgg.pth
load_size: 512
mask_file: ./examples/celeba/mask/
mask_type: 3
mid_layers: 6
model: tc
nThreads: 8
n_decoders: 0
n_encoders: 12
n_layers_D: 3
n_layers_G: 4
name: celeba
ndf: 32
netD: style
netE: diff
netG: diff
netT: original
ngf: 32
no_flip: False
no_shuffle: True
norm: pixel
nsampling: 1
num_embeds: 1024
num_res_blocks: 2
phase: test
preprocess: scale_shortside
results_dir: ./results
reverse_mask: False
top_k: 10
use_pos_G: False
which_iter: 0
word_size: 16
----------------End----------------
testing images = 13
model [TC] was created
creating web directory ./results\celeba\test_latest
how many is 13
Traceback (most recent call last):
  File "E:\1-EngSci3\1-ECE324\Triple-Dots\TFill\test.py", line 25, in <module>
    for i, data in enumerate(islice(dataset, opt.how_many)):
  File "D:\programs\anaconda3\envs\TFill\lib\site-packages\torch\utils\data\dataloader.py", line 359, in __iter__
    return self._get_iterator()
  File "D:\programs\anaconda3\envs\TFill\lib\site-packages\torch\utils\data\dataloader.py", line 305, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
  File "D:\programs\anaconda3\envs\TFill\lib\site-packages\torch\utils\data\dataloader.py", line 918, in __init__
    w.start()
  File "D:\programs\anaconda3\envs\TFill\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "D:\programs\anaconda3\envs\TFill\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "D:\programs\anaconda3\envs\TFill\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "D:\programs\anaconda3\envs\TFill\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "D:\programs\anaconda3\envs\TFill\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'get_transform.<locals>.<lambda>'
(TFill) PS E:\1-EngSci3\1-ECE324\Triple-Dots\TFill> Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:\programs\anaconda3\envs\TFill\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "D:\programs\anaconda3\envs\TFill\lib\multiprocessing\spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

Hello great research! any specific timeline for when you're planning to release the code?