advimman / hidt Goto Github PK

Hello!
I would like to ask about the discriminators used for training of the model. The repository includes only inference time code and the discriminators aren't included so I wanted to implement the discriminators for training based on the paper, but there is hardly any information about the architecture of the discriminators given in the paper. You say that you use a conditional and and unconditional discriminator and they are trained with LSGAN loss, but what are the details of the architecture of the discriminators? It would really help if you could provide the details so that it is possible to reimplement the training code and reproduce the results.

Hi!

Congratulations for the great work!

The results in the paper were amazing!

1. Could you please tell me when can you publish the code and the training dataset？

2. If the code is not ready now, could you please tell me how can I download the dataset?

Thanks!

Thanks for providing source code.

I found that the implement of AdaIN (AdaptiveInstanceNorm2d) use constant value (zero and one) as running_mean and running_var, and they are unlike bias and weight which will be re-assign during forward.

So the input feature of this layer will not be normalize by it's statistic (mean and var), and just will be scale and shift by weight and bias calculated from style_tensor.

I wonder why not normalize the input features with statistics like instance_norm and then apply shift and scale? What effect will it have?

Hello, I am trying to work similar to yours recently. Your work has given me a lot of inspiration, but when I run your wikiart part of the code on windows according to README.md, the following problem appears. Can you help me solve it?

D:\ANACONDA3\envs\pytorch\python.exe D:/HiDT-master/HiDT-master/infer_on_folders.py
result path : .\results/
100%|██████████| 6/6 [00:00<00:00, 14.09it/s]
100%|██████████| 9/9 [00:00<00:00, 138.83it/s]
Traceback (most recent call last):
File "D:/HiDT-master/HiDT-master/infer_on_folders.py", line 114, in
main()
File "D:/HiDT-master/HiDT-master/infer_on_folders.py", line 69, in main
if args.enhancement == 'generator':
AttributeError: 'Namespace' object has no attribute 'enhancement'

Process finished with exit code 1

I am trying to make a timelapse by using frames from another day to night timlapse video but I am getting the following error message in google colab:
`result path : ./results/

100%|██████████| 1/1 [00:00<00:00, 18.86it/s]
100%|██████████| 128/128 [00:02<00:00, 61.73it/s]
Traceback (most recent call last):
File "./bin/infer_on_folders.py", line 112, in
main()
File "./bin/infer_on_folders.py", line 83, in main
return_pil=False)
File "/usr/local/lib/python3.6/dist-packages/torch/autograd/grad_mode.py", line 43, in decorate_no_grad
return func(*args, **kwargs)
File "/content/HiDT/hidt/style_transformer.py", line 204, in transfer_images_to_styles
batch_size=batch_size,
File "/usr/local/lib/python3.6/dist-packages/torch/autograd/grad_mode.py", line 43, in decorate_no_grad
return func(*args, **kwargs)
File "/content/HiDT/hidt/style_transformer.py", line 153, in transfer_decompositions
decomposition_a, decomposition_b)
File "/content/HiDT/hidt/trainers/trainer_base.py", line 75, in _mix_decompositions
assert target['style'].shape[0] == 1
AssertionError
`
I increase the number of frames but quicky ran out of CUDA memory.
The problem does not occur when using a few style images, only when those style images are from a video.
I am sorry if I come off as naive, I am not that experienced and need help.

Kindly teach me how to make smooth timelapses like in demo videos.

Hi
I am reproducing your results but I did not find any link of dataset day time lapsed images. You mentioned in the paper that over 20,000 images were collected from internet to perform experiments. If you provide me some of those images through google drive then it would save my time.
Thanks

I am very excited and pretty interested in the results of your project.
May I ask ... will you be able to release your training code?

In general, the skip-dim is the same as the output-dim of conv to make full use of low-dim feature map.
So i guess, even if the G use Adain norm block to move out the original style of skip-block, it is not enough if we use the full skip-dim.
Thanks for your help~

Hello, I want to reproduce your great job, but to my limited knowledge, I have two questions right now.
Firstly, I'm trying to rewrite the training phrase and beginning to train on the wikiart with content-dir of 'wikiart/Rococo' while style-dir of 'wikiart/Symbolism', but the intermediate result is not good as you, so I want to know what content-dir and style-dir you choose on the wikiart datasets?
Secondly, my loss on style distribution could not converge, it is always around between 4.2-4.4. My code is as below:

class StyleDistLoss(nn.Module):
    '''
    style distribition loss of s and s'
    '''
    def __init__(self, pool_size):
        super(StyleDistLoss, self).__init__()
        self.pool_size = pool_size
        if self.pool_size > 0:
            self.num_style_batch = 0
            self.style_batches = []
        self.loss = nn.L1Loss()

    def __call__(self, sc, st):
        '''
            return the standart Gaussian distribution loss of input 
            style source {sc} and style traget {st} which are respective to s and s' in the paper
        '''
        styles = []
        if self.pool_size == 0:
            styles.extend([sc, st])
        else:
            styles += self.style_batches
            styles.extend([sc, st])

            detach_sc = sc.clone().detach()
            detach_st = st.clone().detach()

            if self.num_style_batch + 2 < self.pool_size:
                self.style_batches.extend([detach_sc, detach_st])
                self.num_style_batch += 2
            else:
                random_idx = [x for x in range(self.num_style_batch)]
                random.shuffle(random_idx)
                self.style_batches[random_idx[0]] = detach_sc
                self.style_batches[random_idx[1]] = detach_st
        tensor_styles = torch.squeeze(torch.cat(styles, 0))
        styles_mean = torch.mean(tensor_styles, dim=0)
        tminuss = tensor_styles - styles_mean
        cov = torch.mm(tminuss.t(), tminuss) / tensor_styles.shape[0]
        std_cov = cov.diag(diagonal=0)
        total_loss = self.loss(styles_mean, torch.zeros_like(styles_mean))
        total_loss += self.loss(cov, torch.ones_like(cov))
        total_loss += self.loss(std_cov, torch.ones_like(std_cov))
        return total_loss

Could you please give me some advice? Thanks!

Hi! I'm so glad to see HiDT.
This is what I was looking for project. But Your Repo is opened only inference code.
How Can I get Training Code? Thanks!
I expect answer