google / tim-gan Goto Github PK

Hi,

It was great to read your paper.

Retrieval script
I was wondering if you are going to release the retrieval script any time soon?

Autoencoder for getting image embeddings for retrieval:
What is the exact architecture of this autoencoder? Is the encoder and decoder the same as the encoder and generator used in TIM-GAN.
could you please explain the process of retrieval pleas, in particular, we have an autoencoder, made of an encoder E1 and decoder D1.
then we pretrain this autoencoder on the dataset. Could you tell me the exact pertaining process, loss functions etc? Can we use the run_pretrain.sh script for training the autoencoder?

While calculating recall for your method, this is written in the paper:

but how do we calculate the recall for other methods? i.e what is the encoder used in that case?

In my understanding, it should have been that there is a separate autoencoder trained on the dataset, which does not have anything to do with TIM-GAN, or any of the other methods, and then after all the models are trained and they are able to generate images, we can use this pretrained autoencoder to get the image representations of the generated images, and use it as a query.
could you tell me if this is happening in the paper, or if not then what is the exact process, because I want to calculate the metrics for these methods on my side.

Thanks you in advance, for the help

Hi,

The CSS dataset in the TIRG repository is not valid for training the TIM-GAN model because they are not re-rendered to solve the misalignment issue of the unchanged objects in the images. Since the attention mechanism is trained based on the differences between the source and the target images, the model could not learn which object to give attention to when the non-target objects are misaligned. Can you please share the dataset generator code or make the re-rendered dataset available?

An example of the misalignment issue is as follows:

The locations of the objects are not stable between the input (source) and the ground truth (target) images.