Trained DALL-E result about dalle-pytorch HOT 5 OPEN

its worked good. Thanks. I found bug.

But, also i checked that it doesn't work with multi-gpu. I tried with PL training. Have you checked this functional?

from dalle-pytorch.

its worked good. Thanks. I found bug.

But, also i checked that it doesn't work with multi-gpu. I tried with PL training. Have you checked this functional?

Hey, what was the bug you found? 👍

from dalle-pytorch.

@adkuln Hi Aleksey! Is this 12 epochs for the VAE or for DALL-E? Can you give more details on your hyperparameters?

from dalle-pytorch.

@adkuln Hi Aleksey! Is this 12 epochs for the VAE or for DALL-E? Can you give more details on your hyperparameters?

Hi! I trained VAE before the main part. VAE training result looks okay. I trained with flickr-8k
After it, i trained DALL-E with flickr-8k also and result looks like this

My params for training:
vae = DiscreteVAE(
image_size = 256,
num_layers = 3,
num_tokens = 8192,
codebook_dim = 256,
hidden_dim = 128,
temperature = 0.9
)

dalle = DALLE(
dim = 256,
vae = vae, # automatically infer (1) image sequence length and (2) number of image tokens
num_text_tokens = 100000, # vocab size for text
text_seq_len = 128, # text sequence length
depth = 6, # should be 64
heads = 8, # attention heads
dim_head = 64, # attention head dimension
attn_dropout = 0.1, # attention dropout
ff_dropout = 0.1 # feedforward dropout
)

10-12 epochs

from dalle-pytorch.

close

from dalle-pytorch.