amirbar / rnn.wgan Goto Github PK

In the fisher GAN implementation, alpha_optimizer_op is not returned and is not run anywhere. Isn't it necessary to add the op as a control_dependency?

I would like to know what hardware you used and its approximate training time.

Could you share your experience of WGAN-GP vs. FisherGAN on learning language generator?
Conceptually, I think 'sampling along the line between real sequence vs. generated sequence' of WGAN-GP sounds not natural for sequence because they can be different length. But I want to know which method works well in practice.

cf) If this is not appropriate topic for Issues, I will delete this post.

Hi,
thanks for the repo, I think it's an original idea to apply GANs.

I would ask for some advice for adapting this code for lyrics. I have large availability of lyrics, but I don't have an idea to adapt the code for my task.
Trying to be more clear: I would create a GAN that, at the end of the training steps, is able to generate lyrics (general format lyrics, not specified on particular artist) and at the same time use the trained discriminator (after training) to recognize lyrics that doesn't match format ( sort of spam detection).

Do you think it's a good idea? Could you suggest something in this way?
Thanks

Simone

@amirbar thanks for releasing this code. I've been doing some experimentation, and I have found the following results:

1. Fisher GAN seems to keep approximately the same performance, but speeds up computation. I personally think it helps the discriminator more because the gradients don't vanish as much compared to WGAN-GP.

2. Simply increasing the GRU's state size to 1024 seems to improve performance as well.

3. As confirmed by @ofirpress on reddit, I can't get any temporal convolution discriminator's to work well.

I don't know if you're team is still pursuing this direction of research but just wanted to notify you in case you were.

HI, I use the default setting for curriculum_training, than use
--GENERATOR_MODEL=Generator_RNN_CL_VL_TH --DISCRIMINATOR_MODEL=Discriminator_RNN --ckpt_path=F:\NLP\rnn-wgan\logs\Generator_RNN_CL_VL_TH-Discriminator_RNN-50-10-512-512-1510731157.353556-\checkpoint
for generate script parameters. While error happens in model.py.
File "F:\NLP\rnn-wgan\model.py", line 52, in Generator_RNN_CL_VL_TH
cells.append(rnn_cell(num_neurons))
TypeError: 'NoneType' object is not callable
Could you plz give me some suggestions?

Hi, I'm interested in your RNN-GANs model but I'm still new to this domain. I'm trying to run your code but I get some problems.
The first one is when I run the generating command:
python generate.py --CKPT_PATH=/logs/Generator_RNN_CL_VL_TH-Discriminator_RNN-50-10-512-512-1509799522.05-/checkpoint --DISC_GRU_LAYERS=2 --GEN_GRU_LAYERS=2
The system outputs this error:

File "generate.py", line 20, in <module>
    _, inference_op = Generator(BATCH_SIZE, charmap_len, seq_len=SEQ_LEN)
  File "/.../rnn.wgan/model.py", line 51, in Generator_RNN_CL_VL_TH
    cells.append(rnn_cell(num_neurons))
TypeError: 'NoneType' object is not callable

The second one is when I train the model by running: python curriculum_training.py. It takes a lot of time, 3 days for training seq-1 to seq-4. I run this code on a server with 256CPU , 1536Core, Intel Xeon E5-4655v3 [6Core, 30M Cache, 2.9GHz], 32TB of RAM (DDR4).

Can you help me about that? Thank you in advance.

It seems current code can only work with fixed length text generation, then generator and discriminator are trained for fixed length input/output. I'm wondering is there a way to generate variable length text?

So I am researching in language models that can generate words.

Your results showed that RNN + GANs improves the quality of generated sequences compared to CNN+GANs.
But do you think that the combination of RNN and GANs performs better than an independent RNN model?(LSTM, GRU)
Because the resulting sentences you got from RNN+GANs are not coherent, and a well-trained RNN model can do the same job. Also, when I was training your model, I feel like using CL+VL+TH is very time-consuming. So is it really worth to train an RNN with GANs? Or the purpose of this project is just to prove that RNN could work well with GANs?

Thanks,
Sida Sun