[Discussion] General tutorial improvements? about tensorflow-seq2seq-tutorials HOT 8 CLOSED

Yes, these tutorials were very helpful. Thank you for writing them!

Using pre-trained embeddings was also an issue I came across. One of the main difficulties I've had with learning to write encoder-decoder models in TF has been the rapidly changing API. I don't think that your tutorials actually use the legacy seq2seq API at any point - this is a selling point that's probably worth noting!

from tensorflow-seq2seq-tutorials.

The best tutorial about RNN in TF that I have ever read.
It would be great if you add:

1. How we can assign ID numbers to the each word from corpus(the best way with TF API)
2. What if would like to download pre-trained embeddings

from tensorflow-seq2seq-tutorials.

These tutorials help a lot. Thanks so much!

But I just consider that how to coding after we change the parameter time_major. It seems that the code would change a lot if we set the parameter time_major as True which means the format of the inputs is [batch_size, time_major].

And how to use MMI in seq2seq model

from tensorflow-seq2seq-tutorials.

How can I implement attention in 'Advanced dynamic seq2seq with TensorFlow'?

from tensorflow-seq2seq-tutorials.

I also want to ask the question that @pushpankar asked, it seems in 2-seq2seq-advnaced ipython notebook there aren't any implementation for attention mechanism

from tensorflow-seq2seq-tutorials.

@pushpankar
@ryh95
Yep, attention implementation is missing as of now. I was going to implement it in tutorial 2.5, but switched to pytorch soon after finishing tutorial 3.
I would definitely like to return to it at some point in the future.

from tensorflow-seq2seq-tutorials.

Matvey:
Thank you for tutorial. I have couple of questions regarding the Tutorial #1.

1. Is the seq2seq trained to output [W, X, Y, Z, ] or just [X,Y,Z, ] ?
2. The vocab_size is set to 10 ? why? and why we have input_embedding_size = 20 which is greater than vocab_size? I would expect that vocab_size should be far greater than embedding_size?
3. Should we include less.close() at the end to close interaction session?

from tensorflow-seq2seq-tutorials.

@ematvey Thank you for your tutorial. Best one I've found yet (and I've looked at lots of them). I am implementing a multi-dimensional regression seq2seq with Mixture Density Networks. If you could explain how to implement attention mechanism then it would be perfect.

Again, thanks for the tutorial. It is awesome as is!!

from tensorflow-seq2seq-tutorials.