oxford-cs-deepnlp-2017 / practical-3 Goto Github PK

(I am an Oxford student taking this course, but I haven't found how to sign up to Piazza.)

I figure could be encoded with all zeros. Is it worth having separate dimensions for and in the embeddings, such that all the words are embedded except those two which are one-hot? Otherwise, should they be random, learned, embeddings? Perhaps I can make only those two embeddings learnable.

Should I just implement all and test?

Thanks for the great course and I am not an oxford student.
I wonder that is there some baselines of task1? Since my model's performance is not good, which is:

Test : 25.60% (size : 250)
Validation : 30.40% (size : 250)
Train : 50.40%

I use word2vec as embedding method. It seems that RNNs work not as well as the perceptions in practice2. I think that there may be some bugs in my networks and I just want to figure out what is the right RNNs' performance.
Thank you.

Hi, I'd like to better understand the input/label format for the LTSM RNN model for language generation. As far as I understand (and correct me if I'm wrong), we have a one-hot-encoding of a word which we feed into the network at each training iteration. My question concerns the training label - do we want a one hot encoding of the next word, or do we want some sort of output word distribution? If so, how would we format these distribution labels that we want to feed into the network? Thanks!