fralotito / pytorch-continuous-bag-of-words Goto Github PK

Thanks for posting the code. A couple of questions/comments below:

1. In the tutorial, it seems the CBOW model has only 1 hidden layer, but in your code, you used 2.

−logp(wi|C)=−logSoftmax(A(∑w∈Cqw)+b)

2. ln72, is it necessary to add the dimension to be summed? Otherwise it'll sum the embeddings along all dimensions and get a scalar?

embeds = sum(self.embeddings(inputs)).view(1,-1)

Hi!

I'm wondering if it's possible to use this model with Glove pre-trained word vectors?
How is it possible to substitute this part in that case?

data = []
for i in range(0, len(raw_text) - 2):
context = [raw_text[i-2], raw_text[i-1]]
target = raw_text[i]
data.append((context, target))

So, I don't have a raw text, but only word vectors and want to predict the next word of the sentence.

what is the biggest you've ever tested 'vocab_size ' value?
if 'vacab_size' is 100,000, emb_dim =128 is this appropriate

    self.linear1 = nn.Linear(embedding_dim, 128)

Hi, why the input is embedding_dim, not 2contextembedding_dim