uzaymacar / attention-mechanisms Goto Github PK

First, Thank you for your work

By following your description, I'm trying to implement the attention layers each with tf 2.1.

I have a question that does the line 221 requires to be add a "squeeze" the inputs ?
attention_score = RepeatVector(source_hidden_states.shape[1])(tf.squeeze(attention_score))
because if i understood the full code correctly, the h_t is already expanded_dim and its attention score is (B, 1, H) before getting in the Repeatvector. However, when i feeding the (B, 1, H) to repeat vector, it rise an error as [repeat_vector is incompatible with the layer: expected ndim=2, found ndim=3.]

Thank you

Dear sir or madam:

Thanks for your time. Recently I have been studying your codes of attention-mechanisms and felt it is very beneficial for me. However, a bug occurs when I run document_classification.py, which throws an error that: 'Dimension' object does not support indexing. It seems that some bugs exist in Class Attention(Layer). The Parameter config was set to 1 and 2 respectively but the same bug appears, and it points out that the codes "self.input_sequence_length, self.hidden_dim = input_shape[0][1], input_shape[0][2]" (line 103, layers.py) lead to the problem. Could you please help me to fix this bug. I'd appreciate it if you are kind to help me.

Best wishes,
Jie Yang