I tried your lm model and another task with your g2lstm.
I can only get 0 1 distribution with all gates value in training, but it didn't work when we chose one sentence of traning data.
Besides,we can not reproduce the effect that input gate tensor 's mean value approaches 1, we got 0.3.

Sorry to disturb, I'm really confused. I'm doing a study based on your experiment.
Looking forward to your reply.

I am trying to follow your code but here is where I get lost:

            self.B = input_.data.new(input_.size()).bernoulli_(self.p)
            self.noise = self.U * self.B

What is the purpose of B? To simulate some kind of dropout for the noise? Is it mentioned in the paper somewhere?

Thanks in advance.

source: https://github.com/zhuohan123/g2-lstm/blob/master/language-modeling/g2_lstm.py#L42

运行到这一句代码
def repackage_hidden(h):
"""Wraps hidden states in new Variables, to detach them from their history."""
if type(h) == Variable:
return Variable(h.data)
else:
return tuple(repackage_hidden(v) for v in h)#这一句

raise TypeError('iteration over a 0-d tensor')
TypeError: iteration over a 0-d tensor

请问这个函数是做什么用的？为什么会出这个错误呢？

Hi,

This is a very interesting work! And I want to follow your work to experiment on the more meaningful gates.

However, the current code is written in PyTorch 0.3, which makes it incompatible with my current codes. So I wonder if you could provide a PyTorch 1.0 version of the g2-lstm for language modeling? (I only need the codes for g2-lstm model, which seems to be all written in g2-lstm.py