philipperemy / tensorflow-multi-dimensional-lstm Goto Github PK

Hello, I have been researching networks like the one in this paper and I am confused about the difference between

1. a multi dimensional lstm here or in https://arxiv.org/pdf/1611.05664.pdf where 2d lstms are part of a larger network of cnns

and

2. convolutional 2d lstms as described https://arxiv.org/abs/1506.04214v1 ,the source of the Keras layer ConvLSTM2D.

What confuses me is that in the network described in the paper in 1) integrates 2d lstms into a cnn architecture, but it seems that this is different in some way from ConvLSTM2D. Could you explain how your project is different from 2), or how they are not?

Thank you

Hello,

First of all thanks for your work, it is to my knowledge the first public attempt at defining a multidimensional while_loop.

If I understood your implementation, each cell of the grid is updated sequentially from left to right, then up to down. But as pointed by your illustration in the ReadMe, this process is parallelizable, as each cell on the diagonal orthogonal to the propagation can be updated independantly. Do you have any lead on performing this processing? I tried to work on this one but could not figure out a solution, Tensorflow really isn't easy to use in this case because the number of cell states to compute at a time is a function of h and w.

Thanks,
Quentin

In the comment section of "def multi_dimensional_rnn_while_loop(rnn_size, input_data, sh, dims=None, scope_n="layer1"):" funtion you have mensioned return tensor shape is [batch,h/sh[0],w/sh[1],channels x sh[0] x sh[1]]
but Actually when you are returning from above mensioned function your return tensor shape is
[batch,h/sh[0],w/sh[1],rnn_size]
Can you please explain why is it so?

Hi, How to use it in keras? Can you give me some advice?please