KnetArray transpose and permutedims missing. about knet.jl HOT 7 CLOSED

Is permutedims for 4D and 5D practical? Necessary?

from knet.jl.

I am leaving this issue open for the general N-D implementation.

from knet.jl.

@ilkerkesen do we actually have a general N-D implementation (beyond 5-D) that works? If so, let's close this.

from knet.jl.

from knet.jl.

So far so good,

1. Current implementation works correctly.
2. Yes, it takes advantage of parallel programming, but ignores its crucial core concepts.
3. I did have a generic implementation, couldn't have ported it into Knet, but let's discuss it later since it is going to be as slower as the current implementation.

I've looked at PyTorch's source code and I think this is the part where transpose magic happens. As different from Julia, PyTorch doesn't have a permutedims actually, it has transpose for multi-dimensional arrays also. Let's see the differences,

• PyTorch equivalent of permutedims(a, (1, 3, 2)) is a.transpose(1, 2) (let's do not forget Python has indexes starting from zero).
• PyTorch equivalent of permutedims(a, (3, 1, 2)) is a.transpose(0, 2).transpose(1, 2) which is cascaded.
• PyTorch arrays are row majored. So, we need to convert that CUBLAS calls as column majored.

As a first step, I'm going to implement a transposedim operation mimicking PyTorch's transpose method. We'll discuss the remaining features later.

from knet.jl.

Caffe2 has a multi-dimensional transpose operation in the way we desire. It takes advantage of cudnn's cudnnTransformTensor kernel. I'm on it.

from knet.jl.

Solved using CuArrays under permutedims branch, please test.

from knet.jl.