Comments (7)
Is permutedims for 4D and 5D practical? Necessary?
from knet.jl.
I am leaving this issue open for the general N-D implementation.
from knet.jl.
@ilkerkesen do we actually have a general N-D implementation (beyond 5-D) that works? If so, let's close this.
from knet.jl.
from knet.jl.
So far so good,
- Current implementation works correctly.
- Yes, it takes advantage of parallel programming, but ignores its crucial core concepts.
- I did have a generic implementation, couldn't have ported it into Knet, but let's discuss it later since it is going to be as slower as the current implementation.
I've looked at PyTorch's source code and I think this is the part where transpose magic happens. As different from Julia, PyTorch doesn't have a permutedims
actually, it has transpose
for multi-dimensional arrays also. Let's see the differences,
- PyTorch equivalent of
permutedims(a, (1, 3, 2))
isa.transpose(1, 2)
(let's do not forget Python has indexes starting from zero). - PyTorch equivalent of
permutedims(a, (3, 1, 2))
isa.transpose(0, 2).transpose(1, 2)
which is cascaded. - PyTorch arrays are row majored. So, we need to convert that CUBLAS calls as column majored.
As a first step, I'm going to implement a transposedim
operation mimicking PyTorch's transpose
method. We'll discuss the remaining features later.
from knet.jl.
Caffe2 has a multi-dimensional transpose operation in the way we desire. It takes advantage of cudnn's cudnnTransformTensor
kernel. I'm on it.
from knet.jl.
Solved using CuArrays under permutedims branch, please test.
from knet.jl.
Related Issues (20)
- dropout seeds the global RNG
- MethodError: no method matching LinearIndices HOT 2
- ERROR: LoadError: KeyError: key :nw not found
- missing docstring error for "File I/O" section
- Making a new CUDA 3 compatible release? HOT 7
- Create a DeepMap.jl package
- Knet 1.4.7: libknet8 library not found HOT 4
- LoadError: Failed to precompile Knet [1902f260-5fb4-5aff-8c31-6271790ab950] to my julia directory HOT 3
- TagBot trigger issue HOT 2
- load/save problem with CuArrays in tutorial/60.rnn.ipynb HOT 2
- Derivative of a Function That Includes @diff Macro HOT 4
- R1 Regularization HOT 9
- type DataType has no field mutable HOT 2
- Quick Start tutorial notebook is broken HOT 1
- Cannot locate artifact 'libknet8' for aarch64-linux-gnu-libgfortran5-cxx11-libstdcxx29-julia_version+1.7.1 in Docker Container on Apple Silicon HOT 9
- Fix deprecations from MLDatasets in tutorial notebooks
- Problem with conv4 on gpu HOT 4
- UndefVarError: accuracy not defined in tutorial 30.lin.ipynb HOT 1
- CuArray{Float32} in ytype raised error HOT 4
- how to use the resnet modules in example folder
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from knet.jl.