Speed of Knet about knet.jl HOT 8 CLOSED

Your input,output,weights matrices aren't KnetArrays; calculations are being done on the CPU.

from knet.jl.

That means that whole package is optimized for GPU and slow on CPU?

from knet.jl.

Yes. GPUs are better at these type of calculations. Although I haven't tested your code, it should run much faster on a GPU.

from knet.jl.

from knet.jl.

Hi Dennis,
thanks for the answer. I think that the core issue is that weights of the model is vector of Any, which causes Julia to make ineffective code. I have copied that from example in ReadMe, but it is not something I am quite content. I have tried to change it to tuple, which should give it a rigid definition and julia can better optimize. Though I run into this error:

no method matching axpy!(::Float64, ::Tuple{Array{Float64,2},Array{Float64,1},Array{Float64,2},Array{Float64,1},Array{Float64,2},Float64}, ::Tuple{Array{Float64,2},Array{Float64,1},Array{Float64,2},Array{Float64,1},Array{Float64,2},Float64})

which I think is caused by missing overloaded function. But I was not able to find, where do you overload axpy! for lists of matrices or how to add handling of tuples such that it will work with update!

from knet.jl.

I have modified the code by replacing the array by tuple (which made the compiler aware of types) and removed the update!, because of the problem mentioned in my previous comment and it flies really nicely.

That said, the update! should be modified to handle tuples, especially if it is encouraged in the documentation to do so.

Tomas

from knet.jl.

Hi,
You can use a tuple to hold your weights, but update! accepts only one weight array. You should iterate over your weights to update them. Your initial examples works since julia can perform the minus operation on list of arrays (e.g. [rand(4,2), zeros(2)] - 0.1 * [rand(4,2), zeros(2)]). Optimization parameters should also be a list or tuple (if you do not use Sgd). Because, the optimization methods other than Sgd need to store some additional arrays to perform the update rule.

You can check this example .

from knet.jl.

Hi,
thanks for the answer. I was thinking about solution like that, but before writing that, I wanted to know if there is some machinery inside that makes this explicit.

Thank you very much for the answer.
I really appreciate it.

Tomas

from knet.jl.