High allocations and getindex about cuda.jl HOT 5 CLOSED

Ok, now I think that the problem might be related to the definition of grad for getindex https://github.com/FluxML/Flux.jl/blob/ca1c73ed352d0557a72e12b663fff20332d9aaff/src/tracker/lib/array.jl#L100

Where you allocate a new zeroed array every time (zero(xs))

It would be nice to be able to create a custom sparse version of this, but sparse cuarrays and views throw errors.

from cuda.jl.

The output of CuArrays.@time has changed (it contained a bug), and there's been some optimizations to the memory allocator, so it might be worth checking your use case again.

We now also have support for pushing objects into the GC pool again, see JuliaGPU/CuArrays.jl#275, so this could be used in Flux to early-free those zero arrays before returning. Would need some abstraction in order to support different kinds of arrays, but that should be straightforward.

from cuda.jl.

Thanks for looking into it. Should I use the do syntax or put some unsafe_frees in the code, or is it expected that the code snippet above should be faster as is?

from cuda.jl.

Both. The GC allocator has improved, so please try the master branch of CuArrays. But it didn't change the last example you posted much, so it might be worth adding some unsafe_free!s/do-block syntaxes.

from cuda.jl.

Let's close this, see #137 (comment). If the issue persists, please open a new issue with an updated MWE.

from cuda.jl.