Taking gradients of traces about tensoroperations.jl HOT 6 CLOSED

Simply from looking at the stack trace (I have not tested it myself), it seems as if the pullback function of tensortrace! gets a Tuple{Float64} as adjoint input ΔC, instead of an Array{Float64,0}. This is strange, as the pullback rule of tensorscalar does transform a scalar back into a Array{Float64,0} value. Is Zygote trying to be smart here, by replacing this zero-dimensional array with a Tuple in between these two steps? Or is it somehow simply sidestepping the pullback rule of tensorscalar?

from tensoroperations.jl.

I just quickly checked: The pullback rule of tensorscalar is being called and it does indeed output an Array{Float64,0}. Then ΔC somehow gets converted to a Tuple{Float64} in tensortrace!. Is this perhaps also related to the @thunk and unthunk of ChainRulesCore?

What's also weird is that pC is just an empty tuple ((), ()) such that numind(pC) in tensortrace! errors as well, when I force ΔC to be of the correct type. So I don't quite understand what is happening under the hood of Zygote here.

from tensoroperations.jl.

So apparently it's not Zygote's fault, but ours.

julia> scale(fill(0.5), 0.3)
(0.15,)

scale is the culprit here that changes the Array{Float64,0} into a Tuple{Float64}. Will investigate further, this should very clearly not happen!

Addendum:
The reason seems to be with broadcasting; broadcasting is used to implement scale(::AbstractArray, ::Number), but it treats zero-dimensional arrays specially and does not preserve them.

from tensoroperations.jl.

Ok, so this fixes it on my side: Jutho/VectorInterface.jl@d4d1129

As soon as the tests turn green, I will tag a new release.

from tensoroperations.jl.

Great, thanks for the fast fix!

from tensoroperations.jl.

Fixed and tagged in v4.1

from tensoroperations.jl.