Issues training models on FluxBench about fluxarchitectures.jl HOT 5 CLOSED

I'll have a look in the next few days. Reminds me a bit of #22, where going from Zygote 0.6.21 to 0.6.22 gave ChainRulesCore.NoTangent errors.

from fluxarchitectures.jl.

Okay, so I am running with 0.6.22 now as well. This needs an mwe so we can inspect it in zygote.

from fluxarchitectures.jl.

The old issue was fixed in a later version of Zygote. The example code to cause the error is still available here. IIRC, it was hard to get it down to a "minimal" example - those seemed to work fine. There is also an old discussion in SliceMaps.jl.

from fluxarchitectures.jl.

Are you sure that this is related to Zygote? I have two branches in my repo:

• faonly_up3, which is your up3 branch, just with everything non FluxArchitectures commented out, and
• faonly, which is an old branch of mine with just the FluxArchitectures benchmarks

I updated my faonly branch to use the latest FluxArchitectures and Zygote 0.6.33, and the CPU benchmarks run fine. When I use the faonly_up3 branch however, I get the error that you saw as well with the same version of Zygote.

It might be worth having a look at the diff, especially the Manifest. I can for example see that there are different versions of CUDA and ChainRules. Do you think that these can be the most likely candidates for the error?

from fluxarchitectures.jl.

It had to do with ChainRules breaking code downstream. The latest FluxBench should be working and has the FA benchmarks in there as well. Thanks for the help!

from fluxarchitectures.jl.