sdobber / fluxarchitectures.jl Goto Github PK
View Code? Open in Web Editor NEWComplex neural network examples for Flux.jl
License: MIT License
Complex neural network examples for Flux.jl
License: MIT License
Updating from Zygote v0.5.5 to v0.5.6 breaks training in the function a -> dropdims(a, dims = (findall(size(a) .== 1)...,))
. Direct workaround would be replacement with a -> a[1,1,:,:]
. See also issue #4.
Ref https://buildkite.com/julialang/fluxbench-dot-jl/builds/84#627a51df-aa8d-4357-963d-877ff8fdef41
some models seem to fail. this is likely due to changes made to Zygote. Is it possible to do a bisect to find out which version of Zygote broke differentiability of certain models? We found this in Metalhead as well, but I would love to have a good idea of what caused this. It is likely aligned with when Zygote started wrapping ChainRules' outputs.
Hello, first of all i want to thank you for this repo because it is the only one i found with some recent "complex" implementation in julia.
im trying the example of tpa-lstm but it seems it is not able to forecast local minima (after a while it turn into a line moving up and down) (I’m not sure if I should try with a larger dataset)
The DSANet Instead output no "pred" (nan32) instead.
any idea?
(julia 1.5.1 and i have all the pkg used updated)
I have tried to run DARNN model on GPU, But some error has occurred.
I modified some code below
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
encoder_lstm = Seq(HiddenRecur(Flux.LSTMCell(inp, encodersize) |> gpu))
decoder_lstm = Seq(HiddenRecur(Flux.LSTMCell(1, decodersize) |> gpu ))
#= @inbounds =# for t in 1:m.poollength # for gpu
input = input |> gpu
target = target |> gpu
model = model |> gpu
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Error message is
Warning: haskey(::TargetIterator, name::String)
is deprecated, use Target(; name = name) !== nothing
instead.
│ caller = llvm_compat(::VersionNumber) at compatibility.jl:176
└ @ CUDAnative ~/.julia/packages/CUDAnative/ierw8/src/compatibility.jl:176
[ Info: CUDA is on
[ Info: device = gpu
[ Info: Training
[ Info: epochs = 1
ERROR: LoadError: MethodError: no method matching (::var"#loss#30")(::CuArrays.CuArray{Float32,4,Nothing}, ::CuArrays.CuArray{Float32,1,Nothing})
Stacktrace:
[1] macro expansion at /home/jee/.julia/packages/Zygote/1GXzF/src/compiler/interface2.jl:0 [inlined]
[2] _pullback(::Zygote.Context, ::var"#loss#30", ::CuArrays.CuArray{Float32,4,Nothing}, ::CuArrays.CuArray{Float32,1,Nothing}) at /home/jee/.julia/packages/Zygote/1GXzF/src/compiler/interface2.jl:13
[3] loss at /home/jee/Projects/Julia/learn/FluxArch/DARNN/e2.jl:82 [inlined]
[4] _pullback(::Zygote.Context, ::var"#loss#30") at /home/jee/.julia/packages/Zygote/1GXzF/src/compiler/interface2.jl:0
[5] pullback(::Function, ::Zygote.Params) at /home/jee/.julia/packages/Zygote/1GXzF/src/compiler/interface.jl:172
[6] gradient(::Function, ::Zygote.Params) at /home/jee/.julia/packages/Zygote/1GXzF/src/compiler/interface.jl:53
[7] train2() at /home/jee/Projects/Julia/learn/FluxArch/DARNN/e2.jl:85
[8] top-level scope at /home/jee/Projects/Julia/learn/FluxArch/DARNN/e2.jl:92
[9] include(::Module, ::String) at ./Base.jl:377
[10] exec_options(::Base.JLOptions) at ./client.jl:288
[11] _start() at ./client.jl:484
Would you to help me to fix the model?
With Zygote v0.6.44, the tests break when doing a gradient calculation on a DARNN network. Might be related to FluxML/Zygote.jl#1111
I am trying to implement TPALSTM using my own data [link](https://www.kaggle.com/datasets/shenba/time-series-datasets/discussion?select=sales-of-shampoo-over-a-three-ye.csv)
. How can I go about it?
Hi,
I would like to check if I see it correctly that in papers researcher refer to the data with 370 users and the data here have 321.
https://archive.ics.uci.edu/ml/datasets/ElectricityLoadDiagrams20112014
If so, do you think it is possible to update?
Thank you very much.
Luboš
Upon training, all output values are NaN
.
Apparently, the construction with Flux.unstack
and Flux.stack
is much slower than the 'slow' Zygote.Buffer
. The latter cannot be used on the GPU due to missing support for array mutation.
Recurrent neural network code will change in a future Flux release:
Recur
gets field init
removed.GRUCell
, LSTMCell
etc. get field h
for the initial hidden state removed.Hi! First of all, thanks a lot for FluxArchitectures.jl! :)
More than an issue, this is a question because of my misunderstanding. While checking the documentation, I realized that I might have been using a wrong implementation of a Stacked LSTM in my codes. However, I’m confused with your current implementation.
I was comparing the description of the Stacked LSTM from your blog with your current implementation and I don’t understand why you are not using the HiddenRecur
anymore. Now it seems like you are chaining the LSTMs just in the regular way, matching the inner dimensions but not feeding the inner cells with the previous hidden states and memories.
Also, in
function (m::StackedLSTMCell)(x)
out = m.chain(x)
m.state = out
return out
end
you are saving only the output in the state of the StackedLSTMCell
, so I don’t understand how this would be fixing the Flux issue that you describe in your blog (Flux.jl's standard setup only allows feeding the output of one cell as the new input
) or is it the case that now Flux behavior has changed fixing this issue?
With horizon
<= poollength
, the RNN has direct access to the value is is supposed to predict.
Thanks @luboshanus for finding this issue.
Hello, i have a few questions regarding the exmaples:
1)from what i understood the model (lets say LSTnet) get as input a target and the data with features (same size of the target + target itself) then update params to make it fit the initial target and the result is an array with the length of the initial size/target
i dont get why the model already get as input most of the target (here m.ConvLayer(x)) (aside from the last few shifted target with the horizon)
so it could just output the input as target with random the last part of the horizon and it would still return an almost "perfect fit"?
assuming it actually forecast without already getting as input the part it should forecast:
3)is the forecast value at time x only last point of the horizon to forecast from time y = x - horizon for any point of the time series?
how is the loss calculated if the last few part in the plot (the horizon of the forecast) are outside the range of the input target (at least in the plot)?
im trying to recreate something like this: https://github.com/jingw2/demand_forecast using these models, and maybe it is the same thing the examples.jl here do when plotting the last part of the horizon outside the target range, but im not sure.
Hi, I didn't find any other way to contact you.
Please, would you be willing to tell me how the network works in a way that you can input batches of different lengths without using reset!
? Is it because the LeNet5()
does some reshaping? I am not much familiar with convolutional networks. I'd like to forecast a basic AR model using just LSTM for example.
I am aware of many examples at Discourse, and it makes sense to use reset!
on hidden states. But you do not in your function that is why I am asking.
Thanks!
@JuliaRegistrator register
In training loop:
ERROR: MethodError: no method matching JuliennedArrays.Slices(::ChainRulesCore.NoTangent, ::Int64)
Include better training data for the models. Could for example be based on the UCI Machine Learning Repository.
Creating a system image with FluxArchitectures fails when using PackageCompiler 2.0.2 and Julia 1.7. It works fine with Julia 1.6.4.
As DA-RNN is basically a encoder-decoder network, how would you do multi step prediction in the decoder phase?
Running Julia 1.8.0 on Windows 11, I copy-pasted the example code at the top of the examples page into a new Julia environment after adding FluxArchitectures and Plots packages. When it reached the line Flux.train!
, it gave the following stack trace:
julia> Flux.train!(loss, Flux.params(model),Iterators.repeated((input, target), 20), ADAM(0.01), cb=cb)
ERROR: BoundsError: attempt to access Tuple{} at index [0]
Stacktrace:
[1] getindex(t::Tuple, i::Int64)
@ Base .\tuple.jl:29
[2] last(a::Tuple{})
@ Base .\abstractarray.jl:479
[3] rrule(config::Zygote.ZygoteRuleConfig{Zygote.Context{true}}, ::typeof(foldl), op::Base.var"#57#58"{typeof(Flux.reset!)}, x::Tuple{}; init::Nothing)
@ ChainRules C:\Users\joel\.julia\packages\ChainRules\fgVxV\src\rulesets\Base\mapreduce.jl:448
[4] chain_rrule_kw(::Zygote.ZygoteRuleConfig{Zygote.Context{true}}, ::Function, ::NamedTuple{(:init,), Tuple{Nothing}}, ::Function, ::Function, ::Vararg{Any})
@ Zygote C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\chainrules.jl:230
[5] macro expansion
@ C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\interface2.jl:0 [inlined]
[6] _pullback(::Zygote.Context{true}, ::Base.var"#foldl##kw", ::NamedTuple{(:init,), Tuple{Nothing}}, ::typeof(foldl), ::Base.var"#57#58"{typeof(Flux.reset!)}, ::Tuple{})
@ Zygote C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\interface2.jl:9
[7] _pullback
@ .\tuple.jl:555 [inlined]
[8] _pullback(::Zygote.Context{true}, ::typeof(foreach), ::typeof(Flux.reset!), ::Tuple{})
@ Zygote C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\interface2.jl:0
[9] _pullback
@ C:\Users\joel\.julia\packages\Flux\EXOFx\src\layers\recurrent.jl:180 [inlined]
[10] _pullback(ctx::Zygote.Context{true}, f::typeof(Flux.reset!), args::typeof(relu))
@ Zygote C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\interface2.jl:0
[11] _pullback
@ .\abstractarray.jl:2774 [inlined]
[12] _pullback(::Zygote.Context{true}, ::typeof(foreach), ::typeof(Flux.reset!), ::NamedTuple{(:σ, :weight, :bias, :stride, :pad, :dilation, :groups), Tuple{typeof(relu), CUDA.CuArray{Float32, 4, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}, Tuple{Int64, Int64}, NTuple{4, Int64}, Tuple{Int64, Int64}, Int64}})
@ Zygote C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\interface2.jl:0
[13] _pullback
@ C:\Users\joel\.julia\packages\Flux\EXOFx\src\layers\recurrent.jl:180 [inlined]
[14] _pullback
@ .\tuple.jl:555 [inlined]
[15] #rrule_via_ad#46
@ C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\chainrules.jl:255 [inlined]
[16] rrule_via_ad
@ C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\chainrules.jl:243 [inlined]
[17] #1678
@ C:\Users\joel\.julia\packages\ChainRules\fgVxV\src\rulesets\Base\mapreduce.jl:444 [inlined]
[18] BottomRF
@ .\reduce.jl:81 [inlined]
[19] #836
@ .\accumulate.jl:291 [inlined]
[20] afoldl(::Base.var"#836#837"{Base.BottomRF{ChainRules.var"#1678#1680"{Zygote.ZygoteRuleConfig{Zygote.Context{true}}, Base.var"#57#58"{typeof(Flux.reset!)}}}}, ::Tuple{Tuple{}, Tuple{Nothing, Nothing}}, ::Conv{2, 4, typeof(relu), CUDA.CuArray{Float32, 4, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}, ::Function)
@ Base .\operators.jl:548
[21] accumulate(op::Function, xs::Tuple{Conv{2, 4, typeof(relu), CUDA.CuArray{Float32, 4, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}, FluxArchitectures.var"#61#63"}; init::Tuple{Nothing, Nothing})
@ Base .\accumulate.jl:290
[22] rrule(config::Zygote.ZygoteRuleConfig{Zygote.Context{true}}, ::typeof(foldl), op::Base.var"#57#58"{typeof(Flux.reset!)}, x::Tuple{Conv{2, 4, typeof(relu), CUDA.CuArray{Float32, 4, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}, FluxArchitectures.var"#61#63"}; init::Nothing)
@ ChainRules C:\Users\joel\.julia\packages\ChainRules\fgVxV\src\rulesets\Base\mapreduce.jl:440
[23] chain_rrule_kw
@ C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\chainrules.jl:230 [inlined]
[24] macro expansion
@ C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\interface2.jl:0 [inlined]
[25] _pullback(::Zygote.Context{true}, ::Base.var"#foldl##kw", ::NamedTuple{(:init,), Tuple{Nothing}}, ::typeof(foldl), ::Base.var"#57#58"{typeof(Flux.reset!)}, ::Tuple{Conv{2, 4, typeof(relu), CUDA.CuArray{Float32, 4, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}, FluxArchitectures.var"#61#63"})
@ Zygote C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\interface2.jl:9
[26] _pullback
@ .\tuple.jl:555 [inlined]
[27] _pullback(::Zygote.Context{true}, ::typeof(foreach), ::typeof(Flux.reset!), ::Tuple{Conv{2, 4, typeof(relu), CUDA.CuArray{Float32, 4, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}, FluxArchitectures.var"#61#63"})
@ Zygote C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\interface2.jl:0
[28] _pullback
@ C:\Users\joel\.julia\packages\Flux\EXOFx\src\layers\recurrent.jl:180 [inlined]
[29] _pullback(ctx::Zygote.Context{true}, f::typeof(Flux.reset!), args::Tuple{Conv{2, 4, typeof(relu), CUDA.CuArray{Float32, 4, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}, FluxArchitectures.var"#61#63"})
@ Zygote C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\interface2.jl:0
[30] _pullback
@ .\abstractarray.jl:2774 [inlined]
[31] _pullback(::Zygote.Context{true}, ::typeof(foreach), ::typeof(Flux.reset!), ::NamedTuple{(:layers,), Tuple{Tuple{Conv{2, 4, typeof(relu), CUDA.CuArray{Float32, 4, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}, FluxArchitectures.var"#61#63"}}})
@ Zygote C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\interface2.jl:0
[32] _pullback
@ C:\Users\joel\.julia\packages\Flux\EXOFx\src\layers\recurrent.jl:180 [inlined]
[33] _pullback
@ .\tuple.jl:555 [inlined]
[34] #rrule_via_ad#46
@ C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\chainrules.jl:255 [inlined]
[35] rrule_via_ad
@ C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\chainrules.jl:243 [inlined]
[36] #1678
@ C:\Users\joel\.julia\packages\ChainRules\fgVxV\src\rulesets\Base\mapreduce.jl:444 [inlined]
[37] BottomRF
@ .\reduce.jl:81 [inlined]
[38] #836
@ .\accumulate.jl:291 [inlined]
[39] afoldl
@ .\operators.jl:548 [inlined]
[40] #accumulate#835
@ .\accumulate.jl:290 [inlined]
[41] #rrule#1677
@ C:\Users\joel\.julia\packages\ChainRules\fgVxV\src\rulesets\Base\mapreduce.jl:440 [inlined]
[42] chain_rrule_kw
@ C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\chainrules.jl:230 [inlined]
[43] macro expansion
@ C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\interface2.jl:0 [inlined]
[44] _pullback
@ C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\interface2.jl:9 [inlined]
[45] _pullback
@ .\tuple.jl:555 [inlined]
[46] _pullback(::Zygote.Context{true}, ::typeof(foreach), ::typeof(Flux.reset!), ::Tuple{Chain{Tuple{Conv{2, 4, typeof(relu), CUDA.CuArray{Float32, 4, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}, FluxArchitectures.var"#61#63"}}, FluxArchitectures.Seq{Flux.Recur{FluxArchitectures.ReluGRUCell{CUDA.CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}}, CUDA.CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}}}, FluxArchitectures.SeqSkip{FluxArchitectures.ReluGRUCell{CUDA.CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}}, Int64}, Chain{Tuple{Dense{typeof(identity), CUDA.CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}}}, Chain{Tuple{FluxArchitectures.var"#62#64", Dense{typeof(identity), CUDA.CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}}}})
@ Zygote C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\interface2.jl:0
[47] _pullback
@ C:\Users\joel\.julia\packages\FluxArchitectures\9iPjE\src\LSTnet.jl:111 [inlined]
[48] _pullback(ctx::Zygote.Context{true}, f::typeof(Flux.reset!), args::FluxArchitectures.LSTnetCell{Chain{Tuple{Conv{2, 4, typeof(relu), CUDA.CuArray{Float32, 4, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}, FluxArchitectures.var"#61#63"}}, FluxArchitectures.Seq{Flux.Recur{FluxArchitectures.ReluGRUCell{CUDA.CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}}, CUDA.CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}}}, FluxArchitectures.SeqSkip{FluxArchitectures.ReluGRUCell{CUDA.CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}}, Int64}, Chain{Tuple{Dense{typeof(identity), CUDA.CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}}}, Chain{Tuple{FluxArchitectures.var"#62#64", Dense{typeof(identity), CUDA.CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}}}})
@ Zygote C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\interface2.jl:0
[49] _pullback
@ .\REPL[16]:2 [inlined]
[50] _pullback(::Zygote.Context{true}, ::typeof(loss), ::CUDA.CuArray{Float32, 4, CUDA.Mem.DeviceBuffer}, ::CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer})
@ Zygote C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\interface2.jl:0
[51] _apply
@ .\boot.jl:816 [inlined]
[52] adjoint
@ C:\Users\joel\.julia\packages\Zygote\qGFGD\src\lib\lib.jl:203 [inlined]
[53] _pullback
@ C:\Users\joel\.julia\packages\ZygoteRules\AIbCs\src\adjoint.jl:65 [inlined]
[54] _pullback
@ C:\Users\joel\.julia\packages\Flux\EXOFx\src\optimise\train.jl:120 [inlined]
[55] _pullback(::Zygote.Context{true}, ::Flux.Optimise.var"#37#40"{typeof(loss), Tuple{CUDA.CuArray{Float32, 4, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}})
@ Zygote C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\interface2.jl:0
[56] pullback(f::Function, ps::Zygote.Params{Zygote.Buffer{Any, Vector{Any}}})
@ Zygote C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\interface.jl:373
[57] gradient(f::Function, args::Zygote.Params{Zygote.Buffer{Any, Vector{Any}}})
@ Zygote C:\Users\joel\.julia\packages\Zygote\qGFGD\src\compiler\interface.jl:96
[58] macro expansion
@ C:\Users\joel\.julia\packages\Flux\EXOFx\src\optimise\train.jl:119 [inlined]
[59] macro expansion
@ C:\Users\joel\.julia\packages\ProgressLogging\6KXlp\src\ProgressLogging.jl:328 [inlined]
[60] train!(loss::Function, ps::Zygote.Params{Zygote.Buffer{Any, Vector{Any}}}, data::Base.Iterators.Take{Base.Iterators.Repeated{Tuple{CUDA.CuArray{Float32, 4, CUDA.Mem.DeviceBuffer}, CUDA.CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}}}, opt::Adam; cb::var"#1#2")
@ Flux.Optimise C:\Users\joel\.julia\packages\Flux\EXOFx\src\optimise\train.jl:117
[61] top-level scope
@ REPL[20]:1
The SkipGRU
layer does not use the correct previous hidden state, but only the current ones.
input, target = get_data(:exchange_rate, poollength, datalength, horizon) |> gpu
works fine on the CPU, but creates some NaNs at the beginning of the dataset (at least on a Jetson Nano).
I'am testing with a test set and want the next value after the time series. Horizon is 1....
So after 800 values, how to fetch the predicted 801 ?
ERROR: MethodError: no method matching iterate(::Nothing)
in Flux.train!
.
DARNN test fail in the backward pass with type Nothing has no field method
. This is probably coming from FluxML/Zygote.jl#1410.
Unfortunately, Zygote cannot be updated currently, see #48
Otherwise, the backward pass on DARNN fails.
Seems to be related to FluxML/Zygote.jl#1304
This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml
to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix
on this issue.
I'll open a PR within a few hours, please be patient!
Change from Diagonal to Scale messes things up.
Hi,
I'm trying to use LSTNet for weather elements forecast.They are 2 dims times mat for a weather station. When I use LSTNet, It out :
the ["PRS", "TEM", "RHU", "U", "V", "PRE_1h", "GST", "lat", "lon", "RHO", "ARED", "AGREEN", "ABLUE"] 1/3
┌ Warning: Size mismatch in loss function! In future this will be an error. In Flux <= 0.12 broadcasting accepts this, but may not give sensible results
│ summary(ŷ) = "1×25 Matrix{Float32}"
│ summary(y) = "13×25 Matrix{Float32}"
Could you give me some advice?
Thank you.
I try to run TPALSTM example on julia 1.9 but it throws error that 'Slices' not defined
exception =
│ UndefVarError: Slices
not defined
│ Stacktrace:
│ [1] (::FluxArchitectures.Seq{FluxArchitectures.StackedLSTMCell{Chain{Tuple{Flux.Recur{Flux.LSTMCell{Matrix{Float32}, Matrix{Float32}, Vector{Float32}, Tuple{Matrix{Float32}, Matrix{Float32}}}, Tuple{Matrix{Float32}, Matrix{Float32}}}, Flux.Recur{Flux.LSTMCell{Matrix{Float32}, Matrix{Float32}, Vector{Float32}, Tuple{Matrix{Float32}, Matrix{Float32}}}, Tuple{Matrix{Float32}, Matrix{Float32}}}}}, Vector{Float32}}})(#unused#::Vector{Float32}, #unused#::FluxArchitectures.StackedLSTMCell{Chain{Tuple{Flux.Recur{Flux.LSTMCell{Matrix{Float32}, Matrix{Float32}, Vector{Float32}, Tuple{Matrix{Float32}, Matrix{Float32}}}, Tuple{Matrix{Float32}, Matrix{Float32}}}, Flux.Recur{Flux.LSTMCell{Matrix{Float32}, Matrix{Float32}, Vector{Float32}, Tuple{Matrix{Float32}, Matrix{Float32}}}, Tuple{Matrix{Float32}, Matrix{Float32}}}}}, Vector{Float32}}, x::Matrix{Float32})
Currently, the hidden state of recurrent network parts is not updated in the right way, as all data fed to them is considerered to be a single timestep. Instead, it needs to be fed as a sequence so that the hidden state gets updated after each timestep.
One way of doing this is by using Flux.unstack
and Flux.stack
. The drawback is that this makes computations extremely slow.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.