enzymead / reactant.jl Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
using Reactant, Lux, Random # , Optimisers
function loss_fn(model, ps, st, data)
y, st_new = model(data, ps, st)
return sum(y) # , st_new, (;)
end
model = Dense(10, 5, tanh)
data = rand(Float32, 10, 3)
ps, st = Lux.setup(Xoshiro(0), model)
reactant_ps = Reactant.make_tracer(IdDict(), ps, (), Reactant.ArrayToConcrete, nothing)
reactant_st = Reactant.make_tracer(IdDict(), st, (), Reactant.ArrayToConcrete, nothing)
reactant_data = Reactant.make_tracer(IdDict(), data, (), Reactant.ArrayToConcrete, nothing)
reactant_loss_fn = Reactant.compile(
loss_fn, (model, reactant_ps, reactant_st, reactant_data))
function gradient_loss_fn(model, ps, st, data)
dps = Enzyme.make_zero(ps)
Enzyme.autodiff(Enzyme.Reverse, loss_fn, Active, Const(model),
Duplicated(ps, dps), Const(st), Const(data))
return dps
end
gradient_loss_fn(model, ps, st, data) # works
reactant_gradient_loss_fn = Reactant.compile(
gradient_loss_fn, (model, reactant_ps, reactant_st, reactant_data))
# Lux.Experimental.single_train_step(AutoReactant(), loss_fn, data, ts)
terminate called after throwing an instance of 'xla::XlaRuntimeError'
what(): UNKNOWN: <unknown>:0: error: Reduction function must return a scalar or tuple of scalars but returns shape: f32[5]:
<unknown>:0: note: see current operation: "func.return"(%11, %10) : (tensor<1x5xf32>, tensor<10x5xf32>) -> ()
I found this code:
Lines 122 to 124 in 1cd144f
This code is wrong: Base.similar
is not used to convert the eltype of an array, but to create a new array (with the same size and eltype by default, but configurable) but with undefined content.
Instead, Base.convert
should be used for this.
Furthermore, this definition just returns the same array?
From Julia documentation,
A closure is simply a callable object with field names corresponding to captured variables. For example, the following code:
function adder(x) return y->x+y endis lowered to (roughly):
struct ##1{T} x::T end (_::##1)(y) = _.x + y function adder(x) return ##1(x) end
It shouldn't be hard to add support for closures. We just need to correctly track the captured variables in Reactant.make_mlir_fn
, right?
I'm randomly running into a crash when printing the output of calling stablehlo.einsum
just after calling it.
I believe it might be a problem with buffer synchronization because...
XLA.await
on the result buffer and XLA.is_ready
returns trueAlso, the problem might be linked with using struct
types, as I've been unable to recreate the error if working directly on arrays.
julia> f(a′, b′)
2048×2048 Tensor{Float64, 2, Reactant.ConcreteRArray{Float64, (2048, 2048), 2}}:
0.0 233.947 239.041 242.471 244.843 233.684 234.742 242.731 241.478 235.834 246.115 228.468 162.586 153.2 … 241.457 238.066 238.748 231.795 250.899 Error showing value of type Tensor{Float64, 2, Reactant.ConcreteRArray{Float64, (2048, 2048), 2}}:
ERROR: ArgumentError: can't repeat a string -1 times
Stacktrace:
[1] repeat(s::String, r::Int64)
@ Base ./strings/substring.jl:263
[2] repeat
@ Base ./strings/substring.jl:260 [inlined]
[3] print_matrix_row(io::IOContext{Base.TTY}, X::AbstractVecOrMat, A::Vector{Tuple{Int64, Int64}}, i::Int64, cols::Vector{Int64}, sep::String, idxlast::Int64)
@ Base ./arrayshow.jl:118
[4] _print_matrix(io::IOContext{Base.TTY}, X::AbstractVecOrMat, pre::String, sep::String, post::String, hdots::String, vdots::String, ddots::String, hmod::Int64, vmod::Int64, rowsA::UnitRange{Int64}, colsA::UnitRange{Int64})
@ Base ./arrayshow.jl:254
[5] print_matrix(io::IOContext{Base.TTY}, X::Tensor{Float64, 2, Reactant.ConcreteRArray{Float64, (2048, 2048), 2}}, pre::String, sep::String, post::String, hdots::String, vdots::String, ddots::String, hmod::Int64, vmod::Int64)
@ Base ./arrayshow.jl:171
[6] print_matrix(io::IO, X::AbstractVecOrMat, pre::AbstractString, sep::AbstractString, post::AbstractString, hdots::AbstractString, vdots::AbstractString, ddots::AbstractString, hmod::Integer, vmod::Integer)
@ Base ./arrayshow.jl:171 [inlined]
[7] print_array
@ ./arrayshow.jl:358 [inlined]
[8] show(io::IOContext{Base.TTY}, ::MIME{Symbol("text/plain")}, X::Tensor{Float64, 2, Reactant.ConcreteRArray{Float64, (2048, 2048), 2}})
@ Base ./arrayshow.jl:399
[9] (::REPL.var"#55#56"{REPL.REPLDisplay{REPL.LineEditREPL}, MIME{Symbol("text/plain")}, Base.RefValue{Any}})(io::Any)
@ REPL /gpfs/apps/MN5/GPP/JULIA/1.10.0/INTEL/share/julia/stdlib/v1.10/REPL/src/REPL.jl:273
[10] with_repl_linfo(f::Any, repl::REPL.LineEditREPL)
@ REPL /gpfs/apps/MN5/GPP/JULIA/1.10.0/INTEL/share/julia/stdlib/v1.10/REPL/src/REPL.jl:569
[11] display(d::REPL.REPLDisplay, mime::MIME{Symbol("text/plain")}, x::Any)
@ REPL /gpfs/apps/MN5/GPP/JULIA/1.10.0/INTEL/share/julia/stdlib/v1.10/REPL/src/REPL.jl:259
[12] display(d::REPL.REPLDisplay, x::Any)
@ REPL /gpfs/apps/MN5/GPP/JULIA/1.10.0/INTEL/share/julia/stdlib/v1.10/REPL/src/REPL.jl:278
[13] display(x::Any)
@ Base.Multimedia ./multimedia.jl:340
[14] #invokelatest#2
@ ./essentials.jl:887 [inlined]
[15] invokelatest
@ ./essentials.jl:884 [inlined]
[16] print_response(errio::IO, response::Any, show_value::Bool, have_color::Bool, specialdisplay::Union{Nothing, AbstractDisplay})
@ REPL /gpfs/apps/MN5/GPP/JULIA/1.10.0/INTEL/share/julia/stdlib/v1.10/REPL/src/REPL.jl:315
[17] (::REPL.var"#57#58"{REPL.LineEditREPL, Pair{Any, Bool}, Bool, Bool})(io::Any)
@ REPL /gpfs/apps/MN5/GPP/JULIA/1.10.0/INTEL/share/julia/stdlib/v1.10/REPL/src/REPL.jl:284
[18] with_repl_linfo(f::Any, repl::REPL.LineEditREPL)
@ REPL /gpfs/apps/MN5/GPP/JULIA/1.10.0/INTEL/share/julia/stdlib/v1.10/REPL/src/REPL.jl:569
[19] print_response(repl::REPL.AbstractREPL, response::Any, show_value::Bool, have_color::Bool)
@ REPL /gpfs/apps/MN5/GPP/JULIA/1.10.0/INTEL/share/julia/stdlib/v1.10/REPL/src/REPL.jl:282
[20] (::REPL.var"#do_respond#80"{Bool, Bool, REPL.var"#93#103"{REPL.LineEditREPL, REPL.REPLHistoryProvider}, REPL.LineEditREPL, REPL.LineEdit.Prompt})(s::REPL.LineEdit.MIState, buf::Any, ok::Bool)
@ REPL /gpfs/apps/MN5/GPP/JULIA/1.10.0/INTEL/share/julia/stdlib/v1.10/REPL/src/REPL.jl:911
[21] (::VSCodeServer.var"#103#106"{REPL.var"#do_respond#80"{Bool, Bool, REPL.var"#93#103"{REPL.LineEditREPL, REPL.REPLHistoryProvider}, REPL.LineEditREPL, REPL.LineEdit.Prompt}})(mi::REPL.LineEdit.MIState, buf::IOBuffer, ok::Bool)
@ VSCodeServer ~/.vscode-server/extensions/julialang.language-julia-1.79.2/scripts/packages/VSCodeServer/src/repl.jl:122
[22] #invokelatest#2
@ Base ./essentials.jl:887 [inlined]
[23] invokelatest
@ Base ./essentials.jl:884 [inlined]
[24] run_interface(terminal::REPL.Terminals.TextTerminal, m::REPL.LineEdit.ModalInterface, s::REPL.LineEdit.MIState)
@ REPL.LineEdit /gpfs/apps/MN5/GPP/JULIA/1.10.0/INTEL/share/julia/stdlib/v1.10/REPL/src/LineEdit.jl:2656
[25] run_frontend(repl::REPL.LineEditREPL, backend::REPL.REPLBackendRef)
@ REPL /gpfs/apps/MN5/GPP/JULIA/1.10.0/INTEL/share/julia/stdlib/v1.10/REPL/src/REPL.jl:1312
[26] (::REPL.var"#62#68"{REPL.LineEditREPL, REPL.REPLBackendRef})()
@ REPL /gpfs/apps/MN5/GPP/JULIA/1.10.0/INTEL/share/julia/stdlib/v1.10/REPL/src/REPL.jl:386
using Reactant
using Cassette
struct Tensor{T,N,A<:AbstractArray{T,N}} <: AbstractArray{T,N}
data::A
inds::Vector{Symbol}
end
Tensor(data::A, inds::AbstractVector{Symbol}) where {T,N,A<:AbstractArray{T,N}} = Tensor{T,N,A}(data, inds)
Base.parent(x::Tensor) = x.data
Base.size(t::Tensor) = size(parent(t))
Base.@propagate_inbounds Base.getindex(t::Tensor, i...) = getindex(parent(t), i...)
n = 2048
a = Tensor(rand(n,n), [:i, :j]);
b = Tensor(rand(n,n), [:j, :k])
a′ = Tensor(Reactant.ConcreteRArray(a.data), a.inds);
b′ = Tensor(Reactant.ConcreteRArray(b.data), b.inds);
contract(a,b) = a.data * b.data
function contract(a::Tensor{Ta,Na,Aa}, b::Tensor{Tb,Nb,Ab}) where {Ta,Na,Aa<:Reactant.TracedRArray,Tb,Nb,Ab<:Reactant.TracedRArray}
ia = collect(a.inds)
ib = collect(b.inds)
i = ∩(ia, ib)
ic::Vector{Symbol} = setdiff(ia ∪ ib, i)
T = Base.promote_eltype(a, b)
mlirty = Reactant.MLIR.IR.Type(T)
op_a = parent(a).mlir_data
op_b = parent(b).mlir_data
rsize = (size(a.data, 1), size(b.data, 2))
result_0 = Reactant.MLIR.IR.TensorType(rsize, mlirty)
einsum_config = Reactant.MLIR.IR.Attribute("$(join(ia)),$(join(ib))->$(join(ic))")
result = Reactant.MLIR.IR.result(Reactant.MLIR.Dialects.stablehlo.einsum(op_a, op_b; result_0, einsum_config))
data = Reactant.TracedRArray{T,rsize,length(ic)}((), result)
_res = Tensor(data, ic)
return _res
end
f = Reactant.compile(contract, (a′,b′))
f(a′,b′)
Tensor
, it seems to workusing Reactant
using Cassette
n = 2048
a = rand(n, n);
b = rand(n, n);
a′ = Reactant.ConcreteRArray(a);
b′ = Reactant.ConcreteRArray(b);
matmul(a,b) = a * b
function matmul(a::Reactant.TracedRArray{Ta,Sa,Na}, b::Reactant.TracedRArray{Tb,Sb,Nb}) where {Ta,Tb,Sa,Sb,Na,Nb}
T = Base.promote_type(Ta,Tb)
mlirty = Reactant.MLIR.IR.Type(T)
rsize = (Sa[1], Sb[2])
result_0 = Reactant.MLIR.IR.TensorType(rsize, mlirty)
einsum_config = Reactant.MLIR.IR.Attribute("ij,jk->ik")
result = Reactant.MLIR.IR.result(Reactant.MLIR.Dialects.stablehlo.einsum(a.mlir_data, b.mlir_data; result_0, einsum_config))
return Reactant.TracedRArray{T,rsize,2}((), result)
end
Cassette.overdub(ctx::Reactant.TraceCtx, f::typeof(matmul), args...; kwargs...) = f(args...; kwargs...)
f = Reactant.compile(matmul, (a′,b′))
f(a′,b′)
alternatively construct RArrays with a device
I don't see a device argument to ConcreteRArray. Is this currently not implemented?
Also, can we directly pass a Ptr to the data for ConcreteRArray, for example, if we have a CuArray input to a Lux model, I want to wrap it as a ConcreteRArray without paying the cost to copy it.
In Julia, exp
, cos
, sin
and other functions, when run with array inputs, do not return the element-wise application of the function but the "matrix"-version of those functions. However, Reactant generates the element-wise application of those functions.
In order to keep consistency with Julia semantics, we should change this.
julia> Reactant.@code_hlo sin(c)
Module:
module attributes {transform.with_named_sequence} {
func.func @main(%arg0: tensor<2x3xf64>) -> tensor<2x3xf64> {
%0 = stablehlo.sine %arg0 : tensor<2x3xf64>
return %0 : tensor<2x3xf64>
}
}
julia> f(x) = sin.(x) # need to define this way because `@code_hlo` doesn't like broadcasting
f (generic function with 1 method)
julia> Reactant.@code_hlo f(c)
Module:
module attributes {transform.with_named_sequence} {
func.func @main(%arg0: tensor<2x3xf64>) -> tensor<2x3xf64> {
%0 = stablehlo.sine %arg0 : tensor<2x3xf64>
return %0 : tensor<2x3xf64>
}
}
Not needed for compilation, but if we want to allow eager more operations.
Broadcasting doesn't preserve Array type
julia> using Reactant
julia> Reactant.ConcreteRArray(rand(10, 3))
10×3 Reactant.ConcreteRArray{Float64, (10, 3), 2}:
0.241899 0.892 0.344609
0.658603 0.978326 0.0684158
0.154474 0.506356 0.737901
0.665978 0.635999 0.312166
0.955006 0.613347 0.834144
0.261397 0.361902 0.758361
0.838547 0.939787 0.191731
0.336063 0.649798 0.305722
0.0539672 0.754429 0.444449
0.964148 0.524124 0.628569
julia> Reactant.ConcreteRArray(rand(10, 3)) .+ 1
10×3 Matrix{Float64}:
1.60667 1.2144 1.15662
1.79119 1.18538 1.81092
1.01788 1.73736 1.10406
1.88946 1.51043 1.7367
1.11927 1.46719 1.82236
1.18602 1.32522 1.6286
1.12319 1.79284 1.85603
1.42769 1.33347 1.96626
1.40952 1.39544 1.52497
1.23689 1.28314 1.1096
Currently expected since we don't have the broadcast style overloads.
When trying to run the test on my aarch64 macOS, I'm getting the following fatal error on line
Line 15 in 6f9c9a1
Precompiling project...
7 dependencies successfully precompiled in 20 seconds. 122 already precompiled.
Testing Running tests...
Test Summary: | Pass Total Time
Layout | 5 5 0.1s
F0504 21:01:25.451342 38805431 simple_orc_jit.cc:310] Check failed: target_machine != nullptr
*** Check failure stack trace: ***
@ 0x1601222b0 absl::lts_20230802::log_internal::LogMessage::SendToLog()
@ 0x160121cac absl::lts_20230802::log_internal::LogMessage::Flush()
@ 0x16012268c absl::lts_20230802::log_internal::LogMessageFatal::~LogMessageFatal()
@ 0x1601226ac absl::lts_20230802::log_internal::LogMessageFatal::~LogMessageFatal()
@ 0x15c6435c0 xla::cpu::SimpleOrcJIT::InferTargetMachineForJIT()
@ 0x15be5668c xla::cpu::CpuCompiler::RunHloPasses()
@ 0x15be2c130 xla::TfrtCpuClient::Compile()
@ 0x15be2ceb8 xla::TfrtCpuClient::Compile()
@ 0x15b960608 ClientCompile
@ 0x122db1ccc 0x0
@ 0x122db27d0 0x0
@ 0x122b80ce4 0x0
@ 0x122b880bc 0x0
@ 0x1015c23b4 do_call
@ 0x1015c08ec eval_body
@ 0x1015c0e08 eval_body
@ 0x1015c0e08 eval_body
@ 0x1015c11d0 jl_interpret_toplevel_thunk
@ 0x1015d96f8 jl_toplevel_eval_flex
@ 0x1015d94c0 jl_toplevel_eval_flex
@ 0x1015da3a8 ijl_toplevel_eval_in
@ 0x130aa1ab8 japi1_include_string_81107.4
@ 0x1015a7cb4 ijl_apply_generic
@ 0x13125fcbc japi1__include_81115.4
@ 0x1015c23b4 do_call
@ 0x1015c0bb0 eval_body
@ 0x1015c11d0 jl_interpret_toplevel_thunk
@ 0x1015d96f8 jl_toplevel_eval_flex
@ 0x1015d94c0 jl_toplevel_eval_flex
@ 0x1015da3a8 ijl_toplevel_eval_in
@ 0x130aa1ab8 japi1_include_string_81107.4
@ 0x1015a7cb4 ijl_apply_generic
@ 0x13125fcbc japi1__include_81115.4
@ 0x1015c23b4 do_call
@ 0x1015c0bb0 eval_body
@ 0x1015c11d0 jl_interpret_toplevel_thunk
@ 0x1015d96f8 jl_toplevel_eval_flex
@ 0x1015d94c0 jl_toplevel_eval_flex
@ 0x1015da3a8 ijl_toplevel_eval_in
@ 0x13172133c jlplt_ijl_toplevel_eval_in_25935.3
@ 0x1016057b0 true_main
@ 0x1016056a4 jl_repl_entrypoint
@ 0x100b93f6c main
@ 0x18d02a0e0 start
[46954] signal (6): Abort trap: 6
in expression starting at /Users/mofeing/Developer/Reactant.jl/test/basic.jl:7
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 5647750 (Pool: 5642220; Big: 5530); GC: 7
ERROR: Package Reactant errored during testing (received signal: 6)
I'm opening an issue because I don't see this error on CI.
When trying to run the tests, I get the following error:
Precompiling project...
✗ Reactant → ReactantNNlibExt
1 dependency successfully precompiled in 7 seconds. 127 already precompiled.
1 dependency had output during precompilation:
┌ Reactant
│ error in running finalizer: TypeError(func=Symbol("ccall: first argument not a pointer or valid constant expression"), context="", expected=Ptr{T} where T, got=(:PjRtBufferFree, 0x000000008fc4a150))
│ ijl_type_error_rt at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-HL2F7YQ3XH.0/build/default-honeycrisp-HL2F7YQ3XH-0/julialang/julia-release-1-dot-10/src/rtutils.c:119
│ ijl_type_error at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-HL2F7YQ3XH.0/build/default-honeycrisp-HL2F7YQ3XH-0/julialang/julia-release-1-dot-10/src/rtutils.c:127
│ #5 at /Users/mofeing/Developer/Reactant.jl/src/XLA.jl:114
│ unknown function (ip: 0x105c080e3)
│ _jl_invoke at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-HL2F7YQ3XH.0/build/default-honeycrisp-HL2F7YQ3XH-0/julialang/julia-release-1-dot-10/src/gf.c:0 [inlined]
│ ijl_apply_generic at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-HL2F7YQ3XH.0/build/default-honeycrisp-HL2F7YQ3XH-0/julialang/julia-release-1-dot-10/src/gf.c:3077
│ run_finalizer at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-HL2F7YQ3XH.0/build/default-honeycrisp-HL2F7YQ3XH-0/julialang/julia-release-1-dot-10/src/gc.c:318
│ jl_gc_run_finalizers_in_list at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-HL2F7YQ3XH.0/build/default-honeycrisp-HL2F7YQ3XH-0/julialang/julia-release-1-dot-10/src/gc.c:408
│ run_finalizers at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-HL2F7YQ3XH.0/build/default-honeycrisp-HL2F7YQ3XH-0/julialang/julia-release-1-dot-10/src/gc.c:454
│ ijl_atexit_hook at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-HL2F7YQ3XH.0/build/default-honeycrisp-HL2F7YQ3XH-0/julialang/julia-release-1-dot-10/src/init.c:299
│ jl_repl_entrypoint at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-HL2F7YQ3XH.0/build/default-honeycrisp-HL2F7YQ3XH-0/julialang/julia-release-1-dot-10/src/jlapi.c:732
└
Testing Running tests...
libc++abi: terminating due to uncaught exception of type xla::XlaRuntimeError: NOT_FOUND: Could not find registered platform with name: "cuda". Available platform names are:
[42937] signal (6): Abort trap: 6
in expression starting at /Users/mofeing/Developer/Reactant.jl/test/runtests.jl:1
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 2908 (Pool: 2900; Big: 8); GC: 0
ERROR: Package Reactant errored during testing (received signal: 6)
When running the example in the README, it crashes with the following error.
julia> f=Reactant.compile(sinsum_add, (input1,input2))
F0626 06:23:06.397895 67363613 simple_orc_jit.cc:311] Check failed: target_machine != nullptr
*** Check failure stack trace: ***
@ 0x30f082cc0 absl::lts_20230802::log_internal::LogMessage::SendToLog()
@ 0x30f0826bc absl::lts_20230802::log_internal::LogMessage::Flush()
@ 0x30f08309c absl::lts_20230802::log_internal::LogMessageFatal::~LogMessageFatal()
@ 0x30f0830bc absl::lts_20230802::log_internal::LogMessageFatal::~LogMessageFatal()
@ 0x30b62c0b0 xla::cpu::SimpleOrcJIT::InferTargetMachineForJIT()
@ 0x30b5976ec xla::cpu::CpuCompiler::RunHloPasses()
@ 0x30b4328f0 xla::TfrtCpuClient::Compile()
@ 0x30b433678 xla::TfrtCpuClient::Compile()
@ 0x30af44610 ClientCompile
@ 0x1463a015c 0x0
@ 0x135b5ccd0 0x0
@ 0x135b840bc 0x0
@ 0x10309a484 do_call
@ 0x103098c80 eval_body
@ 0x1030992a0 jl_interpret_toplevel_thunk
@ 0x1030b17e4 jl_toplevel_eval_flex
@ 0x1030b15ac jl_toplevel_eval_flex
@ 0x1030b15ac jl_toplevel_eval_flex
@ 0x1030b15ac jl_toplevel_eval_flex
@ 0x1030b2494 ijl_toplevel_eval_in
@ 0x1382f2268 japi1_eval_user_input_91784.4
@ 0x136bb6474 julia_YY.run_replYY.59_91871.3
@ 0x13606e5d4 julia_YY.1013_82830.3
@ 0x10308e1c8 jl_f__call_latest
@ 0x138301228 julia_run_main_repl_82800.4
@ 0x1030dd910 true_main
@ 0x1030dd804 jl_repl_entrypoint
@ 0x10266bf6c main
@ 0x1951020e0 start
[92763] signal (6): Abort trap: 6
in expression starting at REPL[5]:1
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 8223320 (Pool: 8215026; Big: 8294); GC: 13
fish: Job 1, 'julia --project=.' terminated by signal SIGABRT (Abort)
The error seems to originate from XLA's and Julia's LLVM symbols getting mixed.
I'm trying out Reactant to compile DifferentiationInterface operators, and running into this error in my test suite: https://github.com/gdalle/DifferentiationInterface.jl/actions/runs/9642905092/job/26591732655?pr=325
I think it's because broadcasting ^
is not supported?
This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml
to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix
on this issue.
I'll open a PR within a few hours, please be patient!
It is something similar to @code_llvm
but that prints the StableHLO-generated code.
This is probably not hard; it is already available inside the compile function but it would be nice to have it.
@code_mlir
macro, like in pangoraw/Coil.jl #17Currently it errors if the result has a NamedTuple. All Lux models return a state (NamedTuple), so it would be nice to add a case for that.
The inplace Lux functions currently fail because it constructs a matrix
using Reactant, Lux, Random
using Test
# Generate some data for the XOR problem: vectors of length 2, as columns of a matrix:
noisy = rand(Float32, 2, 1000) # 2×1000 Matrix{Float32}
truth = [xor(col[1] > 0.5, col[2] > 0.5) for col in eachcol(noisy)] # 1000-element Vector{Bool}
# Define our model, a multi-layer perceptron with one hidden layer of size 3:
model = Chain(Dense(2 => 3, tanh), # activation function inside layer
BatchNorm(3), Dense(3 => 2), softmax)
ps, st = Lux.setup(Xoshiro(123), model)
using BenchmarkTools
origout, _ = model(noisy, ps, st)
@show origout[3]
@btime model($noisy, $ps, $st)
cmodel = Reactant.make_tracer(IdDict(), model, (), Reactant.ArrayToConcrete, nothing)
cps = Reactant.make_tracer(IdDict(), ps, (), Reactant.ArrayToConcrete, nothing)
cst = Reactant.make_tracer(IdDict(), st, (), Reactant.ArrayToConcrete, nothing)
cnoisy = Reactant.ConcreteRArray(noisy)
f = Reactant.compile((a, b, c, d) -> a(b, c, d), (cmodel, cnoisy, cps, CST))
Minimal version
using Reactant
noisy = rand(Float32, 2, 1000)
cnoisy = Reactant.ConcreteRArray(noisy)
similar(cnoisy) # Matrix
Since this package is evolving fast, having releases show up on GitHub (not just in the General registry) would be very useful, especially to track changes
I reduced the Lux version down to just Enzyme + Reactant
using Reactant, Enzyme
function simple_reduce(w, x)
y = sum(w * x; dims=1)
return sum(y)
end
w = Reactant.ConcreteRArray(randn(Float32, 10, 3))
x = Reactant.ConcreteRArray(randn(Float32, 3, 5))
simple_reduce_xla = Reactant.compile(simple_reduce, (w, x))
simple_reduce_xla(w, x) # Works
function simple_reduce_grad(w, x)
dw = Enzyme.make_zero(w)
dx = Enzyme.make_zero(x)
Enzyme.autodiff(
Enzyme.Reverse, simple_reduce, Active, Duplicated(w, dw), Duplicated(x, dx))
return dw, dx
end
simple_reduce_grad_xla = Reactant.compile(simple_reduce_grad, (w, x))
simple_reduce_grad_xla(w, x)
error: size of operand dimension 0 (5) is not equal to 1 or size of result dimension 0 (10)
Pipeline failed
If you replace the function with
function simple_reduce(w, x)
return sum(w * x)
end
I think it is stemming from the reduce operation being generated incorrectly see LuxDL/Lux.jl#665 (comment) (couldn't reduce that yet)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.