turinglang / libtask.jl Goto Github PK
View Code? Open in Web Editor NEWTape based task copying in Turing
License: MIT License
Tape based task copying in Turing
License: MIT License
We should probably tag a new release to enable Turing to be used on Julia v0.7.
The tag name "v0.2" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai
The current Instruction
type
mutable struct Instruction{F}
fun::F
input::Tuple
output
tape
end
carries both the code (i.e. fun::F
) and data (i.e. input
and output
). If we want to support loops suggested here, we likely will need a mechanism to store multiple copies of data in the same instruction. It is because each instruction might run multiple times assuming we copy the control flow IR onto the tape without linearization. Maybe we can turn input
and output
into a vector?
struct Instruction{F}
fun::F
io::Vector{InstructionIO}
tape
end
struct InstructionIO
input::Tuple
output
end
This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml
to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix
on this issue.
I'll open a PR within a few hours, please be patient!
@JuliaRegistrator register()
It would be useful to use Artifacts and jll packages, instead of creating a build.jl and using Binaryprovider to download the depdendencies.
Using Artifacts has a couple of advantages: 1) it reduces duplication in download and storage of the native depdendencies, and 2) it works via the package server mechanism, which provides a globally replicated CDN, better caching, and better support for proxies.
This would mean however that only Julia 1.3 and above are supported, but I noticed that Turing.jl already has 1.3 as minimum bound, so should be fine to limit this pakage to that lower bound.
I have the exact same issue as #67 and #68. However the fix described in #67 does not resolve the problem for me..
My status
Status `~/.julia/environments/v1.4/Project.toml`
[5b7e9947] AdvancedMH v0.5.0
[149ed9f2] AdvancedVI v0.1.0 [`~/.julia/dev/AdvancedVI`]
[27a7e980] Animations v0.4.0
[c52e3926] Atom v0.12.11
[38eea1fd] AugmentedGaussianProcesses v0.8.0 [`~/.julia/dev/AugmentedGaussianProcesses`]
[6e4b80f9] BenchmarkTools v0.5.0
[76274a88] Bijectors v0.6.7
[b99e7846] BinaryProvider v0.5.9
[70588ee8] CSSUtil v0.1.1
[336ed68f] CSV v0.6.1
[13f3f980] CairoMakie v0.2.6
[5ae59095] Colors v0.12.0
[8f4d0f93] Conda v1.4.1
[a93c6f00] DataFrames v0.20.2
[1313f7d8] DataFramesMeta v0.5.0
[328f52d8] DeepGP v0.1.0 [`~/.julia/dev/DeepGP`]
[4d968f93] DeterminantalPointProcesses v0.1.0
[b4f34e82] Distances v0.8.2
[31c24e10] Distributions v0.23.2
[ced4e74d] DistributionsAD v0.4.10
[ffbed154] DocStringExtensions v0.8.1
[35a29f4d] DocumenterTools v0.1.5
[634d3b9d] DrWatson v1.11.0
[442a2c76] FastGaussQuadrature v0.4.2
[26cc04aa] FiniteDifferences v0.10.0
[587475ba] Flux v0.10.4
[f6369f11] ForwardDiff v0.10.10
[713c75ef] Franklin v0.8.0
[e9467ef8] GLMakie v0.1.4
[7073ff75] IJulia v1.21.2
[e5e0dc1b] Juno v0.8.2
[5ab0869b] KernelDensity v0.5.1
[ec8451be] KernelFunctions v0.4.0 [`~/.julia/dev/KernelFunctions`]
[2c470bb0] Kronecker v0.4.0
[b964fa9f] LaTeXStrings v1.1.0
[23fbe1c1] Latexify v0.13.2
[23992714] MAT v0.8.0
[c7f686f2] MCMCChains v3.0.12
[cc2ba9b6] MLDataUtils v0.5.1
[1914dd2f] MacroTools v0.5.5
[ee78f7c6] Makie v0.10.0
[5a521ce4] MakieLayout v0.9.8
[442fdcdd] Measures v0.3.1
[5fb14364] OhMyREPL v0.5.5
[90014a1f] PDMats v0.9.12 [`~/.julia/dev/PDMats`]
[14b8a8f1] PkgTemplates v0.6.4
[91a5bcdd] Plots v1.3.0
[92933f4c] ProgressMeter v1.3.0
[d330b81b] PyPlot v2.9.0
[ce6b1742] RDatasets v0.6.8
[37e2e3b7] ReverseDiff v1.2.0
[699a6c99] SimpleTraits v0.9.2
[82cb661a] SliceMap v0.2.3
[276daf66] SpecialFunctions v0.10.2
[2913bbd2] StatsBase v0.32.2
[4c63d2b9] StatsFuns v0.9.4
[65254759] StatsMakie v0.2.1
[f3b207a7] StatsPlots v0.14.6
[37b6cedf] Traceur v0.3.1
[592b5752] Trapz v2.0.2
[fce5fe82] Turing v0.12.0
[d9a01c3f] Underscores v2.0.0
[e88e6eb3] Zygote v0.4.20
[700de1a5] ZygoteRules v0.2.0
[edit] Forgot to mention I am on Ubuntu 20.04 64-bit
As discussed in TuringLang/Turing.jl#1237, it would be nice to implement the iteration interface for CTask
since then it would be possible to use alternatives such as resumable functions in the particle filter implementations in Turing in a straightforward way (currently produce
and consume
statements are hardcoded). However, currently CTask
is just a function that returns a Task
for which stack copying has been enabled. Implementing the iteration interface for Task
would be pretty bad type piracy (BTW Base.copy(::Task)
in the current code is also type piracy). To avoid this I suggest adding a CTask
struct that is defined as
struct CTask
task::Task
end
Then Base.copy
and Base.iterate
could be implemented without committing type piracy. The downside would be that one would probably have to forward a bunch of methods defined for Task
(such istaskstarted
etc.) to task
.
Do you think I should give it a try, or are there any other suggestions?
Hi-
I get the following error building Libtask:
(v1.3) pkg> build Libtask
Building Libtask → `~/.julia/packages/Libtask/04fjt/deps/build.log`
┌ Error: Error building `Libtask`:
│ fatal: not a git repository (or any of the parent directories): .git
│ ┌ Error: Download failed: curl: (22) The requested URL returned error: 404 Not Found
│ └ @ Base download.jl:43
│ ERROR: LoadError: failed process: Process(`git -C /home/dfish/.julia/packages/Libtask/04fjt tag`, ProcessExited(128)) [128]
│
│ Stacktrace:
│ [1] pipeline_error at ./process.jl:525 [inlined]
│ [2] (::Base.var"#561#562"{Base.Process})() at ./process.jl:319
│ [3] iterate(::Base.EachLine{Base.PipeEndpoint}, ::Nothing) at ./io.jl:964 (repeats 2 times)
│ [4] _collect(::UnitRange{Int64}, ::Base.EachLine{Base.PipeEndpoint}, ::Base.HasEltype, ::Base.SizeUnknown) at ./array.jl:571
│ [5] #readlines#294 at ./array.jl:560 [inlined]
│ [6] readlines at ./io.jl:498 [inlined]
│ [7] find_prev_tag(::String) at /home/dfish/.julia/packages/Libtask/04fjt/deps/build.jl:3
│ [8] include_build_script(::SubString{String}, ::Bool) at /home/dfish/.julia/packages/Libtask/04fjt/deps/build.jl:44
│ [9] top-level scope at /home/dfish/.julia/packages/Libtask/04fjt/deps/build.jl:64
│ [10] include at ./boot.jl:328 [inlined]
│ [11] include_relative(::Module, ::String) at ./loading.jl:1105
│ [12] include(::Module, ::String) at ./Base.jl:31
│ [13] include(::String) at ./client.jl:424
│ [14] top-level scope at none:5
│ in expression starting at /home/dfish/.julia/packages/Libtask/04fjt/deps/build.jl:64
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/backwards_compatible_isolation.jl:649
Version:
(v1.3) pkg> st Libtask
Status `~/.julia/environments/v1.3/Project.toml`
[6f1fad26] Libtask v0.3.2
With Julia 1.7.0-beta3 on Windows and an empty default environment, Libtask
installs cleanly from current master (7c404c4), but the tests throw an error:
(@v1.7) pkg> st
Status `C:\Users\abc\.julia\environments\v1.7\Project.toml` (empty project)
(@v1.7) pkg> add Libtask#7c404c4
Updating registry at `C:\Users\abc\.julia\registries\General.toml`
Resolving package versions...
Updating `C:\Users\abc\.julia\environments\v1.7\Project.toml`
[6f1fad26] + Libtask v0.5.3 `https://github.com/TuringLang/Libtask.jl.git#7c404c4`
Updating `C:\Users\abc\.julia\environments\v1.7\Manifest.toml`
[692b3bcd] + JLLWrappers v1.3.0
[6f1fad26] + Libtask v0.5.3 `https://github.com/TuringLang/Libtask.jl.git#7c404c4`
[21216c6a] + Preferences v1.2.2
[3ae2931a] + Libtask_jll v0.5.1+0
[0dad84c5] + ArgTools
[56f22d72] + Artifacts
[2a0f44e3] + Base64
[ade2ca70] + Dates
[f43a241f] + Downloads
[b77e0a4c] + InteractiveUtils
[b27032c2] + LibCURL
[76f85450] + LibGit2
[8f399da3] + Libdl
[37e2e46d] + LinearAlgebra
[56ddb016] + Logging
[d6f4376e] + Markdown
[ca575930] + NetworkOptions
[44cfe95a] + Pkg
[de0858da] + Printf
[3fa0cd96] + REPL
[9a3f8284] + Random
[ea8e919c] + SHA
[9e88b42a] + Serialization
[6462fe0b] + Sockets
[2f01184e] + SparseArrays
[10745b16] + Statistics
[fa267f1f] + TOML
[a4e569a6] + Tar
[cf7118a7] + UUIDs
[4ec0a83e] + Unicode
[e66e0078] + CompilerSupportLibraries_jll
[deac9b47] + LibCURL_jll
[29816b5a] + LibSSH2_jll
[c8ffd9c3] + MbedTLS_jll
[14a3606d] + MozillaCACerts_jll
[4536629a] + OpenBLAS_jll
[83775a58] + Zlib_jll
[8e850b90] + libblastrampoline_jll
[8e850ede] + nghttp2_jll
[3f19e933] + p7zip_jll
(@v1.7) pkg> st
Status `C:\Users\abc\.julia\environments\v1.7\Project.toml`
[6f1fad26] Libtask v0.5.3 `https://github.com/TuringLang/Libtask.jl.git#7c404c4`
(@v1.7) pkg> test Libtask
Testing Libtask
Status `C:\Users\abc\AppData\Local\Temp\jl_i0IMgj\Project.toml`
[6e4b80f9] BenchmarkTools v1.1.1
[6f1fad26] Libtask v0.5.3 `https://github.com/TuringLang/Libtask.jl.git#7c404c4`
[3ae2931a] Libtask_jll v0.5.1+0
[8f399da3] Libdl `@stdlib/Libdl`
[37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra`
[10745b16] Statistics `@stdlib/Statistics`
[8dfed614] Test `@stdlib/Test`
Status `C:\Users\abc\AppData\Local\Temp\jl_i0IMgj\Manifest.toml`
[6e4b80f9] BenchmarkTools v1.1.1
[692b3bcd] JLLWrappers v1.3.0
[682c06a0] JSON v0.21.2
[6f1fad26] Libtask v0.5.3 `https://github.com/TuringLang/Libtask.jl.git#7c404c4`
[69de0a69] Parsers v2.0.2
[21216c6a] Preferences v1.2.2
[3ae2931a] Libtask_jll v0.5.1+0
[0dad84c5] ArgTools `@stdlib/ArgTools`
[56f22d72] Artifacts `@stdlib/Artifacts`
[2a0f44e3] Base64 `@stdlib/Base64`
[ade2ca70] Dates `@stdlib/Dates`
[f43a241f] Downloads `@stdlib/Downloads`
[b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils`
[b27032c2] LibCURL `@stdlib/LibCURL`
[76f85450] LibGit2 `@stdlib/LibGit2`
[8f399da3] Libdl `@stdlib/Libdl`
[37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra`
[56ddb016] Logging `@stdlib/Logging`
[d6f4376e] Markdown `@stdlib/Markdown`
[a63ad114] Mmap `@stdlib/Mmap`
[ca575930] NetworkOptions `@stdlib/NetworkOptions`
[44cfe95a] Pkg `@stdlib/Pkg`
[de0858da] Printf `@stdlib/Printf`
[3fa0cd96] REPL `@stdlib/REPL`
[9a3f8284] Random `@stdlib/Random`
[ea8e919c] SHA `@stdlib/SHA`
[9e88b42a] Serialization `@stdlib/Serialization`
[6462fe0b] Sockets `@stdlib/Sockets`
[2f01184e] SparseArrays `@stdlib/SparseArrays`
[10745b16] Statistics `@stdlib/Statistics`
[fa267f1f] TOML `@stdlib/TOML`
[a4e569a6] Tar `@stdlib/Tar`
[8dfed614] Test `@stdlib/Test`
[cf7118a7] UUIDs `@stdlib/UUIDs`
[4ec0a83e] Unicode `@stdlib/Unicode`
[e66e0078] CompilerSupportLibraries_jll `@stdlib/CompilerSupportLibraries_jll`
[deac9b47] LibCURL_jll `@stdlib/LibCURL_jll`
[29816b5a] LibSSH2_jll `@stdlib/LibSSH2_jll`
[c8ffd9c3] MbedTLS_jll `@stdlib/MbedTLS_jll`
[14a3606d] MozillaCACerts_jll `@stdlib/MozillaCACerts_jll`
[4536629a] OpenBLAS_jll `@stdlib/OpenBLAS_jll`
[83775a58] Zlib_jll `@stdlib/Zlib_jll`
[8e850b90] libblastrampoline_jll `@stdlib/libblastrampoline_jll`
[8e850ede] nghttp2_jll `@stdlib/nghttp2_jll`
[3f19e933] p7zip_jll `@stdlib/p7zip_jll`
Testing Running tests...
stack allocated objects: Error During Test at C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:13
Test threw exception
Expression: consume(ctask) == 0
ReadOnlyMemoryError()
Stacktrace:
[1] setproperty!
@ .\Base.jl:43 [inlined]
[2] consume(::CTask)
@ Libtask C:\Users\abc\.julia\packages\Libtask\8YyQv\src\ctask.jl:218
[3] macro expansion
@ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\Test\src\Test.jl:445 [inlined]
[4] macro expansion
@ C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:13 [inlined]
[5] macro expansion
@ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\Test\src\Test.jl:1282 [inlined]
[6] macro expansion
@ C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:4 [inlined]
[7] macro expansion
@ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\Test\src\Test.jl:1282 [inlined]
[8] top-level scope
@ C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:3
stack allocated objects: Error During Test at C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:3
Got exception outside of a @test
KeyError: key :_libtask_bt not found
Stacktrace:
[1] getindex(d::IdDict{Any, Any}, key::Any)
@ Base .\iddict.jl:108
[2] showerror(io::IOContext{IOBuffer}, ex::Libtask.CTaskException)
@ Libtask C:\Users\abc\.julia\packages\Libtask\8YyQv\src\ctask.jl:34
[3] showerror(io::IOContext{IOBuffer}, ex::Libtask.CTaskException, bt::Vector{Union{Ptr{Nothing}, Base.InterpreterIP}}; backtrace::Bool)
@ Base .\errorshow.jl:88
[4] show_exception_stack(io::IOContext{IOBuffer}, stack::Vector{Any})
@ Base .\errorshow.jl:862
[5] sprint(f::Function, args::Vector{Any}; context::Base.TTY, sizehint::Int64)
@ Base .\strings\io.jl:110
[6] Test.Error(test_type::Any, orig_expr::Any, value::Any, bt::Any, source::Any)
@ Test C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\Test\src\Test.jl:170
[7] do_test(result::Test.ExecutionResult, orig_expr::Any)
@ Test C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\Test\src\Test.jl:634
[8] macro expansion
@ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\Test\src\Test.jl:445 [inlined]
[9] macro expansion
@ C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:14 [inlined]
[10] macro expansion
@ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\Test\src\Test.jl:1282 [inlined]
[11] macro expansion
@ C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:4 [inlined]
[12] macro expansion
@ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\Test\src\Test.jl:1282 [inlined]
[13] top-level scope
@ C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:3
[14] include(fname::String)
@ Base.MainInclude .\client.jl:451
[15] top-level scope
@ C:\Users\abc\.julia\packages\Libtask\8YyQv\test\runtests.jl:4
[16] include(fname::String)
@ Base.MainInclude .\client.jl:451
[17] top-level scope
@ none:6
[18] eval
@ .\boot.jl:373 [inlined]
[19] exec_options(opts::Base.JLOptions)
@ Base .\client.jl:268
[20] _start()
@ Base .\client.jl:495
Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x3c49bfac -- getproperty at .\task.jl:170 [inlined]
produce at C:\Users\abc\.julia\packages\Libtask\8YyQv\src\ctask.jl:154
in expression starting at C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:1
getproperty at .\task.jl:170 [inlined]
produce at C:\Users\abc\.julia\packages\Libtask\8YyQv\src\ctask.jl:154
#176#f at C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:27
#8 at C:\Users\abc\.julia\packages\Libtask\8YyQv\src\ctask.jl:85
unknown function (ip: 000000003c49cd13)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1787 [inlined]
start_task at /cygdrive/c/buildbot/worker/package_win64/build/src\task.c:878
Allocations: 2390953 (Pool: 2390183; Big: 770); GC: 3
ERROR: Package Libtask errored during testing
Project file:
[deps]
Libtask = "6f1fad26-d15e-5dc8-ae53-837a1d7b8c9f"
Manifest file:
# This file is machine-generated - editing it directly is not advised
julia_version = "1.7.0-beta3.0"
manifest_format = "2.0"
[[deps.ArgTools]]
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"
[[deps.Artifacts]]
uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"
[[deps.Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"
[[deps.CompilerSupportLibraries_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae"
[[deps.Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"
[[deps.Downloads]]
deps = ["ArgTools", "LibCURL", "NetworkOptions"]
uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"
[[deps.InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"
[[deps.JLLWrappers]]
deps = ["Preferences"]
git-tree-sha1 = "642a199af8b68253517b80bd3bfd17eb4e84df6e"
uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210"
version = "1.3.0"
[[deps.LibCURL]]
deps = ["LibCURL_jll", "MozillaCACerts_jll"]
uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"
[[deps.LibCURL_jll]]
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"
[[deps.LibGit2]]
deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"
[[deps.LibSSH2_jll]]
deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"
[[deps.Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
[[deps.Libtask]]
deps = ["Libdl", "Libtask_jll", "LinearAlgebra", "Statistics"]
git-tree-sha1 = "af2d97cb50decc6aa3621f5c02d3cd9cea2d0f30"
repo-rev = "7c404c4"
repo-url = "https://github.com/TuringLang/Libtask.jl.git"
uuid = "6f1fad26-d15e-5dc8-ae53-837a1d7b8c9f"
version = "0.5.3"
[[deps.Libtask_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "3ee8b8e25a657e8f5d4ca702e7b78150521caf0f"
uuid = "3ae2931a-708c-5973-9c38-ccf7496fb450"
version = "0.5.1+0"
[[deps.LinearAlgebra]]
deps = ["Libdl", "libblastrampoline_jll"]
uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
[[deps.Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"
[[deps.Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"
[[deps.MbedTLS_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"
[[deps.MozillaCACerts_jll]]
uuid = "14a3606d-f60d-562e-9121-12d972cd8159"
[[deps.NetworkOptions]]
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"
[[deps.OpenBLAS_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "Libdl"]
uuid = "4536629a-c528-5b80-bd46-f80d51c5b363"
[[deps.Pkg]]
deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
[[deps.Preferences]]
deps = ["TOML"]
git-tree-sha1 = "00cfd92944ca9c760982747e9a1d0d5d86ab1e5a"
uuid = "21216c6a-2e73-6563-6e65-726566657250"
version = "1.2.2"
[[deps.Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"
[[deps.REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"
[[deps.Random]]
deps = ["Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
[[deps.SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"
[[deps.Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"
[[deps.Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"
[[deps.SparseArrays]]
deps = ["LinearAlgebra", "Random"]
uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"
[[deps.Statistics]]
deps = ["LinearAlgebra", "SparseArrays"]
uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
[[deps.TOML]]
deps = ["Dates"]
uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"
[[deps.Tar]]
deps = ["ArgTools", "SHA"]
uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"
[[deps.UUIDs]]
deps = ["Random", "SHA"]
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"
[[deps.Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"
[[deps.Zlib_jll]]
deps = ["Libdl"]
uuid = "83775a58-1f1d-513f-b197-d71354ab007a"
[[deps.libblastrampoline_jll]]
deps = ["Artifacts", "Libdl", "OpenBLAS_jll"]
uuid = "8e850b90-86db-534c-a0d3-1478176c7d93"
[[deps.nghttp2_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"
[[deps.p7zip_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"
Maybe we can modify the build script of Libtask
, so that a tag is automatically created if the tag does not exist? This would allow uploading artefacts for new Libtask versions even before a new version is accepted by JuliaRegistry.
@yebai I've been trying to update LibtaskBuilder to link against Julia 1.0 without much luck. In Julia 1.0 struct _jl_task_t
definition no longer has a consumers
field. Any plans/ideas?
The tag name "v0.2.2-alpha1" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai
I retriggered the PR to the general registry on the master branch, i.e., the updated PR includes also the fix to deps/build.jl
. However, the automerge tests still fail due to the following error:
┌ Error: Error building `Libtask`:
│ fatal: not a git repository (or any of the parent directories): .git
│ ┌ Error: Download failed: curl: (22) The requested URL returned error: 404 Not Found
│ └ @ Base download.jl:43
│ ERROR: LoadError: failed process: Process(`git -C /tmp/jl_NXjesH/packages/Libtask/kykuJ fetch --tags`, ProcessExited(128)) [128]
│
│ Stacktrace:
│ [1] pipeline_error at ./process.jl:525 [inlined]
│ [2] run(::Cmd; wait::Bool) at ./process.jl:440
│ [3] run at ./process.jl:438 [inlined]
│ [4] find_prev_tag(::String) at /tmp/jl_NXjesH/packages/Libtask/kykuJ/deps/build.jl:3
│ [5] include_build_script(::SubString{String}, ::Bool) at /tmp/jl_NXjesH/packages/Libtask/kykuJ/deps/build.jl:45
│ [6] top-level scope at /tmp/jl_NXjesH/packages/Libtask/kykuJ/deps/build.jl:65
│ [7] include(::String) at ./client.jl:439
│ [8] top-level scope at none:5
│ in expression starting at /tmp/jl_NXjesH/packages/Libtask/kykuJ/deps/build.jl:65
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/Operations.jl:892
It seems it's not possible to run
Line 3 in 4e201c5
I guess it would be more appropriate to fix a specific version but that seems impossible in the current setup where the latest version might not exist during testing?
The tag name "v0.2.2-alpha1" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai
Unlike normal arrays in Julia 1.0, TArrays
are created without the undef
keyword. For consistency it would be nice to allow
TArray{Real}(undef, 4)
(v1.1) pkg> add Turing
Resolving package versions...
Updating `C:\Users\accou\.julia\environments\v1.1\Project.toml`
[fce5fe82] + Turing v0.6.17
Updating `C:\Users\accou\.julia\environments\v1.1\Manifest.toml`
[no changes]
julia> using Turing
[ Info: Precompiling Turing [fce5fe82-541a-59a6-adf8-730c64b5f9a0]
ERROR: LoadError: could not open file C:\Users\accou\.julia\packages\Libtask\dWROm\deps\deps.jl
Stacktrace:
[1] include at .\boot.jl:326 [inlined]
[2] include_relative(::Module, ::String) at .\loading.jl:1038
[3] include at .\sysimg.jl:29 [inlined]
[4] include(::String) at C:\Users\accou\.julia\packages\Libtask\dWROm\src\Libtask.jl:1
[5] top-level scope at none:0
[6] include at .\boot.jl:326 [inlined]
[7] include_relative(::Module, ::String) at .\loading.jl:1038
[8] include(::Module, ::String) at .\sysimg.jl:29
[9] top-level scope at none:2
[10] eval at .\boot.jl:328 [inlined]
[11] eval(::Expr) at .\client.jl:404
[12] top-level scope at .\none:3
in expression starting at C:\Users\accou\.julia\packages\Libtask\dWROm\src\Libtask.jl:5
ERROR: LoadError: Failed to precompile Libtask [6f1fad26-d15e-5dc8-ae53-837a1d7b8c9f] to C:\Users\accou\.julia\compiled\v1.1\Libtask\KIaXx.ji.
Stacktrace:
[1] error(::String) at .\error.jl:33
[2] compilecache(::Base.PkgId, ::String) at .\loading.jl:1197
[3] _require(::Base.PkgId) at .\loading.jl:960
[4] require(::Base.PkgId) at .\loading.jl:858
[5] require(::Module, ::Symbol) at .\loading.jl:853
[6] include at .\boot.jl:326 [inlined]
[7] include_relative(::Module, ::String) at .\loading.jl:1038
[8] include(::Module, ::String) at .\sysimg.jl:29
[9] top-level scope at none:2
[10] eval at .\boot.jl:328 [inlined]
[11] eval(::Expr) at .\client.jl:404
[12] top-level scope at .\none:3
in expression starting at C:\Users\accou\.julia\packages\Turing\RZOZ8\src\Turing.jl:14
ERROR: Failed to precompile Turing [fce5fe82-541a-59a6-adf8-730c64b5f9a0] to C:\Users\accou\.julia\compiled\v1.1\Turing\gm4QC.ji.
Stacktrace:
[1] compilecache(::Base.PkgId, ::String) at .\loading.jl:1197
[2] _require(::Base.PkgId) at .\loading.jl:960
[3] require(::Base.PkgId) at .\loading.jl:858
[4] require(::Module, ::Symbol) at .\loading.jl:853
(v1.1) pkg> add Libtask
Resolving package versions...
Updating `C:\Users\accou\.julia\environments\v1.1\Project.toml`
[6f1fad26] + Libtask v0.3.0
Updating `C:\Users\accou\.julia\environments\v1.1\Manifest.toml`
[no changes]
(v1.1) pkg> build Turing
Building Arpack ──────────→ `C:\Users\accou\.julia\packages\Arpack\cu5By\deps\build.log`
Building SpecialFunctions → `C:\Users\accou\.julia\packages\SpecialFunctions\fvheQ\deps\build.log`
Building Rmath ───────────→ `C:\Users\accou\.julia\packages\Rmath\Py9gH\deps\build.log`
Building Conda ───────────→ `C:\Users\accou\.julia\packages\Conda\kLXeC\deps\build.log`
Building FFTW ────────────→ `C:\Users\accou\.julia\packages\FFTW\p7sLQ\deps\build.log`
Building GR ──────────────→ `C:\Users\accou\.julia\packages\GR\TMylY\deps\build.log`
Building Plots ───────────→ `C:\Users\accou\.julia\packages\Plots\oiirH\deps\build.log`
Building Libtask ─────────→ `C:\Users\accou\.julia\packages\Libtask\dWROm\deps\build.log`
The problem is that it's looking in the wrong location for the deps folder?
Hi,
I am getting the following load error with Julia 1.2 and Windows:
(v1.2) pkg> build Libtask
Building Libtask → `C:\Users\user\.julia\packages\Libtask\RjRkK\deps\build.log`
┌ Error: Error building `Libtask`:
│ fatal: not a git repository (or any of the parent directories): .git
│ ┌ Error: Download failed: curl: (35) schannel: next InitializeSecurityContext failed: Unknown error (0x80092012) - The revocation function was unable to check revocation for the certificate.
│ └ @ Base download.jl:43
│ ERROR: LoadError: failed process: Process(`git -C 'C:\Users\user\.julia\packages\Libtask\RjRkK' tag`, ProcessExited(128)) [128]
│
│ Stacktrace:
│ [1] pipeline_error at .\process.jl:813 [inlined]
│ [2] (::getfield(Base, Symbol("##533#534")){Base.Process})() at .\process.jl:621
│ [3] iterate(::Base.EachLine{Base.PipeEndpoint}, ::Nothing) at .\io.jl:893 (repeats 2 times)
│ [4] _collect(::UnitRange{Int64}, ::Base.EachLine{Base.PipeEndpoint}, ::Base.HasEltype, ::Base.SizeUnknown) at .\array.jl:555
│ [5] collect at .\array.jl:544 [inlined]
│ [6] #readlines#290 at .\io.jl:427 [inlined]
│ [7] readlines at .\io.jl:427 [inlined]
│ [8] find_prev_tag(::String) at C:\Users\user\.julia\packages\Libtask\RjRkK\deps\build.jl:3
│ [9] include_build_script(::SubString{String}, ::Bool) at C:\Users\user\.julia\packages\Libtask\RjRkK\deps\build.jl:42
│ [10] top-level scope at C:\Users\user\.julia\packages\Libtask\RjRkK\deps\build.jl:62
│ [11] include at .\boot.jl:328 [inlined]
│ [12] include_relative(::Module, ::String) at .\loading.jl:1094
│ [13] include(::Module, ::String) at .\Base.jl:31
│ [14] include(::String) at .\client.jl:431
│ [15] top-level scope at none:5
│ in expression starting at C:\Users\user\.julia\packages\Libtask\RjRkK\deps\build.jl:62
└ @ Pkg.Operations C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.2\Pkg\src\backwards_compatible_isolation.jl:647
julia> versioninfo()
Julia Version 1.2.0
Commit c6da87ff4b (2019-08-20 00:03 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
The tag name "v0.2.2-alpha2" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai
Idk how difficult it is, but I see there is already support for 32bit arm. Is it possible to add the 64bit libraries as well?
julia> versioninfo()
Julia Version 1.3.0-DEV.0
Platform Info:
OS: Linux (aarch64-unknown-linux-gnu)
CPU: unknown
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-6.0.1 (ORCJIT, kryo)
@JuliaRegistrator register
Installation of Libtask fail on line 5 of LibTask.jl
include("../deps/deps.jl"); check_deps();
the file is really not available. Is there any workaround?
Thanks.
Maintaining a library like Libtask_jll
which links to libjulia isn't super simple as you need to have libjulia_jll available in the first place and then build against all different variations. Why don't you ccall
directly into libjulia from Julia, instead of delegating the same work to an external library? This would have the advantage of having Libtask.jl
more readily available on all platforms and versions of Julia without depending on external libraries.
When I tried to load Turing, it fails since Libtask is missing deps.jl. Here's the stack trace:
julia> using Libtask
[ Info: Precompiling Libtask [6f1fad26-d15e-5dc8-ae53-837a1d7b8c9f]
ERROR: LoadError: could not open file /home/rishi/.julia/packages/Libtask/L5wq8/deps/deps.jl
Stacktrace:
[1] include at ./boot.jl:317 [inlined]
[2] include_relative(::Module, ::String) at ./loading.jl:1044
[3] include at ./sysimg.jl:29 [inlined]
[4] include(::String) at /home/rishi/.julia/packages/Libtask/L5wq8/src/Libtask.jl:1
[5] top-level scope at none:0
[6] top-level scope at none:2
in expression starting at /home/rishi/.julia/packages/Libtask/L5wq8/src/Libtask.jl:5
ERROR: Failed to precompile Libtask [6f1fad26-d15e-5dc8-ae53-837a1d7b8c9f] to /home/rishi/.julia/compiled/v1.0/Libtask/KIaXx.ji.
Stacktrace:
[1] compilecache(::Base.PkgId, ::String) at ./loading.jl:1203
[2] _require(::Base.PkgId) at ./loading.jl:960
[3] require(::Base.PkgId) at ./loading.jl:858
[4] require(::Module, ::Symbol) at ./loading.jl:853
Libtask is version 0.2.0, Julia is 1.0.3, OS is Arch Linux.
It might be possible to create a Julia version of the task struct, based on the memory layout information in #89, then we can further simplify the getfield/setfield
functions for manipulating the task fields. It might require creating some new bit types (or primitive types) to represent arbitrarily sized memory blocks without clear Julia equivalent type.
The TArray
data structure implements a copy-on-write array. This is useful for task copying since standard arrays are allocated on the stack, which is only shallow copied during task copying.
However, the current code for TArray
has several limits:
DenseArray
).Perhaps
TRef
type that can store value of arbitrary data types, e.g. x = TRef([0 1 2])
using Libtask
results in ERROR: LoadError: LoadError: UndefVarError: libtask_v1_3 not defined
The tag name "v0.2.3-alpha" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai
The tag name "v0.2.2-alpha1" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai
See the following minimal example:
using Libtask
function f()
t = 0;
while true
t[2] = 0
produce(t)
end
end
t = Task(f)
consume(t)
I updated to newest version of Turing.jl (v0.12.0) and now get the following error on trying to build it:
[ Info: Precompiling Turing [fce5fe82-541a-59a6-adf8-730c64b5f9a0]
ERROR: LoadError: Libtask is not properly installed. Please run `import Pkg; Pkg.build("Libtask")`
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] top-level scope at /Users/harrisonwilde/.julia/packages/Libtask/Q1ugc/src/Libtask.jl:8
[3] include(::Module, ::String) at ./Base.jl:377
[4] top-level scope at none:2
[5] eval at ./boot.jl:331 [inlined]
[6] eval(::Expr) at ./client.jl:449
[7] top-level scope at ./none:3
in expression starting at /Users/harrisonwilde/.julia/packages/Libtask/Q1ugc/src/Libtask.jl:7
ERROR: LoadError: Failed to precompile Libtask [6f1fad26-d15e-5dc8-ae53-837a1d7b8c9f] to /Users/harrisonwilde/.julia/compiled/v1.4/Libtask/KIaXx_ft1Em.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
[3] _require(::Base.PkgId) at ./loading.jl:1029
[4] require(::Base.PkgId) at ./loading.jl:927
[5] require(::Module, ::Symbol) at ./loading.jl:922
[6] include(::Module, ::String) at ./Base.jl:377
[7] top-level scope at none:2
[8] eval at ./boot.jl:331 [inlined]
[9] eval(::Expr) at ./client.jl:449
[10] top-level scope at ./none:3
in expression starting at /Users/harrisonwilde/.julia/packages/Turing/d4vqQ/src/Turing.jl:14
Upon trying to build Libtask as suggested, I get the following:
(Synthetic Data) pkg> build Libtask
Building Libtask → `~/.julia/packages/Libtask/Q1ugc/deps/build.log`
┌ Error: Error building `Libtask`:
│ ERROR: LoadError: LibraryProduct(nothing, ["libtask_v1_3"], :libtask_v1_3, "Prefix(/Users/harrisonwilde/.julia/packages/Libtask/Q1ugc/deps/usr)") is not satisfied, cannot generate deps.jl!
│ Stacktrace:
│ [1] error(::String) at ./error.jl:33
│ [2] write_deps_file(::String, ::Array{LibraryProduct,1}; verbose::Bool, isolate::Bool) at /Users/harrisonwilde/.julia/packages/BinaryProvider/GeAtj/src/Products.jl:419
│ [3] top-level scope at /Users/harrisonwilde/.julia/packages/Libtask/Q1ugc/deps/build.jl:58
│ [4] include(::String) at ./client.jl:439
│ [5] top-level scope at none:5
│ in expression starting at /Users/harrisonwilde/.julia/packages/Libtask/Q1ugc/deps/build.jl:58
└ @ Pkg.Operations /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.4/Pkg/src/Operations.jl:892
My project looks like:
(Synthetic Data) pkg> st
Status `~/Library/Mobile Documents/com~apple~CloudDocs/PhD/Synthetic Data/Project.toml`
[0bf59076] AdvancedHMC v0.2.24
[c7e460c6] ArgParse v1.1.0
[6e4b80f9] BenchmarkTools v0.5.0
[a74b3585] Blosc v0.7.0
[336ed68f] CSV v0.6.1
[49dc2e85] Calculus v0.5.1
[34f1f09b] ClusterManagers v0.3.2
[944b1d66] CodecZlib v0.7.0
[a93c6f00] DataFrames v0.20.2
[31c24e10] Distributions v0.23.2
[f6369f11] ForwardDiff v0.10.10
[f67ccb44] HDF5 v0.13.2 #master (https://github.com/JuliaIO/HDF5.jl.git)
[4138dd39] JLD v0.10.0
[6f1fad26] Libtask v0.4.0
[c7f686f2] MCMCChains v3.0.12
[f0e99cf1] MLBase v0.8.0
[add582a8] MLJ v0.11.2
[a7f614a8] MLJBase v0.13.5
[6ee0df7b] MLJLinearModels v0.3.2
[429524aa] Optim v0.20.6
[8e8a01fc] ParallelProcessingTools v0.4.1
[91a5bcdd] Plots v0.29.9
[c46f51b8] ProfileView v0.6.5
[92933f4c] ProgressMeter v1.2.0
[ce6b1742] RDatasets v0.6.7
[3646fa90] ScikitLearn v0.5.1
[276daf66] SpecialFunctions v0.10.0
[4c63d2b9] StatsFuns v0.9.4
[f3b207a7] StatsPlots v0.14.5
[bd369af6] Tables v1.0.4
[37b6cedf] Traceur v0.3.0
[fce5fe82] Turing v0.12.0
[e88e6eb3] Zygote v0.4.20
[37e2e46d] LinearAlgebra
We currently treat all instructions on a tape as primitive. There are use-cases (see here) where we need to selectively trace into instructions. One neat way of achieving that is to have a special instruction type that allows us to plug a tape A into another tape B, where tape A is considered as a standard instruction on tape B. This way, we can create a dedicated tape for an instruction that we want to trace into, and use it on another (parent) tape.
The following MWE breaks in the current version of Turing.
using LinearAlgebra
using Turing
# Linear Model
Δt = 0.1
A = [1.0 Δt; 0.0 1.0]
Q = [0.25 * Δt^4 0.5 * Δt^3; 0.5 * Δt^3 Δt^2] + 1e-6 * I
H = [1.0 0.0; 0.0 1.0;]
R = 0.1 * I
@model function hmm(states, ::Type{T} = Float64) where T
# Initialize arrays for the time-series
observations = Matrix{T}(undef, 2, length(states))
# Prior for first step
states[1] ~ MvNormal([0.0, 0.5], sqrt(0.001))
observations[:, 1] ~ MvNormal(states[1], R/Δt)
for t in 2:length(states)
states[t] ~ MvNormal(A * states[t-1], Δt*Q)
observations[:, t] ~ MvNormal(states[t], R/Δt)
end
return states
end
# Sample a trial from the prior to be used as the ground truth
hmm_prior = hmm(Union{Missing,Vector{Float64}}[missing for _ in 1:100])
states = hmm_prior()
# Condition on the sampled trial
hmm_conditioned = hmm(states)
# Perform inference (not working)
chain = sample(hmm_conditioned, SMC(), 1000)
With the stack trace:
ERROR: CTaskException:
Malformed dims
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] TArray{Union{Missing, Array{Float64,1}},1}(::Array{Union{Missing, Array{Float64,1}},1}) at /Users/martin/.julia/packages/Libtask/Zo6uM/src/tarray.jl:36
Any idea why this happens?
When replacing states = hmm_prior()
with states = Array{Array{Float64}}(hmm_prior())
everything seems fine.
The tag name "v0.2.2-alpha2" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai
The tag name "0.1.3" is not of the appropriate SemVer form (vX.Y.Z).
cc: @cpfiffer
after turing update to v0.12.0 on Windows 10 I get:
Building Libtask → C:\Users\**\.julia\packages\Libtask\Q1ugc\deps\build.log
┌ Error: Error building Libtask
:
│ ERROR: LoadError: LibraryProduct(nothing, ["libtask_v1_3"], :libtask_v1_3, "Prefix(C:\Users\**\.julia\packages\Libtask\Q1ugc\deps\usr)") is not satisfied, cannot generate deps.jl!
│ Stacktrace:
│ [1] error(::String) at .\error.jl:33
│ [2] write_deps_file(::String, ::Array{LibraryProduct,1}; verbose::Bool, isolate::Bool) at C:\Users*.julia\packages\BinaryProvider\GeAtj\src\Products.jl:419
│ [3] top-level scope at C:\Users*.julia\packages\Libtask\Q1ugc\deps\build.jl:58
│ [4] include(::String) at .\client.jl:439
│ [5] top-level scope at none:5
│ in expression starting at C:\Users**.julia\packages\Libtask\Q1ugc\deps\build.jl:58
└ @ Pkg.Operations D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Operations.jl:892
Hey I was curious if there was some specific reason as to why the Julia requirement for this package was fixed to 1.0 and not 0.7?
The tag name "v0.2.2-alpha2" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai
using Libtask
results in: ERROR: LoadError: LoadError: UndefVarError: libtask_v1_3 not defined
The tag name "v0.2.3-alpha" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai
@JuliaRegistrator register()
The TArray
-family data structure in Libtask
provides a copy-on-write mechanism for heap-allocated data structures. This is helpful when we want to decouple heap-allocated data structures after tasks are forked, i.e. each task will get an independent copy of heap-allocated variables, and mutating these variables from one task won't affect other tasks. The current implementation requires the user to use TArray
-like constructors instead of Array
s. This is inconvenient and sometimes confusing. It should be possible to completely replace the functionality of TArray
by using a compiler-based implementation.
The basic idea is to detect yield
points in static single assignment (SSA) form of functions, then use an SSA manipulation tool (e.g. IRTools) to explicit insert statements that detect task-copying and deep copy variables that are mutated after the task-copying point.
The following code would halt
using Libtask
function f_ct()
t = 0;
while t<3
println(t)
t = 1 + t
end
return t
end
t = CTask(f_ct)
julia> consume(t) # Never returns.
0
1
2
This is likely due to the consumer
task never gets waken up if the producer
task exit before any produce
statement. Maybe a better behaviour for this kind of tasks is to wake up the producer
task with the producer
task's return
value?
Related TuringLang/AdvancedPS.jl#37
CI tests of DynamicPPL (see https://travis-ci.com/github/TuringLang/DynamicPPL.jl/jobs/306791553 and TuringLang/DynamicPPL.jl#52) fail on Julia 1.4 and Julia master due to the error
ERROR: LoadError: The following package names could not be resolved:
* Libtask (not found in project or manifest)
Stacktrace:
[1] pkgerror(::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/Types.jl:53
[2] ensure_resolved(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; registry::Bool) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/Types.jl:794
[3] ensure_resolved at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/Types.jl:759 [inlined]
[4] build(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; verbose::Bool, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:697
[5] build at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:682 [inlined]
[6] #build#108 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:680 [inlined]
[7] build at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:680 [inlined]
[8] #build#105 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:677 [inlined]
[9] build(::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:677
[10] top-level scope at /home/travis/.julia/packages/Libtask/K7OAU/src/Libtask.jl:7
[11] include(::Module, ::String) at ./Base.jl:377
[12] top-level scope at none:2
[13] eval at ./boot.jl:331 [inlined]
[14] eval(::Expr) at ./client.jl:449
[15] top-level scope at ./none:3
in expression starting at /home/travis/.julia/packages/Libtask/K7OAU/src/Libtask.jl:5
I think the problematic lines are
Lines 5 to 8 in a1ce1ea
Can we remove them completely? I would assume that Pkg should run Pkg.build()
automatically?
I observed this behaviour a few times already. At first I thought I couldn't reproduce the issue but now I figured out the corner case and here is a MWE.
using Turing
using LinearAlgebra
using StatsFuns: logistic
x = vcat(randn(100,3), randn(100,3).+2)
y = vcat(zeros(100), ones(100))
@model lr_nuts(x, y, σ) = begin
N,D = size(x)
α ~ Normal(0, σ)
β ~ MvNormal(zeros(D), ones(D)*σ)
p = Vector{Float64}(undef, D)
s = Vector{Bool}(undef, D)
for d = 1:D
p[d] ~ Beta(1/2, 1/2)
s[d] ~ Bernoulli(p[d])
end
for n = 1:N
y[n] ~ Bernoulli(logistic(dot(x[n,s], β[s]) + α))
end
end
chn = sample(lr_nuts(x,y,sqrt(10)), SMC(1000));
This code produces a chain with variance in the discrete RV's s
if I'm using Julia 1.0.4. If I'm using Julia 1.1 instead, the variance for all discrete RV's (s
) is zero.
The version of Libtask that's currently shipped with Turing.jl (0.5.1, I guess) raises this error when I run sample(model, sampler, 100)
:
CTaskException:
Failed to show error:
UndefVarError: ct not defined
Stacktrace:
[1] showerror(io::IOBuffer, ex::Libtask.CTaskException)
@ Libtask ~/.julia/packages/Libtask/00Il9/src/ctask.jl:30
[2] try_showerror(::IOBuffer, ::Libtask.CTaskException)
@ Main.PlutoRunner ~/.julia/packages/Pluto/yDJcG/src/runner/PlutoRunner.jl:454
[3] sprint(f::Function, args::Libtask.CTaskException; context::Nothing, sizehint::Int64)
@ Base ./strings/io.jl:105
[4] sprint(f::Function, args::Libtask.CTaskException)
@ Base ./strings/io.jl:101
[5] format_output(val::CapturedException; context::Pair{Symbol, Dict{Tuple{UInt64, Int64}, Int64}})
@ Main.PlutoRunner ~/.julia/packages/Pluto/yDJcG/src/runner/PlutoRunner.jl:549
[6] formatted_result_of(cell_id::Base.UUID, ends_with_semicolon::Bool, showmore::Nothing)
@ Main.PlutoRunner ~/.julia/packages/Pluto/yDJcG/src/runner/PlutoRunner.jl:436
[7] top-level scope
@ none:1
[8] eval(m::Module, e::Any)
@ Core ./boot.jl:360
[9] (::Distributed.var"#106#108"{Distributed.CallMsg{:call_fetch}})()
@ Distributed /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:278
[10] run_work_thunk(thunk::Distributed.var"#106#108"{Distributed.CallMsg{:call_fetch}}, print_error::Bool)
@ Distributed /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:63
[11] macro expansion
@ /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:278 [inlined]
[12] (::Distributed.var"#105#107"{Distributed.CallMsg{:call_fetch}, Distributed.MsgHeader, Sockets.TCPSocket})()
@ Distributed ./task.jl:411
This happens because, indeed, ct
is not defined in version 0.5.1:
Lines 25 to 33 in ca8099f
It was fixed in the next commit:
Lines 30 to 35 in c66b8c9
...yet this version was never published, it seems? Turing v0.15.20 uses Libtask 0.5.1:
[[Libtask]]
deps = ["Libtask_jll", "LinearAlgebra", "Statistics"]
git-tree-sha1 = "6088b80fb5017440579ea8113a516ad2807afe19"
uuid = "6f1fad26-d15e-5dc8-ae53-837a1d7b8c9f"
version = "0.5.1"
[[Libtask_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "901fc8752bbc527a6006a951716d661baa9d54e9"
uuid = "3ae2931a-708c-5973-9c38-ccf7496fb450"
version = "0.4.3+0"
I'm a total newbie and trying to use Turing for the very first time. In building the package for my M1 Mac the manager fails and complains about issues with Libtask but I can't say I fully understand what's happening here. Here is the full traceback.
(@v1.7) pkg> add Turing
Resolving package versions...
No Changes to ~/.julia/environments/v1.7/Project.toml
No Changes to ~/.julia/environments/v1.7/Manifest.toml
Precompiling project...
✗ BinaryProvider
✗ Libtask
✗ ThreadingUtilities
✗ SLEEFPirates
✗ LoopVectorization
✗ RecursiveFactorization
✗ NonlinearSolve
✗ Bijectors
✗ DiffEqBase
✗ OrdinaryDiffEq
✗ DynamicPPL
✗ AdvancedVI
✗ GpABC
✗ Turing
80 dependencies successfully precompiled in 51 seconds (174 already precompiled)
14 dependencies errored. To see a full report either run import Pkg; Pkg.precompile()
or load the packages
and this is the traceback when I try to build the package
(@v1.7) pkg> build Turing
Building Libtask → ~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/83e082fccb4e37d93df6440cdbd41dcbe5e46cb6/build.log
ERROR: Error building Libtask
:
┌ Warning: Platform arm64-apple-darwin21.1.0
is not an officially supported platform
└ @ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:450
ERROR: LoadError: KeyError: key "unknown" not found
Stacktrace:
[1] getindex
@ ./dict.jl:481 [inlined]
[2] parse_dl_name_version
@ ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:511 [inlined]
[3] detect_libgfortran_abi(libgfortran_name::String, platform::BinaryProvider.UnknownPlatform) (repeats 2 times)
@ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:555
[4] detect_libgfortran_abi
@ ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:584 [inlined]
[5] detect_compiler_abi()
@ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:667
[6] top-level scope
@ ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:685
[7] include(mod::Module, _path::String)
@ Base ./Base.jl:418
[8] include(x::String)
@ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/BinaryProvider.jl:1
[9] top-level scope
@ ~/.julia/packages/BinaryProvider/U2dKK/src/BinaryProvider.jl:12
[10] include
@ ./Base.jl:418 [inlined]
[11] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
@ Base ./loading.jl:1318
[12] top-level scope
@ none:1
[13] eval
@ ./boot.jl:373 [inlined]
[14] eval(x::Expr)
@ Base.MainInclude ./client.jl:453
[15] top-level scope
@ none:1
in expression starting at /Users/bpajusco/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:685
in expression starting at /Users/bpajusco/.julia/packages/BinaryProvider/U2dKK/src/BinaryProvider.jl:1
ERROR: LoadError: Failed to precompile BinaryProvider [b99e7846-7c00-51b0-8f62-c81ae34c0232] to /Users/bpajusco/.julia/compiled/v1.7/BinaryProvider/jl_vEZwnJ.
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:33
[2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
@ Base ./loading.jl:1466
[3] compilecache(pkg::Base.PkgId, path::String)
@ Base ./loading.jl:1410
[4] _require(pkg::Base.PkgId)
@ Base ./loading.jl:1120
[5] require(uuidkey::Base.PkgId)
@ Base ./loading.jl:1013
[6] require(into::Module, mod::Symbol)
@ Base ./loading.jl:997
[7] include(fname::String)
@ Base.MainInclude ./client.jl:451
[8] top-level scope
@ none:5
in expression starting at /Users/bpajusco/.julia/packages/Libtask/OtgFl/deps/build.jl:9
Just tried whether Turing would work in Julia 1.7.0 on a Mac Air M1 and ran into the problem that Libtask was not properly installed. I then tried building libtask as described in the error message and got the error that currently apple arm architectures are not supported. Are there any plans to change this soon? Especially now that Julia 1.7.0 officially supports that architecture.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.