turinglang / libtask.jl Goto Github PK

View Code? Open in Web Editor NEW

17.0 17.0 9.0 253 KB

Tape based task copying in Turing

License: MIT License

Julia 100.00%

continuations coroutines threads

libtask.jl's People

Contributors

Stargazers

Watchers

Forkers

mohamed82008 cpfiffer devmotion global-localhost global19 global19-atlassian-net willtebbutt topolarity markus7800

libtask.jl's Issues

Error tagging new release

The tag name "v0.2.3-alpha" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai

Error building Libtask

after turing update to v0.12.0 on Windows 10 I get:

Building Libtask → C:\Users\**\.julia\packages\Libtask\Q1ugc\deps\build.log
┌ Error: Error building Libtask:
│ ERROR: LoadError: LibraryProduct(nothing, ["libtask_v1_3"], :libtask_v1_3, "Prefix(C:\Users\**\.julia\packages\Libtask\Q1ugc\deps\usr)") is not satisfied, cannot generate deps.jl!
│ Stacktrace:
│ [1] error(::String) at .\error.jl:33
│ [2] write_deps_file(::String, ::Array{LibraryProduct,1}; verbose::Bool, isolate::Bool) at C:\Users*.julia\packages\BinaryProvider\GeAtj\src\Products.jl:419
│ [3] top-level scope at C:\Users*.julia\packages\Libtask\Q1ugc\deps\build.jl:58
│ [4] include(::String) at .\client.jl:439
│ [5] top-level scope at none:5
│ in expression starting at C:\Users**.julia\packages\Libtask\Q1ugc\deps\build.jl:58
└ @ Pkg.Operations D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Operations.jl:892

Error building Libtask

I have the exact same issue as #67 and #68. However the fix described in #67 does not resolve the problem for me..
My status

Status `~/.julia/environments/v1.4/Project.toml`
  [5b7e9947] AdvancedMH v0.5.0
  [149ed9f2] AdvancedVI v0.1.0 [`~/.julia/dev/AdvancedVI`]
  [27a7e980] Animations v0.4.0
  [c52e3926] Atom v0.12.11
  [38eea1fd] AugmentedGaussianProcesses v0.8.0 [`~/.julia/dev/AugmentedGaussianProcesses`]
  [6e4b80f9] BenchmarkTools v0.5.0
  [76274a88] Bijectors v0.6.7
  [b99e7846] BinaryProvider v0.5.9
  [70588ee8] CSSUtil v0.1.1
  [336ed68f] CSV v0.6.1
  [13f3f980] CairoMakie v0.2.6
  [5ae59095] Colors v0.12.0
  [8f4d0f93] Conda v1.4.1
  [a93c6f00] DataFrames v0.20.2
  [1313f7d8] DataFramesMeta v0.5.0
  [328f52d8] DeepGP v0.1.0 [`~/.julia/dev/DeepGP`]
  [4d968f93] DeterminantalPointProcesses v0.1.0
  [b4f34e82] Distances v0.8.2
  [31c24e10] Distributions v0.23.2
  [ced4e74d] DistributionsAD v0.4.10
  [ffbed154] DocStringExtensions v0.8.1
  [35a29f4d] DocumenterTools v0.1.5
  [634d3b9d] DrWatson v1.11.0
  [442a2c76] FastGaussQuadrature v0.4.2
  [26cc04aa] FiniteDifferences v0.10.0
  [587475ba] Flux v0.10.4
  [f6369f11] ForwardDiff v0.10.10
  [713c75ef] Franklin v0.8.0
  [e9467ef8] GLMakie v0.1.4
  [7073ff75] IJulia v1.21.2
  [e5e0dc1b] Juno v0.8.2
  [5ab0869b] KernelDensity v0.5.1
  [ec8451be] KernelFunctions v0.4.0 [`~/.julia/dev/KernelFunctions`]
  [2c470bb0] Kronecker v0.4.0
  [b964fa9f] LaTeXStrings v1.1.0
  [23fbe1c1] Latexify v0.13.2
  [23992714] MAT v0.8.0
  [c7f686f2] MCMCChains v3.0.12
  [cc2ba9b6] MLDataUtils v0.5.1
  [1914dd2f] MacroTools v0.5.5
  [ee78f7c6] Makie v0.10.0
  [5a521ce4] MakieLayout v0.9.8
  [442fdcdd] Measures v0.3.1
  [5fb14364] OhMyREPL v0.5.5
  [90014a1f] PDMats v0.9.12 [`~/.julia/dev/PDMats`]
  [14b8a8f1] PkgTemplates v0.6.4
  [91a5bcdd] Plots v1.3.0
  [92933f4c] ProgressMeter v1.3.0
  [d330b81b] PyPlot v2.9.0
  [ce6b1742] RDatasets v0.6.8
  [37e2e3b7] ReverseDiff v1.2.0
  [699a6c99] SimpleTraits v0.9.2
  [82cb661a] SliceMap v0.2.3
  [276daf66] SpecialFunctions v0.10.2
  [2913bbd2] StatsBase v0.32.2
  [4c63d2b9] StatsFuns v0.9.4
  [65254759] StatsMakie v0.2.1
  [f3b207a7] StatsPlots v0.14.6
  [37b6cedf] Traceur v0.3.1
  [592b5752] Trapz v2.0.2
  [fce5fe82] Turing v0.12.0
  [d9a01c3f] Underscores v2.0.0
  [e88e6eb3] Zygote v0.4.20
  [700de1a5] ZygoteRules v0.2.0

[edit] Forgot to mention I am on Ubuntu 20.04 64-bit

Register v0.2.8

@JuliaRegistrator register()

Error tagging new release

The tag name "v0.2" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai

Error tagging new release

The tag name "v0.2.2-alpha1" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai

`undef` keyword for TArrays

Unlike normal arrays in Julia 1.0, TArrays are created without the undef keyword. For consistency it would be nice to allow

TArray{Real}(undef, 4)

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

After updating to a new version of Turing I am having issues getting Libtask to build correctly

I updated to newest version of Turing.jl (v0.12.0) and now get the following error on trying to build it:

[ Info: Precompiling Turing [fce5fe82-541a-59a6-adf8-730c64b5f9a0]
ERROR: LoadError: Libtask is not properly installed. Please run `import Pkg; Pkg.build("Libtask")`
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] top-level scope at /Users/harrisonwilde/.julia/packages/Libtask/Q1ugc/src/Libtask.jl:8
 [3] include(::Module, ::String) at ./Base.jl:377
 [4] top-level scope at none:2
 [5] eval at ./boot.jl:331 [inlined]
 [6] eval(::Expr) at ./client.jl:449
 [7] top-level scope at ./none:3
in expression starting at /Users/harrisonwilde/.julia/packages/Libtask/Q1ugc/src/Libtask.jl:7
ERROR: LoadError: Failed to precompile Libtask [6f1fad26-d15e-5dc8-ae53-837a1d7b8c9f] to /Users/harrisonwilde/.julia/compiled/v1.4/Libtask/KIaXx_ft1Em.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
 [3] _require(::Base.PkgId) at ./loading.jl:1029
 [4] require(::Base.PkgId) at ./loading.jl:927
 [5] require(::Module, ::Symbol) at ./loading.jl:922
 [6] include(::Module, ::String) at ./Base.jl:377
 [7] top-level scope at none:2
 [8] eval at ./boot.jl:331 [inlined]
 [9] eval(::Expr) at ./client.jl:449
 [10] top-level scope at ./none:3
in expression starting at /Users/harrisonwilde/.julia/packages/Turing/d4vqQ/src/Turing.jl:14

Upon trying to build Libtask as suggested, I get the following:

(Synthetic Data) pkg> build Libtask
   Building Libtask → `~/.julia/packages/Libtask/Q1ugc/deps/build.log`
┌ Error: Error building `Libtask`:
│ ERROR: LoadError: LibraryProduct(nothing, ["libtask_v1_3"], :libtask_v1_3, "Prefix(/Users/harrisonwilde/.julia/packages/Libtask/Q1ugc/deps/usr)") is not satisfied, cannot generate deps.jl!
│ Stacktrace:
│  [1] error(::String) at ./error.jl:33
│  [2] write_deps_file(::String, ::Array{LibraryProduct,1}; verbose::Bool, isolate::Bool) at /Users/harrisonwilde/.julia/packages/BinaryProvider/GeAtj/src/Products.jl:419
│  [3] top-level scope at /Users/harrisonwilde/.julia/packages/Libtask/Q1ugc/deps/build.jl:58
│  [4] include(::String) at ./client.jl:439
│  [5] top-level scope at none:5
│ in expression starting at /Users/harrisonwilde/.julia/packages/Libtask/Q1ugc/deps/build.jl:58
└ @ Pkg.Operations /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.4/Pkg/src/Operations.jl:892

My project looks like:

(Synthetic Data) pkg> st
Status `~/Library/Mobile Documents/com~apple~CloudDocs/PhD/Synthetic Data/Project.toml`
  [0bf59076] AdvancedHMC v0.2.24
  [c7e460c6] ArgParse v1.1.0
  [6e4b80f9] BenchmarkTools v0.5.0
  [a74b3585] Blosc v0.7.0
  [336ed68f] CSV v0.6.1
  [49dc2e85] Calculus v0.5.1
  [34f1f09b] ClusterManagers v0.3.2
  [944b1d66] CodecZlib v0.7.0
  [a93c6f00] DataFrames v0.20.2
  [31c24e10] Distributions v0.23.2
  [f6369f11] ForwardDiff v0.10.10
  [f67ccb44] HDF5 v0.13.2 #master (https://github.com/JuliaIO/HDF5.jl.git)
  [4138dd39] JLD v0.10.0
  [6f1fad26] Libtask v0.4.0
  [c7f686f2] MCMCChains v3.0.12
  [f0e99cf1] MLBase v0.8.0
  [add582a8] MLJ v0.11.2
  [a7f614a8] MLJBase v0.13.5
  [6ee0df7b] MLJLinearModels v0.3.2
  [429524aa] Optim v0.20.6
  [8e8a01fc] ParallelProcessingTools v0.4.1
  [91a5bcdd] Plots v0.29.9
  [c46f51b8] ProfileView v0.6.5
  [92933f4c] ProgressMeter v1.2.0
  [ce6b1742] RDatasets v0.6.7
  [3646fa90] ScikitLearn v0.5.1
  [276daf66] SpecialFunctions v0.10.0
  [4c63d2b9] StatsFuns v0.9.4
  [f3b207a7] StatsPlots v0.14.5
  [bd369af6] Tables v1.0.4
  [37b6cedf] Traceur v0.3.0
  [fce5fe82] Turing v0.12.0
  [e88e6eb3] Zygote v0.4.20
  [37e2e46d] LinearAlgebra

Create a Julia implementation of the C task struct

It might be possible to create a Julia version of the task struct, based on the memory layout information in #89, then we can further simplify the getfield/setfield functions for manipulating the task fields. It might require creating some new bit types (or primitive types) to represent arbitrarily sized memory blocks without clear Julia equivalent type.

Works only on 1.0

Hey I was curious if there was some specific reason as to why the Julia requirement for this package was fixed to 1.0 and not 0.7?

Error tagging new release

The tag name "v0.2.2-alpha2" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai

Automerge tests in the general registry fail

I retriggered the PR to the general registry on the master branch, i.e., the updated PR includes also the fix to deps/build.jl. However, the automerge tests still fail due to the following error:

┌ Error: Error building `Libtask`: 
│ fatal: not a git repository (or any of the parent directories): .git
│ ┌ Error: Download failed: curl: (22) The requested URL returned error: 404 Not Found
│ └ @ Base download.jl:43
│ ERROR: LoadError: failed process: Process(`git -C /tmp/jl_NXjesH/packages/Libtask/kykuJ fetch --tags`, ProcessExited(128)) [128]
│ 
│ Stacktrace:
│  [1] pipeline_error at ./process.jl:525 [inlined]
│  [2] run(::Cmd; wait::Bool) at ./process.jl:440
│  [3] run at ./process.jl:438 [inlined]
│  [4] find_prev_tag(::String) at /tmp/jl_NXjesH/packages/Libtask/kykuJ/deps/build.jl:3
│  [5] include_build_script(::SubString{String}, ::Bool) at /tmp/jl_NXjesH/packages/Libtask/kykuJ/deps/build.jl:45
│  [6] top-level scope at /tmp/jl_NXjesH/packages/Libtask/kykuJ/deps/build.jl:65
│  [7] include(::String) at ./client.jl:439
│  [8] top-level scope at none:5
│ in expression starting at /tmp/jl_NXjesH/packages/Libtask/kykuJ/deps/build.jl:65
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/Operations.jl:892

It seems it's not possible to run

Libtask.jl/deps/build.jl

Line 3 in 4e201c5

run(`git -C $project_root fetch --tags`)

in the registry CI since the project directory is not downloaded as a git repository there.

I guess it would be more appropriate to fix a specific version but that seems impossible in the current setup where the latest version might not exist during testing?

Error tagging new release

The tag name "0.1.3" is not of the appropriate SemVer form (vX.Y.Z).
cc: @cpfiffer

Error tagging new release

The tag name "v0.2.3-alpha" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai

Implement support for stochastic control flow (e.g. if and unbounded loop)

Related TuringLang/AdvancedPS.jl#37

Why not ccalling directly into libjulia instead of having `Libtask_jll`?

Maintaining a library like Libtask_jll which links to libjulia isn't super simple as you need to have libjulia_jll available in the first place and then build against all different variations. Why don't you ccall directly into libjulia from Julia, instead of delegating the same work to an external library? This would have the advantage of having Libtask.jl more readily available on all platforms and versions of Julia without depending on external libraries.

Error tagging new release

The tag name "v0.2.2-alpha2" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai

Tag a new release

We should probably tag a new release to enable Turing to be used on Julia v0.7.

SMC sampling with Julia 1.1 produces samples with 0 variance for discrete RVs!

I observed this behaviour a few times already. At first I thought I couldn't reproduce the issue but now I figured out the corner case and here is a MWE.

using Turing
using LinearAlgebra
using StatsFuns: logistic

x = vcat(randn(100,3), randn(100,3).+2)
y = vcat(zeros(100), ones(100))

@model lr_nuts(x, y, σ) = begin

    N,D = size(x)

    α ~ Normal(0, σ)
    β ~ MvNormal(zeros(D), ones(D)*σ)

    p = Vector{Float64}(undef, D)
    s = Vector{Bool}(undef, D)
    for d = 1:D
        p[d] ~ Beta(1/2, 1/2)
        s[d] ~ Bernoulli(p[d])
    end

    for n = 1:N
        y[n] ~ Bernoulli(logistic(dot(x[n,s], β[s]) + α))
    end
end

chn = sample(lr_nuts(x,y,sqrt(10)), SMC(1000));

This code produces a chain with variance in the discrete RV's s if I'm using Julia 1.0.4. If I'm using Julia 1.1 instead, the variance for all discrete RV's (s) is zero.

UndefVarError: `ct` not defined

The version of Libtask that's currently shipped with Turing.jl (0.5.1, I guess) raises this error when I run sample(model, sampler, 100):

CTaskException:
Failed to show error:
UndefVarError: ct not defined
Stacktrace:
[1] showerror(io::IOBuffer, ex::Libtask.CTaskException)
@ Libtask ~/.julia/packages/Libtask/00Il9/src/ctask.jl:30
[2] try_showerror(::IOBuffer, ::Libtask.CTaskException)
@ Main.PlutoRunner ~/.julia/packages/Pluto/yDJcG/src/runner/PlutoRunner.jl:454
[3] sprint(f::Function, args::Libtask.CTaskException; context::Nothing, sizehint::Int64)
@ Base ./strings/io.jl:105
[4] sprint(f::Function, args::Libtask.CTaskException)
@ Base ./strings/io.jl:101
[5] format_output(val::CapturedException; context::Pair{Symbol, Dict{Tuple{UInt64, Int64}, Int64}})
@ Main.PlutoRunner ~/.julia/packages/Pluto/yDJcG/src/runner/PlutoRunner.jl:549
[6] formatted_result_of(cell_id::Base.UUID, ends_with_semicolon::Bool, showmore::Nothing)
@ Main.PlutoRunner ~/.julia/packages/Pluto/yDJcG/src/runner/PlutoRunner.jl:436
[7] top-level scope
@ none:1
[8] eval(m::Module, e::Any)
@ Core ./boot.jl:360
[9] (::Distributed.var"#106#108"{Distributed.CallMsg{:call_fetch}})()
@ Distributed /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:278
[10] run_work_thunk(thunk::Distributed.var"#106#108"{Distributed.CallMsg{:call_fetch}}, print_error::Bool)
@ Distributed /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:63
[11] macro expansion
@ /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/Distributed/src/process_messages.jl:278 [inlined]
[12] (::Distributed.var"#105#107"{Distributed.CallMsg{:call_fetch}, Distributed.MsgHeader, Sockets.TCPSocket})()
@ Distributed ./task.jl:411

This happens because, indeed, ct is not defined in version 0.5.1:

Libtask.jl/src/ctask.jl

Lines 25 to 33 in ca8099f

 function Base.showerror(io::IO, ex::CTaskException) 

 println(io, "CTaskException:") 

 bt = @static if VERSION < v"1.6.0-DEV.1145" 

 ct.backtrace 

 else 

 ct.storage[:_libtask_bt] 

 end 

 showerror(io, ex.task.exception, bt) 

 end

It was fixed in the next commit:

Libtask.jl/src/ctask.jl

Lines 30 to 35 in c66b8c9

 ct = ex.task 

 bt = @static if VERSION < v"1.6.0-DEV.1145" 

 ct.backtrace 

 else 

 ct.storage[:_libtask_bt] 

 end

...yet this version was never published, it seems? Turing v0.15.20 uses Libtask 0.5.1:

[[Libtask]]
deps = ["Libtask_jll", "LinearAlgebra", "Statistics"]
git-tree-sha1 = "6088b80fb5017440579ea8113a516ad2807afe19"
uuid = "6f1fad26-d15e-5dc8-ae53-837a1d7b8c9f"
version = "0.5.1"

[[Libtask_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "901fc8752bbc527a6006a951716d661baa9d54e9"
uuid = "3ae2931a-708c-5973-9c38-ccf7496fb450"
version = "0.4.3+0"

julia version 1.6.1
Turing.jl version 0.15.20
macOS

Requires that Libtask is specifically added

(v1.1) pkg> add Turing
 Resolving package versions...
  Updating `C:\Users\accou\.julia\environments\v1.1\Project.toml`
  [fce5fe82] + Turing v0.6.17
  Updating `C:\Users\accou\.julia\environments\v1.1\Manifest.toml`
 [no changes]

julia> using Turing
[ Info: Precompiling Turing [fce5fe82-541a-59a6-adf8-730c64b5f9a0]
ERROR: LoadError: could not open file C:\Users\accou\.julia\packages\Libtask\dWROm\deps\deps.jl
Stacktrace:
 [1] include at .\boot.jl:326 [inlined]
 [2] include_relative(::Module, ::String) at .\loading.jl:1038
 [3] include at .\sysimg.jl:29 [inlined]
 [4] include(::String) at C:\Users\accou\.julia\packages\Libtask\dWROm\src\Libtask.jl:1
 [5] top-level scope at none:0
 [6] include at .\boot.jl:326 [inlined]
 [7] include_relative(::Module, ::String) at .\loading.jl:1038
 [8] include(::Module, ::String) at .\sysimg.jl:29
 [9] top-level scope at none:2
 [10] eval at .\boot.jl:328 [inlined]
 [11] eval(::Expr) at .\client.jl:404
 [12] top-level scope at .\none:3
in expression starting at C:\Users\accou\.julia\packages\Libtask\dWROm\src\Libtask.jl:5
ERROR: LoadError: Failed to precompile Libtask [6f1fad26-d15e-5dc8-ae53-837a1d7b8c9f] to C:\Users\accou\.julia\compiled\v1.1\Libtask\KIaXx.ji.
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at .\loading.jl:1197
 [3] _require(::Base.PkgId) at .\loading.jl:960
 [4] require(::Base.PkgId) at .\loading.jl:858
 [5] require(::Module, ::Symbol) at .\loading.jl:853
 [6] include at .\boot.jl:326 [inlined]
 [7] include_relative(::Module, ::String) at .\loading.jl:1038
 [8] include(::Module, ::String) at .\sysimg.jl:29
 [9] top-level scope at none:2
 [10] eval at .\boot.jl:328 [inlined]
 [11] eval(::Expr) at .\client.jl:404
 [12] top-level scope at .\none:3
in expression starting at C:\Users\accou\.julia\packages\Turing\RZOZ8\src\Turing.jl:14
ERROR: Failed to precompile Turing [fce5fe82-541a-59a6-adf8-730c64b5f9a0] to C:\Users\accou\.julia\compiled\v1.1\Turing\gm4QC.ji.
Stacktrace:
 [1] compilecache(::Base.PkgId, ::String) at .\loading.jl:1197
 [2] _require(::Base.PkgId) at .\loading.jl:960
 [3] require(::Base.PkgId) at .\loading.jl:858
 [4] require(::Module, ::Symbol) at .\loading.jl:853

(v1.1) pkg> add Libtask
 Resolving package versions...
  Updating `C:\Users\accou\.julia\environments\v1.1\Project.toml`
  [6f1fad26] + Libtask v0.3.0
  Updating `C:\Users\accou\.julia\environments\v1.1\Manifest.toml`
 [no changes]

(v1.1) pkg> build Turing
  Building Arpack ──────────→ `C:\Users\accou\.julia\packages\Arpack\cu5By\deps\build.log`
  Building SpecialFunctions → `C:\Users\accou\.julia\packages\SpecialFunctions\fvheQ\deps\build.log`
  Building Rmath ───────────→ `C:\Users\accou\.julia\packages\Rmath\Py9gH\deps\build.log`
  Building Conda ───────────→ `C:\Users\accou\.julia\packages\Conda\kLXeC\deps\build.log`
  Building FFTW ────────────→ `C:\Users\accou\.julia\packages\FFTW\p7sLQ\deps\build.log`
  Building GR ──────────────→ `C:\Users\accou\.julia\packages\GR\TMylY\deps\build.log`
  Building Plots ───────────→ `C:\Users\accou\.julia\packages\Plots\oiirH\deps\build.log`
  Building Libtask ─────────→ `C:\Users\accou\.julia\packages\Libtask\dWROm\deps\build.log`

The problem is that it's looking in the wrong location for the deps folder?

Make a CTask struct

As discussed in TuringLang/Turing.jl#1237, it would be nice to implement the iteration interface for CTask since then it would be possible to use alternatives such as resumable functions in the particle filter implementations in Turing in a straightforward way (currently produce and consume statements are hardcoded). However, currently CTask is just a function that returns a Task for which stack copying has been enabled. Implementing the iteration interface for Task would be pretty bad type piracy (BTW Base.copy(::Task) in the current code is also type piracy). To avoid this I suggest adding a CTask struct that is defined as

struct CTask
	task::Task
end

Then Base.copy and Base.iterate could be implemented without committing type piracy. The downside would be that one would probably have to forward a bunch of methods defined for Task (such istaskstarted etc.) to task.

Do you think I should give it a try, or are there any other suggestions?

Seprate code and data in instructions

The current Instruction type

mutable struct Instruction{F}
    fun::F
    input::Tuple
    output
    tape
end

carries both the code (i.e. fun::F) and data (i.e. input and output). If we want to support loops suggested here, we likely will need a mechanism to store multiple copies of data in the same instruction. It is because each instruction might run multiple times assuming we copy the control flow IR onto the tape without linearization. Maybe we can turn input and output into a vector?

struct Instruction{F}
    fun::F
    io::Vector{InstructionIO}
    tape
end

struct InstructionIO
    input::Tuple
    output
end

New releases

@JuliaRegistrator register()

broken install?

Installation of Libtask fail on line 5 of LibTask.jl
include("../deps/deps.jl"); check_deps();
the file is really not available. Is there any workaround?
Thanks.

Hang under Julia 1.5 and nightly.

Error tagging new release

The tag name "v0.2.2-alpha1" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai

`consume` never returns if task function doesn't contain `produce`

The following code would halt

using Libtask

function f_ct()
   t = 0;
   while t<3
     println(t)
     t = 1 + t
   end
   return t
end

t = CTask(f_ct)

julia> consume(t) # Never returns. 
0
1
2

This is likely due to the consumer task never gets waken up if the producer task exit before any produce statement. Maybe a better behaviour for this kind of tasks is to wake up the producer task with the producer task's return value?

Handling missing

The following MWE breaks in the current version of Turing.

using LinearAlgebra
using Turing

# Linear Model
Δt = 0.1
A = [1.0 Δt; 0.0 1.0]
Q = [0.25 * Δt^4 0.5 * Δt^3; 0.5 * Δt^3 Δt^2] + 1e-6 * I

H = [1.0 0.0; 0.0 1.0;]
R = 0.1 * I

@model function hmm(states, ::Type{T} = Float64) where T
    # Initialize arrays for the time-series
    observations = Matrix{T}(undef, 2, length(states))

    # Prior for first step
    states[1] ~ MvNormal([0.0, 0.5], sqrt(0.001))
    observations[:, 1] ~ MvNormal(states[1], R/Δt)

    for t in 2:length(states)
        states[t] ~ MvNormal(A * states[t-1], Δt*Q)
        observations[:, t] ~ MvNormal(states[t], R/Δt)
    end

    return states
end

# Sample a trial from the prior to be used as the ground truth
hmm_prior = hmm(Union{Missing,Vector{Float64}}[missing for _ in 1:100])
states = hmm_prior()

# Condition on the sampled trial 
hmm_conditioned = hmm(states)

# Perform inference (not working)
chain = sample(hmm_conditioned, SMC(), 1000)

With the stack trace:

ERROR: CTaskException:
Malformed dims
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] TArray{Union{Missing, Array{Float64,1}},1}(::Array{Union{Missing, Array{Float64,1}},1}) at /Users/martin/.julia/packages/Libtask/Zo6uM/src/tarray.jl:36

Any idea why this happens?

When replacing states = hmm_prior() with states = Array{Array{Float64}}(hmm_prior()) everything seems fine.

Fails to compiled on Julia 1.0 (different `struct _jl_task_t`)

@yebai I've been trying to update LibtaskBuilder to link against Julia 1.0 without much luck. In Julia 1.0 struct _jl_task_t definition no longer has a consumers field. Any plans/ideas?

Failing to build Libtask and install Turing

I'm a total newbie and trying to use Turing for the very first time. In building the package for my M1 Mac the manager fails and complains about issues with Libtask but I can't say I fully understand what's happening here. Here is the full traceback.

(@v1.7) pkg> add Turing
Resolving package versions...
No Changes to ~/.julia/environments/v1.7/Project.toml
No Changes to ~/.julia/environments/v1.7/Manifest.toml
Precompiling project...
✗ BinaryProvider
✗ Libtask
✗ ThreadingUtilities
✗ SLEEFPirates
✗ LoopVectorization
✗ RecursiveFactorization
✗ NonlinearSolve
✗ Bijectors
✗ DiffEqBase
✗ OrdinaryDiffEq
✗ DynamicPPL
✗ AdvancedVI
✗ GpABC
✗ Turing
80 dependencies successfully precompiled in 51 seconds (174 already precompiled)
14 dependencies errored. To see a full report either run import Pkg; Pkg.precompile() or load the packages

and this is the traceback when I try to build the package

(@v1.7) pkg> build Turing
Building Libtask → ~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/83e082fccb4e37d93df6440cdbd41dcbe5e46cb6/build.log
ERROR: Error building Libtask:
┌ Warning: Platform arm64-apple-darwin21.1.0 is not an officially supported platform
└ @ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:450
ERROR: LoadError: KeyError: key "unknown" not found
Stacktrace:
[1] getindex
@ ./dict.jl:481 [inlined]
[2] parse_dl_name_version
@ ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:511 [inlined]
[3] detect_libgfortran_abi(libgfortran_name::String, platform::BinaryProvider.UnknownPlatform) (repeats 2 times)
@ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:555
[4] detect_libgfortran_abi
@ ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:584 [inlined]
[5] detect_compiler_abi()
@ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:667
[6] top-level scope
@ ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:685
[7] include(mod::Module, _path::String)
@ Base ./Base.jl:418
[8] include(x::String)
@ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/BinaryProvider.jl:1
[9] top-level scope
@ ~/.julia/packages/BinaryProvider/U2dKK/src/BinaryProvider.jl:12
[10] include
@ ./Base.jl:418 [inlined]
[11] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
@ Base ./loading.jl:1318
[12] top-level scope
@ none:1
[13] eval
@ ./boot.jl:373 [inlined]
[14] eval(x::Expr)
@ Base.MainInclude ./client.jl:453
[15] top-level scope
@ none:1
in expression starting at /Users/bpajusco/.julia/packages/BinaryProvider/U2dKK/src/PlatformNames.jl:685
in expression starting at /Users/bpajusco/.julia/packages/BinaryProvider/U2dKK/src/BinaryProvider.jl:1
ERROR: LoadError: Failed to precompile BinaryProvider [b99e7846-7c00-51b0-8f62-c81ae34c0232] to /Users/bpajusco/.julia/compiled/v1.7/BinaryProvider/jl_vEZwnJ.
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:33
[2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
@ Base ./loading.jl:1466
[3] compilecache(pkg::Base.PkgId, path::String)
@ Base ./loading.jl:1410
[4] _require(pkg::Base.PkgId)
@ Base ./loading.jl:1120
[5] require(uuidkey::Base.PkgId)
@ Base ./loading.jl:1013
[6] require(into::Module, mod::Symbol)
@ Base ./loading.jl:997
[7] include(fname::String)
@ Base.MainInclude ./client.jl:451
[8] top-level scope
@ none:5
in expression starting at /Users/bpajusco/.julia/packages/Libtask/OtgFl/deps/build.jl:9

Error tagging new release

The tag name "v0.2.2-alpha2" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai

Error tagging new release

The tag name "v0.2.2-alpha1" is not of the appropriate SemVer form (vX.Y.Z).
cc: @yebai

Improving artefacts uploading process

Maybe we can modify the build script of Libtask, so that a tag is automatically created if the tag does not exist? This would allow uploading artefacts for new Libtask versions even before a new version is accepted by JuliaRegistry.

Libtask does not support latest Julia version

using Libtask results in ERROR: LoadError: LoadError: UndefVarError: libtask_v1_3 not defined

Replacing `TArray` with compiler-based copy-on-write

The TArray-family data structure in Libtask provides a copy-on-write mechanism for heap-allocated data structures. This is helpful when we want to decouple heap-allocated data structures after tasks are forked, i.e. each task will get an independent copy of heap-allocated variables, and mutating these variables from one task won't affect other tasks. The current implementation requires the user to use TArray-like constructors instead of Arrays. This is inconvenient and sometimes confusing. It should be possible to completely replace the functionality of TArray by using a compiler-based implementation.

The basic idea is to detect yield points in static single assignment (SSA) form of functions, then use an SSA manipulation tool (e.g. IRTools) to explicit insert statements that detect task-copying and deep copy variables that are mutated after the task-copying point.

Libtask does not support Julia 1.4.0

using Libtask results in: ERROR: LoadError: LoadError: UndefVarError: libtask_v1_3 not defined

Allow `TapedFunction` as a special Instruction

We currently treat all instructions on a tape as primitive. There are use-cases (see here) where we need to selectively trace into instructions. One neat way of achieving that is to have a special instruction type that allows us to plug a tape A into another tape B, where tape A is considered as a standard instruction on tape B. This way, we can create a dedicated tape for an instruction that we want to trace into, and use it on another (parent) tape.

Add AArch64 Linux support

Idk how difficult it is, but I see there is already support for 32bit arm. Is it possible to add the 64bit libraries as well?

julia> versioninfo()
Julia Version 1.3.0-DEV.0
Platform Info:
  OS: Linux (aarch64-unknown-linux-gnu)
  CPU: unknown
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, kryo)

Libtask waits for ever is function body has errors.

See the following minimal example:

using Libtask

function f()
    t = 0;
    while true
        t[2] = 0
        produce(t)
    end
end

t = Task(f)
consume(t)

Libtask doesn't work with Julia 1.7

With Julia 1.7.0-beta3 on Windows and an empty default environment, Libtask installs cleanly from current master (7c404c4), but the tests throw an error:

(@v1.7) pkg> st
      Status `C:\Users\abc\.julia\environments\v1.7\Project.toml` (empty project)

(@v1.7) pkg> add Libtask#7c404c4
    Updating registry at `C:\Users\abc\.julia\registries\General.toml`
   Resolving package versions...
    Updating `C:\Users\abc\.julia\environments\v1.7\Project.toml`
  [6f1fad26] + Libtask v0.5.3 `https://github.com/TuringLang/Libtask.jl.git#7c404c4`
    Updating `C:\Users\abc\.julia\environments\v1.7\Manifest.toml`
  [692b3bcd] + JLLWrappers v1.3.0
  [6f1fad26] + Libtask v0.5.3 `https://github.com/TuringLang/Libtask.jl.git#7c404c4`
  [21216c6a] + Preferences v1.2.2
  [3ae2931a] + Libtask_jll v0.5.1+0
  [0dad84c5] + ArgTools
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [f43a241f] + Downloads
  [b77e0a4c] + InteractiveUtils
  [b27032c2] + LibCURL
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [ca575930] + NetworkOptions
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [fa267f1f] + TOML
  [a4e569a6] + Tar
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll
  [deac9b47] + LibCURL_jll
  [29816b5a] + LibSSH2_jll
  [c8ffd9c3] + MbedTLS_jll
  [14a3606d] + MozillaCACerts_jll
  [4536629a] + OpenBLAS_jll
  [83775a58] + Zlib_jll
  [8e850b90] + libblastrampoline_jll
  [8e850ede] + nghttp2_jll
  [3f19e933] + p7zip_jll

(@v1.7) pkg> st
      Status `C:\Users\abc\.julia\environments\v1.7\Project.toml`
  [6f1fad26] Libtask v0.5.3 `https://github.com/TuringLang/Libtask.jl.git#7c404c4`

(@v1.7) pkg> test Libtask
     Testing Libtask
      Status `C:\Users\abc\AppData\Local\Temp\jl_i0IMgj\Project.toml`
  [6e4b80f9] BenchmarkTools v1.1.1
  [6f1fad26] Libtask v0.5.3 `https://github.com/TuringLang/Libtask.jl.git#7c404c4`
  [3ae2931a] Libtask_jll v0.5.1+0
  [8f399da3] Libdl `@stdlib/Libdl`
  [37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra`
  [10745b16] Statistics `@stdlib/Statistics`
  [8dfed614] Test `@stdlib/Test`
      Status `C:\Users\abc\AppData\Local\Temp\jl_i0IMgj\Manifest.toml`
  [6e4b80f9] BenchmarkTools v1.1.1
  [692b3bcd] JLLWrappers v1.3.0
  [682c06a0] JSON v0.21.2
  [6f1fad26] Libtask v0.5.3 `https://github.com/TuringLang/Libtask.jl.git#7c404c4`
  [69de0a69] Parsers v2.0.2
  [21216c6a] Preferences v1.2.2
  [3ae2931a] Libtask_jll v0.5.1+0
  [0dad84c5] ArgTools `@stdlib/ArgTools`
  [56f22d72] Artifacts `@stdlib/Artifacts`
  [2a0f44e3] Base64 `@stdlib/Base64`
  [ade2ca70] Dates `@stdlib/Dates`
  [f43a241f] Downloads `@stdlib/Downloads`
  [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils`
  [b27032c2] LibCURL `@stdlib/LibCURL`
  [76f85450] LibGit2 `@stdlib/LibGit2`
  [8f399da3] Libdl `@stdlib/Libdl`
  [37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra`
  [56ddb016] Logging `@stdlib/Logging`
  [d6f4376e] Markdown `@stdlib/Markdown`
  [a63ad114] Mmap `@stdlib/Mmap`
  [ca575930] NetworkOptions `@stdlib/NetworkOptions`
  [44cfe95a] Pkg `@stdlib/Pkg`
  [de0858da] Printf `@stdlib/Printf`
  [3fa0cd96] REPL `@stdlib/REPL`
  [9a3f8284] Random `@stdlib/Random`
  [ea8e919c] SHA `@stdlib/SHA`
  [9e88b42a] Serialization `@stdlib/Serialization`
  [6462fe0b] Sockets `@stdlib/Sockets`
  [2f01184e] SparseArrays `@stdlib/SparseArrays`
  [10745b16] Statistics `@stdlib/Statistics`
  [fa267f1f] TOML `@stdlib/TOML`
  [a4e569a6] Tar `@stdlib/Tar`
  [8dfed614] Test `@stdlib/Test`
  [cf7118a7] UUIDs `@stdlib/UUIDs`
  [4ec0a83e] Unicode `@stdlib/Unicode`
  [e66e0078] CompilerSupportLibraries_jll `@stdlib/CompilerSupportLibraries_jll`
  [deac9b47] LibCURL_jll `@stdlib/LibCURL_jll`
  [29816b5a] LibSSH2_jll `@stdlib/LibSSH2_jll`
  [c8ffd9c3] MbedTLS_jll `@stdlib/MbedTLS_jll`
  [14a3606d] MozillaCACerts_jll `@stdlib/MozillaCACerts_jll`
  [4536629a] OpenBLAS_jll `@stdlib/OpenBLAS_jll`
  [83775a58] Zlib_jll `@stdlib/Zlib_jll`
  [8e850b90] libblastrampoline_jll `@stdlib/libblastrampoline_jll`
  [8e850ede] nghttp2_jll `@stdlib/nghttp2_jll`
  [3f19e933] p7zip_jll `@stdlib/p7zip_jll`
     Testing Running tests...
stack allocated objects: Error During Test at C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:13
  Test threw exception
  Expression: consume(ctask) == 0
  ReadOnlyMemoryError()
  Stacktrace:
   [1] setproperty!
     @ .\Base.jl:43 [inlined]
   [2] consume(::CTask)
     @ Libtask C:\Users\abc\.julia\packages\Libtask\8YyQv\src\ctask.jl:218
   [3] macro expansion
     @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\Test\src\Test.jl:445 [inlined]
   [4] macro expansion
     @ C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:13 [inlined]
   [5] macro expansion
     @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\Test\src\Test.jl:1282 [inlined]
   [6] macro expansion
     @ C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:4 [inlined]
   [7] macro expansion
     @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\Test\src\Test.jl:1282 [inlined]
   [8] top-level scope
     @ C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:3
stack allocated objects: Error During Test at C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:3
  Got exception outside of a @test
  KeyError: key :_libtask_bt not found
  Stacktrace:
    [1] getindex(d::IdDict{Any, Any}, key::Any)
      @ Base .\iddict.jl:108
    [2] showerror(io::IOContext{IOBuffer}, ex::Libtask.CTaskException)
      @ Libtask C:\Users\abc\.julia\packages\Libtask\8YyQv\src\ctask.jl:34
    [3] showerror(io::IOContext{IOBuffer}, ex::Libtask.CTaskException, bt::Vector{Union{Ptr{Nothing}, Base.InterpreterIP}}; backtrace::Bool)
      @ Base .\errorshow.jl:88
    [4] show_exception_stack(io::IOContext{IOBuffer}, stack::Vector{Any})
      @ Base .\errorshow.jl:862
    [5] sprint(f::Function, args::Vector{Any}; context::Base.TTY, sizehint::Int64)
      @ Base .\strings\io.jl:110
    [6] Test.Error(test_type::Any, orig_expr::Any, value::Any, bt::Any, source::Any)
      @ Test C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\Test\src\Test.jl:170
    [7] do_test(result::Test.ExecutionResult, orig_expr::Any)
      @ Test C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\Test\src\Test.jl:634
    [8] macro expansion
      @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\Test\src\Test.jl:445 [inlined]
    [9] macro expansion
      @ C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:14 [inlined]
   [10] macro expansion
      @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\Test\src\Test.jl:1282 [inlined]
   [11] macro expansion
      @ C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:4 [inlined]
   [12] macro expansion
      @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.7\Test\src\Test.jl:1282 [inlined]
   [13] top-level scope
      @ C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:3
   [14] include(fname::String)
      @ Base.MainInclude .\client.jl:451
   [15] top-level scope
      @ C:\Users\abc\.julia\packages\Libtask\8YyQv\test\runtests.jl:4
   [16] include(fname::String)
      @ Base.MainInclude .\client.jl:451
   [17] top-level scope
      @ none:6
   [18] eval
      @ .\boot.jl:373 [inlined]
   [19] exec_options(opts::Base.JLOptions)
      @ Base .\client.jl:268
   [20] _start()
      @ Base .\client.jl:495

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x3c49bfac -- getproperty at .\task.jl:170 [inlined]
produce at C:\Users\abc\.julia\packages\Libtask\8YyQv\src\ctask.jl:154
in expression starting at C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:1
getproperty at .\task.jl:170 [inlined]
produce at C:\Users\abc\.julia\packages\Libtask\8YyQv\src\ctask.jl:154
#176#f at C:\Users\abc\.julia\packages\Libtask\8YyQv\test\ctask.jl:27
#8 at C:\Users\abc\.julia\packages\Libtask\8YyQv\src\ctask.jl:85
unknown function (ip: 000000003c49cd13)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1787 [inlined]
start_task at /cygdrive/c/buildbot/worker/package_win64/build/src\task.c:878
Allocations: 2390953 (Pool: 2390183; Big: 770); GC: 3
ERROR: Package Libtask errored during testing

Project file:

[deps]
Libtask = "6f1fad26-d15e-5dc8-ae53-837a1d7b8c9f"

Manifest file:

# This file is machine-generated - editing it directly is not advised

julia_version = "1.7.0-beta3.0"
manifest_format = "2.0"

[[deps.ArgTools]]
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"

[[deps.Artifacts]]
uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"

[[deps.Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

[[deps.CompilerSupportLibraries_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae"

[[deps.Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"

[[deps.Downloads]]
deps = ["ArgTools", "LibCURL", "NetworkOptions"]
uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"

[[deps.InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"

[[deps.JLLWrappers]]
deps = ["Preferences"]
git-tree-sha1 = "642a199af8b68253517b80bd3bfd17eb4e84df6e"
uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210"
version = "1.3.0"

[[deps.LibCURL]]
deps = ["LibCURL_jll", "MozillaCACerts_jll"]
uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"

[[deps.LibCURL_jll]]
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"

[[deps.LibGit2]]
deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"

[[deps.LibSSH2_jll]]
deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"

[[deps.Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"

[[deps.Libtask]]
deps = ["Libdl", "Libtask_jll", "LinearAlgebra", "Statistics"]
git-tree-sha1 = "af2d97cb50decc6aa3621f5c02d3cd9cea2d0f30"
repo-rev = "7c404c4"
repo-url = "https://github.com/TuringLang/Libtask.jl.git"
uuid = "6f1fad26-d15e-5dc8-ae53-837a1d7b8c9f"
version = "0.5.3"

[[deps.Libtask_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "3ee8b8e25a657e8f5d4ca702e7b78150521caf0f"
uuid = "3ae2931a-708c-5973-9c38-ccf7496fb450"
version = "0.5.1+0"

[[deps.LinearAlgebra]]
deps = ["Libdl", "libblastrampoline_jll"]
uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"

[[deps.Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"

[[deps.Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"

[[deps.MbedTLS_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"

[[deps.MozillaCACerts_jll]]
uuid = "14a3606d-f60d-562e-9121-12d972cd8159"

[[deps.NetworkOptions]]
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"

[[deps.OpenBLAS_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "Libdl"]
uuid = "4536629a-c528-5b80-bd46-f80d51c5b363"

[[deps.Pkg]]
deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"

[[deps.Preferences]]
deps = ["TOML"]
git-tree-sha1 = "00cfd92944ca9c760982747e9a1d0d5d86ab1e5a"
uuid = "21216c6a-2e73-6563-6e65-726566657250"
version = "1.2.2"

[[deps.Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"

[[deps.REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"

[[deps.Random]]
deps = ["Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"

[[deps.SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"

[[deps.Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"

[[deps.Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"

[[deps.SparseArrays]]
deps = ["LinearAlgebra", "Random"]
uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"

[[deps.Statistics]]
deps = ["LinearAlgebra", "SparseArrays"]
uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"

[[deps.TOML]]
deps = ["Dates"]
uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"

[[deps.Tar]]
deps = ["ArgTools", "SHA"]
uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"

[[deps.UUIDs]]
deps = ["Random", "SHA"]
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[[deps.Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"

[[deps.Zlib_jll]]
deps = ["Libdl"]
uuid = "83775a58-1f1d-513f-b197-d71354ab007a"

[[deps.libblastrampoline_jll]]
deps = ["Artifacts", "Libdl", "OpenBLAS_jll"]
uuid = "8e850b90-86db-534c-a0d3-1478176c7d93"

[[deps.nghttp2_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"

[[deps.p7zip_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"

Refactor `TAarray`

The TArray data structure implements a copy-on-write array. This is useful for task copying since standard arrays are allocated on the stack, which is only shallow copied during task copying.

However, the current code for TArray has several limits:

type definition is over-restrictive (e.g. subtype of DenseArray).
some constructors and array-related functions are either broken or missing
fix display for TArray

Perhaps

create a TRef type that can store value of arbitrary data types, e.g. x = TRef([0 1 2])

Error Building Libtask

Hi-

I get the following error building Libtask:

(v1.3) pkg> build Libtask
  Building Libtask → `~/.julia/packages/Libtask/04fjt/deps/build.log`
┌ Error: Error building `Libtask`: 
│ fatal: not a git repository (or any of the parent directories): .git
│ ┌ Error: Download failed: curl: (22) The requested URL returned error: 404 Not Found
│ └ @ Base download.jl:43
│ ERROR: LoadError: failed process: Process(`git -C /home/dfish/.julia/packages/Libtask/04fjt tag`, ProcessExited(128)) [128]
│ 
│ Stacktrace:
│  [1] pipeline_error at ./process.jl:525 [inlined]
│  [2] (::Base.var"#561#562"{Base.Process})() at ./process.jl:319
│  [3] iterate(::Base.EachLine{Base.PipeEndpoint}, ::Nothing) at ./io.jl:964 (repeats 2 times)
│  [4] _collect(::UnitRange{Int64}, ::Base.EachLine{Base.PipeEndpoint}, ::Base.HasEltype, ::Base.SizeUnknown) at ./array.jl:571
│  [5] #readlines#294 at ./array.jl:560 [inlined]
│  [6] readlines at ./io.jl:498 [inlined]
│  [7] find_prev_tag(::String) at /home/dfish/.julia/packages/Libtask/04fjt/deps/build.jl:3
│  [8] include_build_script(::SubString{String}, ::Bool) at /home/dfish/.julia/packages/Libtask/04fjt/deps/build.jl:44
│  [9] top-level scope at /home/dfish/.julia/packages/Libtask/04fjt/deps/build.jl:64
│  [10] include at ./boot.jl:328 [inlined]
│  [11] include_relative(::Module, ::String) at ./loading.jl:1105
│  [12] include(::Module, ::String) at ./Base.jl:31
│  [13] include(::String) at ./client.jl:424
│  [14] top-level scope at none:5
│ in expression starting at /home/dfish/.julia/packages/Libtask/04fjt/deps/build.jl:64
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/backwards_compatible_isolation.jl:649

Version:

(v1.3) pkg> st Libtask
    Status `~/.julia/environments/v1.3/Project.toml`
  [6f1fad26] Libtask v0.3.2

Use Artifacts instead of BinaryProvider/build.jl

It would be useful to use Artifacts and jll packages, instead of creating a build.jl and using Binaryprovider to download the depdendencies.

Using Artifacts has a couple of advantages: 1) it reduces duplication in download and storage of the native depdendencies, and 2) it works via the package server mechanism, which provides a globally replicated CDN, better caching, and better support for proxies.

This would mean however that only Julia 1.3 and above are supported, but I noticed that Turing.jl already has 1.3 as minimum bound, so should be fine to limit this pakage to that lower bound.

Load error on Windows and Julia 1.2

Hi,

I am getting the following load error with Julia 1.2 and Windows:

(v1.2) pkg> build Libtask
  Building Libtask → `C:\Users\user\.julia\packages\Libtask\RjRkK\deps\build.log`
┌ Error: Error building `Libtask`:
│ fatal: not a git repository (or any of the parent directories): .git
│ ┌ Error: Download failed: curl: (35) schannel: next InitializeSecurityContext failed: Unknown error (0x80092012) - The revocation function was unable to check revocation for the certificate.
│ └ @ Base download.jl:43
│ ERROR: LoadError: failed process: Process(`git -C 'C:\Users\user\.julia\packages\Libtask\RjRkK' tag`, ProcessExited(128)) [128]
│
│ Stacktrace:
│  [1] pipeline_error at .\process.jl:813 [inlined]
│  [2] (::getfield(Base, Symbol("##533#534")){Base.Process})() at .\process.jl:621
│  [3] iterate(::Base.EachLine{Base.PipeEndpoint}, ::Nothing) at .\io.jl:893 (repeats 2 times)
│  [4] _collect(::UnitRange{Int64}, ::Base.EachLine{Base.PipeEndpoint}, ::Base.HasEltype, ::Base.SizeUnknown) at .\array.jl:555
│  [5] collect at .\array.jl:544 [inlined]
│  [6] #readlines#290 at .\io.jl:427 [inlined]
│  [7] readlines at .\io.jl:427 [inlined]
│  [8] find_prev_tag(::String) at C:\Users\user\.julia\packages\Libtask\RjRkK\deps\build.jl:3
│  [9] include_build_script(::SubString{String}, ::Bool) at C:\Users\user\.julia\packages\Libtask\RjRkK\deps\build.jl:42
│  [10] top-level scope at C:\Users\user\.julia\packages\Libtask\RjRkK\deps\build.jl:62
│  [11] include at .\boot.jl:328 [inlined]
│  [12] include_relative(::Module, ::String) at .\loading.jl:1094
│  [13] include(::Module, ::String) at .\Base.jl:31
│  [14] include(::String) at .\client.jl:431
│  [15] top-level scope at none:5
│ in expression starting at C:\Users\user\.julia\packages\Libtask\RjRkK\deps\build.jl:62
└ @ Pkg.Operations C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.2\Pkg\src\backwards_compatible_isolation.jl:647

julia> versioninfo()
Julia Version 1.2.0
Commit c6da87ff4b (2019-08-20 00:03 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

arm64-apple-darwin21.1.0 support

Just tried whether Turing would work in Julia 1.7.0 on a Mac Air M1 and ran into the problem that Libtask was not properly installed. I then tried building libtask as described in the error message and got the error that currently apple arm architectures are not supported. Are there any plans to change this soon? Especially now that Julia 1.7.0 officially supports that architecture.

Missing deps.jl - Can't Precompile Turing

When I tried to load Turing, it fails since Libtask is missing deps.jl. Here's the stack trace:

julia> using Libtask
[ Info: Precompiling Libtask [6f1fad26-d15e-5dc8-ae53-837a1d7b8c9f]
ERROR: LoadError: could not open file /home/rishi/.julia/packages/Libtask/L5wq8/deps/deps.jl
Stacktrace:
 [1] include at ./boot.jl:317 [inlined]
 [2] include_relative(::Module, ::String) at ./loading.jl:1044
 [3] include at ./sysimg.jl:29 [inlined]
 [4] include(::String) at /home/rishi/.julia/packages/Libtask/L5wq8/src/Libtask.jl:1
 [5] top-level scope at none:0
 [6] top-level scope at none:2
in expression starting at /home/rishi/.julia/packages/Libtask/L5wq8/src/Libtask.jl:5
ERROR: Failed to precompile Libtask [6f1fad26-d15e-5dc8-ae53-837a1d7b8c9f] to /home/rishi/.julia/compiled/v1.0/Libtask/KIaXx.ji.
Stacktrace:
 [1] compilecache(::Base.PkgId, ::String) at ./loading.jl:1203
 [2] _require(::Base.PkgId) at ./loading.jl:960
 [3] require(::Base.PkgId) at ./loading.jl:858
 [4] require(::Module, ::Symbol) at ./loading.jl:853

Libtask is version 0.2.0, Julia is 1.0.3, OS is Arch Linux.

Release Libtask

@JuliaRegistrator register

Test errors on DynamicPPL due to Libtask

CI tests of DynamicPPL (see https://travis-ci.com/github/TuringLang/DynamicPPL.jl/jobs/306791553 and TuringLang/DynamicPPL.jl#52) fail on Julia 1.4 and Julia master due to the error

ERROR: LoadError: The following package names could not be resolved:
 * Libtask (not found in project or manifest)
Stacktrace:
 [1] pkgerror(::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/Types.jl:53
 [2] ensure_resolved(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; registry::Bool) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/Types.jl:794
 [3] ensure_resolved at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/Types.jl:759 [inlined]
 [4] build(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; verbose::Bool, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:697
 [5] build at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:682 [inlined]
 [6] #build#108 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:680 [inlined]
 [7] build at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:680 [inlined]
 [8] #build#105 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:677 [inlined]
 [9] build(::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:677
 [10] top-level scope at /home/travis/.julia/packages/Libtask/K7OAU/src/Libtask.jl:7
 [11] include(::Module, ::String) at ./Base.jl:377
 [12] top-level scope at none:2
 [13] eval at ./boot.jl:331 [inlined]
 [14] eval(::Expr) at ./client.jl:449
 [15] top-level scope at ./none:3
in expression starting at /home/travis/.julia/packages/Libtask/K7OAU/src/Libtask.jl:5

I think the problematic lines are

Libtask.jl/src/Libtask.jl

Lines 5 to 8 in a1ce1ea

 if !isfile(joinpath((@__DIR__) |> dirname, "deps/desp.jl")) 

 import Pkg 

 Pkg.build("Libtask") 

 end

Can we remove them completely? I would assume that Pkg should run Pkg.build() automatically?

	function Base.showerror(io::IO, ex::CTaskException)
	println(io, "CTaskException:")
	bt = @static if VERSION < v"1.6.0-DEV.1145"
	ct.backtrace
	else
	ct.storage[:_libtask_bt]
	end
	showerror(io, ex.task.exception, bt)
	end

	ct = ex.task
	bt = @static if VERSION < v"1.6.0-DEV.1145"
	ct.backtrace
	else
	ct.storage[:_libtask_bt]
	end

	if !isfile(joinpath((@__DIR__) \|> dirname, "deps/desp.jl"))
	import Pkg
	Pkg.build("Libtask")
	end

turinglang / libtask.jl Goto Github PK

libtask.jl's People

Contributors

Stargazers

Watchers

Forkers

libtask.jl's Issues

Recommend Projects

Recommend Topics

Recommend Org