julialang / packagecompiler.jl Goto Github PK

View Code? Open in Web Editor NEW

1.4K 48.0 182.0 3.41 MB

Compile your Julia Package

Home Page: https://julialang.github.io/PackageCompiler.jl/dev/

License: MIT License

Julia 93.50% C 6.50%

packagecompiler julia

packagecompiler.jl's Introduction

PackageCompiler

PackageCompiler is a Julia package with three main purposes:

Creating custom sysimages for reduced latency when working locally with packages that have high startup times.
Creating "apps" which are a bundle of files including an executable that can be sent and run on other machines without Julia being installed on that machine.
Creating a relocatable C library bundle form of Julia code.

For installation and usage instructions, see the documentation.

Upgrading from PackageCompiler 1.0

PackageCompiler 2.0 comes with a few breaking changes. Read the upgrade guide here for instructions on how to upgrade to version 2.0.

packagecompiler.jl's People

Contributors

Stargazers

Watchers

Forkers

ken-b hayatoikoma andyferris pallharaldsson transgirlcodes jlapeyre lucatrv nhdaly atsushisakai zgornel kahaaga tsurherman dmoliveira techlord-rce tkoolen bdahon guillempuig-zz noahstier okwh zijuexiansheng libicc vtjnash bajizhh jackdunnnz petr-hlavenka kristofferc tkf cako kamos86 jdchn openprototyping raphbacher john2912 musm tlienart charudatta10 pbellive kharazity sokoltsov dalavancloud 00vareladavid asinghvi17 dasssj chaydont davidanthoff hisopika71199191 daviehh jkrt gunnarfarneback kdheepak kunray hypnus97 relationalai-oss conning j-fu aerdely smsinks stanleyjs jheinen kramsretlow kamleshbhalui aminya xjzpguob mcabbott stjordanis prakhar2b rob-luke pthariensflame visr kmsquire masonprotter matsievskiysv oliverevans96 natema grahamgill newptcai bmasinde paristsai ianbutterworth twavv alakialakiani arthursoprana p1scescom gsir-ying iljak91 mkitti tchigher alperyilmaz roger-luo rdguerrerom tubbz-alt omus petvana rustyconover pankgeorg paulcorcuera permanentrance standardgalactic jtschneider wxyangf

packagecompiler.jl's Issues

Tests failing on 0.7 nightly

Tested on

64 bit julia, 64 bit host
32 bit julia, 64 bit host
32 bit julia, 32 bit host

This might be related to timholy/SnoopCompile.jl#15 which might be some sort of mundane file IO issue.

For the matched ones:

INFO: Linking sys.so
INFO: cc -L/julia/bin/../lib -shared -ljulia -o /root/.julia/v0.7/PackageCompiler/sysimg/sys.so /root/.julia/v0.7/PackageCompiler/sysimg/sys.o
INFO:     Not replacing system image.
    You can start julia with julia -J /root/.julia/v0.7/PackageCompiler/sysimg/sys to load the compiled files.
basic tests: Test Failed at /root/.julia/v0.7/PackageCompiler/test/runtests.jl:16
  Expression: length(readlines(userimg)) > 700
   Evaluated: 2 > 700
Stacktrace:
 [1] macro expansion at /root/.julia/v0.7/PackageCompiler/test/runtests.jl:16 [inlined]
 [2] macro expansion at /buildworker/worker/package_linux32/build/usr/share/julia/site/v0.7/Test/src/Test.jl:964 [inlined]
 [3] top-level scope at /root/.julia/v0.7/PackageCompiler/test/runtests.jl:12

The cross-compiling one had slightly different errors (and maybe this shouldn't be expected to work anyways):

INFO: Linking sys.so
INFO: cc -L/home/alex/julia-nightly-32/bin/../lib -shared -ljulia -o /home/alex/.julia/v0.7/PackageCompiler/sysimg/sys.so /home/alex/.julia/v0.7/PackageCompiler/sysimg/sys.o
/usr/bin/ld: skipping incompatible /home/alex/julia-nightly-32/bin/../lib/libjulia.so when searching for -ljulia
/usr/bin/ld: cannot find -ljulia
collect2: error: ld returned 1 exit status
ERROR: LoadError: failed process: Process(`cc -L/home/alex/julia-nightly-32/bin/../lib -shared -ljulia -o /home/alex/.julia/v0.7/PackageCompiler/sysimg/sys.so /home/alex/.julia/v0.7/PackageCompiler/sysimg/sys.o`, ProcessExited(1)) [1]
Stacktrace:
 [1] error(::String, ::Base.Process, ::String, ::Int64, ::String) at ./error.jl:44
 [2] pipeline_error(::Base.Process) at ./process.jl:704
 [3] run(::Cmd) at ./process.jl:666
 [4] link_sysimg(::String, ::String, ::Bool) at /home/alex/.julia/v0.7/PackageCompiler/src/build_sysimg.jl:119
 [5] (::getfield(PackageCompiler, Symbol("##2#3")){Bool,String,String,String,String})() at /home/alex/.julia/v0.7/PackageCompiler/src/build_sysimg.jl:91
 [6] cd(::getfield(PackageCompiler, Symbol("##2#3")){Bool,String,String,String,String}, ::String) at ./file.jl:70
 [7] #build_sysimg#1(::Bool, ::Function, ::String, ::String, ::String) at /home/alex/.julia/v0.7/PackageCompiler/src/build_sysimg.jl:59
 [8] build_sysimg at /home/alex/.julia/v0.7/PackageCompiler/src/build_sysimg.jl:58 [inlined]
...

Permission Access with PackageCompiler.force_native_image!()

My OS is arch-linux
I'm running PackageCompiler.force_native_image!() as instructed on first run, however it's spitting back a permission error.
Is there currently a way to fix default write permissions of generated files?
error:

julia> PackageCompiler.force_native_image!()
ERROR: open: permission denied (EACCES)
Stacktrace:
 [1] uv_error at ./libuv.jl:68 [inlined]
 [2] open(::String, ::UInt16, ::UInt16) at ./filesystem.jl:81
 [3] sendfile(::String, ::String) at ./file.jl:523
 [4] #cp#10(::Bool, ::Bool, ::Function, ::String, ::String) at ./file.jl:227
 [5] (::Base.Filesystem.#kw##cp)(::Array{Any,1}, ::Base.Filesystem.#cp, ::String, ::String) at ./<missing>:0
 [6] rename(::String, ::String) at ./file.jl:510
 [7] (::Base.Filesystem.#kw##mv)(::Array{Any,1}, ::Base.Filesystem.#mv, ::String, ::String) at ./<missing>:0
 [8] copy_system_image(::String, ::String, ::Bool) at /home/effectz/.julia/v0.6/PackageCompiler/src/PackageCompiler.jl:48
 [9] force_native_image!() at /home/effectz/.julia/v0.6/PackageCompiler/src/api.jl:116
 [10] macro expansion at ./REPL.jl:97 [inlined]
 [11] (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:73

info:

julia> versioninfo()
Julia Version 0.6.4
Commit 9d11f62bcb (2018-07-09 19:09 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i3-4340 CPU @ 3.60GHz
  WORD_SIZE: 64
  BLAS: libopenblas (DYNAMIC_ARCH NO_AFFINITY Haswell MAX_THREADS=16)
  LAPACK: libopenblas
  LIBM: libm
  LLVM: libLLVM-3.9.1 (ORCJIT, haswell)

`PROGRAM_FILE` not set when compiled through `juliac.jl`

Base.PROGRAM_FILE is empty when compiled through juliac.jl. (The global ARGS is empty, which is where PROGRAM_FILE is set from.)

PROGRAM_FILE is set here, by popping the first value off of ARGS, iff ARGS is not empty (checked here).

Currently, the default program.c skips setting the global ARGS, opting instead to pass an Array to julia_main. This means that there is no way to access PROGRAM_FILE from compiled code.

fails Pkg.test with Julia v1.0

ERROR: LoadError: UndefVarError: Test not defined
Stacktrace:
[1] include at .\boot.jl:317 [inlined]
[2] include_relative(::Module, ::String) at .\loading.jl:1038
[3] include(::Module, ::String) at .\sysimg.jl:29
[4] include(::String) at .\client.jl:388
[5] top-level scope at none:0
in expression starting at C:\Users\GEvans.julia\packages\PackageCompiler\5u3Km\test\runtests.jl:2

Need to replace line 2: "using Base.Test" with "using Test"

Now just need to find out why the test are failing...

Simpler Example

Makie is a great example - but would be great to have a simpler example that is easy for people to try out.

PackageCompiler will not compile my (simple) Julia program to a static binary due to ArgParse errors

I am trying to compile my program to an executable on Linux using PackageCompiler - but can't get it to work on either Julia 0.6 or 0.7. In both cases it's ArgParse that creates the error, but different errors for the two versions. See the attached files for detail (I renamed my program to be compiled here from O6.jl to O6.txt due to limitations of uploading .jl files)

O6.txt
julia06.txt
julia07.txt

can't use a Module defined within

if i just modify the example hello.jl to contain

module Goodbye
end

using Goodbye

then when i compile it i get:

ERROR: LoadError: ArgumentError: Module Goodbye not found in current path.
Run `Pkg.add("Goodbye")` to install the Goodbye package.

without this modification it works as advertised. is there a trick to get this to work?

0.7 compile time doesn't work as well anymore

I'll soon push a branch with snoopcompile + sysimage working.
Meanwhile, the compilation speed improvements are not as drastic anymore.
I hope (am pretty sure?), this is just some flag we need to figure out.

userimg = Pkg.dir("Makie", "test", "precompile.jl")
image_path = PackageCompiler.sysimg_folder()
PackageCompiler.build_sysimg(image_path, userimg, compilecache = "yes")

precompile.jl generated with julia --trace-compilation=yes runtests.jl.
Speed improvement is just from 80s to 20s, with a precompile file that should definitely have everything needed in it.

Juju: statically compiling the static compiler!

@lucatrv let's branch out the conversation from #76 to here!

Unable to compile LanguageServer. Error: snooped.csv: No such file or directory

I'm not sure if this is an error related to changes in 0.7. When I try to run compile_package("LanguageServer"), I eventually encountered this error:

ERROR: ArgumentError: Package SnoopCompile not found in current path:
- Run `Pkg.add("SnoopCompile")` to install the SnoopCompile package.

Stacktrace:
 [1] require(::Module, ::Symbol) at ./loading.jl:817
 [2] top-level scope at /home/jx/.julia/packages/PackageCompiler/5u3Km/src/snooping.jl:15
 [3] eval at ./boot.jl:319 [inlined]
 [4] eval(::Module, ::Expr) at ./deprecated.jl:55
 [5] top-level scope at ./none:2
done.
ERROR: SystemError: opening file /home/jx/.julia/packages/PackageCompiler/5u3Km/packages/rYKOb/snooped.csv: No such file or directory

I then ran Pkg.add("SnoopCompile"), but on the retry the following error ensued:

ERROR: LoadError: UndefVarError: @snoop1 not defined
Stacktrace:
 [1] eval at ./boot.jl:319 [inlined]
 [2] eval(::Module, ::Expr) at ./deprecated.jl:55
 [3] top-level scope at ./none:2
in expression starting at /home/jx/.julia/packages/PackageCompiler/5u3Km/src/snooping.jl:20
done.
ERROR: SystemError: opening file /home/jx/.julia/packages/PackageCompiler/5u3Km/packages/rYKOb/snooped.csv: No such file or directory
Stacktrace:
 [1] #systemerror#39(::Nothing, ::Function, ::String, ::Bool) at ./error.jl:106
 [2] systemerror at ./error.jl:106 [inlined]
 [3] #open#297(::Nothing, ::Nothing, ::Nothing, ::Nothing, ::Nothing, ::Function, ::String) at ./iostream.jl:283
 [4] open at ./iostream.jl:275 [inlined]
 [5] #eachline#288 at ./io.jl:891 [inlined]
 [6] eachline at ./io.jl:887 [inlined]
 [7] read(::String) at /home/jx/.julia/packages/SnoopCompile/JDKqS/src/SnoopCompile.jl:67
 [8] snoop(::String, ::String, ::String) at /home/jx/.julia/packages/PackageCompiler/5u3Km/src/snooping.jl:30
 [9] (::getfield(PackageCompiler, Symbol("##25#27")))(::Tuple{String,String}) at /home/jx/.julia/packages/PackageCompiler/5u3Km/src/snooping.jl:105
 [10] map at ./tuple.jl:162 [inlined]
 [11] snoop_userimg at /home/jx/.julia/packages/PackageCompiler/5u3Km/src/snooping.jl:94 [inlined]
 [12] #compile_package#37(::Bool, ::Bool, ::Bool, ::Function, ::Tuple{String,String}) at /home/jx/.julia/packages/PackageCompiler/5u3Km/src/PackageCompiler.jl:132
 [13] #compile_package#34 at /home/jx/.julia/packages/PackageCompiler/5u3Km/src/PackageCompiler.jl:130 [inlined]
 [14] compile_package(::String) at /home/jx/.julia/packages/PackageCompiler/5u3Km/src/PackageCompiler.jl:113
 [15] top-level scope at none:0

I'm on Arch Linux and installed Julia 0.7 from the official binary files manually (since the Arch repo already contains version 1.0)

Gadfly plotting (to file) fails to compile: Undefined symbol `_jl_depwarn_partial_indexing`

# gadfly_test.jl

#  Need this block until this is resolved: https://github.com/JuliaLang/PackageCompiler.jl/issues/47
if VERSION < v"0.7"
    try
        JULIA_HOME
    catch
        warn("JULIA_HOME is not defined, initializing manually")
        Sys.__init__()
        Base.early_init()
        JULIA_HOME
    end
end

using Gadfly

Base.@ccallable function julia_main(args::Vector{String})::Cint
    # plot([sin, cos], 0, 25)  #  Compiles but does nothing (no gui).
    draw(SVG("output.svg", 6inch, 3inch), plot([sin, cos], 0, 25))  # Fails to compile
    return 0
end

This fails while linking the shared library, complaining that it can't find a symbol for _jl_depwarn_partial_indexing:

Build shared library "gadfly_test.dylib" in build directory:
  `cc -m64 -shared -o gadfly_test.dylib /Users/daly/Documents/developer/code/deep/JuliaDeep/builddir/gadfly_test.o -std=gnu99 -I/Applications/Julia-0.6.app/Contents/Resources/julia/include/julia -DJULIA_ENABLE_THREADING=1 -fPIC -L/Applications/Julia-0.6.app/Contents/Resources/julia/lib -Wl,-rpath,/Applications/Julia-0.6.app/Contents/Resources/julia/lib -Wl,-rpath,/Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia -ljulia -Wl,-install_name,@rpath/gadfly_test.dylib`
Undefined symbols for architecture x86_64:
  "_jl_depwarn_partial_indexing", referenced from:
      _japi1_setindex!_38741 in gadfly_test.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ERROR: LoadError: failed process: Process(`cc -m64 -shared -o gadfly_test.dylib /Users/daly/Documents/developer/code/deep/JuliaDeep/builddir/gadfly_test.o -std=gnu99 -I/Applications/Julia-0.6.app/Contents/Resources/julia/include/julia -DJULIA_ENABLE_THREADING=1 -fPIC -L/Applications/Julia-0.6.app/Contents/Resources/julia/lib -Wl,-rpath,/Applications/Julia-0.6.app/Contents/Resources/julia/lib -Wl,-rpath,/Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia -ljulia -Wl,-install_name,@rpath/gadfly_test.dylib`, ProcessExited(1)) [1]

So I assume there is some deprecated behavior going on inside Gadfly's plotting somewhere, presumably when indexing something. But is it a bug in julia that it doesn't produce a C symbol for that dependency warning, _jl_depwarn_partial_indexing? Or is there some setting we're not setting to make this work?

Channels don't seem to compile

The error I got was:

Julia program file:
  "/Users/ranjan/graph/static/buzz.jl"
C program file:
  "/Users/ranjan/.julia/v0.6/PackageCompiler/examples/program.c"
Build directory:
  "/Users/ranjan/graph/static/build"
ERROR: LoadError: UndefVarError: uv_eventloop not defined
Stacktrace:
 [1] enq_work(::Task) at ./event.jl:83
 [2] async_run_thunk(::Function) at ./task.jl:323
 [3] include_from_node1(::String) at ./loading.jl:576
 [4] include(::String) at ./sysimg.jl:14
while loading /Users/ranjan/graph/static/buzz.jl, in expression starting on line 3
ERROR: failed process: Process(`/Applications/Julia-0.6.app/Contents/Resources/julia/bin/julia -Ccore2 -J/Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib --compile=yes --depwarn=yes --startup-file=no --output-o /Users/ranjan/graph/static/build/buzz.o -e 'empty!(Base.LOAD_CACHE_PATH) # reset / remove any builtin paths
        push!(Base.LOAD_CACHE_PATH, abspath("/Users/ranjan/graph/static/build")) # enable usage of precompiled files
        include("/Users/ranjan/graph/static/buzz.jl") # include Julia program file
        empty!(Base.LOAD_CACHE_PATH) # reset / remove build-system-relative paths'`, ProcessExited(1)) [1]
Stacktrace:
 [1] pipeline_error(::Base.Process) at ./process.jl:682
 [2] run(::Cmd) at ./process.jl:651
 [3] build_object(::String, ::String, ::String, ::Bool, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void) at /Users/ranjan/.julia/v0.6/PackageCompiler/src/static_julia.jl:221
 [4] #static_julia#1(::String, ::String, ::String, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::String, ::Void, ::PackageCompiler.#static_julia, ::String) at /Users/ranjan/.julia/v0.6/PackageCompiler/src/static_julia.jl:127
 [5] (::PackageCompiler.#kw##static_julia)(::Array{Any,1}, ::PackageCompiler.#static_julia, ::String) at ./<missing>:0
 [6] build_executable(::String) at /Users/ranjan/.julia/v0.6/PackageCompiler/src/api.jl:84

Here's a minimum reproducible script:

const ch = Channel{Any}(32)
 @async ch
 
 Base.@ccallable function julia_main(ARGS::Vector{String})::Cint
     return 0
 end

How to compile a driver program or a single executable with Julia 0.7

Question: I thought that program.c and program2.c in static-julia were intended to build respectively a driver program or a single executable with Julia 0.7 (see README.md). I have not been able to build a single executable successfully so far, so I could not test this though. Now that the old program.c is gone, I wonder how it will be possible to select either one of the two outputs with Julia 0.7.

Still some problem with force=true

it still doesn't work on windows or mac.
Last I tried it complained about missing sys.o and reverted to some state which I suspect is
not the one I started with.

However PackageCompiler.revert() seems to work and doesn't have any problems copying the files
or locating them

Building executable and then shipping

Hi Simon,

I was able to build executables for my code, but I have a few external dependencies (Blink and Gzip) as well. When I build an executable, it seems to me that everything is properly linked to the external dependencies and the executable can run on my computer. But it fails when I copy it over to another computer, because the dependencies are linked but not shipped.

Is there a way we can fix this? I would also love to hear about your progress trying to ship Makie like we discussed.

cc: @ViralBShah

Check packages before compile them

If you try to compile various packages at the same time or packages with a lot of dependencies, the slow compilation process starts to sometimes fail at some point (after minutes of compilation).
It would be great if compile_package could be able to check the list of packages and their dependencies to see if they can be compiled (to avoid some errors, e.g. packages that cannot be precompiled).
Also, it would be great if that function could catch common problems (e.g. non-const globals).

Best regards,

Can't install package

I am on PackageCompiler v0.4.0.

viralbshah@arctic1:~/julia-1.0.0$ ./bin/julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.0.0 (2018-08-08)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(v1.0) pkg> add PackageCompiler
   Cloning default registries into /home/viralbshah/.julia/registries
   Cloning registry General from "https://github.com/JuliaRegistries/General.git"
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
 Installed SnoopCompile ──── v0.3.0
 Installed PackageCompiler ─ v0.4.0
 Installed LibExpat ──────── v0.5.0
 Installed BinaryProvider ── v0.4.2
 Installed ArgParse ──────── v0.6.0
 Installed Libz ──────────── v1.0.0
 Installed URIParser ─────── v0.4.0
 Installed Compat ────────── v1.1.0
 Installed TextWrap ──────── v0.3.0
 Installed WinRPM ────────── v0.4.2
 Installed BinDeps ───────── v0.8.10
 Installed BufferedStreams ─ v1.0.0
 Installed HTTPClient ────── v0.2.1
 Installed LibCURL ───────── v0.3.2
  Updating `~/.julia/environments/v1.0/Project.toml`
  [9b87118b] + PackageCompiler v0.4.0
  Updating `~/.julia/environments/v1.0/Manifest.toml`
  [c7e460c6] + ArgParse v0.6.0
  [9e28174c] + BinDeps v0.8.10
  [b99e7846] + BinaryProvider v0.4.2
  [e1450e63] + BufferedStreams v1.0.0
  [34da2185] + Compat v1.1.0
  [0862f596] + HTTPClient v0.2.1
  [b27032c2] + LibCURL v0.3.2
  [522f3ed2] + LibExpat v0.5.0
  [2ec943e9] + Libz v1.0.0
  [9b87118b] + PackageCompiler v0.4.0
  [aa65fe97] + SnoopCompile v0.3.0
  [b718987f] + TextWrap v0.3.0
  [30578b45] + URIParser v0.4.0
  [c17dfb99] + WinRPM v0.4.2
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8bb1440f] + DelimitedFiles
  [8ba89e20] + Distributed
  [b77e0a4c] + InteractiveUtils
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [a63ad114] + Mmap
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [1a1011a3] + SharedArrays
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  Building LibCURL ────────→ `~/.julia/packages/LibCURL/Z32eb/deps/build.log`
  Building WinRPM ─────────→ `~/.julia/packages/WinRPM/Y9QdZ/deps/build.log`
  Building PackageCompiler → `~/.julia/packages/PackageCompiler/5u3Km/deps/build.log`
┌ Error: Error building `PackageCompiler`:
│ ERROR: LoadError: syntax: try without catch or finally
│ Stacktrace:
│  [1] include at ./boot.jl:317 [inlined]
│  [2] include_relative(::Module, ::String) at ./loading.jl:1038
│  [3] include(::Module, ::String) at ./sysimg.jl:29
│  [4] include(::String) at ./client.jl:388
│  [5] top-level scope at none:0
│ in expression starting at /home/viralbshah/.julia/packages/PackageCompiler/5u3Km/deps/build.jl:4
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1068

Problem with force=true due to missing .o .ji files

The build process seems to be working right , but when I set force = True
it tries to copy "sys.o" "sys.ji", "inference.o", "inference.ji" from PackageCompiler sysimg folder
to the system sys_img folder.
All of these files are missing.
I located sys.o in tmp_v0.6.2 folder but the rest are missing .

The only file that is present is sys.dylib , which If I copy manually to the system sysimg folder seems
to be working.

Trying to compile Plots dependencies

I am trying to compile Plots dependencies to make Plots a little faster. The first problem that I found was that compiling all the packages in REQUIRE throws an error because GR cannot be compiled:

compile_package("RecipesBase",
                "PlotUtils",
                "PlotThemes",
                "Reexport",
                "StaticArrays",
                "FixedPointNumbers",
                "Measures",
                "Showoff",
                "StatsBase",
                "JSON",
                "NaNMath",
                "Requires",
                "Contour",
                "GR",
                force = false,
                reuse = false)

signal (11): Segmentation fault
while loading /home/zea/.julia/v0.6/PackageCompiler/sysimg/precompile.jl, in expression starting on line 91377

ERROR: failed process: Process(`/home/zea/bin/julia-d55cadc350/bin/julia -Cnative -J/home/zea/.julia/v0.6/PackageCompiler/sysimg/backup/native/sys.so --compile=yes --depwarn=yes --startup-file=no --precompiled=no --compilecache=no --output-o /home/zea/.julia/v0.6/PackageCompiler/sysimg/sys.o -e 'empty!(Base.LOAD_CACHE_PATH) # reset / remove any builtin paths
        push!(Base.LOAD_CACHE_PATH, abspath("/home/zea/.julia/v0.6/PackageCompiler/sysimg")) # enable usage of precompiled files
        include("/home/zea/.julia/v0.6/PackageCompiler/sysimg/precompile.jl") # include Julia program file
        empty!(Base.LOAD_CACHE_PATH) # reset / remove build-system-relative paths'`, ProcessSignaled(11)) [0]
Stacktrace:
 [1] pipeline_error(::Base.Process) at ./process.jl:682
 [2] run(::Cmd) at ./process.jl:651
 [3] build_object(::String, ::String, ::String, ::Bool, ::String, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void) at /home/zea/.julia/v0.6/PackageCompiler/src/static_julia.jl:217
 [4] #static_julia#1(::Void, ::String, ::String, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::String, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::String, ::Void, ::PackageCompiler.#static_julia, ::String) at /home/zea/.julia/v0.6/PackageCompiler/src/static_julia.jl:127
 [5] (::PackageCompiler.#kw##static_julia)(::Array{Any,1}, ::PackageCompiler.#static_julia, ::String) at ./<missing>:0
 [6] #build_sysimg#6(::Bool, ::Bool, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::PackageCompiler.#build_sysimg, ::String, ::String) at /home/zea/.julia/v0.6/PackageCompiler/src/api.jl:22
 [7] #compile_package#25(::Bool, ::Bool, ::Bool, ::Function, ::Tuple{String,String}, ::Vararg{Tuple{String,String},N} where N) at /home/zea/.julia/v0.6/PackageCompiler/src/PackageCompiler.jl:134
 [8] (::PackageCompiler.#kw##compile_package)(::Array{Any,1}, ::PackageCompiler.#compile_package, ::Tuple{String,String}, ::Vararg{Tuple{String,String},N} where N) at ./<missing>:0
 [9] #compile_package#22(::Array{Any,1}, ::Function, ::String, ::Vararg{String,N} where N) at /home/zea/.julia/v0.6/PackageCompiler/src/PackageCompiler.jl:114
 [10] (::PackageCompiler.#kw##compile_package)(::Array{Any,1}, ::PackageCompiler.#compile_package, ::String, ::Vararg{String,N} where N) at ./<missing>:0

I had bad luck of having GR at the end, so it took one hour just to fail (#74).

So, after that, I'd tried it without GR and PackageCompiler ended well.

compile_package("RecipesBase",
                "PlotUtils",
                "PlotThemes",
                "Reexport",
                "StaticArrays",
                "FixedPointNumbers",
                "Measures",
                "Showoff",
                "StatsBase",
                "JSON",
                "NaNMath",
                "Requires",
                "Contour",
                force = false,
                reuse = false)

[...a lot of lines...]

julia_cmd = `/home/zea/bin/julia-d55cadc350/bin/julia -Cnative -J/home/zea/.julia/v0.6/PackageCompiler/sysimg/backup/native/sys.so --compile=yes --depwarn=yes --startup-file=no --precompiled=no --compilecache=no`
done.
INFO: import StaticArrays
INFO: Precompiling module Contour.
INFO: import Contour
INFO: import Compat
WARNING: Module not found: VerticesTests
INFO: import Base
WARNING: Module not found: InterfaceTests
INFO: import Core
WARNING: Not emitted because code couldn't parse: precompile(Tuple{typeof(Base.==), Type{Int}, Type{Int}})
Julia program file:
  "/home/zea/.julia/v0.6/PackageCompiler/sysimg/precompile.jl"
Build directory:
  "/home/zea/.julia/v0.6/PackageCompiler/sysimg"
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
All done
INFO:     Not replacing system image.
    You can start julia with julia -J /home/zea/.julia/v0.6/PackageCompiler/sysimg/sys.so to load the compiled files.
4772.625897 seconds (10.39 M allocations: 794.277 MiB, 0.02% gc time)
"/home/zea/.julia/v0.6/PackageCompiler/sysimg/sys.so"

However, if I use the new image I get a Segmentation fault as soon as I try to use Plots:

julia> using Compat

julia> using Plots
INFO: Precompiling module Plots.

signal (11): Segmentation fault
while loading no file, in expression starting on line 0
jl_typemap_entry_assoc_exact at /buildworker/worker/package_linux64/build/src/typemap.c:774
jl_typemap_assoc_exact at /buildworker/worker/package_linux64/build/src/julia_internal.h:893 [inlined]
jl_lookup_generic_ at /buildworker/worker/package_linux64/build/src/gf.c:1871 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:1923
julia_cmd at ./util.jl:604
create_expr_cache at ./loading.jl:626
compilecache at ./loading.jl:709
_require at ./loading.jl:497
require at ./loading.jl:405
unknown function (ip: 0x7fcd54ceeebb)
jl_call_fptr_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:1926
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1424 [inlined]
eval_import_path_ at /buildworker/worker/package_linux64/build/src/toplevel.c:403
eval_import_path at /buildworker/worker/package_linux64/build/src/toplevel.c:430 [inlined]
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:495
jl_toplevel_eval_in at /buildworker/worker/package_linux64/build/src/builtins.c:496
eval at ./boot.jl:235
unknown function (ip: 0x7fcd53ff9aef)
jl_call_fptr_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:1926
eval_user_input at ./REPL.jl:66
unknown function (ip: 0x7fcd47ea1e66)
jl_call_fptr_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:1926
macro expansion at ./REPL.jl:97 [inlined]
#1 at ./event.jl:73
unknown function (ip: 0x7fcd47e9fabf)
jl_call_fptr_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:1926
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1424 [inlined]
start_task at /buildworker/worker/package_linux64/build/src/task.c:267
unknown function (ip: 0xffffffffffffffff)
Allocations: 2303594 (Pool: 2295404; Big: 8190); GC: 0
Segmentation fault (core dumped)

Julia version:

Julia Version 0.6.3
Commit d55cadc350 (2018-05-28 20:20 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU           E5606  @ 2.13GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Nehalem)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, westmere)

Package versions:

2 required packages:
 - PackageCompiler               0.3.1
 - Plots                         0.17.1
24 additional packages:
 - ArgParse                      0.5.0
 - ColorTypes                    0.6.7
 - Colors                        0.8.2
 - Compat                        0.66.0
 - Contour                       0.4.0
 - DataStructures                0.8.3
 - FixedPointNumbers             0.4.6
 - GR                            0.31.0
 - JSON                          0.17.2
 - Measures                      0.2.0
 - Missings                      0.2.9
 - NaNMath                       0.3.1
 - Nullables                     0.0.5
 - PlotThemes                    0.2.0
 - PlotUtils                     0.4.4
 - RecipesBase                   0.3.1
 - Reexport                      0.1.0
 - Requires                      0.4.3
 - Showoff                       0.2.0
 - SnoopCompile                  0.2.0
 - SortingAlgorithms             0.2.1
 - StaticArrays                  0.7.0
 - StatsBase                     0.23.0
 - TextWrap                      0.2.0

Best regards,

juliac not working

When I run juliac on my script, I get the following errors:

julia juliac.jl -vae monroe21_exp.jl
Julia program file:
"/Users/a598124/.julia/v0.6/PackageCompiler/monroe21_exp.jl"
C program file:
"/Users/a598124/.julia/v0.6/PackageCompiler/src/../examples/program.c"
Build directory:
"/Users/a598124/.julia/v0.6/PackageCompiler/builddir"
Change to build directory
Build module image files ".ji" in subdirectory "tmp_v0.6.2":
/Applications/Julia-0.6.app/Contents/Resources/julia/bin/julia -Cnative -J/Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib --compile=yes --depwarn=yes --startup-file=no -e 'empty!(Base.LOAD_CACHE_PATH) # reset / remove any builtin paths push!(Base.LOAD_CACHE_PATH, abspath("tmp_v0.6.2")) # enable usage of precompiled files include("/Users/a598124/.julia/v0.6/PackageCompiler/monroe21_exp.jl") # include "julia_program" file empty!(Base.LOAD_CACHE_PATH) # reset / remove build-system-relative paths'
Build object file "monroe21_exp.o" in subdirectory "tmp_v0.6.2":
/Applications/Julia-0.6.app/Contents/Resources/julia/bin/julia -Cnative -J/Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib --compile=yes --depwarn=yes --startup-file=no --output-o tmp_v0.6.2/monroe21_exp.o -e 'empty!(Base.LOAD_CACHE_PATH) # reset / remove any builtin paths push!(Base.LOAD_CACHE_PATH, abspath("tmp_v0.6.2")) # enable usage of precompiled files include("/Users/a598124/.julia/v0.6/PackageCompiler/monroe21_exp.jl") # include "julia_program" file empty!(Base.LOAD_CACHE_PATH) # reset / remove build-system-relative paths'
Build shared library "monroe21_exp.dylib" in build directory:
cc -m64 -shared -o monroe21_exp.dylib tmp_v0.6.2/monroe21_exp.o -std=gnu99 -I/Applications/Julia-0.6.app/Contents/Resources/julia/include/julia -DJULIA_ENABLE_THREADING=1 -fPIC -L/Applications/Julia-0.6.app/Contents/Resources/julia/lib -Wl,-rpath,/Applications/Julia-0.6.app/Contents/Resources/julia/lib -Wl,-rpath,/Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia -ljulia -Wl,-install_name,@rpath/monroe21_exp.dylib
Building executable "monroe21_exp" in build directory:
cc -m64 '-DJULIAC_PROGRAM_LIBNAME="monroe21_exp.dylib"' -o monroe21_exp /Users/a598124/.julia/v0.6/PackageCompiler/src/../examples/program.c monroe21_exp.dylib -std=gnu99 -I/Applications/Julia-0.6.app/Contents/Resources/julia/include/julia -DJULIA_ENABLE_THREADING=1 -fPIC -L/Applications/Julia-0.6.app/Contents/Resources/julia/lib -Wl,-rpath,/Applications/Julia-0.6.app/Contents/Resources/julia/lib -Wl,-rpath,/Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia -ljulia -Wl,-rpath,@executable_path
/Users/a598124/.julia/v0.6/PackageCompiler/src/../examples/program.c:32:62: warning:
incompatible pointer types passing 'jl_datatype_t *' (aka 'struct
_jl_datatype_t *') to parameter of type 'jl_value_t *' (aka
'struct _jl_value_t *') [-Wincompatible-pointer-types]
...*ARGS = jl_alloc_array_1d(jl_apply_array_type(jl_string_type, 1), 0);
^~~~~~~~~~~~~~
/Applications/Julia-0.6.app/Contents/Resources/julia/include/julia/julia.h:1199:58: note:
passing argument to parameter 'type' here
JL_DLLEXPORT jl_value_t *jl_apply_array_type(jl_value_t *type, size_t dim);
^
1 warning generated.
Undefined symbols for architecture x86_64:
"_julia_main", referenced from:
_main in program-7c3c8b.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ERROR: LoadError: failed process: Process(cc -m64 '-DJULIAC_PROGRAM_LIBNAME="monroe21_exp.dylib"' -o monroe21_exp /Users/a598124/.julia/v0.6/PackageCompiler/src/../examples/program.c monroe21_exp.dylib -std=gnu99 -I/Applications/Julia-0.6.app/Contents/Resources/julia/include/julia -DJULIA_ENABLE_THREADING=1 -fPIC -L/Applications/Julia-0.6.app/Contents/Resources/julia/lib -Wl,-rpath,/Applications/Julia-0.6.app/Contents/Resources/julia/lib -Wl,-rpath,/Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia -ljulia -Wl,-rpath,@executable_path, ProcessExited(1)) [1]
Stacktrace:
[1] pipeline_error(::Base.Process) at ./process.jl:682
[2] run(::Cmd) at ./process.jl:651
[3] compile_executable(::String, ::String, ::String, ::Bool, ::Void, ::Void) at /Users/a598124/.julia/v0.6/PackageCompiler/src/static_julia.jl:179
[4] #julia_compile#1(::String, ::Void, ::String, ::Bool, ::Bool, ::Bool, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::String, ::PackageCompiler.#julia_compile, ::String) at /Users/a598124/.julia/v0.6/PackageCompiler/src/static_julia.jl:124
[5] (::PackageCompiler.#kw##julia_compile)(::Array{Any,1}, ::PackageCompiler.#julia_compile, ::String) at ./:0
[6] julia_main(::Array{String,1}) at /Users/a598124/.julia/v0.6/PackageCompiler/juliac.jl:121
[7] include_from_node1(::String) at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib:?
[8] include(::String) at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib:?
[9] process_options(::Base.JLOptions) at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib:?
[10] _start() at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib:?
while loading /Users/a598124/.julia/v0.6/PackageCompiler/juliac.jl, in expression starting on line 128

PyPlot attempt

when doing force=true, I meet the result

WARNING: An error has occured while replacing sysimg files:
ERROR: open: no such file or directory (ENOENT)
INFO: Recovering old system image from backup
WARNING: No backup found but restoring. Need to build a new system image from scratch
ERROR: MethodError: no method matching build_sysimg(::String)
Closest candidates are:
  build_sysimg(::Any, ::Any, ::Any; debug) at /Users/abradley/.julia/v0.6/PackageCompiler/src/build_sysimg.jl:58
Stacktrace:
 [1] revert(::Bool) at /Users/abradley/.julia/v0.6/PackageCompiler/src/PackageCompiler.jl:79
 [2] #compile_package#19(::Bool, ::Bool, ::Bool, ::Function, ::Tuple{String,String}, ::Vararg{Tuple{String,String},N} where N) at /Users/abradley/.julia/v0.6/PackageCompiler/src/PackageCompiler.jl:181
 [3] (::PackageCompiler.#kw##compile_package)(::Array{Any,1}, ::PackageCompiler.#compile_package, ::Tuple{String,String}, ::Vararg{Tuple{String,String},N} where N) at ./<missing>:0
 [4] #compile_package#12(::Array{Any,1}, ::Function, ::String, ::Vararg{String,N} where N) at /Users/abradley/.julia/v0.6/PackageCompiler/src/PackageCompiler.jl:131
 [5] (::PackageCompiler.#kw##compile_package)(::Array{Any,1}, ::PackageCompiler.#compile_package, ::String, ::Vararg{String,N} where N) at ./<missing>:0

Avoid calling `julia-config.jl` from `static_julia.jl`

I think we should avoid calling julia-config.jl with Base.shell_split(read(etc etc)) in static_julia.jl. I have discussed this in the past. Of course we should also avoid code duplication in julia-config.jl and PackageCompiler.jl. We could however for now copy functions cflags, ldflags, ldlibs, allflags into PackageCompiler.jl, if we knew that PackageCompiler.jl is going to be added to Julia stdlib, so that julia-config.jl could be modified to use it at a later stage.
What do you think?

Using julia shared libraries as python C exstensions

I will try to post details of my progress here, in the hope to have a few more eyes looking at the problems I encounter.
Here is a start: https://discourse.julialang.org/t/calling-julia-shared-library-from-python-linking-libraries/9169

Let's hope we can do this quickly, this feature would be so awesome! :)

Failed to compile PyCall

Actually the failure is something else, but trying to compile PyCall is the MWE

output is

/Applications/Julia-0.6.app/Contents/Resources/julia/share/julia/base/precompile.jl
LoadError("sysimg.jl", 411, LoadError("/Applications/Julia-0.6.app/Contents/Resources/julia/share/julia/base/userimg.jl", 3, LoadError("/Users/tsurherman/.julia/v0.6/PyCall/src/PyCall.jl", 28, LoadError("/Users/tsurherman/.julia/v0.6/Conda/src/Conda.jl", 309, LoadError("/Users/tsurherman/.julia/v0.6/Conda/src/bindeps_conda.jl", 2, LoadError("/Users/tsurherman/.julia/v0.6/BinDeps/src/BinDeps.jl", 10, UndefVarError(:JULIA_HOME)))))))

What do you make of it?

test errors

Pkg.test("PackageCompiler")
...
docs/helpdb/Base.jl
docs/basedocs.jl
markdown/Markdown.jl
docs/Docs.jl
/Applications/Julia-0.6.app/Contents/Resources/julia/share/julia/base/precompile.jl
LoadError("sysimg.jl", 411, LoadError("/Applications/Julia-0.6.app/Contents/Resources/julia/share/julia/base/userimg.jl", 2, ArgumentError("Module Matcha not found in current path.\nRun `Pkg.add(\"Matcha\")` to install the Matcha package.")))
ERROR: LoadError: failed process: Process(`/Applications/Julia-0.6.app/Contents/Resources/julia/bin/julia -C native --output-ji /Users/abradley/.julia/v0.6/PackageCompiler/sysimg/sys.ji --output-o /Users/abradley/.julia/v0.6/PackageCompiler/sysimg/sys.o -J /Users/abradley/.julia/v0.6/PackageCompiler/sysimg/inference.ji --startup-file=no sysimg.jl`, ProcessExited(1)) [1]
Stacktrace:
 [1] pipeline_error(::Base.Process) at ./process.jl:682
 [2] run(::Cmd) at ./process.jl:651
 [3] (::PackageCompiler.##2#3{Bool,String,String,String,String})() at /Users/abradley/.julia/v0.6/PackageCompiler/src/build_sysimg.jl:89
 [4] cd(::PackageCompiler.##2#3{Bool,String,String,String,String}, ::String) at ./file.jl:70
 [5] #build_sysimg#1(::Bool, ::Function, ::String, ::String, ::String) at /Users/abradley/.julia/v0.6/PackageCompiler/src/build_sysimg.jl:59
 [6] #compile_package#19(::Bool, ::Bool, ::Bool, ::Function, ::Tuple{String,String}, ::Vararg{Tuple{String,String},N} where N) at /Users/abradley/.julia/v0.6/PackageCompiler/src/PackageCompiler.jl:169
 [7] (::PackageCompiler.#kw##compile_package)(::Array{Any,1}, ::PackageCompiler.#compile_package, ::Tuple{String,String}, ::Vararg{Tuple{String,String},N} where N) at ./<missing>:0
 [8] #compile_package#12(::Array{Any,1}, ::Function, ::String, ::Vararg{String,N} where N) at /Users/abradley/.julia/v0.6/PackageCompiler/src/PackageCompiler.jl:131
 [9] (::PackageCompiler.#kw##compile_package)(::Array{Any,1}, ::PackageCompiler.#compile_package, ::String, ::Vararg{String,N} where N) at ./<missing>:0
 [10] include_from_node1(::String) at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 [11] include(::String) at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 [12] process_options(::Base.JLOptions) at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 [13] _start() at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib:?
while loading /Users/abradley/.julia/v0.6/PackageCompiler/test/runtests.jl, in expression starting on line 8
===========================[ ERROR: PackageCompiler ]===========================

failed process: Process(`/Applications/Julia-0.6.app/Contents/Resources/julia/bin/julia -Cnative -J/Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib --compile=yes --depwarn=yes --check-bounds=yes --code-coverage=none --color=yes --compilecache=yes /Users/abradley/.julia/v0.6/PackageCompiler/test/runtests.jl`, ProcessExited(1)) [1]

================================================================================
INFO: No packages to install, update or remove
ERROR: PackageCompiler had test errors

Why is this code _still_ seeing a wrong value for `JULIA_HOME`?

Creating this issue from #84.

Somehow, this function:
https://github.com/JuliaLang/julia/blob/v0.6.3/base/distributed/cluster.jl#L380-L385
still has the wrong value for JULIA_HOME when statically compiled into a binary:

shell> cd ~/tmp/
julia> mktemp() do f,_
                  write(f, "Base.@ccallable function julia_main(ARGS::Vector{String})::Cint
                              println(Base.JULIA_HOME)
                              println(Base.julia_exename())
                              println(Base.julia_cmd())
                              println(Base.Distributed.default_addprocs_params())
                              return 0
                            end")
                  build_executable(f)
                  run(`$(joinpath("builddir", basename(f)))`)
       end
...
All done
WARNING: redefining constant JULIA_HOME
/Applications/Julia-0.6.app/Contents/Resources/julia/bin
julia
`/Applications/Julia-0.6.app/Contents/Resources/julia/bin/julia -Cgeneric -J/Users/daly/tmp/builddir/tmpAsWEFE.dylib --compile=yes --depwarn=yes`
Dict{Any,Any}(Pair{Any,Any}(:topology, :all_to_all),Pair{Any,Any}(:exeflags, ``),Pair{Any,Any}(:enable_threaded_blas, false),Pair{Any,Any}(:exename, "/Users/daly/tmp/builddir/julia"),Pair{Any,Any}(:dir, "/Users/daly/tmp"))

Is it somehow being compiled before we overwrite JULIA_HOME manually?

Cut a version before JuliaCon?

Hi! Would it be possible to tag and release a version of PackageCompiler before JuliaCon?

Specifically, now that the static_julia parts work on julia v0.7, it would be really nice to have that be running live 😊.
Sorry I haven't looked at getting the other parts of this package working on v0.7 yet, though...

Thaaanks for the consideration. I'm sure it's a busy time for everyone, so no pressure either way. :)

compile_package not defined

I get the error below:

julia pc_driver.jl
ERROR: LoadError: UndefVarError: compile_package not defined
Stacktrace:
[1] include_from_node1(::String) at ./loading.jl:576
[2] include(::String) at ./sysimg.jl:14
[3] process_options(::Base.JLOptions) at ./client.jl:305
[4] _start() at ./client.jl:371
while loading /Users/a598124/juliaCode/boxPacking1/pc_driver.jl, in expression starting on line 5

What's the relation between BuildExecutable.jl and PackageCompiler.jl?

Hey,
so far the question is in the heading.
But I'm curious what's the current state of full standalone executables.

Thus, would the compiled executables work on systems entirely without a julia installation?
What are the current filesizes? Are there still those problems of heavily oversized binaries as they were for BuildExecutable?
Is it possible to include other binaries?
And my last question so far would be, to what extent cross compilation already works.

Thanks for making this lib!!

compiling a new sysimg should use vanilla sysimg as base

If I am not wrong, from the look of the precompile lists , it seems to me that
if I build a package into the sysimg and then at a later date update that package and therefore would like to build the package again into the sysimg.. well since the previous sysimg gets loaded into the new snooping process , the end result is a kind of cumulative compilation .. which is not what I intended

Way to specify target architecture for compile_package?

I would like to generate a compiled sysimg for a wider target architecture than the default choice. Currently I have modified compile_package here to pass the desired cpu_target, but it would be good to have this supported in the main package.

I would file a PR but I am not sure which flags you would think to expose or the best way to organize it.

can't compile Makie.jl on MacOS

Not sure if this is supposed to work, but, even though Pkg.test("AbstractPlotting") passes, I get

julia> using PackageCompiler

julia> compile_package("Makie", force = false, reuse = false)
julia_cmd = `/Applications/Julia-0.6.app/Contents/Resources/julia/bin/julia -Cnative -J/Users/abradley/.julia/v0.6/PackageCompiler/sysimg/backup/native/sys.dylib --compile=yes --depwarn=yes --compilecache=no --startup-file=no`
WARNING: Method definition set_theme!() in module AbstractPlotting at /Users/abradley/.julia/v0.6/AbstractPlotting/src/scenes.jl:233 overwritten at /Users/abradley/.julia/v0.6/AbstractPlotting/src/scenes.jl:238.
ERROR: LoadError: LoadError: LoadError: LoadError: LoadError: LoadError: LoadError: TypeError: _setindex!: in typeassert, expected Array{ColorTypes.RGBA{Float64},1}, got Array{ColorTypes.RGBA,1}
Stacktrace:
 [1] _setindex!(::Dict{Symbol,Array{ColorTypes.RGBA{Float64},1}}, ::Array{ColorTypes.RGBA,1}, ::Symbol, ::Int64) at ./dict.jl:396
 [2] setindex!(::Dict{Symbol,Array{ColorTypes.RGBA{Float64},1}}, ::Array{ColorTypes.RGBA,1}, ::Symbol) at ./dict.jl:428
 [3] convert(::Type{Dict{Symbol,Array{ColorTypes.RGBA{Float64},1}}}, ::Dict{Symbol,Any}) at ./dict.jl:201
 [4] PlotUtils.ColorLibrary(::Dict{Symbol,Symbol}, ::Dict{Symbol,Any}) at /Users/abradley/.julia/v0.6/PlotUtils/src/color_gradients.jl:5
 [5] include_from_node1(::String) at ./loading.jl:576
 [6] include(::String) at /Users/abradley/.julia/v0.6/PackageCompiler/sysimg/backup/native/sys.dylib:?
 [7] include_from_node1(::String) at ./loading.jl:576
 [8] eval(::Module, ::Any) at /Users/abradley/.julia/v0.6/PackageCompiler/sysimg/backup/native/sys.dylib:?
 [9] _require(::Symbol) at ./loading.jl:490
 [10] require(::Symbol) at ./loading.jl:405
 [11] include_from_node1(::String) at ./loading.jl:576
 [12] include(::String) at /Users/abradley/.julia/v0.6/PackageCompiler/sysimg/backup/native/sys.dylib:?
 [13] include_from_node1(::String) at ./loading.jl:576
 [14] eval(::Module, ::Any) at /Users/abradley/.julia/v0.6/PackageCompiler/sysimg/backup/native/sys.dylib:?
 [15] _require(::Symbol) at ./loading.jl:490
 [16] require(::Symbol) at ./loading.jl:405
 [17] include_from_node1(::String) at ./loading.jl:576
 [18] eval(::Module, ::Any) at /Users/abradley/.julia/v0.6/PackageCompiler/sysimg/backup/native/sys.dylib:?
 [19] _require(::Symbol) at ./loading.jl:490
 [20] require(::Symbol) at ./loading.jl:405
 [21] include_from_node1(::String) at ./loading.jl:576
 [22] include(::String) at /Users/abradley/.julia/v0.6/PackageCompiler/sysimg/backup/native/sys.dylib:?
 [23] include_from_node1(::String) at /Users/abradley/.julia/v0.6/PackageCompiler/sysimg/backup/native/sys.dylib:?
 [24] include(::String) at /Users/abradley/.julia/v0.6/PackageCompiler/sysimg/backup/native/sys.dylib:?
 [25] macro expansion at /Users/abradley/.julia/v0.6/SnoopCompile/src/SnoopCompile.jl:71 [inlined]
 [26] macro expansion at /Users/abradley/.julia/v0.6/PackageCompiler/src/snooping.jl:20 [inlined]
 [27] anonymous at ./<missing>:?
 [28] anonymous at ./<missing>:2
while loading /Users/abradley/.julia/v0.6/PlotUtils/src/color_gradients.jl, in expression starting on line 69
while loading /Users/abradley/.julia/v0.6/PlotUtils/src/PlotUtils.jl, in expression starting on line 27
while loading /Users/abradley/.julia/v0.6/AbstractPlotting/src/basic_recipes/axis.jl, in expression starting on line 2
while loading /Users/abradley/.julia/v0.6/AbstractPlotting/src/AbstractPlotting.jl, in expression starting on line 48
while loading /Users/abradley/.julia/v0.6/Makie/src/Makie.jl, in expression starting on line 6
while loading /Users/abradley/.julia/v0.6/Makie/test/../examples/library.jl, in expression starting on line 1
while loading /Users/abradley/.julia/v0.6/Makie/test/runtests.jl, in expression starting on line 3
done.
WARNING: failed parse of line: Tuple{typeof(AbstractPlotting.Plot), Type{AbstractArray{#s276, 1} where #s276<:(Base.Complex{T} where T<:Real)}}
INFO: import Reexport
INFO: import BinaryProvider
INFO: import ImageCore
INFO: import MacroTools
INFO: import FileIO
INFO: import Colors
INFO: import StaticArrays
INFO: Recompiling stale cache file /Users/abradley/.julia/lib/v0.6/Ratios.ji for module Ratios.
INFO: Recompiling stale cache file /Users/abradley/.julia/lib/v0.6/Interpolations.ji for module Interpolations.
INFO: Recompiling stale cache file /Users/abradley/.julia/lib/v0.6/Images.ji for module Images.
INFO: import Images
INFO: import Compat
INFO: import FreeType
INFO: import FixedPointNumbers
INFO: import ImageMetadata
INFO: import Base
INFO: import ImageAxes
INFO: import Core
INFO: import Graphics
INFO: import SpecialFunctions
INFO: import Requires
INFO: import SimpleTraits
INFO: import StatsBase
INFO: import JSON
INFO: import ColorTypes
INFO: import PlotUtils
INFO: import DataStructures
INFO: import AbstractPlotting
INFO: import OffsetArrays
INFO: import FreeTypeAbstraction
INFO: import Reactive
INFO: import ImageFiltering
Julia program file:
  "/Users/abradley/.julia/v0.6/PackageCompiler/sysimg/precompile.jl"
Build directory:
  "/Users/abradley/.julia/v0.6/PackageCompiler/sysimg"
WARNING: redefining constant JULIA_HOME
ERROR: LoadError: LoadError: LoadError: TypeError: _setindex!: in typeassert, expected Array{ColorTypes.RGBA{Float64},1}, got Array{ColorTypes.RGBA,1}
Stacktrace:
 [1] _setindex!(::Dict{Symbol,Array{ColorTypes.RGBA{Float64},1}}, ::Array{ColorTypes.RGBA,1}, ::Symbol, ::Int64) at ./dict.jl:396
 [2] setindex!(::Dict{Symbol,Array{ColorTypes.RGBA{Float64},1}}, ::Array{ColorTypes.RGBA,1}, ::Symbol) at ./dict.jl:428
 [3] convert(::Type{Dict{Symbol,Array{ColorTypes.RGBA{Float64},1}}}, ::Dict{Symbol,Any}) at ./dict.jl:201
 [4] PlotUtils.ColorLibrary(::Dict{Symbol,Symbol}, ::Dict{Symbol,Any}) at /Users/abradley/.julia/v0.6/PlotUtils/src/color_gradients.jl:5
 [5] include_from_node1(::String) at ./loading.jl:576
 [6] include(::String) at ./sysimg.jl:14
 [7] include_from_node1(::String) at ./loading.jl:576
 [8] eval(::Module, ::Any) at ./boot.jl:235
 [9] _require(::Symbol) at ./loading.jl:490
 [10] require(::Symbol) at ./loading.jl:405
 [11] include_from_node1(::String) at ./loading.jl:576
 [12] include(::String) at ./sysimg.jl:14
while loading /Users/abradley/.julia/v0.6/PlotUtils/src/color_gradients.jl, in expression starting on line 69
while loading /Users/abradley/.julia/v0.6/PlotUtils/src/PlotUtils.jl, in expression starting on line 27
while loading /Users/abradley/.julia/v0.6/PackageCompiler/sysimg/precompile.jl, in expression starting on line 25
ERROR: failed process: Process(`/Applications/Julia-0.6.app/Contents/Resources/julia/bin/julia -Cnative -J/Users/abradley/.julia/v0.6/PackageCompiler/sysimg/backup/native/sys.dylib --compile=yes --depwarn=yes --compilecache=no --startup-file=no --output-o sys.o -e '
  empty!(Base.LOAD_CACHE_PATH) # reset / remove any builtin paths
  push!(Base.LOAD_CACHE_PATH, "cache_ji_v0.6.4") # enable usage of precompiled files
  Sys.__init__(); Base.early_init(); # JULIA_HOME is not defined, initializing manually
  include("/Users/abradley/.julia/v0.6/PackageCompiler/sysimg/precompile.jl") # include Julia program file
  empty!(Base.LOAD_CACHE_PATH) # reset / remove build-system-relative paths'`, ProcessExited(1)) [1]
Stacktrace:
 [1] pipeline_error(::Base.Process) at ./process.jl:682
 [2] run(::Cmd) at ./process.jl:651
 [3] cd(::PackageCompiler.##5#7, ::String) at ./file.jl:70
 [4] build_object(::String, ::String, ::String, ::Bool, ::String, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void) at /Users/abradley/.julia/v0.6/PackageCompiler/src/static_julia.jl:250
 [5] #static_julia#1(::Void, ::Bool, ::Bool, ::String, ::String, ::Void, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::Void, ::Bool, ::Bool, ::String, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::PackageCompiler.#static_julia, ::String) at /Users/abradley/.julia/v0.6/PackageCompiler/src/static_julia.jl:151
 [6] (::PackageCompiler.#kw##static_julia)(::Array{Any,1}, ::PackageCompiler.#static_julia, ::String) at ./<missing>:0
 [7] #build_sysimg#20(::Bool, ::Bool, ::Bool, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::PackageCompiler.#build_sysimg, ::String, ::String) at /Users/abradley/.julia/v0.6/PackageCompiler/src/api.jl:24
 [8] #compile_package#37(::Bool, ::Bool, ::Bool, ::Function, ::Tuple{String,String}, ::Vararg{Tuple{String,String},N} where N) at /Users/abradley/.julia/v0.6/PackageCompiler/src/PackageCompiler.jl:136
 [9] (::PackageCompiler.#kw##compile_package)(::Array{Any,1}, ::PackageCompiler.#compile_package, ::Tuple{String,String}, ::Vararg{Tuple{String,String},N} where N) at ./<missing>:0
 [10] #compile_package#34(::Array{Any,1}, ::Function, ::String, ::Vararg{String,N} where N) at /Users/abradley/.julia/v0.6/PackageCompiler/src/PackageCompiler.jl:119
 [11] (::PackageCompiler.#kw##compile_package)(::Array{Any,1}, ::PackageCompiler.#compile_package, ::String, ::Vararg{String,N} where N) at ./<missing>:0

julia> versioninfo()
Julia Version 0.6.4
Commit 9d11f62bcb (2018-07-09 19:09 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin14.5.0)
  CPU: Intel(R) Core(TM) i5-5250U CPU @ 1.60GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell MAX_THREADS=16)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, broadwell)

sysfiles undefined

Hi,

I am trying your Makie precompile solution and I have some troubles. Basically

using PackageCompiler; PackageCompiler.revert()
ERROR: UndefVarError: sysfiles not defined
Stacktrace:
 [1] revert(::Bool) at /Users/rveltz/.julia/v0.6/PackageCompiler/src/PackageCompiler.jl:62
 [2] revert() at /Users/rveltz/.julia/v0.6/PackageCompiler/src/PackageCompiler.jl:59

I looked at your code and it seems sysfiles is undefined...

Maybe there is a quick fix...

Best regards

Blosc library loading error.

I am trying to make portable Win x64 binary using package compiler. Julia source itself uses the following libraries:

using StrPack
using DataFrames
using CSV
using MAT
using ArgParse

Compilation looks like this:

build_executable("navconvert.jl"; builddir="win", cpu_target="x86-64")

Resulting executable works on the machine it was compiled on.

After movement of the whole "win" directory to a computer where Julia is not installed, I obtain the following error:

fatal: error thrown and no exception handler available.
Base.InitError(mod=:Blosc, error=ErrorException("C:\Users\user\.julia\v0.6\Blosc\src\..\deps\usr\bin\libblosc.dll does not exist, Please re-run Pkg.build("Blosc"), and restart Julia."))

It is clear that libblosc.dll is not moved to the build directory at compilation stage, but manual addition of the library to the mentioned directory does not change anything.

The same behavior is observed on Mac OS 10.13 with libblosc.1.14.3.dylib.

Conclusion: it seems that full runtime path to the dll is hardcoded into the built executable.

Julia version 0.6.3, PackageCompiler version from June 17, 2018.

Error tagging new release

The URL of this package does not match that stored in METADATA.jl.
cc: @SimonDanisch

package compiler ,should it compile functions involving Anonymous functions?

I see many lines in precompile.jl containing Symbols for example:
precompile(Tuple{getfield(Main, Symbol("##1#2"))})

Do they need to be included? since they represent compilation of something that is only relevant for
the current session.

ERROR: open: permission denied (EACCES)

I installed Julia in /usr/local, so the following fails.

PackageCompiler.compile_package("TrivialPackage", force = false, reuse = true)

Julia Version 0.6.2
Linux (x86_64-linux-gnu)

using shared libraries from other languages

I'm interested in using this package for creating shared libraries that can be used from Fortran (or any other language that supports calling C-exported shared libraries).

Currently the shared libraries created here do contain the @ccallable functions given in the julia script. But if I'm correct these cannot be used directly because you need to call jl_init first, and anything else mentioned in the embedding docs. This is done in program.c, which is compiled to executable.

If I want to be able to call simple julia functions like Base.@ccallable f(x::Float64)::Float64 = 2.0 * x from a shared library, how would I go about that? Modify the program.c to compile as shared library and provide the initialization functions from libjulia there as well? If anyone could provide a minimal example that would be great.

I tried things like:

extern double f(double a);

#ifdef _OS_WINDOWS_
__declspec(dllexport) __cdecl
#endif
void my_jl_init(void)
{
    jl_init();
}

#ifdef _OS_WINDOWS_
__declspec(dllexport) __cdecl
#endif
double my_f(double a)
{
    return f(a);
}

But while my_jl_init seemed to work, if I do my_f after, it segfaults. Is this more or less the right approach however?

`juliac` fails to compile Blink.jl: `JULIA_HOME not defined`

juliac is failing when I try to compile a file that uses Blink.jl, complaining that LoadError: UndefVarError: JULIA_HOME not defined.

Here is the very simple file I was compiling:

# test/blink_test.jl

using Blink

Base.@ccallable function julia_main(args::Vector{String})::Cint
    win = Window(shell(), Dict(:width=>1100))
    return 0
end

And here's the failure:

  (11:17:03) $ julia ~/.julia/v0.6/PackageCompiler/juliac.jl -vaej test/blink_test.jl ~/.julia/v0.6/PackageCompiler/examples/program.c builddir
Julia program file:
  "/Users/daly/Documents/developer/code/deep/JuliaDeep/test/blink_test.jl"
C program file:
  "/Users/daly/.julia/v0.6/PackageCompiler/examples/program.c"
Build directory:
  "/Users/daly/Documents/developer/code/deep/JuliaDeep/test/builddir"
Make build directory
Change to build directory
Build module image files ".ji" in directory "/Users/daly/Documents/developer/code/deep/JuliaDeep/test/builddir":
  `/Applications/Julia-0.6.app/Contents/Resources/julia/bin/julia -Cnative -J/Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib --compile=yes --depwarn=yes --startup-file=no --precompiled=no --compilecache=no -e 'empty!(Base.LOAD_CACHE_PATH) # reset / remove any builtin paths
        push!(Base.LOAD_CACHE_PATH, abspath("/Users/daly/Documents/developer/code/deep/JuliaDeep/test/builddir")) # enable usage of precompiled files
        include("/Users/daly/Documents/developer/code/deep/JuliaDeep/test/blink_test.jl") # include Julia program file
        empty!(Base.LOAD_CACHE_PATH) # reset / remove build-system-relative paths'`
Build object file "blink_test.o" in directory "/Users/daly/Documents/developer/code/deep/JuliaDeep/test/builddir":
  `/Applications/Julia-0.6.app/Contents/Resources/julia/bin/julia -Cnative -J/Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib --compile=yes --depwarn=yes --startup-file=no --precompiled=no --compilecache=no --output-o /Users/daly/Documents/developer/code/deep/JuliaDeep/test/builddir/blink_test.o -e 'empty!(Base.LOAD_CACHE_PATH) # reset / remove any builtin paths
        push!(Base.LOAD_CACHE_PATH, abspath("/Users/daly/Documents/developer/code/deep/JuliaDeep/test/builddir")) # enable usage of precompiled files
        include("/Users/daly/Documents/developer/code/deep/JuliaDeep/test/blink_test.jl") # include Julia program file
        empty!(Base.LOAD_CACHE_PATH) # reset / remove build-system-relative paths'`
ERROR: LoadError: LoadError: LoadError: LoadError: LoadError: UndefVarError: JULIA_HOME not defined
Stacktrace:
 [1] include_from_node1(::String) at ./loading.jl:576
 [2] eval(::Module, ::Any) at ./boot.jl:235
 [3] _require(::Symbol) at ./loading.jl:490
 [4] require(::Symbol) at ./loading.jl:405
 [5] include_from_node1(::String) at ./loading.jl:576
 [6] include(::String) at ./sysimg.jl:14
 [7] include_from_node1(::String) at ./loading.jl:576
 [8] include(::String) at ./sysimg.jl:14
 [9] include_from_node1(::String) at ./loading.jl:576
 [10] eval(::Module, ::Any) at ./boot.jl:235
 [11] _require(::Symbol) at ./loading.jl:490
 [12] require(::Symbol) at ./loading.jl:405
 [13] include_from_node1(::String) at ./loading.jl:576
 [14] include(::String) at ./sysimg.jl:14
while loading /Users/daly/.julia/v0.6/BinDeps/src/BinDeps.jl, in expression starting on line 10
while loading /Users/daly/.julia/v0.6/Blink/src/AtomShell/install.jl, in expression starting on line 1
while loading /Users/daly/.julia/v0.6/Blink/src/AtomShell/AtomShell.jl, in expression starting on line 9
while loading /Users/daly/.julia/v0.6/Blink/src/Blink.jl, in expression starting on line 11
while loading /Users/daly/Documents/developer/code/deep/JuliaDeep/test/blink_test.jl, in expression starting on line 1
ERROR: LoadError: failed process: Process(`/Applications/Julia-0.6.app/Contents/Resources/julia/bin/julia -Cnative -J/Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib --compile=yes --depwarn=yes --startup-file=no --precompiled=no --compilecache=no --output-o /Users/daly/Documents/developer/code/deep/JuliaDeep/test/builddir/blink_test.o -e 'empty!(Base.LOAD_CACHE_PATH) # reset / remove any builtin paths
        push!(Base.LOAD_CACHE_PATH, abspath("/Users/daly/Documents/developer/code/deep/JuliaDeep/test/builddir")) # enable usage of precompiled files
        include("/Users/daly/Documents/developer/code/deep/JuliaDeep/test/blink_test.jl") # include Julia program file
        empty!(Base.LOAD_CACHE_PATH) # reset / remove build-system-relative paths'`, ProcessExited(1)) [1]
Stacktrace:
 [1] pipeline_error(::Base.Process) at ./process.jl:682
 [2] run(::Cmd) at ./process.jl:651
 [3] build_object(::String, ::String, ::String, ::Bool, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void) at /Users/daly/.julia/v0.6/PackageCompiler/src/static_julia.jl:217
 [4] #static_julia#1(::String, ::String, ::String, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::Void, ::String, ::Void, ::PackageCompiler.#static_julia, ::String) at /Users/daly/.julia/v0.6/PackageCompiler/src/static_julia.jl:127
 [5] (::PackageCompiler.#kw##static_julia)(::Array{Any,1}, ::PackageCompiler.#static_julia, ::String) at ./<missing>:0
 [6] julia_main(::Array{String,1}) at /Users/daly/.julia/v0.6/PackageCompiler/juliac.jl:130
 [7] include_from_node1(::String) at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 [8] include(::String) at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 [9] process_options(::Base.JLOptions) at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib:?
 [10] _start() at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib:?
while loading /Users/daly/.julia/v0.6/PackageCompiler/juliac.jl, in expression starting on line 136
  (11:18:23) $

Function `default_sysimg_path`

@SimonDanisch function default_sysimg_path(true) does not work on my system, neither on Linux or Windows, Julia v0.6 or v0.7. Moreover I am not sure to understand it. Can I update it as follows?

function default_sysimg_path()
        joinpath(dirname(JULIA_BINDIR), "lib", "julia")
end

Fails to compile (link DLL) due to undefined symbol `impresetstkoflw`

I've run PackageCompiler.compile_package on one of my still developing packages. As far as I can tell it didn't even get to my package yet when it failed with:

<...>
deprecated.jl
docs/helpdb.jl
docs\helpdb\Base.jl
docs/basedocs.jl
markdown/Markdown.jl
docs/Docs.jl
C:\Users\alexe\AppData\Local\Julia-0.6.2\share\julia\base\precompile.jl
Build shared library "C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.dll" in build directory:
  `cc -m64 -shared -o 'C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.dll' 'C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.o' -std=gnu99 '-IC:\Users\alexe\AppData\Local\Julia-0.6.2\include\julia' -DJULIA_ENABLE_THREADING=1 '-LC:\Users\alexe\AppData\Local\Julia-0.6.2\bin' -Wl,--stack,8388608 -ljulia -lopenlibm -Wl,--export-all-symbols`
C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.o: In function `julia_pure_eval_call_641':
./inference.jl:1636: undefined reference to `__imp__resetstkoflw'
./inference.jl:1636:(.text+0x2377f): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `__imp__resetstkoflw'
C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.o:./inference.jl:1629: undefined reference to `__imp__resetstkoflw'
C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.o:./inference.jl:1629:(.text+0x28197): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `__imp__resetstkoflw'
C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.o:./inference.jl:1629: undefined reference to `__imp__resetstkoflw'
C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.o:./inference.jl:1629:(.text+0x52c0a): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `__imp__resetstkoflw'
C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.o:./inference.jl:1629: undefined reference to `__imp__resetstkoflw'
C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.o:./inference.jl:1629:(.text+0xa1c2d): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `__imp__resetstkoflw'
C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.o:./inference.jl:1629: undefined reference to `__imp__resetstkoflw'
C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.o:./inference.jl:1629:(.text+0xa3b6c): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `__imp__resetstkoflw'
C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.o:./inference.jl:1629: more undefined references to `__imp__resetstkoflw' follow
C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.o:./inference.jl:1629:(.text+0xaa718): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `__imp__resetstkoflw'
C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.o:./inference.jl:1629:(.text+0xd84ad): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `__imp__resetstkoflw'
C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.o:./inference.jl:1629:(.text+0xf6a9b): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `__imp__resetstkoflw'
C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.o:./inference.jl:1629:(.text+0x12a049): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `__imp__resetstkoflw'
C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.o:./inference.jl:1629:(.text+0x1a9437): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `__imp__resetstkoflw'
C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.o:./inference.jl:1629:(.text+0x1dce22): additional relocation overflows omitted from the output
collect2: error: ld returned 1 exit status
ERROR: failed process: Process(`cc -m64 -shared -o 'C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.dll' 'C:\Users\alexe\.julia\v0.6\PackageCompiler\sysimg\backup\x86-64\sys.o' -std=gnu99 '-IC:\Users\alexe\AppData\Local\Julia-0.6.2\include\julia' -DJULIA_ENABLE_THREADING=1 '-LC:\Users\alexe\AppData\Local\Julia-0.6.2\bin' -Wl,--stack,8388608 -ljulia -lopenlibm -Wl,--export-all-symbols`, ProcessExited(1)) [1]
Stacktrace:
 [1] pipeline_error(::Base.Process) at .\process.jl:682
 [2] run(::Cmd) at .\process.jl:651
 [3] build_shared(::String, ::String, ::Bool, ::Void, ::Void, ::Void) at C:\Users\alexe\.julia\v0.6\PackageCompiler\src\static_julia.jl:231
 [4] (::PackageCompiler.##18#19{Bool,String})() at C:\Users\alexe\.julia\v0.6\PackageCompiler\src\system_image.jl:43
 [5] cd(::PackageCompiler.##18#19{Bool,String}, ::String) at .\file.jl:59
 [6] #compile_system_image#17(::Bool, ::Function, ::String, ::String) at C:\Users\alexe\.julia\v0.6\PackageCompiler\src\system_image.jl:13
 [7] (::PackageCompiler.#kw##compile_system_image)(::Array{Any,1}, ::PackageCompiler.#compile_system_image, ::String, ::String) at .\<missing>:0
 [8] get_backup!(::Bool, ::Void) at C:\Users\alexe\.julia\v0.6\PackageCompiler\src\system_image.jl:60
 [9] snoop_vanilla(::String, ::String) at C:\Users\alexe\.julia\v0.6\PackageCompiler\src\snooping.jl:8
 [10] snoop(::String, ::String, ::String) at C:\Users\alexe\.julia\v0.6\PackageCompiler\src\snooping.jl:29
 [11] (::PackageCompiler.##13#15)(::Tuple{String,String}) at C:\Users\alexe\.julia\v0.6\PackageCompiler\src\snooping.jl:105
 [12] map(::PackageCompiler.##13#15, ::Tuple{Tuple{String,String}}) at .\tuple.jl:158
 [13] snoop_userimg(::String, ::Tuple{String,String}, ::Vararg{Tuple{String,String},N} where N) at C:\Users\alexe\.julia\v0.6\PackageCompiler\src\snooping.jl:94
 [14] #compile_package#25(::Bool, ::Bool, ::Bool, ::Function, ::Tuple{String,String}, ::Vararg{Tuple{String,String},N} where N) at C:\Users\alexe\.julia\v0.6\PackageCompiler\src\PackageCompiler.jl:130
 [15] compile_package(::Tuple{String,String}, ::Vararg{Tuple{String,String},N} where N) at C:\Users\alexe\.julia\v0.6\PackageCompiler\src\PackageCompiler.jl:128
 [16] #compile_package#22(::Array{Any,1}, ::Function, ::String, ::Vararg{String,N} where N) at C:\Users\alexe\.julia\v0.6\PackageCompiler\src\PackageCompiler.jl:114
 [17] compile_package(::String, ::Vararg{String,N} where N) at C:\Users\alexe\.julia\v0.6\PackageCompiler\src\PackageCompiler.jl:108

System info:

Julia Version 0.6.2
Commit d386e40c17* (2017-12-13 18:08 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i5-5200U CPU @ 2.20GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, broadwell)

PackageCompiler version: 0.3.1

`contains` function missing on `julia-1.0.0`

(v1.0) pkg> add PackageCompiler
julia> using PackageCompiler
[ Info: Precompiling PackageCompiler [9b87118b-4619-50d2-8e1e-99f35a4d4d9d]
ERROR: LoadError: UndefVarError: contains not defined

The issue is on src/PackageCompiler.jl:13, the line goes:

const contains07 = contains

Function contains is defined for julia-0.6 but not for >=julia-0.7.

REPL modes do not work after compilation

REPL modes from IPython and Rcall don't work after the packages are compiled with the stable version of PackageCompiler (0.3.1). For example:

# compilation
using PackageCompiler
compile_package("RCall", force = false, reuse = false)

# using the new image
~$ julia -J /home/zea/.julia/v0.6/PackageCompiler/sysimg/sys.so

RCall (compiled) REPL mode doesn't work, $ should show the R> prompt:

P.S.: Also, I get a lot of WARNING: redefining constant JULIA_HOME messages using the compiled image.

juliac.jl: command not found

Hi, I'm attempting to generate an executable file from a Julia source file. Thus, I completed the following steps:

From julia REPL, Pkg.add("PackageCompiler")
From Terminal, juliac.jl -vae fib.j

-bash: juliac.jl: command not found

As a side note, it seems that the entire option set for juliac.jl should be added to julia. I'll speak more on that in another ticket.

attempt to revert meets type error

I tried the revert command, finding

julia> PackageCompiler.revert([debug = false])
ERROR: TypeError: non-boolean (Array{Bool,1}) used in boolean context
Stacktrace:
 [1] default_sysimg_path(::Array{Bool,1}) at /Users/abradley/.julia/v0.6/PackageCompiler/src/system_image.jl:2
 [2] revert(::Array{Bool,1}) at /Users/abradley/.julia/v0.6/PackageCompiler/src/PackageCompiler.jl:66

process spawning on MacOS

I am not sure if this is expected behaviour or not, but once I run PackageCompiler (in particular compiling Makie, but not sure it is a Makie issue), julia keeps spawning new julia processes and throwing this warning:

julia> using IJulia
INFO: Recompiling stale cache file /Users/abradley/.julia/lib/v0.6/IJulia.ji for module IJulia.
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME
WARNING: redefining constant JULIA_HOME

the processes sometimes close themselves, but often they stay open and/or appear to freeze.

julia> versioninfo()
Julia Version 0.6.3
Commit d55cadc350 (2018-05-28 20:20 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin14.5.0)
  CPU: Intel(R) Core(TM) i5-5250U CPU @ 1.60GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, broadwell)

How to get parallelism to work?

This is the file I passed to build_executable

Base.@ccallable function julia_main(ARGS::Vector{String})::Cint
     addprocs(2)
     pmap(x -> println(myid()), 1:2)
     return 0
 end

When I run the executable, I get:

➜  static ./build/buzz
ERROR (unhandled task failure): could not spawn setenv(`/Users/ranjan/graph/static/build/julia -Cgeneric -J/Users/ranjan/graph/static/build/buzz.dylib --compile=yes --depwarn=yes --bind-to 127.0.0.1 --worker XJ4ngyA8sbBGqZpR`; dir="/Users/ranjan/graph/static"): no such file or directory (ENOENT)
Stacktrace:
 [1] _jl_spawn(::String, ::Array{String,1}, ::Ptr{Void}, ::Base.Process, ::Base.DevNullStream, ::Base.PipeEndpoint, ::Base.TTY) at ./process.jl:360
 [2] #376 at ./process.jl:512 [inlined]
 [3] setup_stdio(::Base.##376#377{Cmd}, ::Tuple{Base.DevNullStream,Pipe,Base.TTY}) at ./process.jl:499
 [4] #spawn#375(::Nullable{Base.ProcessChain}, ::Function, ::Cmd, ::Tuple{Base.DevNullStream,Pipe,Base.TTY}) at ./process.jl:511
 [5] (::Base.#kw##spawn)(::Array{Any,1}, ::Base.#spawn, ::Cmd, ::Tuple{Base.DevNullStream,Pipe,Base.TTY}) at ./<missing>:0
 [6] #spawn#372(::Nullable{Base.ProcessChain}, ::Function, ::Base.CmdRedirect, ::Tuple{Base.DevNullStream,Pipe,Base.TTY}) at ./process.jl:392
 [7] spawn(::Base.CmdRedirect, ::Tuple{Base.DevNullStream,Pipe,Base.TTY}) at ./process.jl:392
 [8] open(::Base.CmdRedirect, ::String, ::Base.DevNullStream) at /Users/ranjan/graph/static/build/buzz.dylib:?
 [9] launch(::Base.Distributed.LocalManager, ::Dict{Any,Any}, ::Array{WorkerConfig,1}, ::Condition) at /Users/ranjan/graph/static/build/buzz.dylib:?
 [10] (::Base.Distributed.##31#34{Base.Distributed.LocalManager,Dict{Any,Any},Array{WorkerConfig,1},Condition})() at ./event.jl:73
fatal: error thrown and no exception handler available.
Base.UVError(prefix="could not spawn setenv(`/Users/ranjan/graph/static/build/julia -Cgeneric -J/Users/ranjan/graph/static/build/buzz.dylib --compile=yes --depwarn=yes --bind-to 127.0.0.1 --worker XJ4ngyA8sbBGqZpR`; dir="/Users/ranjan/graph/static")", code=-2)
rec_backtrace at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.dylib (unknown line)
jl_throw at /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.dylib (unknown line)
wait at /Users/ranjan/graph/static/build/buzz.dylib (unknown line)
#addprocs_locked#30 at /Users/ranjan/graph/static/build/buzz.dylib (unknown line)
#addprocs#29 at /Users/ranjan/graph/static/build/buzz.dylib (unknown line)
#addprocs#243 at /Users/ranjan/graph/static/build/buzz.dylib (unknown line)
julia_main at /Users/ranjan/graph/static/build/buzz.dylib (unknown line)
julia_main at /Users/ranjan/graph/static/build/buzz.dylib (unknown line)
main at /Users/ranjan/graph/static/./build/buzz (unknown line)

Error building `PackageCompiler`

I'm trying to Pkg.add("PackageCompiler") but got the below error:

┌ Error: Error building PackageCompiler:
│ ERROR: LoadError: syntax: try without catch or finally

My laptop os MacBook Air, macOS High Sierra, v 10.13.6

compile_package not defined

after the update:

julia> using PackageCompiler

julia> compile_package("Makie", force = false, reuse = false) 
ERROR: UndefVarError: compile_package not defined

julia> versioninfo()
Julia Version 0.6.2
Commit d386e40c17 (2017-12-13 18:08 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin14.5.0)
  CPU: Intel(R) Core(TM) i5-5250U CPU @ 1.60GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, broadwell)

What's the best way to incorporate this into a package?

So that compilation is more complete without the user having to do anything. Is it to include in the the Package.jl file under src?