Building executable and then shipping about packagecompiler.jl HOT 11 CLOSED

And you can download the game i made here!:
https://nhdaly.github.io/MadeThis.html

Direct link:
https://nhdaly.github.io/assets/PaddleBattle.dmg

from packagecompiler.jl.

That'd be great :) It seems like you have a custom logic for dealing with assets and binary dependencies.
I think it'd be great, if we'd also have a generic solution for all assets + paths relative to a package.
@KristofferC, @StefanKarpinski any idea about a way forward here?
I think we just need an oficial way of referencing files inside a package, which we can later redirect when moving the packages.

from packagecompiler.jl.

But I think we could close this Issue, in favor of opening specific issues in that repo? :)

from packagecompiler.jl.

I think I'm on the right track!
I've gotten pretty far by replacing all package relative paths with Pkg.dir(...) and then starting the executable as follows:

withenv("JULIA_PKGDIR" => pwd()) do
       run(`makie.exe`)
end

I had some hickups with ccalls in the __init__ function and the binary loading code is now a bit ugly (in the __init__ put the path to the lib into DL_LOAD_PATH and then just have the lib constant be the name of the shared library).

We will need to rethink what to do with the advertised joinpath(@__FILE__, ...) since that's not doing the right thing anymore.
Pkg.dir is not such a nice option, but I don't really see another nice option, that is in base and can be changed via an environment variable.

from packagecompiler.jl.

But it fails when I copy it over to another computer, because the dependencies are linked but not shipped.

Is there a way we can fix this? I would also love to hear about your progress trying to ship Makie like we discussed.

Hey! i've actually been working on this too, and I've gotten something to work!

I've managed to get it to work with the SDL.jl external library, so Blink and Gzip should be no problems at all!!

I actually wrote a tool to wrap up julia code into a Mac .app! You can find it here:
https://github.com/NHDaly/build-jl-app-bundle

If you're hoping to do the same on windows or linux, the process should be really similar. Perhaps you could start from this and then maybe even send over a PR to add that functionality! :D

It basically just copies the libraries you need into the application bundle directory, and then ships them together as a single bundle.

Here's the code for the game that I packaged up:
https://github.com/NHDaly/PaddleBattleJL

I've built a complete executable mac .app from it, and i'm currently in the process of applying for an Apple Developer account so i can try putting it all the way to the app store!!

Sorry I haven't shared this yet. I've been meaning to do a writeup but something kept coming up so i didn't get to it. I'll do it soon i promise!

from packagecompiler.jl.

@SimonDanisch, I was actually thinking that if it makes sense, we could maybe add the functionality from that repo into this repo? Or maybe it makes sense to keep it as a standalone repository? But i'm hoping to be able to create an end-to-end executable bundler that you could use to turn julia code into a distributable, sellable, app!

from packagecompiler.jl.

It seems like you have a custom logic for dealing with assets and binary dependencies.
I think it'd be great, if we'd also have a generic solution for all assets + paths relative to a package.
...
I think we just need an oficial way of referencing files inside a package, which we can later redirect when moving the packages.

Ah, I think it's one step more than that: that's part of why I had made the custom versions of my binary dependencies. At least for mac/unix, the binaries and libraries themselves have a field called the RPATH (or they can use others I think on mac?) which specifies where they expect to find their dependencies.

So for all the libraries you copy into the package, you
A) need to make sure you copy over all the libraries' dependencies as well, and
B) need to then use install_name_tool to -change the rpath for the dependency from its old path to its new path inside the package, relative to the library (I used "@loader_path/<dependencyname>.dylib" <library>.dylib, which means next-to the library.)

However, you don't want to pull all of the library's dependencies, since most of them are provided by the system, so it's at best unnecessary bulk, and at worst could cause incompatibilities when copying the executable to other machines. For example, here are the dependencies I have from SDL2_ttf:

$ otool -L ~/.julia/v0.6/Homebrew/deps/usr/lib/libSDL2_ttf-2.0.0.dylib   # (use `ldd` instead of `otool` on linux)
/Users/daly/.julia/v0.6/Homebrew/deps/usr/lib/libSDL2_ttf-2.0.0.dylib:
        /Users/daly/.julia/v0.6/Homebrew/deps/usr/opt/sdl2_ttf/lib/libSDL2_ttf-2.0.0.dylib (compatibility version 15.0.0, current version 15.0.0)
        /Users/daly/.julia/v0.6/Homebrew/deps/usr/opt/freetype/lib/libfreetype.6.dylib (compatibility version 21.0.0, current version 21.0.0)
        /Users/daly/.julia/v0.6/Homebrew/deps/usr/opt/sdl2/lib/libSDL2-2.0.0.dylib (compatibility version 5.0.0, current version 5.1.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)```

So in order for SDL_ttf to work, I can't just copy it over, I also need to copy libfreetype and libSDL2. But I don't want to copy /usr/lib/libSystem.B.dylib, since it's a system dependency and will be provided on all macs. Then I have to repeat the process for libfreetype, calling otool -L on it and copying its dependencies. But this process is tedious and I agree it is hard to expect users to do this. It would be nice to automate.

It's interesting though because every package's binary deps are different. For Blink.jl, I think all you need to do is copy over ~/.julia/v0.6/Blink/deps/Julia.app, which is itself already a portable executable (it's just a renamed Electron.app).

I had actually been thinking, it would be nice if there was a programmatic way to discover the binary deps of any given package. Like perhaps to register a package there is a required field for your binary dependencies, such that we could simply call Pkg.binary_deps("Blink.jl") to get the above ~/.julia/v0.6/Blink/deps/Julia.app, or Pkg.binary_deps("SDL2.jl") to get a list of the libs we need to copy out of Homebrew... Wdyt, does that sound crazy?

from packagecompiler.jl.

Also for what it's worth, at least on mac, Xcode does this for you. Perhaps this problem is out-of-scope for PackageCompiler -- which is after all a compiler -- and to do this step you should use some other kind of executable bundler. Like, you could probably have done everything I did purely through Xcode... And on linux maybe you use a different build tool? Maybe Meson or something.

But I was hoping to be able to keep it all within julia code, so that way you can do this all without requiring external applications, but maybe that's silly since this is exactly what xcode was built for...

from packagecompiler.jl.

generic solution for all assets

Regarding assets, the best solution I have come up with so far was basically "any path provided via --resource will be copied to AppName.app/Contents/Resources/", and then you can access the resources from there. That felt like the easiest way to keep the release environment most similar to your development environment:
Basically, then all you have to do is cd ../Resources when you start up. I believe this most closely mirrors the way things behave when developing a C++ app through xcode.

But I agree it would be nice if, instead, you could simply refer to the resources via something like resources("assets/music.wav") and it would automatically determine if you are inside a compiled program and need to use a different path or something...

from packagecompiler.jl.

Update: I've changed my application builder repo into a Package, ApplicationBuilder.jl. I'm starting to agree with @ranjanan that it might be separate enough from PackageCompiler to keep it in its own package (especially since I think we'll want to provide utilities that can be called from user code to set it up to be bundled), but I'm also happy to merge it into this package if you think that's best!! 😄

Also, I wrote a writeup about it!:
https://discourse.julialang.org/t/building-and-releasing-a-fully-self-contained-application-written-in-julia/10418

Sorry it's so long, I struggle with being terse. :(

from packagecompiler.jl.

But I think we could close this Issue, in favor of opening specific issues in that repo? :)

I agree with that.

from packagecompiler.jl.