Comments (51)
andrejusc
commented on July 29, 2024
2
@ryanmoran - cool, will be waiting quietly for that then and will be your beta tester ;)
from rust.
andrejusc
commented on July 29, 2024
2
Hi folks, in case you're interested - I've published article in Medium where outcome of our discussion here (i.e. smaller image preparation for Rust use case) is used in bigger solution development picture: https://medium.com/@andrejusc/rust-code-as-serverless-function-and-evaluation-via-openshift-dev-sandbox-a9019d17ca47 . Hopefully that will further allow to evolve things. Thank you once again.
from rust.
robdimsdale
commented on July 29, 2024
1
Great, glad it's working. I was able to run the same command and build the app:
pack build buildpack-sample:1.0.8 \
--path=. --verbose \
--builder=docker.io/paketobuildpacks/builder-jammy-buildpackless-static \
--trust-builder --docker-host=inherit --buildpack=docker.io/paketocommunity/rust \
-e RUSTFLAGS='-C target-feature=+crt-static' \
-e BP_CARGO_INSTALL_ARGS='--target=x86_64-unknown-linux-gnu' \
-e BP_CARGO_TINI_DISABLED=true
I didn't validate that the binary is statically-linked, but given the stack doesn't have libc, I'm fairly confident that it must be statically-linked.
In terms of the size of the image, the breakdown is roughly:
- 14MB for the app
- 2.3MB for the launcher (part of the CNB spec)
- 2.2MB for the base image
- Of this, most of it (1.6MB) is time-zone data
The app is essentially the same size as when you pre-compiled and dropped it onto a scratch image, so that checks out. In order to be CNB-compliant you will need the launcher, so that's just overhead you can't avoid. So in reality there's only about 2MB more you can gain by dropping the timezone packages.
As an aside, we could make a scratch stack which contains absolutely no packages at runtime, but I do not yet see value in maintaining this. Longer-term the Buildpacks spec will allow us to use "run-time extensions" to dynamically install packages during the build process, and at that point it could be helpful to have a scratch as it would allow you to install exactly (and only) the minimal set of packages your particular app needs.
Regardless, I'm glad it's working. I'm glad we validated the static stack/builder, and I'm glad you are able to get an image size that you're satisfied with. I also learned some interesting stuff about static linking, which is a bonus!
from rust.
andrejusc
commented on July 29, 2024
ok, Readme recommends usage of tiny. So tried it for function building this way:
func_linux_amd64 build --image buildpack-smaple --verbose --builder-image=gcr.io/paketo-buildpacks/builder:tiny
but such faced then this issue:
...
Adding buildpack paketo-community/rust version 0.27.0 to builder
Adding buildpack paketo-community/rust-dist version 1.16.1 to builder
Adding buildpack paketo-community/rustup version 1.8.1 to builder
Adding buildpack paketo-buildpacks/procfile version 5.6.0 to builder
Adding buildpack paketo-buildpacks/syft version 1.26.0 to builder
Adding buildpack paketo-community/cargo version 0.9.0 to builder
Setting custom order
Creating builder with the following buildpacks:
-> paketo-buildpacks/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-community/[email protected]
-> paketo-community/[email protected]
-> paketo-community/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-buildpacks/[email protected]
-> paketo-community/[email protected]
Error: validating buildpacks: buildpack paketo-community/[email protected] (Buildpack API 0.7) is incompatible with lifecycle 0.9.3 (Buildpack API(s) 0.2, 0.3, 0.4)
Not sure how to overcome this one yet.
from rust.
andrejusc
commented on July 29, 2024
I'm using such version of func cli:
$ ./func_linux_amd64 --version
v1.9.2
which to my understanding is coming from here:
https://github.com/knative/func/blob/knative-v1.9.2/go.mod
which references this:
github.com/buildpacks/pack v0.28.0
which in turn references this:
github.com/buildpacks/lifecycle v0.15.1
which has this:
https://github.com/buildpacks/lifecycle/blob/v0.15.1/api/apis.go
and such block:
var (
Platform = newApisMustParse([]string{"0.3", "0.4", "0.5", "0.6", "0.7", "0.8", "0.9", "0.10"}, nil)
Buildpack = newApisMustParse([]string{"0.2", "0.3", "0.4", "0.5", "0.6", "0.7", "0.8", "0.9"}, nil)
)
But I'm not familiar with such and how it leads to error in above comment
from rust.
andrejusc
commented on July 29, 2024
Further analysis reveals such:
$ podman inspect gcr.io/paketo-buildpacks/builder:tiny | grep -oP "lifecycle.*}"
lifecycle\":{\"version\":\"0.9.3\",\"api\":{\"buildpack\":\"0.2\",\"platform\":\"0.3\"},\"apis\":{\"buildpack\":{\"deprecated\":[],\"supported\":[\"0.2\",\"0.3\",\"0.4\"]},\"platform\":{\"deprecated\":[],\"supported\":[\"0.3\",\"0.4\"]}}},\"createdBy\":{\"name\":\"Pack CLI\",\"version\":\"0.15.0+git-49ad805.build-1613\"}}
so at least such explains where very old lifecycle 0.9.3 is coming from then
from rust.
andrejusc
commented on July 29, 2024
And for previously used base I have lifecycle 0.16.1:
$ podman inspect gcr.io/paketo-buildpacks/builder:base | grep -oP "lifecycle.*}"
lifecycle\":{\"version\":\"0.16.1\",\"api\":{\"buildpack\":\"0.2\",\"platform\":\"0.3\"},\"apis\":{\"buildpack\":{\"deprecated\":[\"0.2\",\"0.3\",\"0.4\",\"0.5\",\"0.6\"],\"supported\":[\"0.2\",\"0.3\",\"0.4\",\"0.5\",\"0.6\",\"0.7\",\"0.8\",\"0.9\"]},\"platform\":{\"deprecated\":[\"0.3\",\"0.4\",\"0.5\",\"0.6\"],\"supported\":[\"0.3\",\"0.4\",\"0.5\",\"0.6\",\"0.7\",\"0.8\",\"0.9\",\"0.10\",\"0.11\"]}}},\"createdBy\":{\"name\":\"Pack CLI\",\"version\":\"0.29.0+git-95c8060.build-4209\"}}
from rust.
andrejusc
commented on July 29, 2024
@dmikusa - may you know if my observations here are correct and that gcr.io/paketo-buildpacks/builder:tiny image is outdated and of no use by now? Would be still nice to have some common up-to-date common tiny image somewhere.
from rust.
ryanmoran
commented on July 29, 2024
The GCR registry is a legacy location for these builder images. We no longer post new images there. You can see a listing of all the the available builder images which are now hosted at DockerHub here: https://paketo.io/docs/reference/builders-reference/.
from rust.
andrejusc
commented on July 29, 2024
@ryanmoran - I see, confirming that taken from DockerHub tiny image has appropriate lifecycle 0.16.1 version in it:
$ podman inspect docker.io/paketobuildpacks/builder:tiny | grep -oP "lifecycle.*}"
lifecycle\":{\"version\":\"0.16.1\",\"api\":{\"buildpack\":\"0.2\",\"platform\":\"0.3\"},\"apis\":{\"buildpack\":{\"deprecated\":[\"0.2\",\"0.3\",\"0.4\",\"0.5\",\"0.6\"],\"supported\":[\"0.2\",\"0.3\",\"0.4\",\"0.5\",\"0.6\",\"0.7\",\"0.8\",\"0.9\"]},\"platform\":{\"deprecated\":[\"0.3\",\"0.4\",\"0.5\",\"0.6\"],\"supported\":[\"0.3\",\"0.4\",\"0.5\",\"0.6\",\"0.7\",\"0.8\",\"0.9\",\"0.10\",\"0.11\"]}}},\"createdBy\":{\"name\":\"Pack CLI\",\"version\":\"0.29.0+git-95c8060.build-4209\"}}
Would it be possible to update this repo's main Readme and add your note above about CGR registry as a legacy?
from rust.
dmikusa
commented on July 29, 2024
Sorry about the bad docs there. Updated in this PR: #289
from rust.
andrejusc
commented on July 29, 2024
@dmikusa - based on all above discussion - I've switched now to use such builder image: docker.io/paketobuildpacks/builder:tiny
and explicitly set BP_RUSTUP_INIT_LIBC to musl, so initially in output getting such:
Paketo Buildpack for Rustup 1.8.1
https://github.com/paketo-community/rustup
Build Configuration:
$BP_RUSTUP_ENABLED true use rustup to install Rust
$BP_RUSTUP_INIT_LIBC musl libc implementation: gnu or musl
$BP_RUSTUP_INIT_VERSION 1 the rustup version
$BP_RUST_PROFILE minimal the Rust profile to install
$BP_RUST_TARGET an additional Rust target to install
$BP_RUST_TOOLCHAIN stable the Rust toolchain or version number to install
Rustup (musl libc) 1.25.2: Contributing to layer
Downloading from https://static.rust-lang.org/rustup/archive/1.25.2/x86_64-unknown-linux-musl/rustup-init
Verifying checksum
Copying to /layers/paketo-community_rustup/rustup-init-musl/bin
Cargo: Contributing to layer
Rustup: Reusing cached layer
Rust: Contributing to layer
Installing Rust
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
stable-x86_64-unknown-linux-gnu unchanged - rustc 1.68.2 (9eb3afe9e 2023-03-27)
info: checking for self-updates
info: downloading component 'rust-std' for 'x86_64-unknown-linux-musl'
info: installing component 'rust-std' for 'x86_64-unknown-linux-musl'
but then hitting such error:
error: failed to run custom build command for `zstd-sys v2.0.7+zstd.1.5.4`
Caused by:
process didn't exit successfully: `/workspace/target/release/build/zstd-sys-4c214eeedc489111/build-script-build` (exit status: 1)
--- stdout
cargo:rerun-if-env-changed=ZSTD_SYS_USE_PKG_CONFIG
TARGET = Some("x86_64-unknown-linux-musl")
OPT_LEVEL = Some("3")
HOST = Some("x86_64-unknown-linux-gnu")
cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-musl
CC_x86_64-unknown-linux-musl = None
cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_musl
CC_x86_64_unknown_linux_musl = None
cargo:rerun-if-env-changed=TARGET_CC
TARGET_CC = None
cargo:rerun-if-env-changed=CC
CC = None
...
CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
running: "musl-gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-fvisibility=hidden" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-o" "/workspace/target/x86_64-unknown-linux-musl/release/build/zstd-sys-b4af3a3ac6cc6c6e/out/zstd/lib/common/debug.o" "-c" "zstd/lib/common/debug.c"
...
--- stderr
error occurred: Failed to find tool. Is `musl-gcc` installed?
Anything to check in such case? Based on current Readme - my understanding is that tiny builder will install if needed musl specific parts, but maybe that is incorrect understanding?
from rust.
dmikusa
commented on July 29, 2024
Rustup (musl libc) 1.25.2: Contributing to layer
Downloading from https://static.rust-lang.org/rustup/archive/1.25.2/x86_64-unknown-linux-musl/rustup-init
Verifying checksum
Copying to /layers/paketo-community_rustup/rustup-init-musl/bin
Cargo: Contributing to layer
Rustup: Reusing cached layer
Rust: Contributing to layer
Installing Rust
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
I believe this is a caching issue in the buildpack. You've gone from using the base stack to the tiny stack, which for Rust means you need to use musl (see the note in the README for why). That part is working. The buildpack is installing Rustup which is good, but then Rustup doesn't run because the layer is cached. That's a bug. I'll open an issue about this.
What you can do to work around this is to clear the buildpack cache. If you run your next pack build with the --clear-cache flag, that should clear it out and you should see this line go away Rustup: Reusing cached layer. Don't forget to remove that argument on subsequent builds.
Thanks for all your feedback, btw!
from rust.
andrejusc
commented on July 29, 2024
@dmikusa - as in my use case I'm executing not pack directly - is there some analogy of --clear-cache functionality to be activated via something like BP_RUSTUP_CACHE_CLEAR: true (which by default could be false) flag/setting? That would suit my needs then and I think would be awesome from other third-parties utilizing buildpacks functionality. What do you think?
from rust.
andrejusc
commented on July 29, 2024
So I found that caches are basically presented in my case under here ~/.local/share/containers/storage/volumes/:
drwx------ 3 user user 4096 Mar 31 20:37 pack-my-sample_latest-4f438fa4aa91.build
drwx------ 3 user user 4096 Mar 31 20:37 pack-my-sample_latest-4f438fa4aa91.launch
and after trying to play with cleanup - now I'm facing new issue with tiny builder usage:
...
===> BUILDING
Starting build
Running build for buildpack paketo-community/[email protected]
Looking up buildpack
Finding plan
Creating plan directory
Preparing paths
Running build command
Paketo Buildpack for Rustup 1.8.1
https://github.com/paketo-community/rustup
Build Configuration:
$BP_RUSTUP_ENABLED true use rustup to install Rust
$BP_RUSTUP_INIT_LIBC musl libc implementation: gnu or musl
$BP_RUSTUP_INIT_VERSION 1 the rustup version
$BP_RUST_PROFILE minimal the Rust profile to install
$BP_RUST_TARGET an additional Rust target to install
$BP_RUST_TOOLCHAIN stable the Rust toolchain or version number to install
Rustup (musl libc) 1.25.2: Contributing to layer
Downloading from https://static.rust-lang.org/rustup/archive/1.25.2/x86_64-unknown-linux-musl/rustup-init
Verifying checksum
Copying to /layers/paketo-community_rustup/rustup-init-musl/bin
Cargo: Contributing to layer
Rustup: Contributing to layer
Installing Rustup
Rust: Contributing to layer
Installing Rust
info: syncing channel updates for 'stable-x86_64-unknown-linux-musl'
info: latest update on 2023-03-28, rust version 1.68.2 (9eb3afe9e 2023-03-27)
info: downloading component 'cargo'
info: downloading component 'rust-std'
info: downloading component 'rustc'
info: installing component 'cargo'
info: installing component 'rust-std'
info: installing component 'rustc'
stable-x86_64-unknown-linux-musl installed - (error reading rustc version)
info: default toolchain set to 'stable-x86_64-unknown-linux-musl'
info: checking for self-updates
info: component 'rust-std' for target 'x86_64-unknown-linux-musl' is up to date
unable to invoke layer creator
unable to contribute Rust layer
error executing 'rustc --version':
Combined Output: error: command failed: 'rustc': No such file or directory (os error 2)
:
exit status 1
ERROR: failed to build: exit status 1
Error: executing lifecycle: failed with status code: 51
why it's not happy now about rustc?
from rust.
dmikusa
commented on July 29, 2024
By default, it pack will store cached build information in volumes. In Docker you can docker volume ls and see them. You can also delete the volumes that way, it's like clearing the cache. I assume Podman has the same commands but I don't have it installed to check.
You can also just change the name of your application/image. The cache is scoped to the application name, so if the name changes then it should be treated like a new fresh build.
from rust.
dmikusa
commented on July 29, 2024
why it's not happy now about rustc?
That is strange. The buildpack runs rustc --version so that we can include that in the cached information. It enables us to automatically invalidate the cache when a new version of rust is available. It seems that it's unable to run that for some reason.
It's weird, even rustup is having trouble. It should print the version as well.
stable-x86_64-unknown-linux-musl installed - (error reading rustc version)
I'm not sure off-hand what might be causing that and it's hard to say without knowing more about your setup.
I will say that builds are a little trickier on tiny because of using musl. You can try switching to the base stack and builder, which will work with gnu libc.
from rust.
andrejusc
commented on July 29, 2024
I'm not sure off-hand what might be causing that and it's hard to say without knowing more about your setup.
I don't see any tiny image usage under .github/workflows in this repo and if it mentioned as supported in Readme - I think we need at least 1 evaluation flow definition there. Just to prove it works somewhere else as expected and then it's down to someone's setup's specifics.
I could work on adding that if you could point me where base image is used for some example build in workflows.
from rust.
dmikusa
commented on July 29, 2024
I could work on adding that if you could point me where base image is used for some example build in workflows.
Not a base image, sorry. I didn't explain that well. There are three variants of the stack and builder that Paketo publishes: tiny, base, and full. I was suggesting that you give the base image a try instead. Try switching to paketobuildpacks/builder-jammy-base as your builder.
Another possibility is that you could be hitting an issue with the Jammy stack. There are some differences with the Jammy stack, security has been tightened down some. You may be hitting something there. If you want to try using paketobuildpacks/builder:tiny or paketobuildpacks/builder:base that is the older stack. If that works and Jammy fails, let me know and I can look into that more. I did try running builds on both and they both worked for me.
from rust.
andrejusc
commented on July 29, 2024
Ok, culprit here seems in my attempt to set explicitly builder's setting BP_RUSTUP_INIT_LIBC to musl. When it set - both builder-jammy-base and builder-jammy-tiny are failing.
And when it unset - this part of tiny builder output is interesting then in a sense of it using gnu toolchain and not musl:
info: installing component 'rustc'
stable-x86_64-unknown-linux-gnu installed - rustc 1.68.2 (9eb3afe9e 2023-03-27)
info: default toolchain set to 'stable-x86_64-unknown-linux-gnu'
info: checking for self-updates
info: component 'rust-std' for target 'x86_64-unknown-linux-gnu' is up to date
and that Readme says I need to use x86_64-unknown-linux-musl as a target. Seems some disconnect in that statement then.
And when I have BP_RUSTUP_INIT_LIBC set to musl again - then tiny fails on such:
info: installing component 'rustc'
stable-x86_64-unknown-linux-musl installed - (error reading rustc version)
info: default toolchain set to 'stable-x86_64-unknown-linux-musl'
info: checking for self-updates
info: downloading component 'rust-std' for 'x86_64-unknown-linux-gnu'
info: installing component 'rust-std' for 'x86_64-unknown-linux-gnu'
And when now builder-jammy-tiny works - I'm getting not that big 107MB image as before, but now smaller of about 40MB:
$ podman image ls | grep buildpack-sample
localhost/buildpack-sample latest d99a6ccc68e9 43 years ago 39.5 MB
Such size is near comparable to my direct Dockerfile approach, so probably could live for now with such, unless some other optimization possible.
from rust.
andrejusc
commented on July 29, 2024
Additionally now tried with builder paketobuildpacks/builder-jammy-buildpackless-tiny to see if any difference in produced image size - so far same as for builder-jammy-tiny before, i.e. 39.5MB.
from rust.
dmikusa
commented on July 29, 2024
And when I have BP_RUSTUP_INIT_LIBC set to musl again
Ok, yes. I think that would cause the issue. This setting is for something different. It installs the underlying Rust tools compiled using musl libc, but that would require a base image that is also compatible with musl. The Paketo builder images use gnu libc, so that needs to stay as the default for Paketo builder images. That's why it can't run the binary because there's an incompatible libc (my understanding at least).
What you want is to have the tools compiled against gnu libc so it's compatible with the Paketo builders but have a Rust target of musl installed. That will compile your application against musl libc.
and that Readme says I need to use x86_64-unknown-linux-musl as a target. Seems some disconnect in that statement then.
OK, I know what's happening here. This is a Jammy thing. I've logged some issues to get this fixed.
I think you can make this work if you do two things:
-
Add a
rust-toolchain.tomland settargets = ["x86_64-unknown-linux-musl"]. -
Set
BP_CARGO_INSTALL_ARGS='--target=x86_64-unknown-linux-musl'
I think that will make cargo build the right target.
Such size is near comparable to my direct Dockerfile approach, so probably could live for now with such, unless some other optimization possible.
The tiny image is about 22M, so you can see the rest is your app binary. Down the road, I want to add support for upx which can compress binaries. I've seen good results with it compressing binaries up to 50%. If you got that kind of compression ratio upx could take down the image size to maybe 30M.
from rust.
andrejusc
commented on July 29, 2024
The tiny image is about 22M, so you can see the rest is your app binary.
So if looking into this:
$ podman history docker.io/paketobuildpacks/run-jammy-tiny:latest
ID CREATED CREATED BY SIZE COMMENT
61bab0020545 292 years ago 2.56 kB
<missing> 292 years ago 4.1 kB
<missing> 3 days ago COPY /tiny/ / # buildkit 24 MB buildkit.dockerfile.v0
i.e. some buildkit is inside that image. Then last question I have here - why such is part of my final image? Could this buildkit layer be excluded by some means while using some flag/option passed to builder? Hoping that my question makes sense, but please correct me otherwise.
from rust.
dmikusa
commented on July 29, 2024
That is just the base layer from the run image. It looks a little weird but that is just because of the way the packages get installed and then copied into the final run image at build time. It happens like that because there are no tools to install packages in the run image, so you can't just run apt to install the packages. We install them to a prefixed location /tiny running on an image that has the package tools, then copy that prefix location over to the final run image.
from rust.
andrejusc
commented on July 29, 2024
Let me rephrase - what is the purpose of that run layer within my final image if all I have is built static executable, which could run without anything in addition as shown at the beginning of this thread inside my Dockerfile?
from rust.
dmikusa
commented on July 29, 2024
There are a few things that the buildpack spec requires, like the presence of user/groups so if you had a scratch image I don't think that would comply (probably would fail, I haven't tried) with the buildpacks launcher. That said, most of that is glibc, openssl, trusted tls certs, and timezone data. You may or may not need those for your app, but even for statically compiled Rust apps you might need TLS certs or timezone data.
It all boils down to Paketo trying to provide a shared base image that many different apps can use. We don't add much to the tiny stack, but these things are pretty broadly applicable so they are in the tiny stack.
There is a static stack that is even more trimmed down, but it's pretty new. I'm not 100% sure the state of it. See https://github.com/paketo-buildpacks/jammy-static-stack. This only has the files required by buildpacks, trusted TLS cert and timezone data. I don't think there is a builder, but you should be able to try it if you add --run-image docker.io/paketobuildpacks/run-jammy-static:latest to your pack build command. That should swap in the static run image.
from rust.
robdimsdale
commented on July 29, 2024
Just to add to what @dmikusa said, the goal of the static stack is to enable Buildpacks-compliant minimal images for statically-linked binaries. It's experimental at the moment, and we're looking for feedback. I would expect that a statically-linked, self-contained rust application should work on the static stack. If it doesn't, that's useful feedback and we'd appreciate you filing an issue with the error.
from rust.
andrejusc
commented on July 29, 2024
@dmikusa - trying this advice:
I don't think there is a builder, but you should be able to try it if you add --run-image docker.io/paketobuildpacks/run-jammy-static:latest to your pack build command.
and using overall build command in absence of static builder so far like such:
pack build buildpack-sample --path=. --verbose --builder=docker.io/paketobuildpacks/builder-jammy-tiny --trust-builder --docker-host=inherit --run-image docker.io/paketobuildpacks/run-jammy-static:latest
leads to such error:
ERROR: failed to build: invalid run-image 'docker.io/paketobuildpacks/run-jammy-static:latest': run-image stack id 'io.buildpacks.stacks.jammy.static' does not match builder stack 'io.buildpacks.stacks.jammy.tiny'
If such is valid error and you think that it should be allowed to mix stack and runner - I'll then raise separate issue as I'm newbie here and not yet aware fully of what is allowed and what not.
from rust.
robdimsdale
commented on July 29, 2024
Ah, no, I don't think you can replace the run image from a builder with a stack with a different ID.
I think this is an indication that we need to create a builder for the static image cc @ForestEckhardt
from rust.
robdimsdale
commented on July 29, 2024
@andrejusc we should be able to create a static builder for you in the next day or so.
from rust.
dmikusa
commented on July 29, 2024
Thanks @robdimsdale
from rust.
robdimsdale
commented on July 29, 2024
@andrejusc the static buildpackless builder has been released and you can use it directly as follows (using your command from earlier):
pack build buildpack-sample \
--path=. \
--verbose \
--builder=docker.io/paketobuildpacks/builder-jammy-buildpackless-static \
--trust-builder \
--docker-host=inherit
from rust.
andrejusc
commented on July 29, 2024
@robdimsdale - trying that now:
pack build buildpack-sample:1.0.0 --path=. --verbose --builder=docker.io/paketobuildpacks/builder-jammy-buildpackless-static --trust-builder --docker-host=inherit
and getting such error:
===> DETECTING
ERROR: No buildpack groups passed detection.
ERROR: Please check that you are running against the correct path.
ERROR: failed to detect: no buildpacks participating
ERROR: failed to build: executing lifecycle: failed with status code: 20
from rust.
robdimsdale
commented on July 29, 2024
How were you specifying the buildpack in your command above? I just copied that command. I assumed you had specified the buildpack in some config file (e.g. project.toml).
If you aren't doing that, then you'll need to add the rust buildpack explicitly:
pack build buildpack-sample:1.0.0 \
--buildpack=docker.io/paketocommunity/rust \
--builder=docker.io/paketobuildpacks/builder-jammy-buildpackless-static \
<...>
``
from rust.
andrejusc
commented on July 29, 2024
@robdimsdale - ok, probably I've messed something. Retrying...
So again with jammy base builder initially:
pack build buildpack-sample:1.0.1 \
--path=. --verbose \
--builder=docker.io/paketobuildpacks/builder-jammy-base \
--trust-builder --docker-host=inherit --buildpack=docker.io/paketocommunity/rust
and launching container out of produced image - works.
Now doing exactly same, but with jammy static (just incrementing image tag to avoid cache reuse):
pack build buildpack-sample:1.0.2 \
--path=. --verbose \
--builder=docker.io/paketobuildpacks/builder-jammy-buildpackless-static \
--trust-builder --docker-host=inherit --buildpack=docker.io/paketocommunity/rust
and launching container ouf of produced image - leads to such error:
ERROR: failed to launch: bash exec: no such file or directory
Also worth mentioning - at least at the moment I could see both tiny and static builder images of same sizes - I;d imagine static should be smaller:
$ podman image ls | grep paketo | grep buildpackless
docker.io/paketobuildpacks/builder-jammy-buildpackless-tiny latest cd57f6bc96a5 43 years ago 419 MB
docker.io/paketobuildpacks/builder-jammy-buildpackless-static latest f986f14083df 43 years ago 419 MB
from rust.
robdimsdale
commented on July 29, 2024
Ok, I'm confused now. I thought we were trying to get a statically-linked rust binary working via the static builder, so I'm confused as to why we expect bash to exist on the running container.
I'm not familiar enough with the rust buildpack to know what start commands it creates for the app container, but if you got an app working with tiny you should use the same app and same commands for static. The tiny image doesn't have a shell, so you must have done something different to get an app working on that stack/builder. I couldn't see which commands you were using above for that scenario, but whatever you were doing, switch out the tiny builder for the static builder.
The reason why this app works on base and not static is that this app is relying on bash to exist at runtime, which is true for base (and full) and not true for tiny or static.
As far as the image size of the builder, I haven't explored that. I will note that both the tiny and the static images have the effectively the same build image, which is much larger than the run image in both builders. It's surprises me that the images are identical to the MB, but i wouldn't necessarily expect the builder to be significantly smaller.
from rust.
robdimsdale
commented on July 29, 2024
For reference, the packages on the run-time image for static are here:
ca-certificates \
tzdata \
And for tiny are here:
base-files \
ca-certificates \
libc6 \
libssl3 \
netbase \
openssl \
tzdata \
zlib1g \
from rust.
andrejusc
commented on July 29, 2024
@robdimsdale - if such output could be of any help to understand issue with static image.
For base builder case:
pack inspect buildpack-sample:1.0.1
Inspecting image: buildpack-sample:1.0.1
REMOTE:
(not present)
LOCAL:
Stack: io.buildpacks.stacks.jammy
Base Image:
Reference: a2a7e8cfbf95df0019e581ef6f076274cc0b900f1644c318b63a45dbc341d17b
Top Layer: sha256:b96de72e9823de1f35c5f4e172702a9258796c7b98f6745723d95eef6517e74c
Run Images:
index.docker.io/paketobuildpacks/run-jammy-base:latest
Buildpacks:
ID VERSION HOMEPAGE
paketo-community/rustup 1.8.1 https://github.com/paketo-community/rustup
paketo-community/rust-dist 1.16.1 https://github.com/paketo-community/rust-dist
paketo-buildpacks/syft 1.26.0 https://github.com/paketo-buildpacks/syft
paketo-community/cargo 0.9.0 https://github.com/paketo-community/cargo
paketo-buildpacks/procfile 5.6.0 https://github.com/paketo-buildpacks/procfile
Processes:
TYPE SHELL COMMAND ARGS WORK DIR
web (default) bash /workspace/bin/function /workspace
function tini -g -- /workspace/bin/function /workspace
And for static:
pack inspect buildpack-sample:1.0.2
Inspecting image: buildpack-sample:1.0.2
REMOTE:
(not present)
LOCAL:
Stack: io.buildpacks.stacks.jammy.static
Base Image:
Reference: c53b8f93b93c1de4173803f753bf2dfb18d8daecd9a692696103c53da60c3bd4
Top Layer: sha256:6153dd9ac2c7b8b0df5f2e7e365d54e2f8abe4350f1c56f69e70f3ba26565e2b
Run Images:
index.docker.io/paketobuildpacks/run-jammy-static:latest
Buildpacks:
ID VERSION HOMEPAGE
paketo-community/rustup 1.8.1 https://github.com/paketo-community/rustup
paketo-community/rust-dist 1.16.1 https://github.com/paketo-community/rust-dist
paketo-buildpacks/syft 1.26.0 https://github.com/paketo-buildpacks/syft
paketo-community/cargo 0.9.0 https://github.com/paketo-community/cargo
paketo-buildpacks/procfile 5.6.0 https://github.com/paketo-buildpacks/procfile
Processes:
TYPE SHELL COMMAND ARGS WORK DIR
web (default) bash /workspace/bin/function /workspace
function tini -g -- /workspace/bin/function /workspace
My Procfile is such:
web: /workspace/bin/function
and I don't yet fully understand which part dictates to use shell as bash and not have executable as itself.
from rust.
dmikusa
commented on July 29, 2024
Oh, that's a Profile thing. The stack is different so it's not handling that right. Filed an enhancement request, paketo-buildpacks/procfile#175.
If the application's stack is not io.paketo.stacks.tiny the contents of Procfile will be executed as a shell script.
https://github.com/paketo-buildpacks/procfile#behavior
You might not need a Procfile. Have you tried w/out? The cargo buildpack can inspect, it looks at your project targets, and usually set up process types automatically.
from rust.
andrejusc
commented on July 29, 2024
@dmikusa -ok, removed Procfile if that is right now not suitable for static use case. Now inspect output is different and without bash:
pack inspect buildpack-sample:1.0.3
Inspecting image: buildpack-sample:1.0.3
REMOTE:
(not present)
LOCAL:
Stack: io.buildpacks.stacks.jammy.static
Base Image:
Reference: c53b8f93b93c1de4173803f753bf2dfb18d8daecd9a692696103c53da60c3bd4
Top Layer: sha256:6153dd9ac2c7b8b0df5f2e7e365d54e2f8abe4350f1c56f69e70f3ba26565e2b
Run Images:
index.docker.io/paketobuildpacks/run-jammy-static:latest
Buildpacks:
ID VERSION HOMEPAGE
paketo-community/rustup 1.8.1 https://github.com/paketo-community/rustup
paketo-community/rust-dist 1.16.1 https://github.com/paketo-community/rust-dist
paketo-buildpacks/syft 1.26.0 https://github.com/paketo-buildpacks/syft
paketo-community/cargo 0.9.0 https://github.com/paketo-community/cargo
Processes:
TYPE SHELL COMMAND ARGS WORK DIR
function (default) tini -g -- /workspace/bin/function /workspace
But trying to launch container - getting now different error:
ERROR: failed to launch: direct exec: no such file or directory
Anything else I'm dong not right/not expected here?
from rust.
robdimsdale
commented on July 29, 2024
Do you have a sample app we can look at? I could make some guesses but it would be faster if we could iterate on it locally.
Regardless, I'm not sure what is failing to be executed. Unless it's exec itself, which might not be on the static stack. Do you get the same error if you run the following command?
docker run -it --rm --entrypoint=launcher buildpack-sample:1.0.3 tini -g -- /workspace/bin/function
This bypasses the default process list and invokes tini -g -- /workspace/bin/function directly. Whether you get the same error or not could help narrow down the list of possibilities.
from rust.
andrejusc
commented on July 29, 2024
@robdimsdale - so far no luck, tried proposed by you command:
podman run -it --rm --entrypoint=launcher buildpack-sample:1.0.5 tini -g -- /workspace/bin/function
ERROR: failed to launch: bash exec: no such file or directory
Attached is my use case. Unzip it into buildpack-sample folder (ignore func.yaml file inside for now, hopefully it's not affecting regular pack command), navigate into that folder and invoke (use some non-existent image tag to avoid buildpack cache reuse issue):
pack build buildpack-sample:1.0.0 \
--path=. --verbose \
--builder=docker.io/paketobuildpacks/builder-jammy-buildpackless-static \
--trust-builder --docker-host=inherit --buildpack=docker.io/paketocommunity/rust
from rust.
dmikusa
commented on July 29, 2024
When I build with the command you've given:
pack build buildpack-sample:1.0.0 \
--path=. --verbose \
--builder=docker.io/paketobuildpacks/builder-jammy-buildpackless-static \
--trust-builder --docker-host=inherit --buildpack=docker.io/paketocommunity/rust
I get this error:
> docker run -it --platform='linux/amd64' c233bd2dac1c
qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory
Are you sure that you're building a static binary?
When I try to build the sample app with a static binary, the build fails.
Command:
pb rust-sample -b paketo-community/rust --builder=docker.io/paketobuildpacks/builder-jammy-buildpackless-static -e BP_CARGO_INSTALL_ARGS='--target=x86_64-unknown-linux-musl'
Output:
cargo install --target=x86_64-unknown-linux-musl --color=never --root=/layers/paketo-community_cargo/Cargo --path=.
Installing function v0.1.0 (/workspace)
Updating crates.io index
Downloading crates ...
Downloaded alloc-stdlib v0.2.2
Downloaded ppv-lite86 v0.2.17
Downloaded num_cpus v1.15.0
Downloaded bytestring v1.3.0
...
Compiling env_logger v0.10.0
error: failed to run custom build command for `zstd-sys v2.0.8+zstd.1.5.5`
Caused by:
process didn't exit successfully: `/workspace/target/release/build/zstd-sys-85b90089b2485305/build-script-build` (exit status: 1)
--- stdout
cargo:rerun-if-env-changed=ZSTD_SYS_USE_PKG_CONFIG
TARGET = Some("x86_64-unknown-linux-musl")
OPT_LEVEL = Some("3")
HOST = Some("x86_64-unknown-linux-gnu")
cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-musl
CC_x86_64-unknown-linux-musl = None
cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_musl
CC_x86_64_unknown_linux_musl = None
cargo:rerun-if-env-changed=TARGET_CC
TARGET_CC = None
cargo:rerun-if-env-changed=CC
CC = None
RUSTC_LINKER = None
cargo:rerun-if-env-changed=CROSS_COMPILE
CROSS_COMPILE = None
cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-musl
CFLAGS_x86_64-unknown-linux-musl = None
cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_musl
CFLAGS_x86_64_unknown_linux_musl = None
cargo:rerun-if-env-changed=TARGET_CFLAGS
TARGET_CFLAGS = None
cargo:rerun-if-env-changed=CFLAGS
CFLAGS = None
cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
CRATE_CC_NO_DEFAULTS = None
DEBUG = Some("false")
CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-musl
CC_x86_64-unknown-linux-musl = None
cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_musl
CC_x86_64_unknown_linux_musl = None
cargo:rerun-if-env-changed=TARGET_CC
TARGET_CC = None
cargo:rerun-if-env-changed=CC
CC = None
RUSTC_LINKER = None
cargo:rerun-if-env-changed=CROSS_COMPILE
CROSS_COMPILE = None
cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-musl
CFLAGS_x86_64-unknown-linux-musl = None
cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_musl
CFLAGS_x86_64_unknown_linux_musl = None
cargo:rerun-if-env-changed=TARGET_CFLAGS
TARGET_CFLAGS = None
cargo:rerun-if-env-changed=CFLAGS
CFLAGS = None
cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
CRATE_CC_NO_DEFAULTS = None
CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-musl
CC_x86_64-unknown-linux-musl = None
cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_musl
CC_x86_64_unknown_linux_musl = None
cargo:rerun-if-env-changed=TARGET_CC
TARGET_CC = None
cargo:rerun-if-env-changed=CC
CC = None
RUSTC_LINKER = None
cargo:rerun-if-env-changed=CROSS_COMPILE
CROSS_COMPILE = None
cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-musl
CFLAGS_x86_64-unknown-linux-musl = None
cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_musl
CFLAGS_x86_64_unknown_linux_musl = None
cargo:rerun-if-env-changed=TARGET_CFLAGS
TARGET_CFLAGS = None
cargo:rerun-if-env-changed=CFLAGS
CFLAGS = None
cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
CRATE_CC_NO_DEFAULTS = None
CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-musl
CC_x86_64-unknown-linux-musl = None
cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_musl
CC_x86_64_unknown_linux_musl = None
cargo:rerun-if-env-changed=TARGET_CC
TARGET_CC = None
cargo:rerun-if-env-changed=CC
CC = None
RUSTC_LINKER = None
cargo:rerun-if-env-changed=CROSS_COMPILE
CROSS_COMPILE = None
cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-musl
CFLAGS_x86_64-unknown-linux-musl = None
cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_musl
CFLAGS_x86_64_unknown_linux_musl = None
cargo:rerun-if-env-changed=TARGET_CFLAGS
TARGET_CFLAGS = None
cargo:rerun-if-env-changed=CFLAGS
CFLAGS = None
cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
CRATE_CC_NO_DEFAULTS = None
CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
running: "musl-gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-fvisibility=hidden" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-o" "/workspace/target/x86_64-unknown-linux-musl/release/build/zstd-sys-1286060b92b95d15/out/zstd/lib/common/debug.o" "-c" "zstd/lib/common/debug.c"
exit status: 127
running: "musl-gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-fvisibility=hidden" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-o" "/workspace/target/x86_64-unknown-linux-musl/release/build/zstd-sys-1286060b92b95d15/out/zstd/lib/common/entropy_common.o" "-c" "zstd/lib/common/entropy_common.c"
exit status: 127
--- stderr
error occurred: Command "musl-gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-fvisibility=hidden" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-o" "/workspace/target/x86_64-unknown-linux-musl/release/build/zstd-sys-1286060b92b95d15/out/zstd/lib/common/debug.o" "-c" "zstd/lib/common/debug.c" with args "musl-gcc" did not execute successfully (status code exit status: 127).
warning: build failed, waiting for other jobs to finish...
<jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
<jemalloc>: (This is the expected behaviour if you are running under QEMU)
error: failed to compile `function v0.1.0 (/workspace)`, intermediate artifacts can be found at `/workspace/target`
unable to invoke layer creator
unable to contribute application layer
unable to install single
unable to build
exit status 101
Not sure if that's something with my local setup, Mac M1 creates some weird build issues occasionally. Anyway, you're going to need to make sure you manually set the target for the time being. In the future the buildpack can see you're running on the static stack and do that automatically.
from rust.
robdimsdale
commented on July 29, 2024
When I try to run the same command:
pack build rust-static-sample -b paketo-community/rust --builder=docker.io/paketobuildpacks/builder-jammy-buildpackless-static -e BP_CARGO_INSTALL_ARGS='--target=x86_64-unknown-linux-musl'
On a linux machine, I get the following output:
[builder] Rust Cargo Build Pack 0.9.0
[builder] https://github.com/paketo-community/cargo
[builder] Build Configuration:
[builder] $BP_CARGO_INSTALL_ARGS --target=x86_64-unknown-linux-musl additional arguments to pass to Cargo install
[builder] $BP_CARGO_INSTALL_TOOLS additional tools to be add with Cargo install
[builder] $BP_CARGO_INSTALL_TOOLS_ARGS additional arguments to pass to Cargo install for tools
[builder] $BP_CARGO_TINI_DISABLED false Skip installing tini
[builder] $BP_CARGO_WORKSPACE_MEMBERS the subset of workspace members for Cargo to install
[builder] $BP_DISABLE_SBOM false Skip running SBOM scan
[builder] $BP_EXCLUDE_FILES colon separated list of glob patterns, matched source files are removed
[builder] $BP_INCLUDE_FILES static/*:templates/*:public/*:html/* colon separated list of glob patterns, matched source files are included
[builder] Tini 0.19.0: Contributing to layer
[builder] Downloading from https://github.com/krallin/tini/releases/download/v0.19.0/tini-amd64
[builder] Verifying checksum
[builder] Copying to /layers/paketo-community_cargo/tini
[builder] Creating cached target directory /workspace/target
[builder] Rust Application: Contributing to layer
[builder] File modification times not restored
[builder] File modification times not restored
[builder] File modification times not restored
[builder] cargo install --target=x86_64-unknown-linux-musl --color=never --root=/layers/paketo-community_cargo/Cargo --path=.
[builder] Installing function v0.1.0 (/workspace)
<...>
[builder] error[E0463]: can't find crate for `core`
[builder] |
[builder] = note: the `x86_64-unknown-linux-musl` target may not be installed
[builder] = help: consider downloading the target with `rustup target add x86_64-unknown-linux-musl`
[builder]
[builder] error[E0463]: can't find crate for `compiler_builtins`
[builder]
[builder] For more information about this error, try `rustc --explain E0463`.
[builder] error: could not compile `cfg-if` due to 2 previous errors
[builder] warning: build failed, waiting for other jobs to finish...
[builder] error: failed to compile `function v0.1.0 (/workspace)`, intermediate artifacts can be found at `/workspace/target`
When I add a rust-toolchain.toml with the following contents:
[toolchain]
targets = ["x86_64-unknown-linux-musl"]
I get a different error:
[builder] running: "musl-gcc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-fvisibility=hidden" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-o" "/workspace/target/x86_64-unknown-linux-musl/release/build/zstd-sys-1286060b92b95d15/out/zstd/lib/common/entropy_common.o" "-c" "zstd/lib/common/entropy_common.c"
[builder]
[builder] --- stderr
[builder]
[builder]
[builder] error occurred: Failed to find tool. Is `musl-gcc` installed?
[builder]
[builder]
[builder] warning: build failed, waiting for other jobs to finish...
[builder] error: failed to compile `function v0.1.0 (/workspace)`, intermediate artifacts can be found at `/workspace/target`
@dmikusa is musl-gcc something you'd expect the rust buildpacks to provide, or should it be on the stack?
Also, why is the buildpack trying to use musl instead of libc? The static build image has libc on it:
→ docker run -it --rm paketobuildpacks/build-jammy-static /bin/bash -c "dpkg -l | grep libc"
ii libc-bin 2.35-0ubuntu3.1 amd64 GNU C Library: Binaries
ii libc-dev-bin 2.35-0ubuntu3.1 amd64 GNU C Library: Development binaries
ii libc6:amd64 2.35-0ubuntu3.1 amd64 GNU C Library: Shared libraries
ii libc6-dev:amd64 2.35-0ubuntu3.1 amd64 GNU C Library: Development Libraries and Header Files
ii libcap-ng0:amd64 0.7.9-2.2build3 amd64 An alternate POSIX capabilities library
ii libcap2:amd64 1:2.44-1build3 amd64 POSIX 1003.1e capabilities (library)
ii libcc1-0:amd64 12.1.0-2ubuntu1~22.04 amd64 GCC cc1 plugin for GDB
ii libcom-err2:amd64 1.46.5-2ubuntu1.1 amd64 common error description library
ii libcrypt-dev:amd64 1:4.4.27-1 amd64 libcrypt development files
ii libcrypt1:amd64 1:4.4.27-1 amd64 libcrypt shared library
ii libctf-nobfd0:amd64 2.38-4ubuntu2.1 amd64 Compact C Type Format library (runtime, no BFD dependency)
ii libctf0:amd64 2.38-4ubuntu2.1 amd64 Compact C Type Format library (runtime, BFD dependency)
ii libcurl3-gnutls:amd64 7.81.0-1ubuntu1.10 amd64 easy-to-use client-side URL transfer library (GnuTLS flavour)
ii libcurl4:amd64 7.81.0-1ubuntu1.10 amd64 easy-to-use client-side URL transfer library (OpenSSL flavour)
ii linux-libc-dev:amd64 5.15.0-69.76 amd64 Linux Kernel Headers for development
And no packages with musl:
→ docker run -it --rm paketobuildpacks/build-jammy-static /bin/bash -c "dpkg -l | grep musl"
Also, I get the exact same error (musl-gcc not found) when using the tiny builder instead of the static builder (and using the same pack build command above).
from rust.
andrejusc
commented on July 29, 2024
@dmikusa - if it could help by some means. On my Linux x86_64 box doing static Rust building directly without any buildpacks involved:
$ RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-unknown-linux-gnu
and then checking output executable to make sure it's static:
$ file target/x86_64-unknown-linux-gnu/release/function
target/x86_64-unknown-linux-gnu/release/function: ELF 64-bit LSB pie executable, x86-64, version 1 (GNU/Linux), static-pie linked, BuildID[sha1]=3c2a78e971678e6160c927342bc2d20e35a4083d, for GNU/Linux 3.2.0, with debug_info, not stripped
and such:
$ ldd target/x86_64-unknown-linux-gnu/release/function
statically linked
$ ls -l target/x86_64-unknown-linux-gnu/release/function
-rwxrwxr-x 2 user user 13854080 Apr 6 15:37 target/x86_64-unknown-linux-gnu/release/function
So my expectation would be that using static builder should produce similar outcome.
I'm not yet very deep into QEMU emulator usage on ARM64 for x86_64 purposes. Did in opposite way for one recent project, but still a lot to learn.
from rust.
dmikusa
commented on July 29, 2024
Oh, cool. I learned something new today. I've only done static builds previously with musl.
@andrejusc Can you try these arguments?
-b paketo-community/rust --builder=docker.io/paketobuildpacks/builder-jammy-buildpackless-static -e RUSTFLAGS='-C target-feature=+crt-static' -e BP_CARGO_INSTALL_ARGS='--target=x86_64-unknown-linux-gnu' -e BP_CARGO_TINI_DISABLED=true
That will pass through RUSTFLAGS and it built a static binary and it was OK for me. It will also disable tini. I think the tini binary we are installing needs libc. I think that is causing the issue.
I'll log an issue to check if we can get a tini binary that's static. Disabling it is OK too, but you may need to setup some signal handlers in your app to ensure it shutdown properly (it needs to handle SIGTERM).
from rust.
dmikusa
commented on July 29, 2024
@robdimsdale Ohhhh, yes. That makes sense. Rust doesn't necessarily need musl-gcc but there's a particular package in this project that is trying to build a native c library wrapper, so it must need it for that. The buildpack isn't providing it at the moment so it would fail for that reason. I have some ideas on how to fix that in the future, but that's further down the road kind of bundled with cross-compile support.
from rust.
andrejusc
commented on July 29, 2024
@dmikusa - cool, appreciate all our collaboration here.
So here is latest. Using such command in my case:
pack build buildpack-sample:1.0.8 \
--path=. --verbose \
--builder=docker.io/paketobuildpacks/builder-jammy-buildpackless-static \
--trust-builder --docker-host=inherit --buildpack=docker.io/paketocommunity/rust \
-e RUSTFLAGS='-C target-feature=+crt-static' \
-e BP_CARGO_INSTALL_ARGS='--target=x86_64-unknown-linux-gnu' \
-e BP_CARGO_TINI_DISABLED=true
and analyzing then built image:
pack inspect buildpack-sample:1.0.8
Inspecting image: buildpack-sample:1.0.8
REMOTE:
(not present)
LOCAL:
Stack: io.buildpacks.stacks.jammy.static
Base Image:
Reference: c53b8f93b93c1de4173803f753bf2dfb18d8daecd9a692696103c53da60c3bd4
Top Layer: sha256:6153dd9ac2c7b8b0df5f2e7e365d54e2f8abe4350f1c56f69e70f3ba26565e2b
Run Images:
index.docker.io/paketobuildpacks/run-jammy-static:latest
Buildpacks:
ID VERSION HOMEPAGE
paketo-community/rustup 1.8.1 https://github.com/paketo-community/rustup
paketo-community/rust-dist 1.16.1 https://github.com/paketo-community/rust-dist
paketo-buildpacks/syft 1.26.0 https://github.com/paketo-buildpacks/syft
paketo-community/cargo 0.9.0 https://github.com/paketo-community/cargo
Processes:
TYPE SHELL COMMAND ARGS WORK DIR
function (default) /workspace/bin/function /workspace
and size of it (which is now good enough):
$ podman image ls | grep sample | grep 1.0.8
docker.io/library/buildpack-sample 1.0.8 211189eee732 43 years ago 19.7 MB
Could confirm that I could launch container out of it and it behaves as expected.
I'm happy then to conclude all this long thread. Hope you've enjoyed it as well. Thank you for all your prompt help and comments @dmikusa @robdimsdale
from rust.
dmikusa
commented on July 29, 2024
Woohoo! Glad it's working. Thanks for patiently working through this with us and all the feedback!
from rust.
dmikusa
commented on July 29, 2024
@andrejusc FYI
Note: image creation date here of 43y is due to made on purpose decision by buildpacks to count from Jan 1st 1980. Discussed here.
You can set --creation-time=now as an argument to pack build and it will use the current time at build for the creation date, if you prefer usable creation dates. I like that when I'm doing work locally.
from rust.
andrejusc
commented on July 29, 2024
@dmikusa - appreciate extra feedback.
from rust.
Related Issues (19)
- Use Test Binary | Help Wanted HOT 6
- Unable to find process HOT 7
- Update builder workflows
- Unable to launch after build HOT 2
- Dockerized rust applications not receiving signals from command line HOT 4
- draft-release isn't working after recent update
- Mark step in tldr as optional
- Rename to "rust" to better align with Paketo naming convention HOT 1
- Update Package Dependency workflow failing
- Getting hang around buildpacksio/lifecycle:cache.sbom layer HOT 2
- Add support for `upx`
- Move default branch from "master" to "main" HOT 1
- failed to build: Missing label HOT 13
- Wrong url for cargo-install in builder.toml HOT 1
- Outdated docs info in the builder section HOT 1
- Exception: ERROR: failed to launch: determine start command: when there is no default process a command is required HOT 8
- Add integration tests HOT 1
- jam summarize no long works HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rust.