Comments (11)
justsmth
commented on August 15, 2024
4
I was able to get your repo to cross-build using aws-lc-rs. I posted my changes on this branch. Feel free to adapt parts of that change for your own purposes.
I also updated this PR as ready for review. Feel free to contact us if you have any other problems. Thanks!
from aws-lc-rs.
zonyitoo
commented on August 15, 2024
1
Ok.. I know why you cannot reproduce. shadowsocks-rust could compile with default features, but fails with --features "full-extra" and --features "full".
@justsmth Please try again with --features "full" or --features "full-extra".
I am still investigating what's the key differences between the dependencies.
from aws-lc-rs.
justsmth
commented on August 15, 2024
I'm unable to reproduce the error you're reporting using using the aws-lc-rs repo. I believe the problem related to the Cross.toml configuration that you have here.
For reference, you can see the configuration that we use for Cross here, which extends the configuration from our Cross Dockerfile.
It's not clear to me what a few of the steps in your configuration are attempting, but I suspect your configuration could be much simpler. Possibly something like:
[build]
pre-build = [
"apt-get install --assume-yes --no-install-recommends build-essential cmake nasm clang",
"cargo install --force --locked bindgen-cli"
]
EDIT:
I experimented more with this. The "cargo install --force --locked bindgen-cli" doesn't work the way I'd expect, which likely explains the "rustup" commands you were using in your Cross.toml. I'll see if I can find a better solution.
from aws-lc-rs.
zonyitoo
commented on August 15, 2024
Yes, there is no cargo in the base image. So I have to install one.
The Dockerfile in this repo, installs build-essential cmake golang-go, and Cross.toml also installs nasm clang wget and also a winehq-stable.
Comparing to my own Cross.toml, I only installed build-essential cmake nasm clang, the missing one are golang-go winehq-stable. Do they necessary while building aws-lc?
from aws-lc-rs.
justsmth
commented on August 15, 2024
the missing one are golang-go winehq-stable. Do they necessary while building aws-lc?
golang-go is only needed for FIPS builds. Installing the latest winehq-stable was (and still is?) needed for testing on x86_64-pc-windows-gnu.
I put up a draft PR that follows the pattern you use for installing bindgen-cli inside the container.
from aws-lc-rs.
zonyitoo
commented on August 15, 2024
I have just merged your changed into my master branch, but still not working well.
$ cross build --target "x86_64-pc-windows-gnu" --features "full-extra"
...
The following warnings were emitted during compilation:
warning: [email protected]: Generating bindings - external bindgen. Platform: x86_64-pc-windows-gnu
error: failed to run custom build command for `aws-lc-sys v0.17.0`
Caused by:
process didn't exit successfully: `/target/debug/build/aws-lc-sys-9800ca95c1e3d479/build-script-main` (exit status: 101)
--- stdout
cargo:rerun-if-env-changed=AWS_LC_SYS_NO_PREFIX
cargo:rerun-if-env-changed=AWS_LC_SYS_INTERNAL_BINDGEN
cargo:rerun-if-env-changed=AWS_LC_SYS_EXTERNAL_BINDGEN
cargo:rerun-if-env-changed=AWS_LC_SYS_NO_ASM
cargo:rerun-if-env-changed=AWS_LC_SYS_CMAKE_BUILDER
cargo:rerun-if-env-changed=AWS_LC_SYS_STATIC
cargo:warning=Generating bindings - external bindgen. Platform: x86_64-pc-windows-gnu
cargo:rerun-if-env-changed=AWS_LC_SYS_INCLUDES
cargo:rustc-cfg=use_bindgen_generated
cargo:rerun-if-env-changed=AWS_LC_SYS_STATIC
CMAKE_TOOLCHAIN_FILE_x86_64-pc-windows-gnu = None
CMAKE_TOOLCHAIN_FILE_x86_64_pc_windows_gnu = None
TARGET_CMAKE_TOOLCHAIN_FILE = None
CMAKE_TOOLCHAIN_FILE = None
CMAKE_GENERATOR_x86_64-pc-windows-gnu = None
CMAKE_GENERATOR_x86_64_pc_windows_gnu = None
TARGET_CMAKE_GENERATOR = None
CMAKE_GENERATOR = None
CMAKE_PREFIX_PATH_x86_64-pc-windows-gnu = None
CMAKE_PREFIX_PATH_x86_64_pc_windows_gnu = None
TARGET_CMAKE_PREFIX_PATH = None
CMAKE_PREFIX_PATH = None
CMAKE_x86_64-pc-windows-gnu = None
CMAKE_x86_64_pc_windows_gnu = None
TARGET_CMAKE = None
CMAKE = Some("cmake")
running: cd "/target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out/build" && CMAKE_PREFIX_PATH="" "cmake" "/cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0" "-DBUILD_SHARED_LIBS=0" "-DCMAKE_BUILD_TYPE=debug" "-DBORINGSSL_PREFIX=aws_lc_0_17_0_" "-DBORINGSSL_PREFIX_HEADERS=/cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/generated-include" "-DBUILD_TESTING=OFF" "-DBUILD_LIBSSL=OFF" "-DDISABLE_PERL=ON" "-DDISABLE_GO=ON" "-DCMAKE_SYSTEM_NAME=Windows" "-DCMAKE_SYSTEM_PROCESSOR=AMD64" "-DCMAKE_INSTALL_PREFIX=/target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -m64" "-DCMAKE_C_COMPILER=/usr/bin/x86_64-w64-mingw32-gcc-posix" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -m64" "-DCMAKE_CXX_COMPILER=/usr/bin/x86_64-w64-mingw32-g++-posix" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -m64" "-DCMAKE_ASM_COMPILER=/usr/bin/x86_64-w64-mingw32-gcc-posix" "--no-warn-unused-cli"
Not searching for unused variables given on the command line.
-- The C compiler identification is GNU 7.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/x86_64-w64-mingw32-gcc-posix - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Go not found. Disabling some code generation and using pre-generated code in generated-src/
-- Perl not found. Disabling some code generation and using pre-generated code in generated-src/
-- stdalign_check.c probe is positive, enabling AWS_LC_STDALIGN_AVAILABLE
-- builtin_swap_check.c probe is positive, enabling AWS_LC_BUILTIN_SWAP_SUPPORTED
-- The ASM_NASM compiler identification is NASM
-- Found assembler: /usr/bin/nasm
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Configuring done
-- Generating done
-- Build files have been written to: /target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out/build
running: cd "/target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out/build" && MAKEFLAGS="-j --jobserver-fds=8,9 --jobserver-auth=8,9" "cmake" "--build" "." "--target" "install" "--config" "Debug"
[ 0%] Built target boringssl_prefix_symbols
[ 1%] Building C object aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/bcm.c.obj
[ 1%] Building C object aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/fips_shared_support.c.obj
[ 2%] Building C object aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/cpucap/cpucap.c.obj
[ 2%] Building ASM_NASM object aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/aesni-gcm-avx512.asm.obj
aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/build.make:120: recipe for target 'aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/aesni-gcm-avx512.asm.obj' failed
CMakeFiles/Makefile2:257: recipe for target 'aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/all' failed
Makefile:135: recipe for target 'all' failed
--- stderr
Copying platform assembly files from /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/aws-lc/generated-src/win-x86_64/crypto/ to /target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out/build/aws-lc/crypto
/target/x86_64-pc-windows-gnu/debug/build/aws-lc-sys-156e9005524d4eed/out/build/aws-lc/crypto/fipsmodule/aesni-gcm-avx512.asm:10: fatal: unable to open include file `openssl/boringssl_prefix_symbols_nasm.inc'
make[2]: *** [aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/aesni-gcm-avx512.asm.obj] Error 1
make[1]: *** [aws-lc/crypto/fipsmodule/CMakeFiles/fipsmodule.dir/all] Error 2
make: *** [all] Error 2
thread 'main' panicked at /cargo/registry/src/index.crates.io-6f17d22bba15001f/cmake-0.1.50/src/lib.rs:1098:5:
command did not execute successfully, got: exit status: 2
build script failed, must exit now
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
CI should always passed. These failed targets are not included in normal CI tests.
- x86_64-pc-windows-gnu
- armv7-unknown-linux-musleabihf
- armv7-unknown-linux-gnueabihf
- arm-unknown-linux-gnueabihf
Interestingly CI tests in your PR are all passed. WHY??
from aws-lc-rs.
zonyitoo
commented on August 15, 2024
clang should also explicitly install the latest version clang-8 like this:
diff --git a/docker/linux-cross/Dockerfile b/docker/linux-cross/Dockerfile
index d17c46cb..5f17c9a5 100644
--- a/docker/linux-cross/Dockerfile
+++ b/docker/linux-cross/Dockerfile
@@ -4,7 +4,7 @@ FROM $CROSS_BASE_IMAGE
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
- apt-get install --assume-yes --no-install-recommends build-essential cmake nasm llvm-dev libclang-dev clang && \
+ apt-get install --assume-yes --no-install-recommends build-essential cmake nasm llvm-8-dev libclang-8-dev clang-8 && \
git config --global --add safe.directory '*' && \
rm -rf /tmp/*Otherwise some of the targets may failed with these errors:
The following warnings were emitted during compilation:
warning: [email protected]: Generating bindings - external bindgen. Platform: x86_64-unknown-freebsd
error: failed to run custom build command for `aws-lc-sys v0.17.0`
Caused by:
process didn't exit successfully: `/target/debug/build/aws-lc-sys-9800ca95c1e3d479/build-script-main` (exit status: 101)
--- stdout
cargo:rerun-if-env-changed=AWS_LC_SYS_NO_PREFIX
cargo:rerun-if-env-changed=AWS_LC_SYS_INTERNAL_BINDGEN
cargo:rerun-if-env-changed=AWS_LC_SYS_EXTERNAL_BINDGEN
cargo:rerun-if-env-changed=AWS_LC_SYS_NO_ASM
cargo:rerun-if-env-changed=AWS_LC_SYS_CMAKE_BUILDER
cargo:rerun-if-env-changed=AWS_LC_SYS_STATIC
cargo:warning=Generating bindings - external bindgen. Platform: x86_64-unknown-freebsd
cargo:rerun-if-env-changed=AWS_LC_SYS_INCLUDES
--- stderr
Failure invoking external bindgen!
bindgen-PARAMS: --prefix-link-name aws_lc_0_17_0_ --rust-target 1.59 --with-derive-default --with-derive-eq --allowlist-file .*(/|\\)openssl(/|\\)[^/\\]+\.h --allowlist-file .*(/|\\)rust_wrapper\.h --rustified-enum point_conversion_form_t --default-macro-constant-type signed --formatter rustfmt --output /target/x86_64-unknown-freebsd/debug/build/aws-lc-sys-6ed5f5aacfdd0e3e/out/bindings.rs --raw-line
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0 OR ISC
/cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/include/rust_wrapper.h -- -I /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/include -I /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/aws-lc/include
bindgen-STDOUT: panicked at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/clang-sys-1.4.0/src/lib.rs:1731:1:
`libclang` function not loaded: `clang_getTranslationUnitTargetInfo`. This crate requires that `libclang` 3.9 or later be installed on your system. For more information on how to accomplish this, see here: https://rust-lang.github.io/rust-bindgen/requirements.html#installing-clang-39
bindgen-STDERR: /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/aws-lc/include/openssl/base.h:61:10: fatal error: 'stdlib.h' file not found
thread 'main' panicked at /cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.17.0/builder/main.rs:459:5:
aws-lc-sys build failed. Please enable the 'bindgen' feature on aws-lc-rs or aws-lc-sys.For more information, see the aws-lc-rs User Guide: https://aws.github.io/aws-lc-rs/index.html
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
from aws-lc-rs.
zonyitoo
commented on August 15, 2024
Here is the key differences between the two dependency trees: If there are rustls 0.23.* and rustls 0.22.* (or any other older version of rustls) in the dependency tree, then compilation will definately get errors on those specific platforms.
from aws-lc-rs.
justsmth
commented on August 15, 2024
clang should also explicitly install the latest version clang-8 like this:
The latest version of clang is actually clang-18: https://releases.llvm.org/download.html
The libclang version installed by a recent LTS release of Ubuntu will be newer than clang-8.
Otherwise some of the targets may failed with these errors:
This output shows that the problem is that bindgen is unable to locate libclang:
...
`libclang` function not loaded: `clang_getTranslationUnitTargetInfo`. This crate requires that `libclang` 3.9 or later be installed on your system. For more information on how to accomplish this, see here: https://rust-lang.github.io/rust-bindgen/requirements.html#installing-clang-39
...
I also had trouble with libclang being located when it's being installed from a "pre-build" in Cross.toml (as you have it). I've had more success by using a dedicated Dockerfile to setup the required libraries as we have here.
from aws-lc-rs.
justsmth
commented on August 15, 2024
Oh! I just now noticed the target your compiling for here:
warning: [email protected]: Generating bindings - external bindgen. Platform: x86_64-unknown-freebsd
We've not tested our build for the x86_64-unknown-freebsd target, so it's possibly broken.
If this is a platform you'd like us to support, please submit a separate issue requesting support for it.
from aws-lc-rs.
zonyitoo
commented on August 15, 2024
Ok, never mind, this was the platform I was running my build tests.
clang-8 is the latest version could be installed inside the Docker container. apt-get will install clang 3.5 by default.
On the other hand, I think the key issue here is that: If compiling with older version of rustls, aws-ls-sys will get different errors on every platforms that is not "pregenerated".
from aws-lc-rs.
Related Issues (20)
- AWS-LC’s libcrypto means, is it support for libcrypto of OpenSSL library added?
- Remove NASM requirement for windows builds
- Link to guide on bindgen requirement HOT 2
- Build fails for some targets because of missing _ISOC11_SOURCE preprocessor definition HOT 2
- Consolidate FFI bindings HOT 4
- Build fails for iOS simulator on x86_64 HOT 2
- Build failed at target: aarch64-pc-windows-msvc HOT 16
- Support streaming API for CBC HOT 1
- Compilation Errors on `x86_64-pc-windows-gnu` Target due to Uninitialized Variables in `aws-lc` HOT 4
- Linux -> Mac cannot cross compile aws-lc-sys HOT 6
- Cross build for target x86_64-pc-windows-gnu failed HOT 6
- Android build fails HOT 2
- Many warnings from cmake and can't build with `cc1.exe: all warnings being treated as errors` HOT 10
- armv7-unknown-linux-gnueabihf build failed. HOT 2
- arm-unknown-linux-gnueabihf build failed HOT 8
- Exported symbols conflict with boring-sys crate
- Linkage error when cross-compiling for `aarch64-pc-windows-msvc` HOT 43
- Cross compilation support for arm32 HOT 1
- Support for P521 curve with SHA256/SHA384 hash algorithms. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aws-lc-rs.