Giter Club home page Giter Club logo

optimizer's Introduction

CosmWasm Optimizing Compiler

This is a Docker build with a locked set of dependencies to produce reproducible builds of cosmwasm smart contracts. It also does heavy optimization on the build size, using binary stripping and wasm-opt.

Image Description x86_64 images (default) ARM images (experimental1)
optimizer Combines rust-optimizer and workspace-optimizer in a single image cosmwasm/optimizer
DockerHub
cosmwasm/optimizer-arm64
DockerHub
rust-optimizer Single contract builds cosmwasm/rust-optimizer
DockerHub
cosmwasm/rust-optimizer-arm64
DockerHub
workspace-optimizer Multi-contract workspaces (e.g. cosmwasm-plus) cosmwasm/workspace-optimizer
DockerHub
cosmwasm/workspace-optimizer-arm64
DockerHub

1 ARM images do not produce the same output as the default images and are discouraged for production use. See Notice below.

Usage

This works for most cases, for monorepo builds see advanced

The easiest way is to simply use the published docker image. You must set the local path to the smart contract you wish to compile and it will produce an artifacts directory with <crate_name>.wasm and checksums.txt containing the hashes. This is just one file.

Run it a few times on different computers and use sha256sum to prove to yourself that this is consistent. I challenge you to produce a smaller build that works with the cosmwasm integration tests (and if you do, please make an issue/PR):

docker run --rm -v "$(pwd)":/code \
  --mount type=volume,source="$(basename "$(pwd)")_cache",target=/target \
  --mount type=volume,source=registry_cache,target=/usr/local/cargo/registry \
  cosmwasm/optimizer:0.15.0

Demo this with cosmwasm-examples (going into eg. erc20 subdir before running), with cosmwasm-plus, or with a sample app from cosmwasm-template.

Note that we use one registry cache (to avoid excessive downloads), but the target cache is a different volume per contract that we compile. This means no interference between contracts, but very fast recompile times when making minor adjustments to a contract you had previously created an optimized build for.

Mono Repos

Contracts as Workspace Members

This is designed for cosmwasm-plus samples. We use a separate docker image

Sometime you want many contracts to be related and import common functionality. This is exactly the case of cosmwasm-plus. In such a case, we can often not just compile from root, as the compile order is not deterministic and there are feature flags shared among the repos. This has lead to issues in the past.

For this use-case we made a second docker image, which will compile all the contracts/* folders inside the workspace and do so one-by-one in alphabetical order. It will then add all the generated wasm files to an artifacts directory with a checksum, just like the basic docker image (same output format).

To compile all contracts in the workspace deterministically, you can run:

docker run --rm -v "$(pwd)":/code \
  --mount type=volume,source="$(basename "$(pwd)")_cache",target=/target \
  --mount type=volume,source=registry_cache,target=/usr/local/cargo/registry \
  cosmwasm/optimizer:0.15.0

The downside is that to verify one contract in the workspace, you need to compile them all, but the majority of the build time is in dependencies, which are shared and cached between the various contracts and thus the time is sub-linear with respect to number of contracts.

Contracts excluded from Workspace

This is designed for cosmwasm samples. You cannot provide automatic verification for these

If you have a more complex build environment, you need to pass a few more arguments to define how to run the build process.

cosmwasm has a root workspace and many contracts under ./contracts/*, which are excluded in the top-level Cargo.toml. In this case, we compile each contract separately with it's own cache. However, since they may refer to packages via path (../../packages/std), we need to run the script in the repo root. In this case, we can use the optimize.sh command:

docker run --rm -v "$(pwd)":/code \
  --mount type=volume,source="devcontract_cache_burner",target=/code/contracts/burner/target \
  --mount type=volume,source=registry_cache,target=/usr/local/cargo/registry \
  cosmwasm/optimizer:0.15.0 ./contracts/burner

Caches

The build system uses the folder /target in its local file system for all Rust compilation results. This ensures there is no conflict with the target folder of the source repository. It also means that for each compilation, Cargo will have an empty target folder and have to re-compile all contracts and all dependencies. In order to avoid this, you can optionally mount a Docker volume into the file system, as highlighted here:

 docker run --rm -v "$(pwd)":/code \
+  --mount type=volume,source="$(basename "$(pwd)")_cache",target=/target \
   --mount type=volume,source=registry_cache,target=/usr/local/cargo/registry \
   cosmwasm/rust-optimizer:0.15.0

Using this cache is considered best practice and included in all our example call snippets.

Before version 0.13.0, the target folder was located at /code/target. This caused situations in which rust-optimizer/workspace-optimizer wrote to the target folder of the host in an unintended way. By ensuring the target location is not a subfolder of the mounted source code we can avoid those sort of problems.

Development

Take a look at the Makefile You can edit the Dockerfile (in a fork), and run make build to compile it.

Notice

This has been tested on Linux (Ubuntu / Debian). There are currently versions of both optimizers for two processor architectures: Intel/Amd 64-bits, and Arm 64-bits (these run natively on Mac M1 machines).

However, the native Arm version produces different wasm artifacts than the Intel version. Given that that impacts reproducibility, non-Intel images and build artifacts contain a "-arm64" suffix, to differentiate and flag them.

Arm images are released to ease development and testing on Mac M1 machines. For release / production use, only contracts built with the Intel optimizers must be used.

optimizer's People

Contributors

aelesbao avatar ethanfrey avatar harryscholes avatar kerber0x avatar larry0x avatar mandrean avatar maurolacy avatar snoyberg avatar ueco-jb avatar webmaster128 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

optimizer's Issues

Allow the containers to use more than one core

I haven't gone deep into the details here, but noticed that when running worspace-optimizer, it maxes out one of the CPUs while the rest are idle.

I tried passing --cpus 3.5 but it didn't make a difference.

Find new name for this repo

In deus-labs/cw-contracts#60 a discussion about this repo's name and the docker image name evolved. Let's move this discussion here.

Right now, we have

  • Repo: CosmWasm/cosmwasm-opt
  • Docker: confio/cosmwasm-opt

I'd like to implement the following changes for the new name

  1. Use the docker organization cosmwasm
  2. Add the term "rust" to prepare for a similar project for assemblyscript
  3. (bonus) Keep the term "optimize" somehow, as this remains the goal here and differentiates this from simply running cargo

Suggestion 1 (by Ethan):

  • Repo: CosmWasm/rust-builder
  • Docker: cosmwasm/rust-builder

Alternative:

  • Repo: CosmWasm/rust-optimizer
  • Docker: cosmwasm/rust-optimizer

Other ideas? Other aspects to consider?

Ceate a base image for *-optimizer

Given that we have two optimizers (rust-optimizer and workspace-optimizer) and that they share most of their docker setup (i.e. SHARED blocks in *.Dockerfile) let's consider creating a base-optimizer docker image, to build a base image with the shared part, and then re-use it for both optimizers. This will also ease maintencence / updates, as most of the regular changes (rust version and so) would need to be done in just one place.

Alternatively, let's consider merging both optimizers into one. workspace-optimizer functionality could be triggered either by passing a flag ([-w|--workspace], or be automatically detected.

Check memory usage when emulating x86 images on M1/M2 macs

When cross compiling the 0.12.10 x86 image on ARM64 macs, the RUSTFLAGS='-C link-arg=-s' cargo build --release step in the builder image required a lot of RAM. 20 GB was not enought but 25 GB worked.

For maintaining this repository this is fine. However, I wonder what the users of rust-optimizer/workspace-optimizer have to configure when using the resulting image. We need to measure this (test a few settings with a few contracts) and document it.

It is likely the memory consumtion of the Rust compiler changed between 0.12.10 (Rust 1.65.0) and 0.12.9 (Rust 1.64.0).

error: could not find Cargo.toml

Hello,

I tryed rust-optimizer after compiling the template from cw-template but when I executed it I get
error: could not find Cargo.toml in /code or any parent directory

For information I'm working on wsl, i don't know if that can help.
Thanks in advance !

Error when running optimizer on cw-template.

Hey,
I am trying to optimize my smart contact, but couldn't get behind some errors the rust-optimizer returns.

When I run rust-optimizer in the root folder of a clean copy of https://github.com/InterWasm/cw-template I get these errors:

$ docker run --rm -v "$(pwd)":/code --mount type=volume,source="$(basename "$(pwd)")_cache",target=/code/target --mount type=volume,source=registry_cache,target=/usr/local/cargo/registry cosmwasm/rust-optimizer:0.11.3

info: RUSTC_WRAPPER=sccache
Info: sccache stats before build
Compile requests                      0
Compile requests executed             0
Cache hits                            0
Cache misses                          0
Cache timeouts                        0
Cache read errors                     0
Forced recaches                       0
Cache write errors                    0
Compilation failures                  0
Cache errors                          0
Non-cacheable compilations            0
Non-cacheable calls                   0
Non-compilation calls                 0
Unsupported compiler calls            0
Average cache write               0.000 s
Average cache read miss           0.000 s
Average cache read hit            0.000 s
Failed distributed compilations       0
Cache location                  Local disk: "/root/.cache/sccache"
Cache size                            0 bytes
Max cache size                       10 GiB
Building contract in /code ...
   Compiling quote v1.0.9
   Compiling uint v0.9.3
error: edition 2021 is unstable and only available with -Z unstable-options.

error: could not compile `uint`

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed
$ rustup update
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
info: checking for self-updates

   stable-x86_64-unknown-linux-gnu unchanged - rustc 1.60.0 (7737e0b5c 2022-04-04)
  nightly-x86_64-unknown-linux-gnu unchanged - rustc 1.62.0-nightly (4dd8b420c 2022-05-01)

info: cleaning up downloads & tmp directories

I tried changing rustup default to stable and nightly, but neither worked.

Would be greate if someone knows how to fix this or has any hints.

Cheers,
Matthiaas.

error: lock file

Hi! I'm stuck trying to optimize a smart contract.
When I run rust-optimizer in the root folder of a clean copy of https://github.com/InterWasm/cw-template I get this problem:
docker run --rm -v "$(pwd)":/code --mount type=volume,source="$(basename "$(pwd)")_cache",target=/code/target --mount type=volume,source=registry_cache,target=/usr/local/cargo/registry cosmwasm/rust-optimizer:0.12.10

Info: RUSTC_WRAPPER=sccache
Info: sccache stats before build
Compile requests                      0
Compile requests executed             0
Cache hits                            0
Cache misses                          0
Cache timeouts                        0
Cache read errors                     0
Forced recaches                       0
Cache write errors                    0
Compilation failures                  0
Cache errors                          0
Non-cacheable compilations            0
Non-cacheable calls                   0
Non-compilation calls                 0
Unsupported compiler calls            0
Average cache write               0.000 s
Average cache read miss           0.000 s
Average cache read hit            0.000 s
Failed distributed compilations       0
Cache location                  Local disk: "/root/.cache/sccache"
Cache size                            0 bytes
Max cache size                       10 GiB
Building contract in /code ...
    Updating crates.io index
error: the lock file /code/Cargo.lock needs to be updated but --locked was passed to prevent this
If you want to try to generate the lock file without accessing the network, remove the --locked flag and use --offline instead.

Can somebody help me please?

Finished, status of exit status: 137

I am trying to optimize an Archway contract built from the following guide: https://docs.archway.io/docs/create/guides/my-first-dapp/start but the process seems to be killed before completiton with the following message: Finished, status of exit status: 137

The command executed in the root project folder: cargo run-script optimize

The full print out is as follows:

cargo run-script optimize
Running script 'optimize': 'docker run --rm -v "$(pwd)":/code -e CARGO_TERM_COLOR=always --mount type=volume,source="$(basename "$(pwd)")_cache",target=/code/target --mount type=volume,source=registry_cache,target=/usr/local/cargo/registry cosmwasm/rust-optimizer:0.12.5
'
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Info: RUSTC_WRAPPER=sccache
Info: sccache stats before build
Compile requests                      0
Compile requests executed             0
Cache hits                            0
Cache misses                          0
Cache timeouts                        0
Cache read errors                     0
Forced recaches                       0
Cache write errors                    0
Compilation failures                  0
Cache errors                          0
Non-cacheable compilations            0
Non-cacheable calls                   0
Non-compilation calls                 0
Unsupported compiler calls            0
Average cache write               0.000 s
Average cache read miss           0.000 s
Average cache read hit            0.000 s
Failed distributed compilations       0
Cache location                  Local disk: "/root/.cache/sccache"
Cache size                            0 bytes
Max cache size                       10 GiB
Building contract in /code ...
    Updating crates.io index
Killed
Finished, status of exit status: 137

Updating the `wasm-opt` tool

The newest version of the wasm-opt tool was released ten months ago so it is sometimes already. Apparently, it allows saving 25% of the binary size in some cases. We should consider migrating to it especially considering existing problems like: CosmWasm/wasmd#1038.

Missing sense of checksums_intermediate.txt

The checksums_intermediate.txt file is generated in the artifacts folder with the following data:

ARBITRARY_SHA_256_HASH  ./target/wasm32-unknown-unknown/release/CRATE_NAME.wasm

However, the ARBITRARY_SHA_256_HASH is not hash of the file ./target/wasm32-unknown-unknown/release/CRATE_NAME.wasm.

I'm really not sure how should I use it, and it is not mentioned in readme what the file is responsible for. Please, let me know why the file is needed, or maybe another hash should be written there.

Publish CI built images to docker hub

It would be nice to publish CI images built on release tags automatically to docker hub.

That way we can also leverage the CI's native Intel and Arm builders.

thread 'main' panicked at 'assertion failed: error_code.success()', src/main.rs:78:9

Using cosmwasm/workspace-optimizer:0.12.9

Local

$ rustup show                             
Default host: aarch64-apple-darwin
rustup home:  /Users/xx/.rustup

installed targets for active toolchain
--------------------------------------

aarch64-apple-darwin
wasm32-unknown-unknown

active toolchain
----------------

stable-aarch64-apple-darwin (default)
rustc 1.65.0 (897e37553 2022-11-02)

It errors out as following

1.64.0-x86_64-unknown-linux-musl (default)
cargo 1.64.0 (387270bc7 2022-09-16)
Building artifacts in workspace...
Found workspace member entries: ["packages/*", "contracts/*"]
Package directories: ["xx..."]
Contracts to be built: ["xx..."]
Building "contracts/xx" ...
warning: profiles for the non root package will be ignored, specify profiles at the workspace root:
package:   /code/contracts/xx/Cargo.toml
workspace: /code/Cargo.toml
warning: profiles for the non root package will be ignored, specify profiles at the workspace root:
package:   /code/contracts/xx/Cargo.toml
workspace: /code/Cargo.toml
warning: profiles for the non root package will be ignored, specify profiles at the workspace root:
package:   /code/contracts/xx/Cargo.toml
workspace: /code/Cargo.toml
warning: profiles for the non root package will be ignored, specify profiles at the workspace root:
package:   /code/contracts/xx/Cargo.toml
workspace: /code/Cargo.toml
    Updating crates.io index
thread 'main' panicked at 'assertion failed: error_code.success()', src/main.rs:78:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
make: *** [optimize] Error 101

"missing field `workspace`" when running in docker

I have a cosmwasm contract and I'm running the optimizer via docker like this in the contract directory:

docker run --rm -v "$(pwd)":/code   --mount type=volume,source="$(basename "$(pwd)")_cache",target=/code/target   --mount type=volume,source=registry_cache,target=/usr/local/cargo/registry -e RUST_BACKTRACE=full   cosmwasm/workspace-optimizer:0.12.6

And it gives the following error:

1.60.0-x86_64-unknown-linux-musl (default)
cargo 1.60.0 (d1fd9fe2c 2022-03-01)
Building artifacts in workspace ...
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { inner: ErrorInner { kind: Custom, line: None, col: 0, message: "missing field `workspace`", key: [] } }', src/main.rs:35:55

The contract code is here

Test builds in CI

It would be nice to test all images in the CI by compiling an example contract with them.

Lock the dependencies when building

cargo build (and wasm-pack build) will always use the deps referenced in Cargo.toml, but may update Cargo.lock with eg. patch releases. In some cases, this can change the actual code in the build path and thus the wasm bytecode/hash. We want to avoid this, so a contract can be safely compiled 6 months later with the same hash.

We should fix the line https://github.com/confio/cosmwasm-opt/blob/master/optimize.sh#L8 to pass in a flag to cargo. I think adding -- --locked will do this, but need to verify

Processing custom cargo environment

Problem

Provide some special cargo env variables into the docker container crashes the script.

Background

I use custom CARGO_TARGET_DIR and CARGO_BUILD_TARGET_DIR variables that are responsible for target folder name that is target by default.
It is important to provide the variables to docker container as the project folder in mounted as volume and cargo shouldn't create additional target directory.

Solution

Use corresponding env variables when constructing paths to files.
Here optimize.sh#L42 change is needed: replace target with "$CARGO_TARGET_DIR".

Adapt rust optimizer for cosmwasm-plus

We currently support two modes:

  • The simple mode is when the repo has exactly one contract in it, no arguments provided.
  • The multi-repo mode was designed for cosmwasm-examples, where each contract is not in the workspace

However, it fails when we try to build cosmwasm-plus:

  • Multi-repo with all contracts in the same workspace (needed as we import from other packages)

There are two key differences here that cause the build error:

  1. (Stack trace below) target where the code goes is not in the directory with the contract's Cargo.toml, but in the repo root
  2. (May cause an issue) multiple contracts will all compile to the same target dir. This means *.wasm will not be unique, but we need the name of the contract we compile.

Output of a failed build (see bottom):

$ docker run --rm -v "$(pwd)":/code \
>   --mount type=volume,source="cosmwasm_plus_cache",target=/code/target \
>   --mount type=volume,source=registry_cache,target=/usr/local/cargo/registry \
>   cosmwasm/rust-optimizer:0.9.0 ./contracts/cw1-subkeys
Building contract in /code/contracts/cw1-subkeys
    Updating crates.io index
 Downloading crates ...
   ....
   Compiling cw20 v0.1.0 (/code/packages/cw20)
   Compiling cw1-whitelist v0.1.0 (/code/contracts/cw1-whitelist)
   Compiling cw1-subkeys v0.1.0 (/code/contracts/cw1-subkeys)
    Finished release [optimized] target(s) in 1m 08s
Failed opening 'target/wasm32-unknown-unknown/release/*.wasm'

That failed as it looked for /code/contracts/cw1-subkeys/target not /code/target

Upgrade rust-optimizer Rust version

Rust-optimizer compilation is failing.

This is because contracts now require 1.49.0 at least.

After the CosmWasm 0.14.0-alpha1 release, let's upgrade rust-optimizer to use the latest Rust version (1.50.0). We can then use the generated alpha2 contract artifacts in alpha1 too.

wasm-opt flags -Os vs. -Oz

Locally I'm testing the two optimization modes with wasm-opt version 110:

$ wasm-opt --help
  …
   -Os                                          execute default optimization 
                                                passes, focusing on code size 

   -Oz                                          execute default optimization 
                                                passes, super-focusing on code 
                                                size 
  …

For my tests, there is a difference in the order of magnitute of 1-2%:

# -Os

-rwxr-xr-x  1 me  staff  192130 16 Okt 17:27 nois_demo.wasm
-rwxr-xr-x  1 me  staff  480903 16 Okt 17:27 nois_oracle.wasm
-rwxr-xr-x  1 me  staff  225705 16 Okt 17:27 nois_proxy.wasm

# -Oz

-rwxr-xr-x  1 me  staff  189263 16 Okt 17:29 nois_demo.wasm
-rwxr-xr-x  1 me  staff  472586 16 Okt 17:29 nois_oracle.wasm
-rwxr-xr-x  1 me  staff  223632 16 Okt 17:29 nois_proxy.wasm

So for now I don't see a strong need to move away from the well established -Os.

please push a later version of arm64

rust 1.64 has some changes in the cargo/workspace stuff, which my projects uses.. I just switch to a M1 and now I can't optimize contracts anymore ;-(

unresolved import cosmwasm_std::testing on Cosmwasm/cw-plus

Inside https://github.com/CosmWasm/cw-plus/tree/v0.13.0, upon running

docker run --rm -v "$(pwd)":/code \
  --mount type=volume,source="$(basename "$(pwd)")_cache",target=/code/target \
  --mount type=volume,source=registry_cache,target=/usr/local/cargo/registry \
  cosmwasm/rust-optimizer-arm64:0.12.5
Info: RUSTC_WRAPPER=sccache
Info: sccache stats before build
Compile requests                      0
Compile requests executed             0
Cache hits                            0
Cache misses                          0
Cache timeouts                        0
Cache read errors                     0
Forced recaches                       0
Cache write errors                    0
Compilation failures                  0
Cache errors                          0
Non-cacheable compilations            0
Non-cacheable calls                   0
Non-compilation calls                 0
Unsupported compiler calls            0
Average cache write               0.000 s
Average cache read miss           0.000 s
Average cache read hit            0.000 s
Failed distributed compilations       0
Cache location                  Local disk: "/root/.cache/sccache"
Cache size                            0 bytes
Max cache size                       10 GiB
Building contract in /code ...
   Compiling cw-storage-plus v0.13.0 (/code/packages/storage-plus)
   Compiling cw2 v0.13.0 (/code/packages/cw2)
   Compiling cw4 v0.13.0 (/code/packages/cw4)
   Compiling cw-controllers v0.13.0 (/code/packages/controllers)
   Compiling cw-multi-test v0.13.0 (/code/packages/multi-test)
error[E0432]: unresolved import `cosmwasm_std::testing`
 --> packages/multi-test/src/app.rs:5:19
  |
5 | use cosmwasm_std::testing::{mock_env, MockApi, MockStorage};
  |                   ^^^^^^^ could not find `testing` in `cosmwasm_std`

error[E0432]: unresolved import `cosmwasm_std::testing`
  --> packages/multi-test/src/wasm.rs:24:19
   |
24 | use cosmwasm_std::testing::mock_wasmd_attr;
   |                   ^^^^^^^ could not find `testing` in `cosmwasm_std`

warning: ignoring -C extra-filename flag due to -o flag

For more information about this error, try `rustc --explain E0432`.
warning: `cw-multi-test` (lib) generated 1 warning
error: could not compile `cw-multi-test` due to 2 previous errors; 1 warning emitted
warning: build failed, waiting for other jobs to finish...
error: build failed

cargo build however works just fine.

See CosmWasm/cw-plus#681 for additional context

ARM build not published

In 0.12.2 / #56 an ARM build of this project was created. This makes the CosmWasm development experience of ARM users much smother as the contract compilation does not need to go through an emulator. However, during our testing it was discovered that this ARM build produces different compilation results than the Intel build. As a consequence the build is not proproducible.

I see a few things to move forward:

  1. Ignore reproducible builds (probably bad)
  2. Find a way to ensure both images produce the same results
  3. Have different image names in order to make explicit on which system a contract is built on

Input welcome. For now, the ARM build is not published to DockerHub.

ErrorInner { kind: Custom, line: None, col: 0, message: "missing field `workspace`", key: [] } }

Facing this issue while using rust-optimizer:

docker run --rm -v "$(pwd)":/code   --mount type=volume,source="$(basename "$(pwd)")_cache",target=/code/target   --mount type=volume,source=registry_cache,target=/usr/local/cargo/registry   cosmwasm/workspace-optimizer:0.12.5
1.58.1-x86_64-unknown-linux-musl (default)
cargo 1.58.0 (f01b232bc 2022-01-19)
Building artifacts in workspace ...
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { inner: ErrorInner { kind: Custom, line: None, col: 0, message: "missing field `workspace`", key: [] } }', src/main.rs:35:55
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Any suggestions on how to resolve this?

Add tooling to check crate matches hash

We want to be able to validate the source code behind a repo.

We can now produce the hash deterministically from a local repo, we need to integrate this more with the publishing system:

  • Take crate name and version, as well as a claimed wasm hash`. Download them and return error unless hash matches

Follow-up:

  • We then need to integrate this script with wasmcli to validate a code sample on the blockchain. (TODO: open issue)

Update for cosmwasm 0.7

Update some dependencies, any desired cleanup.

cosmwasm-opt:0.6.x should continue to work with new code, but it is nice to do an update to use newer rust versions, etc.

[workspace-optimizer]: build fails if workspaces inside `contracts/*` hold path dependencies other than `contracts/*` paths

If internal crates exist other than contracts/*, it seems that workspace-optimizer doesn't build with wasm target.
It appears to be building with no_std, not wasm.

ex path tree)

├── contracts
│   └── counter-contract
├── crates
│   ├── msg

Here workspace-optimizer will fail if counter-contract is holding msg as path dependency.

Does anyone know anything about this?

Update to binaryen 102

Just because v102 (latest) should in principle have better / more optimizations.

As part of this, let's compile binaryen from sources for both archs (Intel and Arm). And, let's run binaryen tests.

Ensure log messages are flushed

In a console log like this

Building contracts/cw3-flex-multisig ...
   Compiling cw4 v0.6.0-beta1 (/code/packages/cw4)
   Compiling cw3-flex-multisig v0.6.0-beta1 (/code/contracts/cw3-flex-multisig)
    Finished release [optimized] target(s) in 30.73s
Building contracts/cw4-group ...
   Compiling cw-controllers v0.6.0-beta1 (/code/packages/controllers)
   Compiling cw4-group v0.6.0-beta1 (/code/contracts/cw4-group)
    Finished release [optimized] target(s) in 21.40s
Building contracts/cw4-stake ...
   Compiling cw4-stake v0.6.0-beta1 (/code/contracts/cw4-stake)
    Finished release [optimized] target(s) in 15.80s
Building contracts/cw721-base ...
   Compiling cw721 v0.6.0-beta1 (/code/packages/cw721)
   Compiling cw721-base v0.6.0-beta1 (/code/contracts/cw721-base)
    Finished release [optimized] target(s) in 26.00s
done.
Optimizing artifacts in workspace...Optimizing ../target/wasm32-unknown-unknown/release/deps/cw1155_base.wasm...done.
Optimizing ../target/wasm32-unknown-unknown/release/deps/cw1_subkeys.wasm...done.
Optimizing ../target/wasm32-unknown-unknown/release/deps/cw1_whitelist.wasm...done.
Optimizing ../target/wasm32-unknown-unknown/release/deps/cw20_atomic_swap.wasm...done.

Too long with no output (exceeded 10m0s): context deadline exceeded

it is hard to understand at which point exactly the build hangs. Part of it is because it is not shown which file is currently optimized. It is very unlikely that the echo command causes problems. But because we use echo -n it is likely the console log is printed but not flushed (by the shell, by docker or by cirlceci). I think we should remove the -ns in order to get partial information directly.

Add flag for disabling default features when building contracts

multichain projects may want to enable/disable contract features for different chains they deploy on.

for example, as cw20s are phased out in favor of TokenFactory some deployments will need cw20 support, and some will use only native tokens. for example, Juno makes extensive use of the cw20 standard, but Osmosis deals entirely in native tokens.

in DAO DAO we're moving cw20 support behind a cargo feature flag that is enabled by default. it would be great to be able to enable / disable that as an argument to workspace-optimizer to make reproducible builds easier.

Fix how cosmwasm-plus compiles

We made a great simplification by just running the compile step one time on the top level folder even with multple workspaces: https://github.com/CosmWasm/rust-optimizer/blob/v0.10.1/optimize.sh#L25

However, when we run this on cosmwasm-plus we get cw1_whitelist with a size of 13kb. That is due to the 'library' feature being enabled when imported by subkeys, so no exports and almost everything gets optimized out.

Features are the union of all crates referencing it. In order to allow these crates to be imported as libraries and also compiled alone, we need to run the cargo build script once per contract. We also decided we cannot pass argument on the command-line, so I propose something like the following:

  • cd $1
  • check Cargo.toml exists
  • if there is a [package]name entry then compile there
  • if there is a [package]members entry, then go into each directory listed there and run the build script in each one

This should work the same for cosmwasm and cosmwasm-examples and cosmwasm-template. The only change would be cosmwasm-plus, where we get the previous behavior (listing .contracts/*) without passing that explicitly. We may need a tool to parse the toml file beyond grep. https://github.com/freshautomations/stoml is interesting but not sure if we can apt install it.

Getting base16ct error

I am not sure if this is my Rust setup, but I am consistently trying to run:

docker run --rm -v "$(pwd)":/code \
  --mount type=volume,source="$(basename "$(pwd)")_cache",target=/code/target \
  --mount type=volume,source=registry_cache,target=/usr/local/cargo/registry \
  cosmwasm/rust-optimizer:0.12.3 ./contracts/*/

in the cw-contracts repo and getting:

error: failed to download `base16ct v0.1.1`

Caused by:
  unable to get packages from source

Caused by:
  failed to parse manifest at `/usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/base16ct-0.1.1/Cargo.toml`

Caused by:
  feature `edition2021` is required

  The package requires the Cargo feature called `edition2021`, but that feature is not stabilized in this version of Cargo (1.55.0 (32da73ab1 2021-08-23)).

and indeed, I see the manifest has this:
https://github.com/RustCrypto/formats/blame/7cadd9d3696b6ec06b79281e08805625dd8f3187/base64ct/Cargo.toml#L16

edition = "2021"

but that's been there for 10 months, so maybe that's not as new as I'd think.

Anyway, writing this issue but again, not positive if it's my setup.

Add a suffix to artifacts built on Arm64

Given that ARM builds are different than Intel builds, and that impacts reproducibility, let's add a suffix to ARM built artifacts.

Along with a notice in the README.md, stating clearly that it does not produce the same builds than the Intel rust-optimizer. And, that for release / production use, only contracts built with the Intel one must be used.

warning: spurious network error

Hi! I'm stuck trying to optimize a smart contract.
When I run rust-optimizer in the root folder of a clean copy of https://github.com/InterWasm/cw-template I get some network problems...

docker run --rm -v "$(pwd)":/code \
  --mount type=volume,source="$(basename "$(pwd)")_cache",target=/code/target \
  --mount type=volume,source=registry_cache,target=/usr/local/cargo/registry \
  cosmwasm/rust-optimizer:0.12.4
Info: RUSTC_WRAPPER=sccache
Info: sccache stats before build
Compile requests                      0
Compile requests executed             0
Cache hits                            0
Cache misses                          0
Cache timeouts                        0
Cache read errors                     0
Forced recaches                       0
Cache write errors                    0
Compilation failures                  0
Cache errors                          0
Non-cacheable compilations            0
Non-cacheable calls                   0
Non-compilation calls                 0
Unsupported compiler calls            0
Average cache write               0.000 s
Average cache read miss           0.000 s
Average cache read hit            0.000 s
Failed distributed compilations       0
Cache location                  Local disk: "/root/.cache/sccache"
Cache size                            0 bytes
Max cache size                       10 GiB
Building contract in /code ...
    Updating crates.io index
warning: spurious network error (2 tries remaining): failed to resolve address for github.com: Name does not resolve; class=Net (12)
warning: spurious network error (1 tries remaining): failed to resolve address for github.com: Name does not resolve; class=Net (12)
error: failed to get `cosmwasm-std` as a dependency of package `c01 v0.1.0 (/code)`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  network failure seems to have happened
  if a proxy or similar is necessary `net.git-fetch-with-cli` may help here
  https://doc.rust-lang.org/cargo/reference/config.html#netgit-fetch-with-cli

Caused by:
  failed to resolve address for github.com: Name does not resolve; class=Net (12)

I haven't a proxy or similar...

Can somebody help me please?

Docker "latest" tag is not found

Overview

Docker tag latest is useful as it is possible for scripts using the docker image to be always up-to-date.

Problem

No latest tag found:

docker: Error response from daemon: manifest for \
cosmwasm/rust-optimizer:latest not found: manifest unknown: manifest unknown.

Note

If it may happen that latest tag may be unstable, provide additional stable tag for marking container version tested by time.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.