Comments (4)
Sounds good! Would you like to put up a PR for this?
from coreutils.
I'm currently doing some tests (WIP to add CI for OpenBSD :), GH workflow similar to the FreeBSD one).
I will propose a fix/PR to support Rust cache in the FreeBSD workflow and some improvements inspired by the FreeBSD workflow from the rustup project.
from coreutils.
Status of my work on FreeBSD workflow
After some analysis and (a lot of) tests, I have a working "rework" of the FreeBSD workflow using Swatinem/rust-cache
and mozilla-actions/sccache-action
:)
Reorg of FreeBSD workflow
The workflow have 2 jobs :
- "Style and Lint" to check Rust code with
cargo fmt
andcargo clippy
- "Tests" to build and test code with
cargo nextest
The both jobs are executed with the vmactions/freebsd-vm action : FreeBSD stable (current version 14.0) VM runned in Ubuntu latest
GH runner.
I have reorg the code of the workflow, inspired by the job build-freebsd
from the CI workflow of the rustup project => https://github.com/rust-lang/rustup/blob/master/.github/workflows/ci.yaml#L1025
Both jobs started with prepare
task to install necessary packages (with pkg
FreeBSD command), create a temporary user tester
(via shell) then run a script shell with sudo -E -u ${TEST_USER}
to execute further tasks:
- job "Style and Lint" => exec
ci/freebsd_style_lint.sh
script - job "Tests" => exec
ci/freebsd_tests.sh
script
With this separation between jobs defined in workflow (YAML format) and shell scripts, it's easier to modify them if needed.
I have also remove the definition of the jobs matrix (GH runner is always ubuntu-latest
) and define feature = "unix"
in workflow environment.
Action mozilla-actions/sccache-action
(The easiest part)
This action is only used in "Tests" job (not used for cargo fmt
and cargo clippy
in "Style and Lint" job).
- define
SCCACHE_GHA_ENABLED: "true"
andRUSTC_WRAPPER: "sccache"
in job env. - export
FEATURES SCCACHE_GHA_ENABLED RUSTC_WRAPPER ACTIONS_CACHE_URL ACTIONS_RUNTIME_TOKEN
variables for FreeBSD VM (variableenvs
for this action) - install
sccache
package for the FreeBSD VM - at the end of the
ci/freebsd_tests.sh
script, exec commandsccache --show-stats > "${WORKSPACE}/sccache-stats.txt" 2>&1
to get the sccache stats in a text file => copyback to the Ubuntu host (see the section below for thecopyback
enabled option). - add a final step to display the sccache stats, reading the outputs in the copied file.
During the "Tests" job, sccache is correctly used by rustc
and the cached files are read/written using ACTIONS_CACHE_URL/ACTIONS_RUNTIME_TOKEN
parameters.
After some tests, I found that this action is not very useful : cache hits are low => only 3 / 193 compile requests.
I'm waiting your comments to decide if we must keep this action or suppress it.
Action for Rust cache Swatinem/rust-cache
Action used in the both jobs : "Style and Lint" and "Tests".
In the "Configure" step of this action, a cache key is created using Rust version (get via rust -vV
command). To have a functional cache used in FreeBSD VM, this version must be the same as used on FreeBSD.
I created a shell script to fake the output of rustc -vV
command => ci/fake_rustc_freebsd-stable.sh
, returns rustc version rustc 1.75.0 (82e1608df 2023-12-21)
for FreeBSD (host: x86_64-unknown-freebsd
).
- before the "Configure" step of rust-cache action, the
rustc
binary on Ubuntu runner is copied for backup - the shell script
fake_rustc_freebsd-stable.sh
is copied as~/.cargo/bin/rustc
- during "Configure" step, the cache key is created with the fake rustc version for FreeBSD (exec
~/.cargo/bin/rustc -vV
) - I add a post-step to copy back the rustc binary on the Ubuntu runner (needed for the "Post Configure Rust cache" step at the end of the job).
- the parameter
prefix-key: "v0-rust-freebsd"
is defined for the action to add a specific prefix when creating the cache key.
In the FreeBSD VM action, I enabled the copyback
parameter to use the Rust cache:
- after start of the FreeBSD VM, sources and files for the Rust cache are copied on it
- tasks are run on the FreeBSD VM using Rust cache
- in both CI shell scripts, remove execution of
cargo clean
to keep the created files byrustc/cargo
- at the end of the step, files from the Rust cache are copied back from the FreeBSD guest to the Ubuntu host
- the "Post Configure Rust cache" step uses these copied files to create the new version of the cache and upload it in GH cache.
After some runs, using Rust cache is efficient:
- job "Style and Lint" running time: 4m51 with cold cache vs 4m10 with cache
- job "Tests" running time: 8m01 with cold cache vs 7m12 with cache
You can check a complete run of this modified workflow on my own repository https://github.com/lcheylus/rust-coreutils/actions/runs/7552906886
You could also check my modified code on it (ugly main
branch without rebase and also my ongoing work to add CI for OpenBSD).
I'm waiting your comments before splitting/rebasing my work in a separate branch and submitting a PR for merge.
from coreutils.
@sylvestre @tertsdiepraam Comments welcome.
from coreutils.
Related Issues (20)
- date: `iso-8601` argument format discrepancy HOT 2
- date: YYYY-MM-DD parsed incorrectly
- uptime: should accept a filename as argument HOT 2
- date.exe -u argument doesn't work. HOT 1
- excessive memory usage in more HOT 6
- date: date setting format error HOT 1
- date: `--date` doesn't recognize negative timestamps HOT 4
- coreutils: Get (parseable) list of available commands HOT 2
- flaky CI macos: stat reports year 1677 HOT 1
- 'make install' still compiles something instead of just installing files
- env -u=var should fail if the variable does not exist HOT 1
- install: insecure mode handling
- hashsum: Should not accept --strict without --check HOT 1
- ln: ln doesn't turn `/` in link target paths into `\` on Windows
- 'ls' display bug HOT 7
- uucore docs: features aren't referenced in the doc
- btrfs causes four test failures
- hashsum: should refuse to run with contradictory bitlengths
- tr: GNU refuses sensible longer-than-class input, should we refuse it, too?
- csplit: issues reading from pipe
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from coreutils.