ctsrd-cheri / cheribuild Goto Github PK

Easily build and run CHERI related projects

License: Other

Python 98.47% Shell 1.10% CMake 0.31% C 0.03% Dockerfile 0.09%

cheribuild's Introduction

`cheribuild.py` - A script to build CHERI-related software (requires Python 3.8+)

This script automates all the steps required to build various CHERI-related software. For example cheribuild.py [options] sdk-riscv64-purecap will create an SDK that can be used to compile software as pure-capability code for a CHERI-RISC-V CPU, and cheribuild.py [options] run-riscv64-purecap will start an instance of CheriBSD built as pure-capability code for CHERI-RISC-V in QEMU.

cheribuild.py also allows building software for Arm's adaption of CHERI, the Morello platform, however not all targets are supported yet.

Supported operating systems

cheribuild.py has been tested and should work on FreeBSD 12 and 13. On Linux, Ubuntu 20.04/22.04, Debian 10/11/12 and OpenSUSE Tumbleweed are supported. macOS 10.14 and newer is also supported.

Pre-Build Setup

macOS

When building on macOS the following commands will install the packages required for the most commonly used cheribuild targets:

brew install cmake ninja libarchive git glib gnu-sed automake autoconf coreutils llvm make wget pixman pkg-config xz texinfo mercurial
# Install samba for shared mounts between host and CheriBSD on QEMU
brew install samba
# If you intend to run the morello FVP model you will also need the following:
brew install homebrew/cask/docker homebrew/cask/xquartz socat dtc

Debian/Ubuntu

If you are building CHERI on a Debian/Ubuntu-based machine, the following command will install the packages required for the most commonly used cheribuild targets:

apt install autoconf automake libtool pkg-config clang bison cmake mercurial ninja-build samba flex texinfo time libglib2.0-dev libpixman-1-dev libarchive-dev libarchive-tools libbz2-dev libattr1-dev libcap-ng-dev libexpat1-dev libgmp-dev

Older versions of Ubuntu may report errors when trying to install libarchive-tools. In this case try using apt install bsdtar instead.

RHEL/Fedora

If you are building CHERI on a RHEL/Fedora-based machine, the following command will install the packages required for the most commonly used cheribuild targets:

dnf install libtool clang-devel bison cmake mercurial ninja-build samba flex texinfo glib2-devel pixman-devel libarchive-devel bsdtar bzip2-devel libattr-devel libcap-ng-devel expat-devel

FreeBSD

If you are building CHERI on a FreeBSD machine, the following command will install the packages required for the most commonly used cheribuild targets:

pkg install autoconf automake bison cmake expat glib gsed llvm mercurial meson ninja pkgconf pixman samba

Arch Linux

If you are building CHERI on an Arch Linux machine, the following command will install the packages required for the most commonly used cheribuild targets:

pacman -Syu autoconf automake libtool pkgconf clang bison cmake ninja samba flex texinfo time glib2 pixman libarchive bzip2 attr libcap-ng inetutils mercurial expat gmp

These package used approx 350MiB, though you probably have most already installed.

Contributing

If you would like to make a change to cheribuild, please submit it as a GitHub pull request to https://github.com/CTSRD-CHERI/cheribuild. Cheribuild includes some pre-commit and pre-push hooks (using pre-commit) that can catch issues that will fail the pull request CI checks. To install the hooks, run the following command python3 -m pip install -U --user pre-commit && pre-commit install -t pre-commit -t pre-push.

Basic usage

In these examples the cheribuild.py program is run from within the cheribuild/ directory as ./cheribuild.py with options following the main command. Please note the leading ./ before the command name. It is also possible to add the cheribuild/ directory to your PATH or to link the binary somewhere such as in a bin/ directory of your choosing.

If you want to start up a QEMU VM running CheriBSD on top of the CHERI-RISC-V ISA run the following command:

./cheribuild.py run-riscv64-purecap -d (-d means build all dependencies).

This command will build the CHERI compiler, QEMU, CheriBSD, create a disk image and boot that in QEMU.

If you want to start up a QEMU VM running CheriBSD on top of the Morello ISA use the following command:

./cheribuild.py run-morello-purecap -d

By default cheribuild.py will clone all projects in ~/cheri, use ~/cheri/build for build directories and install into ~/cheri/output. However, these directories are all configurable (see below for details). When building for the first time, cheribuild.py will request user input multiple times, but the --force/-f flag can be used to accept the default.

If you would like to see what the script would do run it with the --pretend or -p option. For even more detail you can also pass --verbose or -v.

It is also possible to run this script on a remote host by using the remote-cheribuild.py script that is included in this repository: remote-cheribuild.py my.remote.server [options] <targets...> will run this script on my.remote.server.

Usage

cheribuild.py [options...] targets...

Example: to build and run a pure-capability CheriBSD for RISC-V: cheribuild.py --include-dependencies run-riscv64-purecap and for a clean verbose build of LLVM cheribuild.py -v --clean llvm

Available Targets

When selecting a target you can also build all the targets that it depends on by passing the --include-dependencies or -d option. However, some targets (e.g. all, sdk-*) will always build their dependencies because running them without building the dependencies does not make sense (see the list of targets for details).

Overview

There are many targets in cheribuild, and they fall into two categories: single targets and multiarch targets. Single targets, such as qemu, sail and install-morello-fvp, are for targets that only apply natively to the host machine. Most targets, however, are multiarch targets. These are always of the form <name>-<architecture>, where <architecture> indicates the architecture to compile/run code for, which will be cross-compiling/emulating for anything other than native. For example, cheribsd-riscv64-purecap will cross-compile CheriBSD for pure-capability CHERI-RISC-V, disk-image-morello-purecap will create a CheriBSD disk image for pure-capability Morello and gdb-native will build a version of CHERI-GDB that runs natively on the host machine and be used to remote debug CHERI-RISC-V and CHERI-MIPS.

Note that the <architecture> in the target says nothing about what it can do, only where it can do it. For example, llvm-native and llvm-riscv64 both exist; the former is a CHERI-LLVM that will run natively on the host and can be used to cross compile for both CHERI-MIPS and CHERI-RISC-V, whilst the latter is a CHERI-LLVM built as a set of RISC-V binaries that will run on CheriBSD itself.

Where there are multiple variants of the source, there are multiple cheribuild targets with different names which, in the case of multiarch targets, means a different <name>. For example, llvm refers to CHERI-LLVM, whereas morello-llvm refers to Arm's fork of CHERI-LLVM adding Morello support, and upstream-llvm refers to upstream's LLVM. Thus, a target of morello-llvm-riscv64 would be Arm's Morello-extended CHERI-LLVM fork built as a RISC-V binary to run on CheriBSD.

The following main targets are available

qemu builds and installs CTSRD-CHERI/qemu
llvm builds and installs the CTSRD-CHERI/llvm-project toolchain (includes a compiler, linker, and all required binutils).
cheribsd-<architecture> builds and installs CTSRD-CHERI/cheribsd and creates a sysroot for cross-compilation.
disk-image-<architecture> creates a CheriBSD disk-image.
run-<architecture> launches QEMU with the CheriBSD disk image.
freestanding-sdk builds everything required to build and run -ffreestanding binaries: compiler, linker and qemu
cheribsd-sdk-<architecture> builds everything required to compile binaries for CheriBSD: freestanding-sdk and cheribsd-sysroot
sdk-<architecture> is an alias for cheribsd-sdk-<architecture>
all-<architecture>: runs all the targets listed so far (run-<architecture> comes last so that you can interact with QEMU)

Supported architectures

riscv64: RISC-V without CHERI support
riscv64-purecap: pure-capability RISC-V: all pointers are CHERI capabilities.
mips64: MIPS without CHERI support
mips64-purecap: pure-capability MIPS: all pointers are CHERI capabilities.
aarch64: AArch64 without CHERI support
morello-purecap: pure-capability AArch64 (Morello): all pointers are CHERI capabilities.
amd64: 64-bit Intel x86.

Most projects (the ones that don't build a full OS, but just a program or library) also support -native configuration that builds for the host. This can be useful to verify that changes made for CHERI have not broken the native builds.

The following targets are also supported, but discouraged, as they serve little benefit over and above their non-CHERI counterparts, and so should not be used unless absolutely sure. If you still wish to use them despite that, they can be enabled with --enable-hybrid-targets.

riscv64-hybrid: RISC-V with CHERI support: pointers are integers by default but can be annotated with __capability to use CHERI capabilities.
mips64-hybrid: MIPS with CHERI support: pointers are integers by default but can be annotated with __capability to use CHERI capabilities.
morello-hybrid: AArch64 with CHERI (Morello) support: pointers are integers by default but can be annotated with __capability to use CHERI capabilities.

For the cheribsd, disk-image and run targets the hybrid vs purecap distinction applies solely to userspace (see below for more details).

Other targets

freebsd-<architecture> builds and installs freebsd/freebsd.
disk-image-freebsd-<architecture> creates a FreeBSD disk-image.
run-freebsd-<architecture> launches QEMU with the FreeBSD disk image.
cmake builds and installs latest CMake
cherios builds and installs CTSRD-CHERI/cherios
cheritrace builds and installs CTSRD-CHERI/cheritrace
sqlite-<architecture> builds and installs CTSRD-CHERI/sqlite
nginx-<architecture> builds and installs CTSRD-CHERI/nginx
postgres-<architecture> builds and installs CTSRD-CHERI/postgres

Building the compiler and QEMU

In order to run CheriBSD you will first need to compile QEMU (cheribuild.py qemu). You will also need to build LLVM (this includes a compiler and linker suitable for CHERI) using cheribuild.py llvm. The compiler can generate CHERI code for MIPS (64-bit only) and RISCV (32 and 64-bit). All binaries will by default be installed to ~/cheri/sdk/bin.

Building and running CheriBSD

To build CheriBSD run cheribuild.py cheribsd-<architecture>, with architecture being one of

riscv64: Kernel and userspace are RISC-V without CHERI support.
riscv64-purecap: Kernel is RISC-V with CHERI support (hybrid), and all userspace programs built as pure-capability CHERI binaries.
mips64: Kernel and userspace are MIPS without CHERI support.
mips64-purecap: Kernel is MIPS with CHERI support (hybrid), and all userspace programs built as pure-capability CHERI binaries.
aarch64: Kernel and userspace are AArch64 without CHERI support.
morello-purecap: Kernel is AArch64 with CHERI (Morello) support (hybrid), and all userspace programs built as pure-capability CHERI binaries.
amd64: Kernel and userspace are 64-bit Intel x86.

The following targets also exist but are disabled by default and discouraged (see Supported architectures for more details):

riscv64-hybrid: Kernel is RISC-V with CHERI support (hybrid), but most programs built as plain RISC-V.
mips64-hybrid: Kernel is MIPS with CHERI support (hybrid), but most programs built as plain RISC-V.
morello-hybrid: Kernel is AArch64 with CHERI (Morello) support (hybrid), but most programs built as plain AArch64.

Disk image

The disk image is created by the cheribuild.py disk-image-<architecture> target and can then be used as a boot disk by QEMU.

In order to customize the disk image it will add all files under (by default) ~/cheri/extra-files/ to the resulting image. When building the image cheribuild will ask you whether it should add your SSH public keys to the /root/.ssh/authorized_keys file in the CheriBSD image. It will also generate SSH host keys for the image so that those don't change everytime the image is rebuilt. A suitable /etc/rc.conf and /etc/fstab will also be added to this directory and can then be customized.

The default path for the disk image is ~/cheri/output/cheribsd-<architecture>.img, i.e. cheribsd-riscv64-purecap.img for pure-capability RISC-V or cheribsd-mips64.img for MIPS without CHERI support.

CheriBSD SSH ports

Since cheribuild.py was designed to be run by multiple users on a shared build system, it will tell QEMU to listen on a port on localhost that depends on the user ID to avoid conflicts. It will print a message such as Listening for SSH connections on localhost:12374, i.e. you will need to use ssh -p 12374 root@localhost to connect to CheriBSD. This can be changed using cheribuild.py --run/ssh-forwarding-port <portno> run-<architecture> or be made persistent with the following configuration file (see below for more details on the config file format and path):

{
    "run-riscv64-hybrid": {
        "ssh-forwarding-port": 12345
    },
    "run-riscv64-purecap": {
        "ssh-forwarding-port": 12346
    }
}

Speeding up SSH connections

Connecting to CheriBSD via ssh can take a few seconds. Further connections after the first can be sped up by using the openssh ControlMaster setting:

Host cheribsd-riscv
  User root
  Port 12345
  HostName localhost
  ControlPath ~/.ssh/controlmasters/%r@%h:%p
  ControlMaster auto
  StrictHostKeyChecking no

Host cheribsd-riscv-purecap
  User root
  Port 12346
  HostName localhost
  ControlPath ~/.ssh/controlmasters/%r@%h:%p
  ControlMaster auto
  StrictHostKeyChecking no

Building GDB

You can also build a version of GDB that understands CHERI capabilities either as a binary for the host (cheribuild.py gdb-native) to debug coredumps or as a gues binary to use for live debugging in CheriBSD (cheribuild.py gdb-mips64-hybrid for MIPS and cheribuild.py gdb-riscv64-hybrid for RISC-V). The guest binary will be installed in usr/local/bin/gdb under your CheriBSD rootfs and will be included when you build a new disk image (cheribuild.py disk-image-<arch>). The native GDB will be installed to your SDK binary directory (~/cheri/sdk/bin by default).

Cross-compiling for CheriBSD

In order to cross-compile projects such as NGINX or PostgreSQL for CheriBSD you will first need a full SDK: cheribuild.py cheribsd-sdk-mips64-purecap. Then you can then run cheribuild.py postgres-mips64-purecap or cheribuild.py nginx-mips64-purecap, etc. By default, these projects will be installed into your CheriBSD rootfs under /opt and will therefore be automatically included the next time you build a disk image.

See cheribuild.py --list-targets for a full list of targets.

Cross-compiling baremetal MIPS/CHERI

There is currently experimental support to build libcxx as a baremetal library running on top of newlib. This can be done by running cheribuild.py libcxx-baremetal -d.

Adapting the build configuration

There are a lot of options to customize the behaviour of this script: e.g. the directory for the cloned sources can be changed from the default of $HOME/cheri using the --source-root= option. A full list of the available options with descriptions can be found towards the end of this document.

The options can also be made persistent by storing them in a JSON config file (~/.config/cheribuild.json). Options passed on the command line will override those read from the config file. The key in the JSON config file is the same as the long option name without the intial --. For example if you want cheribuild.py to behave as if you had passed --source-root /foo/bar/cheri/src --output-root /foo/bar/cheri/output --build-root /build -j 4 --cheribsd/build-options "-DWITHOUT_ZFS FOO=bar", you can write the following JSON to ~/.config/cheribuild.json:

{
  "source-root": "/foo/bar/cheri/src",
  // build-root defaults to <source-root>/build
  "build-root": "/build",
  // output-root defaults to <source-root>/output
  "output-root": "/foo/bar/cheri/output",
  "make-jobs": 4,
  "cheribsd": {
    "build-options": ["-DWITHOUT_ZFS", "FOO=bar"]
  }
}

Prefixed cheribuild.py symlinks to select config file

If you invoke cheribuild.py as a prefixed command (e.g. debug-cheribuild.py, stable-cheribuild.py) it will read the file ~/.config/{prefix}-cheribuild.json instead. This makes it easy to build debug and release builds of e.g. LLVM or build CheriBSD with various different flags.

Including config files

If you have many config files (e.g. cheribsd-stable, -debug, -release, etc.) it is also possible to #include a base config file and only write the settings that are different.

For example a ~/.config/stable-cheribuild.json could look like this:

{
	"build-root": "/build-stable",
	"#include": "cheribuild-common.json",
	"cheribsd": {
		"source-directory": "/my/other/cheribsd/worktree/with/the/stable/branch"
	}
}

Getting shell completion

You will need to install python3-argcomplete:

pip3 install --user argcomplete

# Or install latest version from git:
git clone https://github.com/kislyuk/argcomplete.git
cd argcomplete
python3 setup.py install --user

NOTE: On FreeBSD pip and setuptools are not installed by default, so you need to run python3 -m ensurepip --user first.

BASH

# NOTE: the next command doesn't seem to work on FreeBSD
~/.local/bin/activate-global-python-argcomplete --user
# On FreeBSD (or if the above work for some other reason) do this:
echo 'eval "$(register-python-argcomplete cheribuild.py)"' >> ~/.bashrc

TCSH:

With tcsh add the following line to ~/.cshrc:

eval "`register-python-argcomplete --shell tcsh cheribuild.py`"

Note: python-argcomplete-tcsh must be in $PATH (should be in ~/.local/bin/). I would also suggest using set autolist to display all options.

List of options (--help output)

NOTE: Since there are so many per-project options that are identical between all projects they are not all shown when running --help. To see the full list of options that can be specified, run cheribuild.py --help-all. Since this will generate lots of output it probably makes more sense to run cheribuild.py --help-all | grep <target_name>.

usage: cheribuild.py [-h] [--help-all] [--config-file FILE] [--pretend] [--build] [--test] [--benchmark]
                     [--build-and-test] [--list-targets] [--dump-configuration] [--print-targets-only]
                     [--clang-path CLANG-PATH] [--clang++-path CLANG++-PATH] [--clang-cpp-path CLANG-CPP-PATH]
                     [--pass-k-to-make] [--with-libstatcounters | --no-with-libstatcounters]
                     [--skip-world | --no-skip-world | --skip-buildworld | --no-skip-buildworld]
                     [--skip-kernel | --no-skip-kernel | --skip-buildkernel | --no-skip-buildkernel]
                     [--freebsd-subdir SUBDIRS] [--buildenv] [--libcompat-buildenv] [--debug-output]
                     [--mips-float-abi {soft,hard}] [--aarch64-fp-and-simd-options {default,nosimd,soft,soft_simd}]
                     [--cross-compile-linkage {default,dynamic,static}]
                     [--subobject-bounds {conservative,subobject-safe,aggressive,very-aggressive,everywhere-unsafe}]
                     [--use-cheri-ubsan | --no-use-cheri-ubsan]
                     [--use-cheri-ubsan-runtime | --no-use-cheri-ubsan-runtime]
                     [--subobject-debug | --no-subobject-debug]
                     [--clang-colour-diags | --no-clang-colour-diags | --clang-color-diags | --no-clang-color-diags]
                     [--use-sdk-clang-for-native-xbuild | --no-use-sdk-clang-for-native-xbuild]
                     [--configure-only | --no-configure-only] [--skip-install | --no-skip-install]
                     [--skip-build | --no-skip-build] [--skip-sdk | --no-skip-sdk] [--skip-dependency-filter REGEX]
                     [--trap-on-unrepresentable | --no-trap-on-unrepresentable]
                     [--qemu-gdb-break-on-cheri-trap | --no-qemu-gdb-break-on-cheri-trap]
                     [--qemu-gdb-debug-userspace-program QEMU-GDB-DEBUG-USERSPACE-PROGRAM]
                     [--only-dependencies | --no-only-dependencies] [--docker | --no-docker]
                     [--docker-container DOCKER-CONTAINER] [--docker-reuse-container | --no-docker-reuse-container]
                     [--compilation-db] [--wait-for-debugger | --no-wait-for-debugger]
                     [--debugger-in-tmux-pane | --no-debugger-in-tmux-pane] [--gdb-random-port | --no-gdb-random-port]
                     [--run-under-gdb | --no-run-under-gdb] [--test-ssh-key TEST-SSH-KEY]
                     [--use-minimal-benchmark-kernel | --no-use-minimal-benchmark-kernel] [--test-extra-args ARGS]
                     [--interact-after-tests] [--test-environment-only] [--test-ld-preload TEST-LD-PRELOAD]
                     [--benchmark-fpga-extra-args ARGS] [--benchmark-clean-boot | --no-benchmark-clean-boot]
                     [--benchmark-extra-args ARGS] [--benchmark-ssh-host BENCHMARK-SSH-HOST]
                     [--benchmark-csv-suffix BENCHMARK-CSV-SUFFIX] [--benchmark-ld-preload BENCHMARK-LD-PRELOAD]
                     [--benchmark-with-debug-kernel | --no-benchmark-with-debug-kernel]
                     [--benchmark-lazy-binding | --no-benchmark-lazy-binding]
                     [--benchmark-iterations BENCHMARK-ITERATIONS] [--benchmark-with-qemu | --no-benchmark-with-qemu]
                     [--shallow-clone | --no-shallow-clone] [--beri-fpga-env-setup-script BERI-FPGA-ENV-SETUP-SCRIPT]
                     [--arm-none-eabi-prefix ARM-NONE-EABI-PREFIX]
                     [--build-morello-firmware-from-source | --no-build-morello-firmware-from-source]
                     [--list-kernels | --no-list-kernels] [--remote-morello-board REMOTE-MORELLO-BOARD]
                     [--get-config-option KEY] [--quiet | --no-quiet | -q] [--verbose | --no-verbose | -v]
                     [--clean | --no-clean | -c] [--force | --no-force | -f] [--logfile | --no-logfile]
                     [--skip-update | --no-skip-update] [--confirm-clone | --no-confirm-clone]
                     [--force-update | --no-force-update]
                     [--skip-configure | --no-skip-configure | --reconfigure | --no-reconfigure | --force-configure | --no-force-configure]
                     [--include-dependencies] [--include-toolchain-dependencies | --no-include-toolchain-dependencies]
                     [--start-with TARGET | --start-after TARGET] [--compilation-db-in-source-dir]
                     [--generate-cmakelists | --no-generate-cmakelists] [--make-without-nice | --no-make-without-nice]
                     [--make-jobs MAKE-JOBS] [--source-root SOURCE-ROOT] [--output-root OUTPUT-ROOT]
                     [--build-root BUILD-ROOT] [--tools-root TOOLS-ROOT] [--morello-sdk-root MORELLO-SDK-ROOT]
                     [--sysroot-install-root SYSROOT-INSTALL-ROOT] [--upstream-qemu/targets UPSTREAM-QEMU/TARGETS]
                     [--qemu/targets QEMU/TARGETS] [--qemu/statistics | --qemu/no-statistics]
                     [--cheri-syzkaller/run-sysgen | --cheri-syzkaller/no-run-sysgen]
                     [--run-syzkaller/syz-config RUN-SYZKALLER/SYZ-CONFIG]
                     [--run-syzkaller/ssh-privkey syzkaller_id_rsa] [--run-syzkaller/workdir DIR]
                     [--freebsd/build-tests | --freebsd/no-build-tests] [--freebsd/build-options OPTIONS]
                     [--freebsd/debug-info | --freebsd/no-debug-info] [--freebsd/subdir SUBDIRS]
                     [--cheribsd/build-tests | --cheribsd/no-build-tests] [--cheribsd/build-options OPTIONS]
                     [--cheribsd/debug-info | --cheribsd/no-debug-info] [--cheribsd/subdir SUBDIRS]
                     [--cheribsd/build-fpga-kernels | --cheribsd/no-build-fpga-kernels]
                     [--cheribsd/default-kernel-abi {hybrid,purecap}]
                     [--cheribsd/build-alternate-abi-kernels | --cheribsd/no-build-alternate-abi-kernels]
                     [--cheribsd/build-bench-kernels | --cheribsd/no-build-bench-kernels]
                     [--cheribsd/caprevoke-kernel | --cheribsd/no-caprevoke-kernel]
                     [--cheribsd-mfs-root-kernel/build-tests | --cheribsd-mfs-root-kernel/no-build-tests]
                     [--cheribsd-mfs-root-kernel/build-options OPTIONS]
                     [--cheribsd-mfs-root-kernel/debug-info | --cheribsd-mfs-root-kernel/no-debug-info]
                     [--cheribsd-mfs-root-kernel/build-fpga-kernels | --cheribsd-mfs-root-kernel/no-build-fpga-kernels]
                     [--cheribsd-mfs-root-kernel/default-kernel-abi {hybrid,purecap}]
                     [--cheribsd-mfs-root-kernel/build-alternate-abi-kernels | --cheribsd-mfs-root-kernel/no-build-alternate-abi-kernels]
                     [--cheribsd-mfs-root-kernel/build-bench-kernels | --cheribsd-mfs-root-kernel/no-build-bench-kernels]
                     [--cheribsd-mfs-root-kernel/caprevoke-kernel | --cheribsd-mfs-root-kernel/no-caprevoke-kernel]
                     [--freebsd-release/build-tests | --freebsd-release/no-build-tests]
                     [--freebsd-release/build-options OPTIONS]
                     [--freebsd-release/debug-info | --freebsd-release/no-debug-info] [--freebsd-release/subdir SUBDIRS]
                     [--cheribsd-release/build-tests | --cheribsd-release/no-build-tests]
                     [--cheribsd-release/build-options OPTIONS]
                     [--cheribsd-release/debug-info | --cheribsd-release/no-debug-info]
                     [--cheribsd-release/subdir SUBDIRS]
                     [--cheribsd-release/build-fpga-kernels | --cheribsd-release/no-build-fpga-kernels]
                     [--cheribsd-release/default-kernel-abi {hybrid,purecap}]
                     [--cheribsd-release/build-alternate-abi-kernels | --cheribsd-release/no-build-alternate-abi-kernels]
                     [--cheribsd-release/build-bench-kernels | --cheribsd-release/no-build-bench-kernels]
                     [--cheribsd-release/caprevoke-kernel | --cheribsd-release/no-caprevoke-kernel]
                     [--cheribsd-sysroot/remote-sdk-path PATH] [--disk-image-minimal/extra-files DIR]
                     [--disk-image-minimal/rootfs-type {ufs,zfs}] [--disk-image-minimal/path IMGPATH]
                     [--disk-image-mfs-root/extra-files DIR] [--disk-image-mfs-root/rootfs-type {ufs,zfs}]
                     [--disk-image-mfs-root/path IMGPATH] [--disk-image/extra-files DIR]
                     [--disk-image/rootfs-type {ufs,zfs}] [--disk-image/path IMGPATH] [--rootfs-tarball/extra-files DIR]
                     [--rootfs-tarball/rootfs-type {ufs,zfs}] [--rootfs-tarball/path IMGPATH]
                     [--disk-image-freebsd/extra-files DIR] [--disk-image-freebsd/rootfs-type {ufs,zfs}]
                     [--disk-image-freebsd/path IMGPATH] [--run/ssh-forwarding-port PORT]
                     [--run/ephemeral | --run/no-ephemeral] [--run/remote-kernel-path RUN/REMOTE-KERNEL-PATH]
                     [--run/alternative-kernel RUN/ALTERNATIVE-KERNEL] [--run/kernel-abi {hybrid,purecap}]
                     [--run-minimal/ssh-forwarding-port PORT] [--run-minimal/ephemeral | --run-minimal/no-ephemeral]
                     [--run-minimal/remote-kernel-path RUN-MINIMAL/REMOTE-KERNEL-PATH]
                     [--run-minimal/alternative-kernel RUN-MINIMAL/ALTERNATIVE-KERNEL]
                     [--run-minimal/kernel-abi {hybrid,purecap}] [--run-mfs-root/ssh-forwarding-port PORT]
                     [--run-mfs-root/remote-kernel-path RUN-MFS-ROOT/REMOTE-KERNEL-PATH]
                     [--run-mfs-root/alternative-kernel RUN-MFS-ROOT/ALTERNATIVE-KERNEL]
                     [--run-mfs-root/kernel-abi {hybrid,purecap}] [--sslproc/build-tests | --sslproc/no-build-tests]
                     [--bash/set-as-root-shell | --bash/no-set-as-root-shell] [--freertos/demo DEMO]
                     [--freertos/prog PROG] [--freertos/bsp BSP] [--run-freertos/demo DEMO] [--run-freertos/prog PROG]
                     [--run-freertos/bsp BSP] [--qtbase-dev/build-tests | --qtbase-dev/no-build-tests]
                     [--qtbase-dev/build-examples | --qtbase-dev/no-build-examples]
                     [--qtbase-dev/assertions | --qtbase-dev/no-assertions]
                     [--qtbase-dev/minimal | --qtbase-dev/no-minimal]
                     [--qtwebkit/build-jsc-only | --qtwebkit/no-build-jsc-only]
                     [--morello-webkit/backend {cloop,tier1asm,tier2asm}]
                     [--morello-webkit/tier2ptrliterals | --morello-webkit/no-tier2ptrliterals]
                     [--morello-webkit/jsheapoffsets | --morello-webkit/no-jsheapoffsets]
                     [TARGET ...]

positional arguments:
  TARGET                The targets to build

options:
  -h, --help            show this help message and exit
  --help-all, --help-hidden
                        Show all help options, including the target-specific ones.
  --pretend, -p         Only print the commands instead of running them (default: 'False')
  --pass-k-to-make, -k  Pass the -k flag to make to continue after the first error (default: 'False')
  --debug-output, -vv   Extremely verbose output (default: 'False')
  --clang-colour-diags, --no-clang-colour-diags, --clang-color-diags, --no-clang-color-diags
                        Force CHERI clang to emit coloured diagnostics (default: 'True')
  --configure-only, --no-configure-only
                        Only run the configure step (skip build and install) (default: 'False')
  --skip-install, --no-skip-install
                        Skip the install step (only do the build) (default: 'False')
  --skip-build, --no-skip-build
                        Skip the build step (only do the install) (default: 'False')
  --only-dependencies, --no-only-dependencies
                        Only build dependencies of targets, not the targets themselves (default: 'False')
  --compilation-db, --cdb
                        Create a compile_commands.json file in the build dir (requires Bear for non-CMake projects)
                        (default: 'False')
  --shallow-clone, --no-shallow-clone
                        Perform a shallow `git clone` when cloning new projects. This can save a lot of time for
                        largerepositories such as FreeBSD or LLVM. Use `git fetch --unshallow` to convert to a non-
                        shallow clone (default: 'True')
  --build-morello-firmware-from-source, --no-build-morello-firmware-from-source
                        Build the firmware from source instead of downloading the latest release. (default: 'False')
  --remote-morello-board REMOTE-MORELLO-BOARD
                        SSH hostname of a Morello board. When set, some projects will run their test suites on the
                        remote board instead of QEMU.
  --quiet, --no-quiet, -q
                        Don't show stdout of the commands that are executed (default: 'False')
  --verbose, --no-verbose, -v
                        Print all commmands that are executed (default: 'False')
  --clean, --no-clean, -c
                        Remove the build directory before build (default: 'False')
  --force, --no-force, -f
                        Don't prompt for user input but use the default action (default: 'False')
  --logfile, --no-logfile
                        Write a logfile for the build steps (default: 'False')
  --skip-update, --no-skip-update
                        Skip the git pull step (default: 'False')
  --confirm-clone, --no-confirm-clone
                        Ask for confirmation before cloning repositories. (default: 'False')
  --force-update, --no-force-update
                        Always update (with autostash) even if there are uncommitted changes (default: 'False')
  --skip-configure, --no-skip-configure
                        Skip the configure step (default: 'False')
  --reconfigure, --no-reconfigure, --force-configure, --no-force-configure
                        Always run the configure step, even for CMake projects with a valid cache. (default: 'False')
  --compilation-db-in-source-dir
                        Generate a compile_commands.json and also copy it to the source directory (default: 'False')
  --generate-cmakelists, --no-generate-cmakelists
                        Generate a CMakeLists.txt that just calls cheribuild. Useful for IDEs that only support CMake
                        (default: 'False')
  --make-without-nice, --no-make-without-nice
                        Run make/ninja without nice(1) (default: 'False')
  --make-jobs MAKE-JOBS, -j MAKE-JOBS
                        Number of jobs to use for compiling (default: '<system-dependent>')

Actions to be performed:
  --build               Run (usually build+install) chosen targets (default)
  --test, --run-tests   Run tests for the passed targets instead of building them
  --benchmark           Run tests for the passed targets instead of building them
  --build-and-test      Run chosen targets and then run any tests afterwards
  --list-targets        List all available targets and exit
  --dump-configuration  Print the current configuration as JSON. This can be saved to ~/.config/cheribuild.json to make
                        it persistent
  --print-targets-only  Don't run the build but instead only print the targets that would be executed (default: 'False')
  --list-kernels, --no-list-kernels
                        List available kernel configs to run and exit (default: 'False')
  --get-config-option KEY
                        Print the value of config option KEY and exit

Selecting which dependencies are built:
  --skip-sdk, --no-skip-sdk
                        When building with --include-dependencies ignore the SDK dependencies. Saves a lot of time when
                        building libc++, etc. with dependencies but the sdk is already up-to-date. This is like --no-
                        include-toolchain-depedencies but also skips the target that builds the sysroot. (default:
                        'False')
  --skip-dependency-filter REGEX
                        A regular expression to match against to target names that should be skipped when using--
                        include-dependency. Can be passed multiple times to add more patterns. (default: '[]')
  --include-dependencies, -d
                        Also build the dependencies of targets passed on the command line. Targets passed on the command
                        line will be reordered and processed in an order that ensures dependencies are built before the
                        real target. (run --list-targets for more information). By default this does not build toolchain
                        targets such as LLVM. Pass --include-toolchain-dependencies to also build those. (default:
                        'False')
  --include-toolchain-dependencies, --no-include-toolchain-dependencies
                        Include toolchain targets such as LLVM and QEMU when --include-dependencies is set. (default:
                        'True')
  --start-with TARGET   Start building at TARGET (useful when resuming an interrupted --include-depedencies build)
  --start-after TARGET  Start building after TARGET (useful when resuming an interrupted --include-depedencies build)

Configuration of default paths:
  --config-file FILE    The config file that is used to load the default settings (default:
                        '$HOME/.config/cheribuild.json')
  --clang-path CLANG-PATH, --cc-path CLANG-PATH
                        The C compiler to use for host binaries (must be compatible with Clang >= 3.7)
  --clang++-path CLANG++-PATH, --c++-path CLANG++-PATH
                        The C++ compiler to use for host binaries (must be compatible with Clang >= 3.7)
  --clang-cpp-path CLANG-CPP-PATH, --cpp-path CLANG-CPP-PATH
                        The C preprocessor to use for host binaries (must be compatible with Clang >= 3.7)
  --beri-fpga-env-setup-script BERI-FPGA-ENV-SETUP-SCRIPT
                        Custom script to source to setup PATH and quartus, default to using cheri-cpu/cheri/setup.sh
  --arm-none-eabi-prefix ARM-NONE-EABI-PREFIX
                        Prefix for arm-none-eabi-gcc binaries (e.g. /usr/bin/arm-none-eabi-). Available
                        athttps://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-
                        toolchain/gnu-rm/downloads (default: '')
  --source-root SOURCE-ROOT
                        The directory to store all sources (default: '$HOME/cheri')
  --output-root OUTPUT-ROOT
                        The directory to store all output (default: '<SOURCE_ROOT>/output')
  --build-root BUILD-ROOT
                        The directory for all the builds (default: '<SOURCE_ROOT>/build')
  --tools-root TOOLS-ROOT
                        The directory to find sdk and bootstrap tools (default: '<OUTPUT_ROOT>')
  --morello-sdk-root MORELLO-SDK-ROOT
                        The directory to find/install the Morello SDK (default: ''<TOOLS_ROOT>/morello-sdk'')
  --sysroot-install-root SYSROOT-INSTALL-ROOT, --sysroot-install-dir SYSROOT-INSTALL-ROOT
                        Sysroot prefix (default: '<TOOLS_ROOT>')

Adjust flags used when compiling MIPS/CHERI projects:
  --with-libstatcounters, --no-with-libstatcounters
                        Link cross compiled CHERI project with libstatcounters. (default: 'False')
  --mips-float-abi {soft,hard}
                        The floating point ABI to use for building MIPS+CHERI programs (default: 'soft')
  --aarch64-fp-and-simd-options {default,nosimd,soft,soft_simd}
                        The floating point/SIMD mode to use for building AArch64 programs (default: 'default')
  --cross-compile-linkage {default,dynamic,static}
                        Whether to link cross-compile projects static or dynamic by default (default: 'default')
  --subobject-bounds {conservative,subobject-safe,aggressive,very-aggressive,everywhere-unsafe}
                        Whether to add additional CSetBounds to subobject references/&-operator
  --use-cheri-ubsan, --no-use-cheri-ubsan
                        Add compiler flags to detect certain undefined CHERI behaviour at runtime (default: 'False')
  --use-cheri-ubsan-runtime, --no-use-cheri-ubsan-runtime
                        Use the UBSan runtime to provide more detailed information on undefined CHERI behaviour.If false
                        (the default) the compiler will generate a trap instruction instead. (default: 'False')
  --subobject-debug, --no-subobject-debug
                        Clear software permission bit 2 when subobject bounds reduced size (Note: this should be turned
                        off for benchmarks!) (default: 'True')
  --use-sdk-clang-for-native-xbuild, --no-use-sdk-clang-for-native-xbuild
                        Compile cross-compile project with CHERI clang from the SDK instead of host compiler (default:
                        'False')

Configuration for running tests:
  --test-ssh-key TEST-SSH-KEY
                        The SSH key to used to connect to the QEMU instance when running tests on CheriBSD. If not
                        specified a key will be generated in the build-root directory on-demand.
  --use-minimal-benchmark-kernel, --no-use-minimal-benchmark-kernel
                        Use a CHERI BENCHMARK version of the cheribsd-mfs-root-kernel (without INVARIATES) for the run-
                        minimal target and for tests. This can speed up longer running tests. This is the default for
                        PostgreSQL and libc++ tests (passing use-minimal-benchmark-kernel can force these tests to use
                        an INVARIANTS kernel). (default: 'False')
  --test-extra-args ARGS
                        Additional flags to pass to the test script in --test
  --interact-after-tests
                        Interact with the CheriBSD instance after running the tests on QEMU (only for --test) (default:
                        'False')
  --test-environment-only
                        Don't actually run the tests. Instead setup a QEMU instance with the right paths set up.
                        (default: 'False')
  --test-ld-preload TEST-LD-PRELOAD
                        Preload the given library before running tests

Configuration for running benchmarks:
  --benchmark-fpga-extra-args ARGS
                        Extra options for the FPGA management script
  --benchmark-clean-boot, --no-benchmark-clean-boot
                        Reboot the FPGA with a new bitfile and kernel before running benchmarks. If not set, assume the
                        FPGA is running. (default: 'False')
  --benchmark-extra-args ARGS
                        Additional flags to pass to the program executed in --benchmark
  --benchmark-ssh-host BENCHMARK-SSH-HOST
                        The SSH hostname/IP for the benchmark FPGA (default: 'cheri-fpga')
  --benchmark-csv-suffix BENCHMARK-CSV-SUFFIX
                        Add a custom suffix for the statcounters CSV.
  --benchmark-ld-preload BENCHMARK-LD-PRELOAD
                        Preload the given library before running benchmarks
  --benchmark-with-debug-kernel, --no-benchmark-with-debug-kernel
                        Run the benchmark with a kernel that has assertions enabled. (default: 'False')
  --benchmark-lazy-binding, --no-benchmark-lazy-binding
                        Run the benchmark without setting LD_BIND_NOW. (default: 'False')
  --benchmark-iterations BENCHMARK-ITERATIONS
                        Override the number of iterations for the benchmark. Note: not all benchmarks support this
                        option
  --benchmark-with-qemu, --no-benchmark-with-qemu
                        Run the benchmarks on QEMU instead of the FPGA (only useful to collect instruction counts or
                        test the benchmarks) (default: 'False')

Configuration for launching QEMU (and other simulators):
  --trap-on-unrepresentable, --no-trap-on-unrepresentable
                        Raise a CHERI exception when capabilities become unreprestable instead of detagging. Useful for
                        debugging, but deviates from the spec, and therefore off by default. (default: 'False')
  --qemu-gdb-break-on-cheri-trap, --no-qemu-gdb-break-on-cheri-trap
                        Drop into GDB attached to QEMU when a CHERI exception is triggered (QEMU only). (default:
                        'False')
  --qemu-gdb-debug-userspace-program QEMU-GDB-DEBUG-USERSPACE-PROGRAM
                        Print the command to debug the following userspace program in GDB attaced to QEMU
  --wait-for-debugger, --no-wait-for-debugger
                        Start QEMU in the 'wait for a debugger' state whenlaunching CheriBSD,FreeBSD, etc. (default:
                        'False')
  --debugger-in-tmux-pane, --no-debugger-in-tmux-pane
                        Start Qemu and gdb in another tmux split (default: 'False')
  --gdb-random-port, --no-gdb-random-port
                        Wait for gdb using a random port (default: 'True')
  --run-under-gdb, --no-run-under-gdb
                        Run tests/benchmarks under GDB. Note: currently most targets ignore this flag. (default:
                        'False')

FreeBSD and CheriBSD build configuration:
  --skip-world, --no-skip-world, --skip-buildworld, --no-skip-buildworld
                        Skip the buildworld-related steps when building FreeBSD or CheriBSD (default: 'False')
  --skip-kernel, --no-skip-kernel, --skip-buildkernel, --no-skip-buildkernel
                        Skip the buildkernel step when building FreeBSD or CheriBSD (default: 'False')
  --freebsd-subdir SUBDIRS, --subdir SUBDIRS
                        Only build subdirs SUBDIRS of FreeBSD/CheriBSD instead of the full tree. Useful for quickly
                        rebuilding individual programs/libraries. If more than one dir is passed they will be processed
                        in order. Note: This will break if not all dependencies have been built.
  --buildenv            Open a shell with the right environment for building the project. Currently only works for
                        FreeBSD/CheriBSD (default: 'False')
  --libcompat-buildenv, --libcheri-buildenv
                        Open a shell with the right environment for building compat libraries. (default: 'False')

Options controlling the use of docker for building:
  --docker, --no-docker
                        Run the build inside a docker container (default: 'False')
  --docker-container DOCKER-CONTAINER
                        Name of the docker container to use (default: 'ctsrd/cheribuild-docker')
  --docker-reuse-container, --no-docker-reuse-container
                        Attach to the same container again (note: docker-container option must be an id rather than a
                        container name (default: 'False')

Options for target 'upstream-qemu':
  --upstream-qemu/targets UPSTREAM-QEMU/TARGETS
                        Build QEMU for the following targets (default: 'arm-
                        softmmu,aarch64-softmmu,mips64-softmmu,riscv64-softmmu,riscv32-softmmu,x86_64-softmmu')

Options for target 'qemu':
  --qemu/targets QEMU/TARGETS
                        Build QEMU for the following targets (default: 'aarch64-softmmu,morello-
                        softmmu,mips64-softmmu,mips64cheri128-softmmu,riscv64-softmmu,riscv64cheri-
                        softmmu,riscv32-softmmu,riscv32cheri-softmmu,x86_64-softmmu')
  --qemu/statistics, --qemu/no-statistics
                        Collect statistics on out-of-bounds capability creation. (default: 'False')

Options for target 'cheri-syzkaller':
  --cheri-syzkaller/run-sysgen, --cheri-syzkaller/no-run-sysgen
                        Rerun syz-extract and syz-sysgen to rebuild generated Go syscall descriptions. (default:
                        'False')

Options for target 'run-syzkaller':
  --run-syzkaller/syz-config RUN-SYZKALLER/SYZ-CONFIG
                        Path to the syzkaller configuration file to use.
  --run-syzkaller/ssh-privkey syzkaller_id_rsa
                        A directory with additional files that will be added to the image (default: '$SOURCE_ROOT/extra-
                        files/syzkaller_id_rsa')
  --run-syzkaller/workdir DIR
                        Working directory for syzkaller output.

Options for target 'freebsd':
  --freebsd/build-tests, --freebsd/no-build-tests
                        Build the tests (default: 'True')
  --freebsd/build-options OPTIONS
                        Additional make options to be passed to make when building FreeBSD/CheriBSD. See `man src.conf`
                        for more info. (default: '[]')
  --freebsd/debug-info, --freebsd/no-debug-info
                        pass make flags for building with debug info (default: 'True')
  --freebsd/subdir SUBDIRS
                        Only build subdirs SUBDIRS instead of the full tree. Useful for quickly rebuilding individual
                        programs/libraries. If more than one dir is passed, they will be processed in order. Note: This
                        will break if not all dependencies have been built. (default: 'the value of the global
                        --freebsd-subdir options')

Options for target 'cheribsd':
  --cheribsd/build-tests, --cheribsd/no-build-tests
                        Build the tests (default: 'True')
  --cheribsd/build-options OPTIONS
                        Additional make options to be passed to make when building FreeBSD/CheriBSD. See `man src.conf`
                        for more info. (default: '[]')
  --cheribsd/debug-info, --cheribsd/no-debug-info
                        pass make flags for building with debug info (default: 'True')
  --cheribsd/subdir SUBDIRS
                        Only build subdirs SUBDIRS instead of the full tree. Useful for quickly rebuilding individual
                        programs/libraries. If more than one dir is passed, they will be processed in order. Note: This
                        will break if not all dependencies have been built. (default: 'the value of the global
                        --freebsd-subdir options')
  --cheribsd/build-fpga-kernels, --cheribsd/no-build-fpga-kernels
                        Also build kernels for the FPGA. (default: 'False')
  --cheribsd/default-kernel-abi {hybrid,purecap}
                        Select default kernel to build (default: 'hybrid')
  --cheribsd/build-alternate-abi-kernels, --cheribsd/no-build-alternate-abi-kernels
                        Also build kernels with non-default ABI (purecap or hybrid) (default: 'True')
  --cheribsd/build-bench-kernels, --cheribsd/no-build-bench-kernels
                        Also build benchmark kernels (default: 'False')
  --cheribsd/caprevoke-kernel, --cheribsd/no-caprevoke-kernel
                        Build kernel with caprevoke support (experimental) (default: 'False')

Options for target 'cheribsd-mfs-root-kernel':
  --cheribsd-mfs-root-kernel/build-tests, --cheribsd-mfs-root-kernel/no-build-tests
                        Build the tests (default: 'True')
  --cheribsd-mfs-root-kernel/build-options OPTIONS
                        Additional make options to be passed to make when building FreeBSD/CheriBSD. See `man src.conf`
                        for more info. (default: '[]')
  --cheribsd-mfs-root-kernel/debug-info, --cheribsd-mfs-root-kernel/no-debug-info
                        pass make flags for building with debug info (default: 'True')
  --cheribsd-mfs-root-kernel/build-fpga-kernels, --cheribsd-mfs-root-kernel/no-build-fpga-kernels
                        Also build kernels for the FPGA. (default: 'False')
  --cheribsd-mfs-root-kernel/default-kernel-abi {hybrid,purecap}
                        Select default kernel to build (default: 'hybrid')
  --cheribsd-mfs-root-kernel/build-alternate-abi-kernels, --cheribsd-mfs-root-kernel/no-build-alternate-abi-kernels
                        Also build kernels with non-default ABI (purecap or hybrid) (default: 'True')
  --cheribsd-mfs-root-kernel/build-bench-kernels, --cheribsd-mfs-root-kernel/no-build-bench-kernels
                        Also build benchmark kernels (default: 'False')
  --cheribsd-mfs-root-kernel/caprevoke-kernel, --cheribsd-mfs-root-kernel/no-caprevoke-kernel
                        Build kernel with caprevoke support (experimental) (default: 'False')

Options for target 'freebsd-release':
  --freebsd-release/build-tests, --freebsd-release/no-build-tests
                        Build the tests (default: 'True')
  --freebsd-release/build-options OPTIONS
                        Additional make options to be passed to make when building FreeBSD/CheriBSD. See `man src.conf`
                        for more info. (default: '[]')
  --freebsd-release/debug-info, --freebsd-release/no-debug-info
                        pass make flags for building with debug info (default: 'True')
  --freebsd-release/subdir SUBDIRS
                        Only build subdirs SUBDIRS instead of the full tree. Useful for quickly rebuilding individual
                        programs/libraries. If more than one dir is passed, they will be processed in order. Note: This
                        will break if not all dependencies have been built. (default: 'the value of the global
                        --freebsd-subdir options')

Options for target 'cheribsd-release':
  --cheribsd-release/build-tests, --cheribsd-release/no-build-tests
                        Build the tests (default: 'True')
  --cheribsd-release/build-options OPTIONS
                        Additional make options to be passed to make when building FreeBSD/CheriBSD. See `man src.conf`
                        for more info. (default: '[]')
  --cheribsd-release/debug-info, --cheribsd-release/no-debug-info
                        pass make flags for building with debug info (default: 'True')
  --cheribsd-release/subdir SUBDIRS
                        Only build subdirs SUBDIRS instead of the full tree. Useful for quickly rebuilding individual
                        programs/libraries. If more than one dir is passed, they will be processed in order. Note: This
                        will break if not all dependencies have been built. (default: 'the value of the global
                        --freebsd-subdir options')
  --cheribsd-release/build-fpga-kernels, --cheribsd-release/no-build-fpga-kernels
                        Also build kernels for the FPGA. (default: 'False')
  --cheribsd-release/default-kernel-abi {hybrid,purecap}
                        Select default kernel to build (default: 'hybrid')
  --cheribsd-release/build-alternate-abi-kernels, --cheribsd-release/no-build-alternate-abi-kernels
                        Also build kernels with non-default ABI (purecap or hybrid) (default: 'True')
  --cheribsd-release/build-bench-kernels, --cheribsd-release/no-build-bench-kernels
                        Also build benchmark kernels (default: 'False')
  --cheribsd-release/caprevoke-kernel, --cheribsd-release/no-caprevoke-kernel
                        Build kernel with caprevoke support (experimental) (default: 'False')

Options for target 'cheribsd-sysroot':
  --cheribsd-sysroot/remote-sdk-path PATH
                        The path to the CHERI SDK on the remote FreeBSD machine (e.g. vica:~foo/cheri/output/sdk)

Options for target 'disk-image-minimal':
  --disk-image-minimal/extra-files DIR
                        A directory with additional files that will be added to the image (default: '$SOURCE_ROOT/extra-
                        files-minimal')
  --disk-image-minimal/rootfs-type {ufs,zfs}
                        Select the type of the root file system image. (default: 'ufs')
  --disk-image-minimal/path IMGPATH
                        The output path for the disk image (default: '$OUTPUT_ROOT/cheribsd-minimal-<TARGET>-disk.img
                        depending on architecture')

Options for target 'disk-image-mfs-root':
  --disk-image-mfs-root/extra-files DIR
                        A directory with additional files that will be added to the image (default: '$SOURCE_ROOT/extra-
                        files-minimal')
  --disk-image-mfs-root/rootfs-type {ufs,zfs}
                        Select the type of the root file system image. (default: 'ufs')
  --disk-image-mfs-root/path IMGPATH
                        The output path for the disk image (default: '$OUTPUT_ROOT/cheribsd-mfs-root-<TARGET>-disk.img
                        depending on architecture')

Options for target 'disk-image':
  --disk-image/extra-files DIR
                        A directory with additional files that will be added to the image (default: '$SOURCE_ROOT/extra-
                        files')
  --disk-image/rootfs-type {ufs,zfs}
                        Select the type of the root file system image. (default: 'ufs')
  --disk-image/path IMGPATH
                        The output path for the disk image (default: '$OUTPUT_ROOT/cheribsd-<TARGET>-disk.img depending
                        on architecture')

Options for target 'rootfs-tarball':
  --rootfs-tarball/extra-files DIR
                        A directory with additional files that will be added to the image (default: '$SOURCE_ROOT/extra-
                        files')
  --rootfs-tarball/rootfs-type {ufs,zfs}
                        Select the type of the root file system image. (default: 'ufs')
  --rootfs-tarball/path IMGPATH
                        The output path for the disk image (default: '$OUTPUT_ROOT/cheribsd-<TARGET>.tar.xz depending on
                        architecture')

Options for target 'disk-image-freebsd':
  --disk-image-freebsd/extra-files DIR
                        A directory with additional files that will be added to the image (default: '$SOURCE_ROOT/extra-
                        files')
  --disk-image-freebsd/rootfs-type {ufs,zfs}
                        Select the type of the root file system image. (default: 'ufs')
  --disk-image-freebsd/path IMGPATH
                        The output path for the disk image (default: '$OUTPUT_ROOT/freebsd-<TARGET>-disk.img depending
                        on architecture')

Options for target 'run':
  --run/ssh-forwarding-port PORT
                        The port on localhost to forward to the QEMU ssh port. You can then use `ssh root@localhost -p
                        $PORT` to connect to the VM (default: '<UID-dependent>')
  --run/ephemeral, --run/no-ephemeral
                        Run qemu in 'snapshot' mode, changes to the disk image are non-persistent (default: 'False')
  --run/remote-kernel-path RUN/REMOTE-KERNEL-PATH
                        When set rsync will be used to update the kernel image from a remote host before launching QEMU.
                        Useful when building and running on separate machines.
  --run/alternative-kernel RUN/ALTERNATIVE-KERNEL
                        Select the kernel to run by specifying the kernel build configuration name.The list of available
                        kernel configurations is given by --list-kernels
  --run/kernel-abi {hybrid,purecap}
                        Select extra kernel variant with the given ABI to run.

Options for target 'run-minimal':
  --run-minimal/ssh-forwarding-port PORT
                        The port on localhost to forward to the QEMU ssh port. You can then use `ssh root@localhost -p
                        $PORT` to connect to the VM (default: '<UID-dependent>')
  --run-minimal/ephemeral, --run-minimal/no-ephemeral
                        Run qemu in 'snapshot' mode, changes to the disk image are non-persistent (default: 'False')
  --run-minimal/remote-kernel-path RUN-MINIMAL/REMOTE-KERNEL-PATH
                        When set rsync will be used to update the kernel image from a remote host before launching QEMU.
                        Useful when building and running on separate machines.
  --run-minimal/alternative-kernel RUN-MINIMAL/ALTERNATIVE-KERNEL
                        Select the kernel to run by specifying the kernel build configuration name.The list of available
                        kernel configurations is given by --list-kernels
  --run-minimal/kernel-abi {hybrid,purecap}
                        Select extra kernel variant with the given ABI to run.

Options for target 'run-mfs-root':
  --run-mfs-root/ssh-forwarding-port PORT
                        The port on localhost to forward to the QEMU ssh port. You can then use `ssh root@localhost -p
                        $PORT` to connect to the VM (default: '<UID-dependent>')
  --run-mfs-root/remote-kernel-path RUN-MFS-ROOT/REMOTE-KERNEL-PATH
                        When set rsync will be used to update the kernel image from a remote host before launching QEMU.
                        Useful when building and running on separate machines.
  --run-mfs-root/alternative-kernel RUN-MFS-ROOT/ALTERNATIVE-KERNEL
                        Select the kernel to run by specifying the kernel build configuration name.The list of available
                        kernel configurations is given by --list-kernels
  --run-mfs-root/kernel-abi {hybrid,purecap}
                        Select extra kernel variant with the given ABI to run.

Options for target 'sslproc':
  --sslproc/build-tests, --sslproc/no-build-tests
                        Build the tests (default: 'False')

Options for target 'bash':
  --bash/set-as-root-shell, --bash/no-set-as-root-shell
                        Set root's shell to bash (in the target rootfs) (default: 'False')

Options for target 'freertos':
  --freertos/demo DEMO  The FreeRTOS Demo build. (default: 'RISC-V-Generic')
  --freertos/prog PROG  The FreeRTOS program to build. (default: 'main_blinky')
  --freertos/bsp BSP    The FreeRTOS BSP to build. This is only valid for the paramterized RISC-V-Generic. The BSP
                        option chooses platform, RISC-V arch and RISC-V abi in the $platform-$arch-$abi format. See
                        RISC-V-Generic/README for more details (default: 'target-dependent default')

Options for target 'run-freertos':
  --run-freertos/demo DEMO
                        The FreeRTOS Demo to run. (default: 'RISC-V-Generic')
  --run-freertos/prog PROG
                        The FreeRTOS program to run. (default: 'main_blinky')
  --run-freertos/bsp BSP
                        The FreeRTOS BSP to run. This is only valid for the paramterized RISC-V-Generic. The BSP option
                        chooses platform, RISC-V arch and RISC-V abi in the $platform-$arch-$abi format. See RISC-V-
                        Generic/README for more details (default: 'target-dependent default')

Options for target 'qtbase-dev':
  --qtbase-dev/build-tests, --qtbase-dev/no-build-tests
                        build the Qt unit tests (default: 'True')
  --qtbase-dev/build-examples, --qtbase-dev/no-build-examples
                        build the Qt examples (default: 'False')
  --qtbase-dev/assertions, --qtbase-dev/no-assertions
                        Include assertions (default: 'True')
  --qtbase-dev/minimal, --qtbase-dev/no-minimal
                        Don't build QtWidgets or QtGui, etc (default: 'True')

Options for target 'qtwebkit':
  --qtwebkit/build-jsc-only, --qtwebkit/no-build-jsc-only
                        only build the JavaScript interpreter executable (default: 'False')

Options for target 'morello-webkit':
  --morello-webkit/backend {cloop,tier1asm,tier2asm}
                        The JavaScript backend to use for building WebKit (default: 'cloop')
  --morello-webkit/tier2ptrliterals, --morello-webkit/no-tier2ptrliterals
                        When true pointers are represented as atomic literals and loaded as data and when false pointers
                        are represented as numeric values which can be splitted and are encoded into instructions. This
                        option only affects the non-purecap tier2 backend. (default: 'True')
  --morello-webkit/jsheapoffsets, --morello-webkit/no-jsheapoffsets
                        Use offsets into the JS heap for object references instead of capabilities. This option only
                        affects the purecap backends. (default: 'False')

cheribuild's People

Contributors

Stargazers

Watchers

cheribuild's Issues

Allow building 128 bit SDK

freebsd-mips target fails to build due to -mcpu=beri

We're adding -mcpu=beri even when using upstream-llvm. This hack seems to be sufficient to work, but obviously breaks most CheriBSD targets:

diff --git a/pycheribuild/projects/cross/cheribsd.py b/pycheribuild/projects/cross/cheribsd.py
index a7b35cae..91c4039c 100644
--- a/pycheribuild/projects/cross/cheribsd.py
+++ b/pycheribuild/projects/cross/cheribsd.py
@@ -529,7 +529,7 @@ class BuildFreeBSD(BuildFreeBSDBase):
             self.linker_for_kernel = "lld"  # bfd won't work here
             self.linker_for_world = "lld"
         elif self.compiling_for_mips(include_purecap=True):
-            target_flags = " -integrated-as -fcolor-diagnostics -mcpu=beri"
+            target_flags = " -integrated-as -fcolor-diagnostics"
             self.cross_toolchain_config.set_with_options(RESCUE=False,  # Won't compile with CHERI clang yet
                                                          BOOT=False)  # bootloaders won't link with LLD yet
         elif self.compiling_for_riscv(include_purecap=True):

Allow Projects to specify dependencies

Per-project options

Currently there is no way to specify certain build options on a per-project basis. One of those is --skip-update, which if specified per-project would allow us to do things such as upgrade LLVM and rebuild CheriBSD using the new LLVM.

postgres build fails

PostgreSQL build fails like this:

gmake[2]: /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-ar: Command not found
/home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-ar cr libpgcommon.a config_info.o controldata_utils.o exec.o keywords.o pg_lzcompress.o pgfnames.o psprintf.o relpath.o rmtree.o string.o username.o wait_error.o fe_memutils.o restricted_token.o 
gmake[2]: *** [Makefile:61: libpgcommon.a] Error 127
gmake[2]: *** Waiting for unfinished jobs....
gmake[2]: Leaving directory '/usr/home/en322/cheri/postgres/src/common' 
gmake[1]: *** [Makefile:36: all-common-recurse] Error 2
gmake[1]: Leaving directory '/usr/home/en322/cheri/postgres/src' 
gmake: *** [GNUmakefile:11: all-src-recurse] Error 2
Command "nice gmake -j8" failed with exit code 2.
See /home/en322/cheri/postgres/gmake.log for details.
[en322@thyme:~/git/cheribuild]% 
[en322@thyme:~/git/cheribuild]% 
[en322@thyme:~/git/cheribuild]% /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-ar
zsh: no such file or directory: /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-ar
[en322@thyme:~/git/cheribuild]% ls -al /home/en322/cheri/output/sdk256/bin/*                       
-rwxr-xr-x  1 en322  wheel  16736312 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/bugpoint
-rwxr-xr-x  1 en322  wheel  19978352 Aug  4 13:31 /home/en322/cheri/output/sdk256/bin/c-index-test
-rwxr-xr-x  1 en322  wheel    759936 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/capsizefix
lrwx------  1 en322  wheel         7 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-c++ -> clang++
lrwx------  1 en322  wheel         5 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-cc -> clang
lrwx------  1 en322  wheel         5 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-clang -> clang
lrwx------  1 en322  wheel         9 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-clang-cpp -> clang-cpp
lrwx------  1 en322  wheel         7 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-clang++ -> clang++
lrwx------  1 en322  wheel         9 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-cpp -> clang-cpp
lrwx------  1 en322  wheel         6 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-ld -> ld.lld
lrwx------  1 en322  wheel         6 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-ld.lld -> ld.lld
lrwx------  1 en322  wheel         3 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-llc -> llc
lrwx------  1 en322  wheel         7 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-llvm-mc -> llvm-mc
lrwx------  1 en322  wheel        12 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-llvm-objdump -> llvm-objdump
lrwx------  1 en322  wheel        12 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-llvm-readobj -> llvm-readobj
lrwx------  1 en322  wheel         9 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-llvm-size -> llvm-size
lrwx------  1 en322  wheel         9 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/clang -> clang-5.0
-rwxr-xr-x  1 en322  wheel  74538944 Aug  4 13:31 /home/en322/cheri/output/sdk256/bin/clang-5.0
-rwxr-xr-x  1 en322  wheel  55321352 Aug  4 13:31 /home/en322/cheri/output/sdk256/bin/clang-check
lrwx------  1 en322  wheel         5 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/clang-cl -> clang
lrwx------  1 en322  wheel         5 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/clang-cpp -> clang
-rwxr-xr-x  1 en322  wheel   1994848 Aug  4 13:31 /home/en322/cheri/output/sdk256/bin/clang-format
-rwxr-xr-x  1 en322  wheel  20993592 Aug  4 13:31 /home/en322/cheri/output/sdk256/bin/clang-import-test
-rwxr-xr-x  1 en322  wheel   2809848 Aug  4 13:31 /home/en322/cheri/output/sdk256/bin/clang-offload-bundler
lrwx------  1 en322  wheel         5 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/clang++ -> clang
-rwx------  1 en322  wheel      9894 Aug  4 13:55 /home/en322/cheri/output/sdk256/bin/fixlinks
-rwxr-xr-x  1 en322  wheel     21718 Jun 19 13:50 /home/en322/cheri/output/sdk256/bin/git-clang-format
lrwx------  1 en322  wheel         6 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/ld -> ld.lld
lrwx------  1 en322  wheel         3 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/ld.lld -> lld
-rwxr-xr-x  1 en322  wheel  44329416 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llc
-rwxr-xr-x  1 en322  wheel  47792304 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/lld
lrwx------  1 en322  wheel         3 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/lld-link -> lld
-rwxr-xr-x  1 en322  wheel  21855008 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/lli
-rwxr-xr-x  1 en322  wheel  14001824 Aug  4 13:21 /home/en322/cheri/output/sdk256/bin/llvm-ar
-rwxr-xr-x  1 en322  wheel   2328896 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-as
-rwxr-xr-x  1 en322  wheel    412840 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-bcanalyzer
-rwxr-xr-x  1 en322  wheel  34398440 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-c-test
-rwxr-xr-x  1 en322  wheel   2295328 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-cat
-rwxr-xr-x  1 en322  wheel    346560 Aug  4 13:21 /home/en322/cheri/output/sdk256/bin/llvm-config
-rwxr-xr-x  1 en322  wheel   2920168 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-cov
-rwxr-xr-x  1 en322  wheel   2538568 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-cvtres
-rwxr-xr-x  1 en322  wheel   2579088 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-cxxdump
-rwxr-xr-x  1 en322  wheel    363664 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-cxxfilt
-rwxr-xr-x  1 en322  wheel   2105400 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-diff
-rwxr-xr-x  1 en322  wheel   1778256 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-dis
-rwxr-xr-x  1 en322  wheel  33498904 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-dsymutil
-rwxr-xr-x  1 en322  wheel   2774088 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-dwarfdump
-rwxr-xr-x  1 en322  wheel  33101552 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-dwp
-rwxr-xr-x  1 en322  wheel   3207584 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-extract
lrwx------  1 en322  wheel         7 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/llvm-lib -> llvm-ar
-rwxr-xr-x  1 en322  wheel   2787024 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-link
-rwxr-xr-x  1 en322  wheel  40485216 Aug  4 13:22 /home/en322/cheri/output/sdk256/bin/llvm-lto
-rwxr-xr-x  1 en322  wheel  41569112 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-lto2
-rwxr-xr-x  1 en322  wheel  15064936 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-mc
-rwxr-xr-x  1 en322  wheel    374336 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-mcmarkup
-rwxr-xr-x  1 en322  wheel   1968640 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-modextract
-rwxr-xr-x  1 en322  wheel  14026824 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-nm
-rwxr-xr-x  1 en322  wheel  14323096 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-objdump
-rwxr-xr-x  1 en322  wheel    441552 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-opt-report
-rwxr-xr-x  1 en322  wheel   2096680 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-pdbdump
-rwxr-xr-x  1 en322  wheel   1284536 Aug  4 13:21 /home/en322/cheri/output/sdk256/bin/llvm-profdata
lrwx------  1 en322  wheel         7 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/llvm-ranlib -> llvm-ar
-rwxr-xr-x  1 en322  wheel   3740264 Aug  4 13:33 /home/en322/cheri/output/sdk256/bin/llvm-readobj
-rwxr-xr-x  1 en322  wheel  12623976 Aug  4 13:33 /home/en322/cheri/output/sdk256/bin/llvm-rtdyld
-rwxr-xr-x  1 en322  wheel   2579072 Aug  4 13:33 /home/en322/cheri/output/sdk256/bin/llvm-size
-rwxr-xr-x  1 en322  wheel   2682544 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-split
-rwxr-xr-x  1 en322  wheel   3278912 Aug  4 13:33 /home/en322/cheri/output/sdk256/bin/llvm-stress
-rwxr-xr-x  1 en322  wheel    368928 Aug  4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-strings
-rwxr-xr-x  1 en322  wheel   2835136 Aug  4 13:33 /home/en322/cheri/output/sdk256/bin/llvm-symbolizer
-rwxr-xr-x  1 en322  wheel   2287440 Aug  4 13:07 /home/en322/cheri/output/sdk256/bin/llvm-tblgen
-rwxr-xr-x  1 en322  wheel   3214880 Aug  4 13:33 /home/en322/cheri/output/sdk256/bin/llvm-xray
lrwx------  1 en322  wheel         7 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-c++ -> clang++
lrwx------  1 en322  wheel         5 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-cc -> clang
lrwx------  1 en322  wheel         5 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-clang -> clang
lrwx------  1 en322  wheel         9 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-clang-cpp -> clang-cpp
lrwx------  1 en322  wheel         7 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-clang++ -> clang++
lrwx------  1 en322  wheel         9 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-cpp -> clang-cpp
lrwx------  1 en322  wheel         6 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-ld -> ld.lld
lrwx------  1 en322  wheel         6 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-ld.lld -> ld.lld
lrwx------  1 en322  wheel         3 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-llc -> llc
lrwx------  1 en322  wheel         7 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-llvm-mc -> llvm-mc
lrwx------  1 en322  wheel        12 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-llvm-objdump -> llvm-objdump
lrwx------  1 en322  wheel        12 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-llvm-readobj -> llvm-readobj
lrwx------  1 en322  wheel         9 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-llvm-size -> llvm-size
lrwx------  1 en322  wheel         7 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-c++ -> clang++
lrwx------  1 en322  wheel         5 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-cc -> clang
lrwx------  1 en322  wheel         5 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-clang -> clang
lrwx------  1 en322  wheel         9 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-clang-cpp -> clang-cpp
lrwx------  1 en322  wheel         7 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-clang++ -> clang++
lrwx------  1 en322  wheel         9 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-cpp -> clang-cpp
lrwx------  1 en322  wheel         6 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-ld -> ld.lld
lrwx------  1 en322  wheel         6 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-ld.lld -> ld.lld
lrwx------  1 en322  wheel         3 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-llc -> llc
lrwx------  1 en322  wheel         7 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-llvm-mc -> llvm-mc
lrwx------  1 en322  wheel        12 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-llvm-objdump -> llvm-objdump
lrwx------  1 en322  wheel        12 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-llvm-readobj -> llvm-readobj
lrwx------  1 en322  wheel         9 Aug  4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-llvm-size -> llvm-size
-rwxr-xr-x  1 en322  wheel   3200024 Aug  4 13:33 /home/en322/cheri/output/sdk256/bin/obj2yaml
-rwxr-xr-x  1 en322  wheel  46395144 Aug  4 13:33 /home/en322/cheri/output/sdk256/bin/opt
-rwxr-xr-x  1 en322  wheel    352984 Aug  4 14:22 /home/en322/cheri/output/sdk256/bin/qemu-ga
-rwxr-xr-x  1 en322  wheel    874168 Aug  4 14:22 /home/en322/cheri/output/sdk256/bin/qemu-img
-rwxr-xr-x  1 en322  wheel    910240 Aug  4 14:22 /home/en322/cheri/output/sdk256/bin/qemu-io
-rwxr-xr-x  1 en322  wheel    845368 Aug  4 14:22 /home/en322/cheri/output/sdk256/bin/qemu-nbd
-rwxr-xr-x  1 en322  wheel   5727568 Aug  4 14:22 /home/en322/cheri/output/sdk256/bin/qemu-system-cheri
-rwxr-xr-x  1 en322  wheel  13887928 Aug  4 13:33 /home/en322/cheri/output/sdk256/bin/sancov
-rwxr-xr-x  1 en322  wheel   2812248 Aug  4 13:33 /home/en322/cheri/output/sdk256/bin/sanstats
-rwxr-xr-x  1 en322  wheel     53444 Mar  3 10:50 /home/en322/cheri/output/sdk256/bin/scan-build
-rwxr-xr-x  1 en322  wheel      4504 Mar  3 10:50 /home/en322/cheri/output/sdk256/bin/scan-view
-rwxr-xr-x  1 en322  wheel   2580208 Aug  4 13:33 /home/en322/cheri/output/sdk256/bin/verify-uselistorder
-rwxr-xr-x  1 en322  wheel    925776 Aug  4 13:33 /home/en322/cheri/output/sdk256/bin/yaml2obj

--include-dependencies, -d not recognised by jenkins-cheri-build.py?

This works fine with cheribuild.py but gives the following error with jenkins-cheri-build.py:

jenkins-cheri-build.py: error: unrecognized arguments: -d

copying of SSH public key fails because .ssh folder doesn't exist

Copying of the current user's SSH public key to the image's root/.ssh/authorized_files fails because the root/.ssh directories don't exist.

Here is a quick hack: https://gist.github.com/kgudka/a3da5e1b8a8c37d83ed9f7ac563062c8

I'm not sure if there is some other underlying problem though, as it never used to do this.

Support for -march, -mabi and -mcmodel RISC-V command line flags

Providing those in the command line is gonna be very useful especially when we're doing multilib

qtwebkit fails to build because cheribuild thinks the CMake version is 0.0.0

Looks like it is due to _get_cmake_version:

1043     def _get_cmake_version(self):
1044         if not Path(self.configureCommand).exists():
1045             return (0, 0, 0)
1046         return get_program_version(self.configureCommand, program_name=b"cmake")

self.configureCommand is set to "cmake" and so the if test succeeds, as it's not an absolute path. I can work around this for now by setting the CMAKE_COMMAND environment variable to the full path to cmake.

disk image code refactor breaks cheribsd builds?

With aba62a2 in tree, attempting to make disk images fails with

Traceback (most recent call last):
  File "/home/nwf20/cheri/mainline/cheribuild/cheribuild.py", line 42, in <module>
    main()
  File "/local/scratch/nwf20/cheri/mainline/cheribuild/pycheribuild/__main__.py", line 227, in main
    real_main()
  File "/local/scratch/nwf20/cheri/mainline/cheribuild/pycheribuild/__main__.py", line 219, in real_main
    targetManager.run(cheriConfig)
  File "/local/scratch/nwf20/cheri/mainline/cheribuild/pycheribuild/targets.py", line 320, in run
    target.execute(config)
  File "/local/scratch/nwf20/cheri/mainline/cheribuild/pycheribuild/targets.py", line 92, in execute
    project.process()
  File "/local/scratch/nwf20/cheri/mainline/cheribuild/pycheribuild/projects/disk_image.py", line 473, in process
    self.__process()
  File "/local/scratch/nwf20/cheri/mainline/cheribuild/pycheribuild/projects/disk_image.py", line 535, in __process
    pathInImage = p.relative_to(self.extraFilesDir)
  File "/usr/lib/python3.6/pathlib.py", line 874, in relative_to
    .format(str(self), str(formatted)))
ValueError: '/home/nwf20/cheri/mainline/host-output/kyua-cheribsd-pkgcache/usr/lib/libarchive.so.6' does not start with '/home/nwf20/cheribsd-extra-files'

I'm invoking

cheribuild.py --disk-image/extra-files=${HOME}/cheribsd-extra-files --disk-image-path=... --disk-image/auto-add-files

Randomise port used for SSH forwarding

If multiple people use this script on the same host then a hard coded port for SSH will not work. Consider using a user's uid or something that would randomise it.

Unable to build qtwebkit

I'm having issues building qtwebkit as of commit c72abd6 receiving the following errors during Command "nice ninja jsc -j8" failed with exit code 1.

/home/sean/cheri/qt5/qtwebkit/Source/WTF/wtf/RedBlackTree.h:106:48: error: use of undeclared identifier 'qClearLowPointerBits'
            return reinterpret_cast<NodeType*>(qClearLowPointerBits<1u>(m_parentAndRed));
                                               ^
/home/sean/cheri/qt5/qtwebkit/Source/WTF/wtf/RedBlackTree.h:111:71: error: use of undeclared identifier 'qGetLowPointerBits'
            m_parentAndRed = reinterpret_cast<uintptr_t>(newParent) | qGetLowPointerBits<1u>(m_parentAndRed);
                                                                      ^
/home/sean/cheri/qt5/qtwebkit/Source/WTF/wtf/RedBlackTree.h:136:17: error: use of undeclared identifier 'qGetLowPointerBits'
            if (qGetLowPointerBits<1u>(m_parentAndRed))
                ^
/home/sean/cheri/qt5/qtwebkit/Source/WTF/wtf/RedBlackTree.h:146:34: error: use of undeclared identifier 'qClearLowPointerBits'
                m_parentAndRed = qClearLowPointerBits<1u>(m_parentAndRed);
                                 ^

Git blame indicates that the following line and commit introduced qClearLowPointerBits: CTSRD-CHERI/qtwebkit@d6cd620

However I'm unable to find qClearLowPointerBits in any commit of CTSRD-CHERI/qtbase.

Could someone please advise on how best to proceed with compiling qtwebkit?

My repos are in the following states:
cheri/icu4c : commit 900f5dcf87beca158ea62987482cbd4e517bb07d
cheri/libxml2: commit c138001afa0bce70d3b9f84f12914e22bce18aa6
cheri/qt5/qtwebkit: commit e78ae7ab924d27e866e5ba6649ec093cd8f03ee2
cheri/qt5/qtbase: commit 08a110622c228046e1d1c2f48504c4ea544cb18b with the following diff

diff --git a/src/3rdparty/harfbuzz-ng/src/hb-buffer.cc b/src/3rdparty/harfbuzz-ng/src/hb-buffer.cc
index 3940a3dbf8..28e164cf74 100644
--- a/src/3rdparty/harfbuzz-ng/src/hb-buffer.cc
+++ b/src/3rdparty/harfbuzz-ng/src/hb-buffer.cc
@@ -83,9 +83,13 @@ hb_segment_properties_equal (const hb_segment_properties_t *a,
 unsigned int
 hb_segment_properties_hash (const hb_segment_properties_t *p)
 {
+  typedef size_t cheri_fake_ptr;
+  // static_assert(sizeof(cheri_fake_ptr) == sizeof(intptr_t));
   return (unsigned int) p->direction ^
         (unsigned int) p->script ^
-        (intptr_t) (p->language);
+     // i'm not sure if this is a valid fix;
+     // based off https://github.com/svaarala/duktape/issues/530;w
+        (cheri_fake_ptr) (p->language);
 }
diff --git a/src/3rdparty/libpng/pngpriv.h b/src/3rdparty/libpng/pngpriv.h                                                                                                                       [0/48184]
index 922fe80c29..b04764b350 100644
--- a/src/3rdparty/libpng/pngpriv.h
+++ b/src/3rdparty/libpng/pngpriv.h
@@ -20,6 +20,9 @@
  * they should be well aware of the issues that may arise from doing so.
  */

+// Added to silence compiler warnings about uintptr_t
+#include <stdint.h>
+
 #ifndef PNGPRIV_H
 #define PNGPRIV_H

@@ -501,7 +504,8 @@
          typedef unsigned __int64 png_ptruint;
 #     endif
 #  else
-      typedef unsigned long png_ptruint;
+      typedef uintptr_t png_ptruint;
+      // typedef unsigned long png_ptruint;
 #  endif
 #  define png_constcast(type, value) ((type)(png_ptruint)(const void*)(value))
 #  define png_aligncast(type, value) ((void*)(value))
diff --git a/src/3rdparty/sqlite/sqlite3.c b/src/3rdparty/sqlite/sqlite3.c
index ea5ba16b6f..69ac619a61 100644
--- a/src/3rdparty/sqlite/sqlite3.c
+++ b/src/3rdparty/sqlite/sqlite3.c
@@ -11987,7 +11987,7 @@ struct fts5_api {
 ** compiler.
 */
 #if defined(__PTRDIFF_TYPE__)  /* This case should work for GCC */
-# define SQLITE_INT_TO_PTR(X)  ((void*)(__PTRDIFF_TYPE__)(X))
+# define SQLITE_INT_TO_PTR(X)  ((void*)((intptr_t)(__PTRDIFF_TYPE__)(X)))
 # define SQLITE_PTR_TO_INT(X)  ((int)(__PTRDIFF_TYPE__)(X))
 #elif !defined(__GNUC__)       /* Works for compilers other than LLVM */
 # define SQLITE_INT_TO_PTR(X)  ((void*)&((char*)0)[X])
diff --git a/src/testlib/qtestcase.cpp b/src/testlib/qtestcase.cpp
index 15fd5b1504..59d83a1a0e 100644
--- a/src/testlib/qtestcase.cpp
+++ b/src/testlib/qtestcase.cpp
@@ -2480,7 +2480,7 @@ template <> Q_TESTLIB_EXPORT char *QTest::toString<__uintcap_t>(const __uintcap_
     char *msg = new char[128];
     // XXXAR: not sure how the compiler will react to passing a reference to a variadic function
     __uintcap_t tmp = t;
-    snprintf(msg, 128, "%lu (%#p)", (qvaddr)tmp, (void *__capability)tmp);
+    snprintf(msg, 128, "%llu (%#p)", (qvaddr)tmp, (void *__capability)tmp);
     return msg;
 }

@@ -2489,7 +2489,7 @@ template <> Q_TESTLIB_EXPORT char *QTest::toString<__intcap_t>(const __intcap_t
     char *msg = new char[128];
     // XXXAR: not sure how the compiler will react to passing a reference to a variadic function
     __intcap_t tmp = t;
-    snprintf(msg, 128, "%ld (%#p)", (qvaddr)tmp, (void *__capability)tmp);
+    snprintf(msg, 128, "%llu (%#p)", (qvaddr)tmp, (void *__capability)tmp);
     return msg;
 }
 #endif

CMake cross-compiler projects should use CMAKE_CROSSCOMPILING_EMULATOR

This way it might be possible to run ctest if CMAKE_CROSSCOMPILING_EMULATOR is set to a script that copies to QEMU and runs it there

It should be possible to create a unified SDK for both 128 and 256

We should now be able to create just a single SDK since LLVM can target both 128 and 256

Clang multicapsize has been merged -> stop appending cheribits to build dir (not possible until the install dir is also the same)
Install the sdk into $OUTPUT_ROOT/sdk instead of a suffixed dir
CheriBSD sysroot should be installed to $OUTPUT_ROOT/sdk/sysroot{128,256}
Baremetal newlib sysroot should be installed to $OUTPUT_ROOT/sdk/baremetal
QEMU needs to install qemu-system-cheri{128,256} binaries (CTSRD-CHERI/qemu#61, CTSRD-CHERI/qemu#62)
The run targets need to select the appropriate qemu-system-cheri
Probably a bunch of other stuff

Automatically generate symlinks for each configuration in .conf

Audit pexpect calls for dot and newline matches

pexpect compiles all regular expressions with re.DOTALL which means that .+ at the end of a regex will match everything until the end of the internal buffer.

Also the docs say that we can't use the $ char (https://pexpect.readthedocs.io/en/stable/overview.html#find-the-end-of-line-cr-lf-conventions) so we should change all of those (if there are any) to \r\n

Cheribuild asks about update despite "skip-update": "true"

With the following in ~/.config/cheribuild.json, ./cheribuild sdk still asks for update of cheribuild itself (updating other things is disabled without questions, as expected):

% ./cheribuild.py sdk run Sources will be stored in /home/en322/cheri Build artifacts will be stored in /home/en322/cheri/output
Current CheriBuild checkout can be updated: Your branch is behind 'origin/master' by 16 commits, and can be fast-forwarded. Would you like to update before continuing? y/[n] (Enter to skip)

opam-2.0 installation fails to create target directory

I tried to install opam-2.0 (the one in Ubuntu 18.04 is too old):

$ ./cheribuild.py opam-2.0
Configuration file /home/atm26/.config/cheribuild.json does not exist, using only command line arguments.
Sources will be stored in /home/atm26/cheri
Build artifacts will be stored in /home/atm26/cheri/output
Will execute the following targets:
   opam-2.0
wget https://github.com/ocaml/opam/releases/download/2.0.1/opam-2.0.1-x86_64-linux -O /home/atm26/cheri/output/bootstrap/bin/opam
/home/atm26/cheri/output/bootstrap/bin/opam: No such file or directory
Fatal error:Command `wget https://github.com/ocaml/opam/releases/download/2.0.1/opam-2.0.1-x86_64-linux -O /home/atm26/cheri/output/bootstrap/bin/opam` failed with non-zero exit code 1. Working directory was /local/scratch/atm26/ecats/cheribuild

I think it needs to mkdir -p $CHERI_SDK/bootstrap/bin but I'm not sure where in the code that should go.

CrossCompileCMakeProject missing some compiler flags

Due to a recent change, CrossCompileCMakeProject doesn't seem to pass the following compiler flags anymore:

-mabi=purecap -mcpu=mips4 -cheri=128|256

I noticed this with QtWebKit. Does qt5.py need to be updated or is it a bug in CrossCompileCMakeProject?

Set CMake library suffix for static libraries

These CMake variables need to be set to ensure that CMake only finds static libraries when doing a static build:

set(CMAKE_SHARED_LIBRARY_SUFFIX ".a")
set(CMAKE_FIND_LIBRARY_SUFFIXES ".a")
set(CMAKE_EXTRA_SHARED_LIBRARY_SUFFIXES ".a")

cheribuild assumes gdb folder exists when checking URL

When building gdb for the first time (no build folder exists for it), I get the following error:

cd /home/CTSRD/CHERI/cheri-build/gdb && git remote get-url origin
Fatal error:Command `git remote get-url origin` failed with non-zero exit code 2. Working directory was /home/CTSRD/CHERI/cheri-build/gdb

Should consistently use python style names

The current mixture of camel case and underscores is extremely confusing.

Properly setup ssh access to qemu

used port on host machine must be configurable
Install a ssh host key that doesn't change when disk image is rebuilt
setup the right authorized_keys entries so passwordless login works?
add mtree entries
add PermitRootLogin without-password to sshd_config

postgres target fails to do distclean in some cases

Differently from llvm or cheribsd targets, the postgres target doesn't use separate build directories for 128 and 256 bit build. This means that doing "./cheribuild --128 postgres" can result in previously built 256 binaries to be installed.

Warning flags seem to leak from gdb.py, postgres.py and nginx.py

When building QtWebkit, the compiler flags seem have warning flags that are set in gdb.py, postgres.py and nginx.py:

-Wno-mismatched-tags -Wno-unknown-warning-option -Wno-mismatched-tags -Wno-unknown-warning-option -Wno-mismatched-tags -Wno-unknown-warning-option -pedantic -Wno-gnu-statement-expression -Wno-flexible-array-extensions -Wno-format-pedantic

Cleaning build directories should use move + background delete

Otherwise it will block for a really long time (especially on vica) before doing anything useful.

Also cleaning the cheribsd rootfs should keep the root directory otherwise NFS reroot breaks

Testing for compiler lld support writes a file named -

The test in pycheribuild/projects/project.py's canUseLLd causes (some versions of?) clang (at least) to generate a file named - in the current working directory, which, while not fatal, is unsightly. It's not clear to me if it's worth using a temporary path, which is unlink()ed on succss, or if we can get away with "/dev/null" rather than "-", since some compiler might conceivably try invoking unlink() before open()ing an existing output. Eliminating "-o" entirely results in, as traditional but unfortunate, a.out.

Qemu build needs pixman dependency

Hi,

I just tried to build the sdk via cheribuild.py sdk on FreeBSD 11

FreeBSD  11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0 
r306420: Thu Sep 29 01:43:23 UTC 2016   
[email protected]:/usr/obj/usr/src/sys/GENERIC  amd64

Apparently qemu is missing a dependency to pixman.
After installing it via pkg install pixman qemu compiled without any further issues.

Perhaps add it as an additional dependency in https://github.com/CTSRD-CHERI/cheribuild/blob/master/pycheribuild/projects/build_qemu.py
See attached console output.

Cloning into '/home/ar/cheri/qemu'...                                  
remote: Counting objects: 337590, done.                                      
remote: Total 337590 (delta 0), reused 0 (delta 0), pack-reused 337589                                                                                                                        
Receiving objects: 100% (337590/337590), 163.86 MiB | 2.05 MiB/s, done.                                                                                                                        
Resolving deltas: 100% (269961/269961), done.                                                                                                                                                  
Already up-to-date.                                                                                                          
Configuring QEMU ...                                                       
cd /home/ar/cheri/build/qemu-build && /home/ar/cheri/qemu/configure --target-list=cheri256-softmmu,cheri128-softmmu,cheri128magic-softmmu --disable-linux-user --disable-bsd-user --disable-xen
 --disable-docs --disable-rdma --disable-werror '--extra-cflags=-O3 -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/usr/local/include' --disable-stack-protector --disable-l
inux-aio --disable-kvm --prefix=/home/ar/cheri/output/sdk             
Saving build log to /home/ar/cheri/build/qemu-build/configure.log      
                                                                    
ERROR: pixman >= 0.21.8 not present. Your options:                    
         (1) Preferred: Install the pixman devel package (any recent      
             distro should have packages as Xorg needs pixman too).      
         (2) Fetch the pixman submodule, using:                     
             git submodule update --init pixman                        
                                                                     
Command "/home/ar/cheri/qemu/configure --target-list=cheri256-softmmu,cheri128-softmmu,cheri128magic-softmmu --disable-linux-user --disable-bsd-user --disable-xen --disable-docs --disable-rdm
a --disable-werror '--extra-cflags=-O3 -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/usr/local/include' --disable-stack-protector --disable-linux-aio --disable-kvm --pref
ix=/home/ar/cheri/output/sdk" failed with exit code 1.
See /home/ar/cheri/build/qemu-build/configure.log for details.

Git repos should be cloned if they don't exist

makefs command fails because manifest file is missing entry

When building a disk image, makefs complains that firstly, the METALOG manifest file is missing the /usr/tests folder and secondly about the entry ./usr/tests/cheri/lib/libc/tls/.debug/ which should be ./usr/tests/cheri/lib/libc/tls/.debug

Make it possible to do a non-shallow git clone

By default, the git clone depth is 1 but one might want the full repo history.

Would be good to make this configurable.

EDIT:
a shallow clone can be converted to an unshallow one with this command:
git fetch --unshallow

cheribuild.py hangs on Ubuntu 18.04

Hello!

Can I please get some guidance on getting past the hang (no progress for the past 11 hrs) that I see while running cheribuild.py on Ubuntu.py? Thanks for any help.

Following is the output of pstree - which I hope will provide the context and commands used.

$ pstree -ahlAt 29358
python3 ./cheribuild.py run-purecap -d --config-file /home/bharadwaj/datadrive/github/src/CHERI/.config/cheribuild.json --source-root /home/bharadwaj/datadrive/github/src/CHERI
  `-bmake -DDB_FROM_SRC -DNO_CLEAN -DI_REALLY_MEAN_NO_CLEAN -DNO_ROOT -DBUILD_WITH_STRICT_TMPPATH TARGET mips TARGET_ARCH mips64c128 CHERI 128 TARGET_CPUTYPE cheri -DWITH_CHERI DEBUG_FLAGS -g LOCAL_XTOOL_DIRS lib/libnetbsd usr.sbin/makefs usr.bin/mkimg CHERI_USE_CAP_TABLE pcrel XAR /home/bharadwaj/datadrive/github/src/CHERI/output/sdk/bin/llvm-ar XNM /home/bharadwaj/datadrive/github/src/CHERI/output/sdk/bin/llvm-nm XSIZE /home/bharadwaj/datadrive/github/src/CHERI/output/sdk/bin/llvm-size XSTRIPBIN /home/bharadwaj/datadrive/github/src/CHERI/output/sdk/bin/llvm-strip XSTRINGS /home/bharadwaj/datadrive/github/src/CHERI/output/sdk/bin/llvm-strings XOBJCOPY /home/bharadwaj/datadrive/github/src/CHERI/output/sdk/bin/llvm-objcopy XRANLIB /home/bharadwaj/datadrive/github/src/CHERI/output/sdk/bin/llvm-ranlib RANLIBFLAGS  -DNO_SHARE METALOG /dev/null -DWITH_TESTS -DWITH_BINUTILS -DWITHOUT_CLANG -DWITHOUT_GCC -DWITHOUT_GDB -DWITHOUT_LLD -DWITHOUT_LLDB -DWITHOUT_MAN -DWITHOUT_MAIL -DWITHOUT_SENDMAIL -DWITHOUT_VT -DWITHOUT_DEBUG_FILES -DWITHOUT_BOOT -DWITH_AUTO_OBJ -DWITHOUT_GCC_BOOTSTRAP -DWITHOUT_CLANG_BOOTSTRAP -DWITHOUT_LLD_BOOTSTRAP -DWITHOUT_LIB32 -DWITHOUT_RESCUE -DWITH_ELFTOOLCHAIN_BOOTSTRAP -DWITH_TOOLCHAIN -DWITHOUT_BINUTILS_BOOTSTRAP -s -de installsysroot -j12 -DWITH_AUTO_OBJ -DNO_CLEAN -DBUILD_WITH_OPIPEFAIL
      `-sh
          `-bmake -m /mnt/datadrive/github/src/CHERI/cheribsd/share/mk -f Makefile.inc1 -DDB_FROM_SRC -DMALLOC_PRODUCTION LOCAL_DIRS ctsrd tools/tools/atsectl LOCAL_LIB_DIRS ctsrd/lib LOCAL_MTREE ctsrd/ctsrd.mtree -DWITH_CHERI128 TARGET mips TARGET_ARCH mips64c128 installsysroot
              `-sh -e
                  `-bmake -f Makefile.inc1 INSTALL install -N /mnt/datadrive/github/src/CHERI/cheribsd/etc -U -M /dev/null -D /home/bharadwaj/datadrive/github/src/CHERI/output/sdk/sysroot128 MTREE_CMD mtree -N /mnt/datadrive/github/src/CHERI/cheribsd/etc -W -DSYSROOT_ONLY MK_TESTS no __MAKE_SHELL /tmp/install.XXyp6ryB/sh -DNO_ROOT METALOG /dev/null reinstallsysroot
                      `-sh -e
                          `-bmake -f Makefile.inc1 LOCAL_MTREE ctsrd/ctsrd.mtree hierarchy
                              `-sh -e
                                  `-bmake LOCAL_MTREE ctsrd/ctsrd.mtree PATH /mnt/datadrive/github/src/CHERI/build/cheribsd-purecap-128-build/mnt/datadrive/github/src/CHERI/cheribsd/mips.mips64c128/tmp/usr/sbin:/mnt/datadrive/github/src/CHERI/build/cheribsd-purecap-128-build/mnt/datadrive/github/src/CHERI/cheribsd/mips.mips64c128/tmp/usr/bin:/mnt/datadrive/github/src/CHERI/build/cheribsd-purecap-128-build/mnt/datadrive/github/src/CHERI/cheribsd/mips.mips64c128/tmp/legacy/usr/sbin:/mnt/datadrive/github/src/CHERI/build/cheribsd-purecap-128-build/mnt/datadrive/github/src/CHERI/cheribsd/mips.mips64c128/tmp/legacy/usr/bin:/mnt/datadrive/github/src/CHERI/build/cheribsd-purecap-128-build/mnt/datadrive/github/src/CHERI/cheribsd/mips.mips64c128/tmp/legacy/bin:/mnt/datadrive/github/src/CHERI/build/cheribsd-purecap-128-build/mnt/datadrive/github/src/CHERI/cheribsd/mips.mips64c128/tmp/legacy/usr/libexec: METALOG /dev/null -DNO_ROOT distrib-dirs
                                      `-sh -e
                                          `-cat -l


$ ll /home/bharadwaj/datadrive/github/src/CHERI
total 32
drwxr-xr-x  8 bharadwaj bharadwaj 4096 May 14 22:52 ./
drwxrwxr-x  3 bharadwaj bharadwaj 4096 May 14 22:09 ../
drwxr-xr-x  5 bharadwaj bharadwaj 4096 May 14 22:54 build/
drwxr-xr-x 26 bharadwaj bharadwaj 4096 May 14 22:54 cheribsd/
drwxr-xr-x  9 bharadwaj bharadwaj 4096 May 14 22:09 cheribuild/
drwxr-xr-x 21 bharadwaj bharadwaj 4096 May 14 22:29 llvm-project/
drwxr-xr-x  4 bharadwaj bharadwaj 4096 May 14 23:12 output/
drwxr-xr-x 56 bharadwaj bharadwaj 4096 May 14 22:13 qemu/

QtWebkit platform plugin error on QtTestBrowser

I'm trying to run QtTestBrowser on Cheribsd however I receive the following error after building and running the binary

root@qemu-cheri128-sean:/mnt/build/qtwebkit-128-build/bin # ./QtTestBrowser
QtTestBrowser: _rtld_thread_init is not available when statically linked.
This application failed to start because it could not find or load the Qt platform plugin "bsdfb"
in "".

Reinstalling the application may fix this problem.
Abort (core dumped)

Qtwebkit was built with

python3 debug-cheribuild.py qtwebkit --qtwebkit/linkage=static --cross-compile-linkage=static --force-update --reconfigure -c --qtwebkit/debug-info

My repositories are in the following state
Cheribuild - 97bfd25
icu4c - 900f5dcf87beca158ea62987482cbd4e517bb07d
qt5/ - ac0e04fc867c636d3a16effa6c7a85d3677bd57f
qt5/qtbase - 6c2f62170ab9973f2a969b2facdc0ca50ea4a8f2
qt5/qtwebkit - e78ae7ab924d27e866e5ba6649ec093cd8f03ee2
libxml2 - c138001afa0bce70d3b9f84f12914e22bce18aa6

Support for RISC-V LLVM

I was trying to to build a freestanding CHERI RISC-V toolchain.
Some targets (e.g. FreeRTOS) suggest that this is possible, but I can't seem to find the correct options to built a crosscompiling LLVM which targets CHERI RISC-V.
Do I need to compile a LLVM which supports RISC-V manually to be able to build dependent targets (such as libcxx, compiler-rt and so on)?

Thanks!

'bsdtar has no installation candidate' on ubuntu 19.10

It might be handy to update the debian/ubuntu directions to include a different incantation for different versions. In 19.10, Eoan Ermine, there is no bsdtar; the installation aborts with an error : "has no installation candidate".

It appears that, after some poking, libarchive-tools replaces it, and was sufficient to continue installation.

Freestanding-sdk target no longer builds

The freestanding-sdk target no longer builds on FreeBSD. Fails at the beginning with the message:

./cheribuild.py --cheri-128 --source-root=./src freestanding-sdk                
Configuration file /home/hx242/.config/cheribuild.json does not exist, using only command line arguments.
Sources will be stored in /usr/home/hx242/cheribuild/src
Build artifacts will be stored in /usr/home/hx242/cheribuild/src/output
cc '-DLLVM_PATH="/usr/home/hx242/cheribuild/src/output/sdk128/bin/"' -x c - -o /usr/home/hx242/cheribuild/src/output/sdk128/bin/cheridis
Fatal error: CheriBSD build directory is missing! (Tried [PosixPath('/usr/home/hx242/cheribuild/src/build/cheribsd-obj-128/mips.mips64/usr/home/hx242/cheribuild/src/cheribsd'), PosixPath('/usr/home/hx242/cheribuild/src/build/cheribsd-obj-128/mips.mips64/usr/home/hx242/cheribuild/src/cheribsd')] )

The workaround is to build the full sdk, but can be painful and unnecessary as CheriOS work does not use CheriBSD bits at all

Q: Is there a flag to suppress warnings during initial installation?

During my first run of cheribuild.py to setup & configure the world for the first time, a large number of warnings cropped up, most about disk space usage, slowing down installation. It was rather annoying. Is there an argument like the classic '-f' to suppress?

If not, it would be very nice to have one to make on-boarding and set up smoother and add it to the getting started directions.

Add ability to specify git revisions

Given the nature of this project and things often being broken, it would be really useful to be able to specify git revisions for some of the git repos being used (llvm, clang, cheri-qemu, cheribsd).

Allow running the script on a non-FreeBSD system

It should run the other steps via SSH on another host (using e.g. --freebsd-server= command line option)

'libarchive-dev' listed twice in prebuild instructions

I noticed that 'libarchive-dev' is listed twice in the prerequisite installations (It's actually listed three time in the initial instructions I received out of band). I deleted the two duplicates when I did my install, and it appears to be fine. Is there a reason it is listed twice in the apt-get install invocation? Build is often black magic to me. :)

Allow building a CHERI sdk

Install host tools into /sdk and fix clang DEFAULT_SYSROOT
Add the necessary sysroot files from cheribsd to /sdk/sysroot and fix symlinks
Add cheridis utility to sdk
Remove unecessary files from sdk

Enable journaling for UFS images

rm -rf cheri/output/sdk/sysroot128.delete-me-pls fails

Missing a chflags -R noschg ?

Warning: Previous async cleanup of  /root/cheri/output/sdk/sysroot128 failed. Cleaning up now
Fatal error:Command `rm -rf /root/cheri/output/sdk/sysroot128.delete-me-pls` failed with non-zero exit code 1. Working directory was /cheri/cheribuild

Dev branch requested for changes | Master becomes unstable sometimes?

Dear Cheribuild Team,

Hope you are doing well. Thank you for your support.

Recently, we have been trying to compile Cheri using Cheribuild. Few times, it was successful and we were able to compile and ssh to the cheribsd. However, few times, because of the recent changes, we got compilation errors. Would it be possible for you to create a dev branch or any other branch for changes? and once it is stable, we can merge to master.

This would make master branch more stable and compilation error-free.

Thank you

Yours Sincerely,
Vijay

Should error out if explicitly specified --config-file does not exist

Right now, it just raises a warning and continues on, which is the right idea if it hasn't been explicitly set, but is unexpected if it has.

[Ubuntu 18.04] Segfault while running qemu-system-cheri128

Hello!

I'm see a segmentation fault upon executing the following command:

$ ./cheribuild.py run-mips-purecap -d --config-file /home/bharadwaj/datadrive/github/src/CHERI/.config/cheribuild.json --source-root /home/bharadwaj/datadrive/github/src/CHERI

...

Listening for SSH connections on localhost:10005
...
Fatal error: Command `/home/bharadwaj/datadrive/github/src/CHERI/output/sdk/bin/qemu-system-cheri128 -M malta -kernel /home/bharadwaj/datadrive/github/src/CHERI/output/rootfs-purecap128/boot/kernel/kernel -m 2048 -nographic -drive file=/home/bharadwaj/datadrive/github/src/CHERI/output/cheribsd-mips-purecap128.img,format=raw,index=0,media=disk -net nic -net 'user,id=net0,ipv6=off,smb=/home/bharadwaj/datadrive/github/src/CHERI<<<source_root@ro:/home/bharadwaj/datadrive/github/src/CHERI/build<<<build_root:/home/bharadwaj/datadrive/github/src/CHERI/output<<<output_root@ro:/home/bharadwaj/datadrive/github/src/CHERI/output/rootfs-purecap128<<<rootfs,hostfwd=tcp::10005-:22' -device virtio-rng-pci` failed with non-zero exit code -11. Working directory was /mnt/datadrive/github/src/CHERI/cheribuild

Tip of cheribuild repo used:

commit 2a799388e87562913801cba33f5199d251bc596c (HEAD -> master, origin/master, origin/HEAD)
Author: Alex Richardson <[email protected]>
Date:   Wed May 20 10:42:14 2020 +0100

    Try to handle failures during --test better

Appreciate any suggestions to address this.

Thanks!

Add target to show useful .ssh/config

Something like

Host *cheribsd*
       User root
       AddressFamily inet
       HostName localhost
       IdentityFile ~/.ssh/insecure_id_ed25519
       ControlPath ~/.ssh/controlmasters/%r@%h:%p
       ControlMaster auto
       PasswordAuthentication no
       StrictHostKeyChecking no
       UserKnownHostsFile /dev/null

Host cheribsd-mips-purecap
        Port 12374
Host cheribsd-mips-hybrid
        Port 12375

etc.

Seems like there is an Include directive so maybe this should be created as ~/.config/cheribuild_ssh_config and users can choose to include it.

--skip-buildworld doesn't

... or rather, not entirely. It does skip building the world, but it still goes through it for some reason, which takes time:

--- obj_crunchdir_mount_cd9660 --- cd /usr/home/en322/cheri/cheribsd/rescue/rescue/../../sbin/mount_cd9660 && MK_AUTO_OBJ=no MK_TESTS=no UPDATE_DEPENDFILE=no _RECURSING_CRUNCH=1 MAKEOBJDIRPREFIX=/home/en322/cheri/build/cheribsd-obj-256/mips.mips64/usr/home/en322/cheri/cheribsd/rescue/rescue make DIRPRFX=rescue/rescue/mount_cd9660/ -DRESCUE CRUNCH_CFLAGS=-DRESCUE MK_AUTO_OBJ=no obj --- obj_crunchdir_kldunload --- --- obj --- --- obj_crunchdir_mount_msdosfs --- cd /usr/home/en322/cheri/cheribsd/rescue/rescue/../../sbin/mount_msdosfs && MK_AUTO_OBJ=no MK_TESTS=no UPDATE_DEPENDFILE=no _RECURSING_CRUNCH=1 MAKEOBJDIRPREFIX=/home/en322/cheri/build/cheribsd-obj-256/mips.mips64/usr/home/en322/cheri/cheribsd/rescue/rescue make DIRPRFX=rescue/rescue/mount_msdosfs/ -DRESCUE CRUNCH_CFLAGS=-DRESCUE MK_AUTO_OBJ=no obj --- obj_crunchdir_ldconfig --- --- obj ---

It then tars things up, which takes time as well.

"Some tests failed!" even if there where no failures

I get "Some tests failed", and an exit code != 0 even if all that tests passed. The command is at the end of the log.

ran 'rm -f /tmp/results.db' successfully (in 0.150537s)
--> b'[PEXPECT\\PROMPT]>kyua test --results-file=/tmp/results.db -k /usr/tests/cddl/usr \x08.sbin/dtrace/common/arithmetic/Kyuafile'
--> b't_dtrace_contrib:err_D_DIV_ZERO_divby0_1_d  ->  passed  [2.533s]'
--> b't_dtrace_contrib:err_D_DIV_ZERO_divby0_2_d  ->  passed  [2.017s]'
--> b't_dtrace_contrib:err_D_DIV_ZERO_divby0_d  ->  passed  [2.028s]'
--> b't_dtrace_contrib:err_D_DIV_ZERO_modby0_d  ->  passed  [2.034s]'
--> b't_dtrace_contrib:err_D_SYNTAX_addmin_d  ->  passed  [2.032s]'
--> b't_dtrace_contrib:err_D_SYNTAX_divmin_d  ->  passed  [2.035s]'
--> b't_dtrace_contrib:err_D_SYNTAX_muladd_d  ->  passed  [2.027s]'
--> b't_dtrace_contrib:err_D_SYNTAX_muldiv_d  ->  passed  [2.046s]'
--> b't_dtrace_contrib:tst_basics_d  ->  passed  [2.183s]'
--> b't_dtrace_contrib:tst_compcast_d  ->  passed  [2.213s]'
--> b't_dtrace_contrib:tst_compnarrowassign_d  ->  passed  [2.147s]'
--> b't_dtrace_contrib:tst_execcast_d  ->  passed  [2.217s]'
--> b''
--> b'Results saved to /tmp/results.db'
--> b''
--> b'12/12 passed (0 failed)'

ran 'kyua test --results-file=/tmp/results.db -k /usr/tests/cddl/usr.sbin/dtrace/common/arithmetic/Kyuafile' successfully (in 33.644513s)
--> b"[PEXPECT\\PROMPT]>cp -v /tmp/results.db /test-results/test-results.db ;if test $? \x08 -eq 0; then echo '__COMMAND' 'SUCCESSFUL__'; else echo '__COMMAND' 'FAILED__';  \x08fi"
--> b'/tmp/results.db -> /test-results/test-results.db'
--> b'__COMMAND SUCCESSFUL__'

ran 'cp -v /tmp/results.db /test-results/test-results.db' successfully (in 0.474903s)
--> b"[PEXPECT\\PROMPT]>fsync /test-results/test-results.db ;if test $? -eq 0; then ech \x08o '__COMMAND' 'SUCCESSFUL__'; else echo '__COMMAND' 'FAILED__'; fi"
--> b'__COMMAND SUCCESSFUL__'

ran 'fsync /test-results/test-results.db' successfully (in 0.177571s)

Running tests for /usr/tests/cddl/usr.sbin/dtrace/common/arithmetic/Kyuafile took: 0:00:34.354746
KYUA installed on the host, no need to do slow conversion in QEMU
Converting kyua databases to JUNitXML in output directory /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204
Running kyua report-junit --results-file=/home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/test-results.db with {'stdout': <_io.TextIOWrapper name='/home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/test-results.xml' mode='w' encoding='UTF-8'>}
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/test-results.xml
Running grep '<testsuite' /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/test-results.xml
--> b'[PEXPECT\\PROMPT]><testsuite errors="0" failures="0" skipped="0" tests="12" time="25.511999999999997">'
Updating statistics in JUnit output directory /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheritest-dynamic-mt.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheritest-mt.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/test-results.xml
Running grep '<testsuite' /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/test-results.xml
--> b'<testsuite errors="0" failures="0" skipped="0" tests="12" time="25.511999999999997">'
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheriabitest-dynamic.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheriabitest-dynamic-mt.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheritest-dynamic.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheritest.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheriabitest-mt.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheriabitest.xml
--> b'poweroff'
--> b'Shutdown NOW!'
--> b'poweroff: [pid 1356]'
--> b'[PEXPECT\\PROMPT]>'
--> b'\x07*** FINAL System shutdown message from root@qemu-cheri-nm712 ***\x07'
--> b''
--> b'System going down IMMEDIATELY'
--> b''
--> b''
--> b''
--> b'System shutdown time has arrived\x07\x07'
--> b'May 26 02:09:15 qemu-cheri-nm712 shutdown[1356]: power-down by root:'
--> b'Stopping sshd.'
--> b'Stopping devd.'
--> b'Writing entropy file: .'
--> b'Writing early boot entropy file: .'
--> b'Terminated'
--> b'.'
--> b'<testsuite errors="0" failures="0" skipped="0" tests="12" time="25.511999999999997">'                                [25/7403]
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheriabitest-dynamic.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheriabitest-dynamic-mt.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheritest-dynamic.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheritest.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheriabitest-mt.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheriabitest.xml
--> b'poweroff'
--> b'Shutdown NOW!'
--> b'poweroff: [pid 1356]'
--> b'[PEXPECT\\PROMPT]>'
--> b'\x07*** FINAL System shutdown message from root@qemu-cheri-nm712 ***\x07'
--> b''
--> b'System going down IMMEDIATELY'
--> b''
--> b''
--> b''
--> b'System shutdown time has arrived\x07\x07'
--> b'May 26 02:09:15 qemu-cheri-nm712 shutdown[1356]: power-down by root:'
--> b'Stopping sshd.'
--> b'Stopping devd.'
--> b'Writing entropy file: .'
--> b'Writing early boot entropy file: .'
--> b'Terminated'
--> b'.'
--> b'May 26 02:09:27 qemu-cheri-nm712 syslogd: exiting on signal 15'
--> b"Waiting (max 60 seconds) for system process `vnlru' to stop... done"
--> b"Waiting (max 60 seconds) for system process `syncer' to stop..."
--> b'Syncing disks, vnodes remaining... 0 0 done'
--> b"Waiting (max 60 seconds) for system thread `bufdaemon' to stop... done"
--> b"Waiting (max 60 seconds) for system thread `bufspacedaemon-0' to stop... done"
--> b'All buffers synced.'
--> b'Uptime: 1m53s'
--> b'qemu-system-cheri128: info: CPU0 executed instructions: 10292767567 (4148864782 user, 6143902785 kernel) in 119.15s KIPS: 8638
4.70'
--> b''

Poweroff took: 0:00:20.850024

Tests failed after 0:01:03.638314 

===> DONE
Total execution time: 0:02:01.314988

ERROR: Some tests failed!
Running rm -fv /home/nm712/cheri/output/cheribsd-mips-nocheri.img.runtests.20200526032204.pid45913
--> b'/home/nm712/cheri/output/cheribsd-mips-nocheri.img.runtests.20200526032204.pid45913'
Fatal error: Command `/home/nm712/cheribuild/test-scripts/run_cheribsd_tests.py --ssh-key /home/nm712/.ssh/id_ed25519.pub --kernel /home/nm712/cheri/output/rootfs-mips/boot/kernel/kernel --qemu-cmd /home/nm712/cheri/output/sdk/bin/qemu-system-cheri128 --disk-image /home/nm712/cheri/output/cheribsd-mips-nocheri.img --kyua-tests-files /usr/tests/cddl/usr.sbin/dtrace/common/arithmetic/Kyuafile` failed with non-zero exit code 2. Working directory was /home/nm712/dtrace_tests

Add mtree entry for fstab and rc.conf

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

ctsrd-cheri / cheribuild Goto Github PK

cheribuild's Introduction

cheribuild.py - A script to build CHERI-related software (requires Python 3.8+)

Supported operating systems

Pre-Build Setup

macOS

Debian/Ubuntu

RHEL/Fedora

FreeBSD

Arch Linux

Contributing

Basic usage

Usage

Available Targets

Overview

The following main targets are available

Supported architectures

Other targets

Building the compiler and QEMU

Building and running CheriBSD

Disk image

CheriBSD SSH ports

Speeding up SSH connections

Building GDB

Cross-compiling for CheriBSD

Cross-compiling baremetal MIPS/CHERI

Adapting the build configuration

Prefixed cheribuild.py symlinks to select config file

Including config files

Getting shell completion

BASH

TCSH:

List of options (--help output)

cheribuild's People

Contributors

Stargazers

Watchers

Forkers

cheribuild's Issues

Recommend Projects

Recommend Topics

Recommend Org

`cheribuild.py` - A script to build CHERI-related software (requires Python 3.8+)