ctsrd-cheri / cheribuild Goto Github PK
View Code? Open in Web Editor NEWEasily build and run CHERI related projects
Home Page: http://www.chericpu.com
License: Other
cheribuild's Issues
'libarchive-dev' listed twice in prebuild instructions
I noticed that 'libarchive-dev' is listed twice in the prerequisite installations (It's actually listed three time in the initial instructions I received out of band). I deleted the two duplicates when I did my install, and it appears to be fine. Is there a reason it is listed twice in the apt-get install invocation? Build is often black magic to me. :)
Support for -march, -mabi and -mcmodel RISC-V command line flags
Providing those in the command line is gonna be very useful especially when we're doing multilib
Should consistently use python style names
The current mixture of camel case and underscores is extremely confusing.
Git repos should be cloned if they don't exist
Q: Is there a flag to suppress warnings during initial installation?
During my first run of cheribuild.py to setup & configure the world for the first time, a large number of warnings cropped up, most about disk space usage, slowing down installation. It was rather annoying. Is there an argument like the classic '-f' to suppress?
If not, it would be very nice to have one to make on-boarding and set up smoother and add it to the getting started directions.
disk image code refactor breaks cheribsd builds?
With aba62a2 in tree, attempting to make disk images fails with
Traceback (most recent call last):
File "/home/nwf20/cheri/mainline/cheribuild/cheribuild.py", line 42, in <module>
main()
File "/local/scratch/nwf20/cheri/mainline/cheribuild/pycheribuild/__main__.py", line 227, in main
real_main()
File "/local/scratch/nwf20/cheri/mainline/cheribuild/pycheribuild/__main__.py", line 219, in real_main
targetManager.run(cheriConfig)
File "/local/scratch/nwf20/cheri/mainline/cheribuild/pycheribuild/targets.py", line 320, in run
target.execute(config)
File "/local/scratch/nwf20/cheri/mainline/cheribuild/pycheribuild/targets.py", line 92, in execute
project.process()
File "/local/scratch/nwf20/cheri/mainline/cheribuild/pycheribuild/projects/disk_image.py", line 473, in process
self.__process()
File "/local/scratch/nwf20/cheri/mainline/cheribuild/pycheribuild/projects/disk_image.py", line 535, in __process
pathInImage = p.relative_to(self.extraFilesDir)
File "/usr/lib/python3.6/pathlib.py", line 874, in relative_to
.format(str(self), str(formatted)))
ValueError: '/home/nwf20/cheri/mainline/host-output/kyua-cheribsd-pkgcache/usr/lib/libarchive.so.6' does not start with '/home/nwf20/cheribsd-extra-files'
I'm invoking
cheribuild.py --disk-image/extra-files=${HOME}/cheribsd-extra-files --disk-image-path=... --disk-image/auto-add-files
--include-dependencies, -d not recognised by jenkins-cheri-build.py?
This works fine with cheribuild.py but gives the following error with jenkins-cheri-build.py:
jenkins-cheri-build.py: error: unrecognized arguments: -d
Add ability to specify git revisions
Given the nature of this project and things often being broken, it would be really useful to be able to specify git revisions for some of the git repos being used (llvm, clang, cheri-qemu, cheribsd).
postgres build fails
PostgreSQL build fails like this:
gmake[2]: /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-ar: Command not found
/home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-ar cr libpgcommon.a config_info.o controldata_utils.o exec.o keywords.o pg_lzcompress.o pgfnames.o psprintf.o relpath.o rmtree.o string.o username.o wait_error.o fe_memutils.o restricted_token.o
gmake[2]: *** [Makefile:61: libpgcommon.a] Error 127
gmake[2]: *** Waiting for unfinished jobs....
gmake[2]: Leaving directory '/usr/home/en322/cheri/postgres/src/common'
gmake[1]: *** [Makefile:36: all-common-recurse] Error 2
gmake[1]: Leaving directory '/usr/home/en322/cheri/postgres/src'
gmake: *** [GNUmakefile:11: all-src-recurse] Error 2
Command "nice gmake -j8" failed with exit code 2.
See /home/en322/cheri/postgres/gmake.log for details.
[en322@thyme:~/git/cheribuild]%
[en322@thyme:~/git/cheribuild]%
[en322@thyme:~/git/cheribuild]% /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-ar
zsh: no such file or directory: /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-ar
[en322@thyme:~/git/cheribuild]% ls -al /home/en322/cheri/output/sdk256/bin/*
-rwxr-xr-x 1 en322 wheel 16736312 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/bugpoint
-rwxr-xr-x 1 en322 wheel 19978352 Aug 4 13:31 /home/en322/cheri/output/sdk256/bin/c-index-test
-rwxr-xr-x 1 en322 wheel 759936 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/capsizefix
lrwx------ 1 en322 wheel 7 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-c++ -> clang++
lrwx------ 1 en322 wheel 5 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-cc -> clang
lrwx------ 1 en322 wheel 5 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-clang -> clang
lrwx------ 1 en322 wheel 9 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-clang-cpp -> clang-cpp
lrwx------ 1 en322 wheel 7 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-clang++ -> clang++
lrwx------ 1 en322 wheel 9 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-cpp -> clang-cpp
lrwx------ 1 en322 wheel 6 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-ld -> ld.lld
lrwx------ 1 en322 wheel 6 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-ld.lld -> ld.lld
lrwx------ 1 en322 wheel 3 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-llc -> llc
lrwx------ 1 en322 wheel 7 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-llvm-mc -> llvm-mc
lrwx------ 1 en322 wheel 12 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-llvm-objdump -> llvm-objdump
lrwx------ 1 en322 wheel 12 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-llvm-readobj -> llvm-readobj
lrwx------ 1 en322 wheel 9 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/cheri-unknown-freebsd-llvm-size -> llvm-size
lrwx------ 1 en322 wheel 9 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/clang -> clang-5.0
-rwxr-xr-x 1 en322 wheel 74538944 Aug 4 13:31 /home/en322/cheri/output/sdk256/bin/clang-5.0
-rwxr-xr-x 1 en322 wheel 55321352 Aug 4 13:31 /home/en322/cheri/output/sdk256/bin/clang-check
lrwx------ 1 en322 wheel 5 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/clang-cl -> clang
lrwx------ 1 en322 wheel 5 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/clang-cpp -> clang
-rwxr-xr-x 1 en322 wheel 1994848 Aug 4 13:31 /home/en322/cheri/output/sdk256/bin/clang-format
-rwxr-xr-x 1 en322 wheel 20993592 Aug 4 13:31 /home/en322/cheri/output/sdk256/bin/clang-import-test
-rwxr-xr-x 1 en322 wheel 2809848 Aug 4 13:31 /home/en322/cheri/output/sdk256/bin/clang-offload-bundler
lrwx------ 1 en322 wheel 5 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/clang++ -> clang
-rwx------ 1 en322 wheel 9894 Aug 4 13:55 /home/en322/cheri/output/sdk256/bin/fixlinks
-rwxr-xr-x 1 en322 wheel 21718 Jun 19 13:50 /home/en322/cheri/output/sdk256/bin/git-clang-format
lrwx------ 1 en322 wheel 6 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/ld -> ld.lld
lrwx------ 1 en322 wheel 3 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/ld.lld -> lld
-rwxr-xr-x 1 en322 wheel 44329416 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llc
-rwxr-xr-x 1 en322 wheel 47792304 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/lld
lrwx------ 1 en322 wheel 3 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/lld-link -> lld
-rwxr-xr-x 1 en322 wheel 21855008 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/lli
-rwxr-xr-x 1 en322 wheel 14001824 Aug 4 13:21 /home/en322/cheri/output/sdk256/bin/llvm-ar
-rwxr-xr-x 1 en322 wheel 2328896 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-as
-rwxr-xr-x 1 en322 wheel 412840 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-bcanalyzer
-rwxr-xr-x 1 en322 wheel 34398440 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-c-test
-rwxr-xr-x 1 en322 wheel 2295328 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-cat
-rwxr-xr-x 1 en322 wheel 346560 Aug 4 13:21 /home/en322/cheri/output/sdk256/bin/llvm-config
-rwxr-xr-x 1 en322 wheel 2920168 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-cov
-rwxr-xr-x 1 en322 wheel 2538568 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-cvtres
-rwxr-xr-x 1 en322 wheel 2579088 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-cxxdump
-rwxr-xr-x 1 en322 wheel 363664 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-cxxfilt
-rwxr-xr-x 1 en322 wheel 2105400 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-diff
-rwxr-xr-x 1 en322 wheel 1778256 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-dis
-rwxr-xr-x 1 en322 wheel 33498904 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-dsymutil
-rwxr-xr-x 1 en322 wheel 2774088 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-dwarfdump
-rwxr-xr-x 1 en322 wheel 33101552 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-dwp
-rwxr-xr-x 1 en322 wheel 3207584 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-extract
lrwx------ 1 en322 wheel 7 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/llvm-lib -> llvm-ar
-rwxr-xr-x 1 en322 wheel 2787024 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-link
-rwxr-xr-x 1 en322 wheel 40485216 Aug 4 13:22 /home/en322/cheri/output/sdk256/bin/llvm-lto
-rwxr-xr-x 1 en322 wheel 41569112 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-lto2
-rwxr-xr-x 1 en322 wheel 15064936 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-mc
-rwxr-xr-x 1 en322 wheel 374336 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-mcmarkup
-rwxr-xr-x 1 en322 wheel 1968640 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-modextract
-rwxr-xr-x 1 en322 wheel 14026824 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-nm
-rwxr-xr-x 1 en322 wheel 14323096 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-objdump
-rwxr-xr-x 1 en322 wheel 441552 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-opt-report
-rwxr-xr-x 1 en322 wheel 2096680 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-pdbdump
-rwxr-xr-x 1 en322 wheel 1284536 Aug 4 13:21 /home/en322/cheri/output/sdk256/bin/llvm-profdata
lrwx------ 1 en322 wheel 7 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/llvm-ranlib -> llvm-ar
-rwxr-xr-x 1 en322 wheel 3740264 Aug 4 13:33 /home/en322/cheri/output/sdk256/bin/llvm-readobj
-rwxr-xr-x 1 en322 wheel 12623976 Aug 4 13:33 /home/en322/cheri/output/sdk256/bin/llvm-rtdyld
-rwxr-xr-x 1 en322 wheel 2579072 Aug 4 13:33 /home/en322/cheri/output/sdk256/bin/llvm-size
-rwxr-xr-x 1 en322 wheel 2682544 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-split
-rwxr-xr-x 1 en322 wheel 3278912 Aug 4 13:33 /home/en322/cheri/output/sdk256/bin/llvm-stress
-rwxr-xr-x 1 en322 wheel 368928 Aug 4 13:32 /home/en322/cheri/output/sdk256/bin/llvm-strings
-rwxr-xr-x 1 en322 wheel 2835136 Aug 4 13:33 /home/en322/cheri/output/sdk256/bin/llvm-symbolizer
-rwxr-xr-x 1 en322 wheel 2287440 Aug 4 13:07 /home/en322/cheri/output/sdk256/bin/llvm-tblgen
-rwxr-xr-x 1 en322 wheel 3214880 Aug 4 13:33 /home/en322/cheri/output/sdk256/bin/llvm-xray
lrwx------ 1 en322 wheel 7 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-c++ -> clang++
lrwx------ 1 en322 wheel 5 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-cc -> clang
lrwx------ 1 en322 wheel 5 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-clang -> clang
lrwx------ 1 en322 wheel 9 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-clang-cpp -> clang-cpp
lrwx------ 1 en322 wheel 7 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-clang++ -> clang++
lrwx------ 1 en322 wheel 9 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-cpp -> clang-cpp
lrwx------ 1 en322 wheel 6 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-ld -> ld.lld
lrwx------ 1 en322 wheel 6 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-ld.lld -> ld.lld
lrwx------ 1 en322 wheel 3 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-llc -> llc
lrwx------ 1 en322 wheel 7 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-llvm-mc -> llvm-mc
lrwx------ 1 en322 wheel 12 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-llvm-objdump -> llvm-objdump
lrwx------ 1 en322 wheel 12 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-llvm-readobj -> llvm-readobj
lrwx------ 1 en322 wheel 9 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips4-unknown-freebsd-llvm-size -> llvm-size
lrwx------ 1 en322 wheel 7 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-c++ -> clang++
lrwx------ 1 en322 wheel 5 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-cc -> clang
lrwx------ 1 en322 wheel 5 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-clang -> clang
lrwx------ 1 en322 wheel 9 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-clang-cpp -> clang-cpp
lrwx------ 1 en322 wheel 7 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-clang++ -> clang++
lrwx------ 1 en322 wheel 9 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-cpp -> clang-cpp
lrwx------ 1 en322 wheel 6 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-ld -> ld.lld
lrwx------ 1 en322 wheel 6 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-ld.lld -> ld.lld
lrwx------ 1 en322 wheel 3 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-llc -> llc
lrwx------ 1 en322 wheel 7 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-llvm-mc -> llvm-mc
lrwx------ 1 en322 wheel 12 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-llvm-objdump -> llvm-objdump
lrwx------ 1 en322 wheel 12 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-llvm-readobj -> llvm-readobj
lrwx------ 1 en322 wheel 9 Aug 4 13:37 /home/en322/cheri/output/sdk256/bin/mips64-unknown-freebsd-llvm-size -> llvm-size
-rwxr-xr-x 1 en322 wheel 3200024 Aug 4 13:33 /home/en322/cheri/output/sdk256/bin/obj2yaml
-rwxr-xr-x 1 en322 wheel 46395144 Aug 4 13:33 /home/en322/cheri/output/sdk256/bin/opt
-rwxr-xr-x 1 en322 wheel 352984 Aug 4 14:22 /home/en322/cheri/output/sdk256/bin/qemu-ga
-rwxr-xr-x 1 en322 wheel 874168 Aug 4 14:22 /home/en322/cheri/output/sdk256/bin/qemu-img
-rwxr-xr-x 1 en322 wheel 910240 Aug 4 14:22 /home/en322/cheri/output/sdk256/bin/qemu-io
-rwxr-xr-x 1 en322 wheel 845368 Aug 4 14:22 /home/en322/cheri/output/sdk256/bin/qemu-nbd
-rwxr-xr-x 1 en322 wheel 5727568 Aug 4 14:22 /home/en322/cheri/output/sdk256/bin/qemu-system-cheri
-rwxr-xr-x 1 en322 wheel 13887928 Aug 4 13:33 /home/en322/cheri/output/sdk256/bin/sancov
-rwxr-xr-x 1 en322 wheel 2812248 Aug 4 13:33 /home/en322/cheri/output/sdk256/bin/sanstats
-rwxr-xr-x 1 en322 wheel 53444 Mar 3 10:50 /home/en322/cheri/output/sdk256/bin/scan-build
-rwxr-xr-x 1 en322 wheel 4504 Mar 3 10:50 /home/en322/cheri/output/sdk256/bin/scan-view
-rwxr-xr-x 1 en322 wheel 2580208 Aug 4 13:33 /home/en322/cheri/output/sdk256/bin/verify-uselistorder
-rwxr-xr-x 1 en322 wheel 925776 Aug 4 13:33 /home/en322/cheri/output/sdk256/bin/yaml2obj
Allow running the script on a non-FreeBSD system
It should run the other steps via SSH on another host (using e.g. --freebsd-server= command line option)
Testing for compiler lld support writes a file named -
The test in pycheribuild/projects/project.py's canUseLLd causes (some versions of?) clang (at least) to generate a file named - in the current working directory, which, while not fatal, is unsightly. It's not clear to me if it's worth using a temporary path, which is unlink()ed on succss, or if we can get away with "/dev/null" rather than "-", since some compiler might conceivably try invoking unlink() before open()ing an existing output. Eliminating "-o" entirely results in, as traditional but unfortunate, a.out.
CMake cross-compiler projects should use CMAKE_CROSSCOMPILING_EMULATOR
This way it might be possible to run ctest if CMAKE_CROSSCOMPILING_EMULATOR is set to a script that copies to QEMU and runs it there
Cheribuild asks about update despite "skip-update": "true"
With the following in ~/.config/cheribuild.json, ./cheribuild sdk still asks for update of cheribuild itself (updating other things is disabled without questions, as expected):
% ./cheribuild.py sdk run Sources will be stored in /home/en322/cheri Build artifacts will be stored in /home/en322/cheri/output
Current CheriBuild checkout can be updated: Your branch is behind 'origin/master' by 16 commits, and can be fast-forwarded. Would you like to update before continuing? y/[n] (Enter to skip)
Allow building 128 bit SDK
qtwebkit fails to build because cheribuild thinks the CMake version is 0.0.0
Looks like it is due to _get_cmake_version:
1043 def _get_cmake_version(self):
1044 if not Path(self.configureCommand).exists():
1045 return (0, 0, 0)
1046 return get_program_version(self.configureCommand, program_name=b"cmake")self.configureCommand is set to "cmake" and so the if test succeeds, as it's not an absolute path. I can work around this for now by setting the CMAKE_COMMAND environment variable to the full path to cmake.
Randomise port used for SSH forwarding
If multiple people use this script on the same host then a hard coded port for SSH will not work. Consider using a user's uid or something that would randomise it.
Unable to build qtwebkit
I'm having issues building qtwebkit as of commit c72abd6 receiving the following errors during Command "nice ninja jsc -j8" failed with exit code 1.
/home/sean/cheri/qt5/qtwebkit/Source/WTF/wtf/RedBlackTree.h:106:48: error: use of undeclared identifier 'qClearLowPointerBits'
return reinterpret_cast<NodeType*>(qClearLowPointerBits<1u>(m_parentAndRed));
^
/home/sean/cheri/qt5/qtwebkit/Source/WTF/wtf/RedBlackTree.h:111:71: error: use of undeclared identifier 'qGetLowPointerBits'
m_parentAndRed = reinterpret_cast<uintptr_t>(newParent) | qGetLowPointerBits<1u>(m_parentAndRed);
^
/home/sean/cheri/qt5/qtwebkit/Source/WTF/wtf/RedBlackTree.h:136:17: error: use of undeclared identifier 'qGetLowPointerBits'
if (qGetLowPointerBits<1u>(m_parentAndRed))
^
/home/sean/cheri/qt5/qtwebkit/Source/WTF/wtf/RedBlackTree.h:146:34: error: use of undeclared identifier 'qClearLowPointerBits'
m_parentAndRed = qClearLowPointerBits<1u>(m_parentAndRed);
^
Git blame indicates that the following line and commit introduced qClearLowPointerBits: CTSRD-CHERI/qtwebkit@d6cd620
However I'm unable to find qClearLowPointerBits in any commit of CTSRD-CHERI/qtbase.
Could someone please advise on how best to proceed with compiling qtwebkit?
My repos are in the following states:
cheri/icu4c : commit 900f5dcf87beca158ea62987482cbd4e517bb07d
cheri/libxml2: commit c138001afa0bce70d3b9f84f12914e22bce18aa6
cheri/qt5/qtwebkit: commit e78ae7ab924d27e866e5ba6649ec093cd8f03ee2
cheri/qt5/qtbase: commit 08a110622c228046e1d1c2f48504c4ea544cb18b with the following diff
diff --git a/src/3rdparty/harfbuzz-ng/src/hb-buffer.cc b/src/3rdparty/harfbuzz-ng/src/hb-buffer.cc
index 3940a3dbf8..28e164cf74 100644
--- a/src/3rdparty/harfbuzz-ng/src/hb-buffer.cc
+++ b/src/3rdparty/harfbuzz-ng/src/hb-buffer.cc
@@ -83,9 +83,13 @@ hb_segment_properties_equal (const hb_segment_properties_t *a,
unsigned int
hb_segment_properties_hash (const hb_segment_properties_t *p)
{
+ typedef size_t cheri_fake_ptr;
+ // static_assert(sizeof(cheri_fake_ptr) == sizeof(intptr_t));
return (unsigned int) p->direction ^
(unsigned int) p->script ^
- (intptr_t) (p->language);
+ // i'm not sure if this is a valid fix;
+ // based off https://github.com/svaarala/duktape/issues/530;w
+ (cheri_fake_ptr) (p->language);
}
diff --git a/src/3rdparty/libpng/pngpriv.h b/src/3rdparty/libpng/pngpriv.h [0/48184]
index 922fe80c29..b04764b350 100644
--- a/src/3rdparty/libpng/pngpriv.h
+++ b/src/3rdparty/libpng/pngpriv.h
@@ -20,6 +20,9 @@
* they should be well aware of the issues that may arise from doing so.
*/
+// Added to silence compiler warnings about uintptr_t
+#include <stdint.h>
+
#ifndef PNGPRIV_H
#define PNGPRIV_H
@@ -501,7 +504,8 @@
typedef unsigned __int64 png_ptruint;
# endif
# else
- typedef unsigned long png_ptruint;
+ typedef uintptr_t png_ptruint;
+ // typedef unsigned long png_ptruint;
# endif
# define png_constcast(type, value) ((type)(png_ptruint)(const void*)(value))
# define png_aligncast(type, value) ((void*)(value))
diff --git a/src/3rdparty/sqlite/sqlite3.c b/src/3rdparty/sqlite/sqlite3.c
index ea5ba16b6f..69ac619a61 100644
--- a/src/3rdparty/sqlite/sqlite3.c
+++ b/src/3rdparty/sqlite/sqlite3.c
@@ -11987,7 +11987,7 @@ struct fts5_api {
** compiler.
*/
#if defined(__PTRDIFF_TYPE__) /* This case should work for GCC */
-# define SQLITE_INT_TO_PTR(X) ((void*)(__PTRDIFF_TYPE__)(X))
+# define SQLITE_INT_TO_PTR(X) ((void*)((intptr_t)(__PTRDIFF_TYPE__)(X)))
# define SQLITE_PTR_TO_INT(X) ((int)(__PTRDIFF_TYPE__)(X))
#elif !defined(__GNUC__) /* Works for compilers other than LLVM */
# define SQLITE_INT_TO_PTR(X) ((void*)&((char*)0)[X])
diff --git a/src/testlib/qtestcase.cpp b/src/testlib/qtestcase.cpp
index 15fd5b1504..59d83a1a0e 100644
--- a/src/testlib/qtestcase.cpp
+++ b/src/testlib/qtestcase.cpp
@@ -2480,7 +2480,7 @@ template <> Q_TESTLIB_EXPORT char *QTest::toString<__uintcap_t>(const __uintcap_
char *msg = new char[128];
// XXXAR: not sure how the compiler will react to passing a reference to a variadic function
__uintcap_t tmp = t;
- snprintf(msg, 128, "%lu (%#p)", (qvaddr)tmp, (void *__capability)tmp);
+ snprintf(msg, 128, "%llu (%#p)", (qvaddr)tmp, (void *__capability)tmp);
return msg;
}
@@ -2489,7 +2489,7 @@ template <> Q_TESTLIB_EXPORT char *QTest::toString<__intcap_t>(const __intcap_t
char *msg = new char[128];
// XXXAR: not sure how the compiler will react to passing a reference to a variadic function
__intcap_t tmp = t;
- snprintf(msg, 128, "%ld (%#p)", (qvaddr)tmp, (void *__capability)tmp);
+ snprintf(msg, 128, "%llu (%#p)", (qvaddr)tmp, (void *__capability)tmp);
return msg;
}
#endif
cheribuild.py hangs on Ubuntu 18.04
Hello!
Can I please get some guidance on getting past the hang (no progress for the past 11 hrs) that I see while running cheribuild.py on Ubuntu.py? Thanks for any help.
Following is the output of pstree - which I hope will provide the context and commands used.
$ pstree -ahlAt 29358
python3 ./cheribuild.py run-purecap -d --config-file /home/bharadwaj/datadrive/github/src/CHERI/.config/cheribuild.json --source-root /home/bharadwaj/datadrive/github/src/CHERI
`-bmake -DDB_FROM_SRC -DNO_CLEAN -DI_REALLY_MEAN_NO_CLEAN -DNO_ROOT -DBUILD_WITH_STRICT_TMPPATH TARGET mips TARGET_ARCH mips64c128 CHERI 128 TARGET_CPUTYPE cheri -DWITH_CHERI DEBUG_FLAGS -g LOCAL_XTOOL_DIRS lib/libnetbsd usr.sbin/makefs usr.bin/mkimg CHERI_USE_CAP_TABLE pcrel XAR /home/bharadwaj/datadrive/github/src/CHERI/output/sdk/bin/llvm-ar XNM /home/bharadwaj/datadrive/github/src/CHERI/output/sdk/bin/llvm-nm XSIZE /home/bharadwaj/datadrive/github/src/CHERI/output/sdk/bin/llvm-size XSTRIPBIN /home/bharadwaj/datadrive/github/src/CHERI/output/sdk/bin/llvm-strip XSTRINGS /home/bharadwaj/datadrive/github/src/CHERI/output/sdk/bin/llvm-strings XOBJCOPY /home/bharadwaj/datadrive/github/src/CHERI/output/sdk/bin/llvm-objcopy XRANLIB /home/bharadwaj/datadrive/github/src/CHERI/output/sdk/bin/llvm-ranlib RANLIBFLAGS -DNO_SHARE METALOG /dev/null -DWITH_TESTS -DWITH_BINUTILS -DWITHOUT_CLANG -DWITHOUT_GCC -DWITHOUT_GDB -DWITHOUT_LLD -DWITHOUT_LLDB -DWITHOUT_MAN -DWITHOUT_MAIL -DWITHOUT_SENDMAIL -DWITHOUT_VT -DWITHOUT_DEBUG_FILES -DWITHOUT_BOOT -DWITH_AUTO_OBJ -DWITHOUT_GCC_BOOTSTRAP -DWITHOUT_CLANG_BOOTSTRAP -DWITHOUT_LLD_BOOTSTRAP -DWITHOUT_LIB32 -DWITHOUT_RESCUE -DWITH_ELFTOOLCHAIN_BOOTSTRAP -DWITH_TOOLCHAIN -DWITHOUT_BINUTILS_BOOTSTRAP -s -de installsysroot -j12 -DWITH_AUTO_OBJ -DNO_CLEAN -DBUILD_WITH_OPIPEFAIL
`-sh
`-bmake -m /mnt/datadrive/github/src/CHERI/cheribsd/share/mk -f Makefile.inc1 -DDB_FROM_SRC -DMALLOC_PRODUCTION LOCAL_DIRS ctsrd tools/tools/atsectl LOCAL_LIB_DIRS ctsrd/lib LOCAL_MTREE ctsrd/ctsrd.mtree -DWITH_CHERI128 TARGET mips TARGET_ARCH mips64c128 installsysroot
`-sh -e
`-bmake -f Makefile.inc1 INSTALL install -N /mnt/datadrive/github/src/CHERI/cheribsd/etc -U -M /dev/null -D /home/bharadwaj/datadrive/github/src/CHERI/output/sdk/sysroot128 MTREE_CMD mtree -N /mnt/datadrive/github/src/CHERI/cheribsd/etc -W -DSYSROOT_ONLY MK_TESTS no __MAKE_SHELL /tmp/install.XXyp6ryB/sh -DNO_ROOT METALOG /dev/null reinstallsysroot
`-sh -e
`-bmake -f Makefile.inc1 LOCAL_MTREE ctsrd/ctsrd.mtree hierarchy
`-sh -e
`-bmake LOCAL_MTREE ctsrd/ctsrd.mtree PATH /mnt/datadrive/github/src/CHERI/build/cheribsd-purecap-128-build/mnt/datadrive/github/src/CHERI/cheribsd/mips.mips64c128/tmp/usr/sbin:/mnt/datadrive/github/src/CHERI/build/cheribsd-purecap-128-build/mnt/datadrive/github/src/CHERI/cheribsd/mips.mips64c128/tmp/usr/bin:/mnt/datadrive/github/src/CHERI/build/cheribsd-purecap-128-build/mnt/datadrive/github/src/CHERI/cheribsd/mips.mips64c128/tmp/legacy/usr/sbin:/mnt/datadrive/github/src/CHERI/build/cheribsd-purecap-128-build/mnt/datadrive/github/src/CHERI/cheribsd/mips.mips64c128/tmp/legacy/usr/bin:/mnt/datadrive/github/src/CHERI/build/cheribsd-purecap-128-build/mnt/datadrive/github/src/CHERI/cheribsd/mips.mips64c128/tmp/legacy/bin:/mnt/datadrive/github/src/CHERI/build/cheribsd-purecap-128-build/mnt/datadrive/github/src/CHERI/cheribsd/mips.mips64c128/tmp/legacy/usr/libexec: METALOG /dev/null -DNO_ROOT distrib-dirs
`-sh -e
`-cat -l
$ ll /home/bharadwaj/datadrive/github/src/CHERI
total 32
drwxr-xr-x 8 bharadwaj bharadwaj 4096 May 14 22:52 ./
drwxrwxr-x 3 bharadwaj bharadwaj 4096 May 14 22:09 ../
drwxr-xr-x 5 bharadwaj bharadwaj 4096 May 14 22:54 build/
drwxr-xr-x 26 bharadwaj bharadwaj 4096 May 14 22:54 cheribsd/
drwxr-xr-x 9 bharadwaj bharadwaj 4096 May 14 22:09 cheribuild/
drwxr-xr-x 21 bharadwaj bharadwaj 4096 May 14 22:29 llvm-project/
drwxr-xr-x 4 bharadwaj bharadwaj 4096 May 14 23:12 output/
drwxr-xr-x 56 bharadwaj bharadwaj 4096 May 14 22:13 qemu/
Dev branch requested for changes | Master becomes unstable sometimes?
Dear Cheribuild Team,
Hope you are doing well. Thank you for your support.
Recently, we have been trying to compile Cheri using Cheribuild. Few times, it was successful and we were able to compile and ssh to the cheribsd. However, few times, because of the recent changes, we got compilation errors. Would it be possible for you to create a dev branch or any other branch for changes? and once it is stable, we can merge to master.
This would make master branch more stable and compilation error-free.
Thank you
Yours Sincerely,
Vijay
Add mtree entry for fstab and rc.conf
copying of SSH public key fails because .ssh folder doesn't exist
Copying of the current user's SSH public key to the image's root/.ssh/authorized_files fails because the root/.ssh directories don't exist.
Here is a quick hack: https://gist.github.com/kgudka/a3da5e1b8a8c37d83ed9f7ac563062c8
I'm not sure if there is some other underlying problem though, as it never used to do this.
"Some tests failed!" even if there where no failures
I get "Some tests failed", and an exit code != 0 even if all that tests passed. The command is at the end of the log.
ran 'rm -f /tmp/results.db' successfully (in 0.150537s)
--> b'[PEXPECT\\PROMPT]>kyua test --results-file=/tmp/results.db -k /usr/tests/cddl/usr \x08.sbin/dtrace/common/arithmetic/Kyuafile'
--> b't_dtrace_contrib:err_D_DIV_ZERO_divby0_1_d -> passed [2.533s]'
--> b't_dtrace_contrib:err_D_DIV_ZERO_divby0_2_d -> passed [2.017s]'
--> b't_dtrace_contrib:err_D_DIV_ZERO_divby0_d -> passed [2.028s]'
--> b't_dtrace_contrib:err_D_DIV_ZERO_modby0_d -> passed [2.034s]'
--> b't_dtrace_contrib:err_D_SYNTAX_addmin_d -> passed [2.032s]'
--> b't_dtrace_contrib:err_D_SYNTAX_divmin_d -> passed [2.035s]'
--> b't_dtrace_contrib:err_D_SYNTAX_muladd_d -> passed [2.027s]'
--> b't_dtrace_contrib:err_D_SYNTAX_muldiv_d -> passed [2.046s]'
--> b't_dtrace_contrib:tst_basics_d -> passed [2.183s]'
--> b't_dtrace_contrib:tst_compcast_d -> passed [2.213s]'
--> b't_dtrace_contrib:tst_compnarrowassign_d -> passed [2.147s]'
--> b't_dtrace_contrib:tst_execcast_d -> passed [2.217s]'
--> b''
--> b'Results saved to /tmp/results.db'
--> b''
--> b'12/12 passed (0 failed)'
ran 'kyua test --results-file=/tmp/results.db -k /usr/tests/cddl/usr.sbin/dtrace/common/arithmetic/Kyuafile' successfully (in 33.644513s)
--> b"[PEXPECT\\PROMPT]>cp -v /tmp/results.db /test-results/test-results.db ;if test $? \x08 -eq 0; then echo '__COMMAND' 'SUCCESSFUL__'; else echo '__COMMAND' 'FAILED__'; \x08fi"
--> b'/tmp/results.db -> /test-results/test-results.db'
--> b'__COMMAND SUCCESSFUL__'
ran 'cp -v /tmp/results.db /test-results/test-results.db' successfully (in 0.474903s)
--> b"[PEXPECT\\PROMPT]>fsync /test-results/test-results.db ;if test $? -eq 0; then ech \x08o '__COMMAND' 'SUCCESSFUL__'; else echo '__COMMAND' 'FAILED__'; fi"
--> b'__COMMAND SUCCESSFUL__'
ran 'fsync /test-results/test-results.db' successfully (in 0.177571s)
Running tests for /usr/tests/cddl/usr.sbin/dtrace/common/arithmetic/Kyuafile took: 0:00:34.354746
KYUA installed on the host, no need to do slow conversion in QEMU
Converting kyua databases to JUNitXML in output directory /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204
Running kyua report-junit --results-file=/home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/test-results.db with {'stdout': <_io.TextIOWrapper name='/home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/test-results.xml' mode='w' encoding='UTF-8'>}
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/test-results.xml
Running grep '<testsuite' /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/test-results.xml
--> b'[PEXPECT\\PROMPT]><testsuite errors="0" failures="0" skipped="0" tests="12" time="25.511999999999997">'
Updating statistics in JUnit output directory /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheritest-dynamic-mt.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheritest-mt.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/test-results.xml
Running grep '<testsuite' /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/test-results.xml
--> b'<testsuite errors="0" failures="0" skipped="0" tests="12" time="25.511999999999997">'
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheriabitest-dynamic.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheriabitest-dynamic-mt.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheritest-dynamic.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheritest.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheriabitest-mt.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheriabitest.xml
--> b'poweroff'
--> b'Shutdown NOW!'
--> b'poweroff: [pid 1356]'
--> b'[PEXPECT\\PROMPT]>'
--> b'\x07*** FINAL System shutdown message from root@qemu-cheri-nm712 ***\x07'
--> b''
--> b'System going down IMMEDIATELY'
--> b''
--> b''
--> b''
--> b'System shutdown time has arrived\x07\x07'
--> b'May 26 02:09:15 qemu-cheri-nm712 shutdown[1356]: power-down by root:'
--> b'Stopping sshd.'
--> b'Stopping devd.'
--> b'Writing entropy file: .'
--> b'Writing early boot entropy file: .'
--> b'Terminated'
--> b'.'
--> b'<testsuite errors="0" failures="0" skipped="0" tests="12" time="25.511999999999997">' [25/7403]
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheriabitest-dynamic.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheriabitest-dynamic-mt.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheritest-dynamic.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheritest.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheriabitest-mt.xml
Updating statistics in JUnit file /home/nm712/dtrace_tests/cheribsd-test-results/20200526-032204/cheriabitest.xml
--> b'poweroff'
--> b'Shutdown NOW!'
--> b'poweroff: [pid 1356]'
--> b'[PEXPECT\\PROMPT]>'
--> b'\x07*** FINAL System shutdown message from root@qemu-cheri-nm712 ***\x07'
--> b''
--> b'System going down IMMEDIATELY'
--> b''
--> b''
--> b''
--> b'System shutdown time has arrived\x07\x07'
--> b'May 26 02:09:15 qemu-cheri-nm712 shutdown[1356]: power-down by root:'
--> b'Stopping sshd.'
--> b'Stopping devd.'
--> b'Writing entropy file: .'
--> b'Writing early boot entropy file: .'
--> b'Terminated'
--> b'.'
--> b'May 26 02:09:27 qemu-cheri-nm712 syslogd: exiting on signal 15'
--> b"Waiting (max 60 seconds) for system process `vnlru' to stop... done"
--> b"Waiting (max 60 seconds) for system process `syncer' to stop..."
--> b'Syncing disks, vnodes remaining... 0 0 done'
--> b"Waiting (max 60 seconds) for system thread `bufdaemon' to stop... done"
--> b"Waiting (max 60 seconds) for system thread `bufspacedaemon-0' to stop... done"
--> b'All buffers synced.'
--> b'Uptime: 1m53s'
--> b'qemu-system-cheri128: info: CPU0 executed instructions: 10292767567 (4148864782 user, 6143902785 kernel) in 119.15s KIPS: 8638
4.70'
--> b''
Poweroff took: 0:00:20.850024
Tests failed after 0:01:03.638314
===> DONE
Total execution time: 0:02:01.314988
ERROR: Some tests failed!
Running rm -fv /home/nm712/cheri/output/cheribsd-mips-nocheri.img.runtests.20200526032204.pid45913
--> b'/home/nm712/cheri/output/cheribsd-mips-nocheri.img.runtests.20200526032204.pid45913'
Fatal error: Command `/home/nm712/cheribuild/test-scripts/run_cheribsd_tests.py --ssh-key /home/nm712/.ssh/id_ed25519.pub --kernel /home/nm712/cheri/output/rootfs-mips/boot/kernel/kernel --qemu-cmd /home/nm712/cheri/output/sdk/bin/qemu-system-cheri128 --disk-image /home/nm712/cheri/output/cheribsd-mips-nocheri.img --kyua-tests-files /usr/tests/cddl/usr.sbin/dtrace/common/arithmetic/Kyuafile` failed with non-zero exit code 2. Working directory was /home/nm712/dtrace_tests
rm -rf cheri/output/sdk/sysroot128.delete-me-pls fails
Missing a chflags -R noschg ?
Warning: Previous async cleanup of /root/cheri/output/sdk/sysroot128 failed. Cleaning up now
Fatal error:Command `rm -rf /root/cheri/output/sdk/sysroot128.delete-me-pls` failed with non-zero exit code 1. Working directory was /cheri/cheribuild
freebsd-mips target fails to build due to -mcpu=beri
We're adding -mcpu=beri even when using upstream-llvm. This hack seems to be sufficient to work, but obviously breaks most CheriBSD targets:
diff --git a/pycheribuild/projects/cross/cheribsd.py b/pycheribuild/projects/cross/cheribsd.py
index a7b35cae..91c4039c 100644
--- a/pycheribuild/projects/cross/cheribsd.py
+++ b/pycheribuild/projects/cross/cheribsd.py
@@ -529,7 +529,7 @@ class BuildFreeBSD(BuildFreeBSDBase):
self.linker_for_kernel = "lld" # bfd won't work here
self.linker_for_world = "lld"
elif self.compiling_for_mips(include_purecap=True):
- target_flags = " -integrated-as -fcolor-diagnostics -mcpu=beri"
+ target_flags = " -integrated-as -fcolor-diagnostics"
self.cross_toolchain_config.set_with_options(RESCUE=False, # Won't compile with CHERI clang yet
BOOT=False) # bootloaders won't link with LLD yet
elif self.compiling_for_riscv(include_purecap=True):
opam-2.0 installation fails to create target directory
I tried to install opam-2.0 (the one in Ubuntu 18.04 is too old):
$ ./cheribuild.py opam-2.0
Configuration file /home/atm26/.config/cheribuild.json does not exist, using only command line arguments.
Sources will be stored in /home/atm26/cheri
Build artifacts will be stored in /home/atm26/cheri/output
Will execute the following targets:
opam-2.0
wget https://github.com/ocaml/opam/releases/download/2.0.1/opam-2.0.1-x86_64-linux -O /home/atm26/cheri/output/bootstrap/bin/opam
/home/atm26/cheri/output/bootstrap/bin/opam: No such file or directory
Fatal error:Command `wget https://github.com/ocaml/opam/releases/download/2.0.1/opam-2.0.1-x86_64-linux -O /home/atm26/cheri/output/bootstrap/bin/opam` failed with non-zero exit code 1. Working directory was /local/scratch/atm26/ecats/cheribuild
I think it needs to mkdir -p $CHERI_SDK/bootstrap/bin but I'm not sure where in the code that should go.
QtWebkit platform plugin error on QtTestBrowser
I'm trying to run QtTestBrowser on Cheribsd however I receive the following error after building and running the binary
root@qemu-cheri128-sean:/mnt/build/qtwebkit-128-build/bin # ./QtTestBrowser
QtTestBrowser: _rtld_thread_init is not available when statically linked.
This application failed to start because it could not find or load the Qt platform plugin "bsdfb"
in "".
Reinstalling the application may fix this problem.
Abort (core dumped)
Qtwebkit was built with
python3 debug-cheribuild.py qtwebkit --qtwebkit/linkage=static --cross-compile-linkage=static --force-update --reconfigure -c --qtwebkit/debug-info
My repositories are in the following state
Cheribuild - 97bfd25
icu4c - 900f5dcf87beca158ea62987482cbd4e517bb07d
qt5/ - ac0e04fc867c636d3a16effa6c7a85d3677bd57f
qt5/qtbase - 6c2f62170ab9973f2a969b2facdc0ca50ea4a8f2
qt5/qtwebkit - e78ae7ab924d27e866e5ba6649ec093cd8f03ee2
libxml2 - c138001afa0bce70d3b9f84f12914e22bce18aa6
It should be possible to create a unified SDK for both 128 and 256
We should now be able to create just a single SDK since LLVM can target both 128 and 256
- Clang multicapsize has been merged -> stop appending cheribits to build dir (not possible until the install dir is also the same)
- Install the sdk into $OUTPUT_ROOT/sdk instead of a suffixed dir
- CheriBSD sysroot should be installed to $OUTPUT_ROOT/sdk/sysroot{128,256}
- Baremetal newlib sysroot should be installed to $OUTPUT_ROOT/sdk/baremetal
- QEMU needs to install qemu-system-cheri{128,256} binaries (CTSRD-CHERI/qemu#61, CTSRD-CHERI/qemu#62)
- The run targets need to select the appropriate qemu-system-cheri
- Probably a bunch of other stuff
Support for RISC-V LLVM
I was trying to to build a freestanding CHERI RISC-V toolchain.
Some targets (e.g. FreeRTOS) suggest that this is possible, but I can't seem to find the correct options to built a crosscompiling LLVM which targets CHERI RISC-V.
Do I need to compile a LLVM which supports RISC-V manually to be able to build dependent targets (such as libcxx, compiler-rt and so on)?
Thanks!
Enable journaling for UFS images
Audit pexpect calls for dot and newline matches
pexpect compiles all regular expressions with re.DOTALL which means that .+ at the end of a regex will match everything until the end of the internal buffer.
Also the docs say that we can't use the $ char (https://pexpect.readthedocs.io/en/stable/overview.html#find-the-end-of-line-cr-lf-conventions) so we should change all of those (if there are any) to \r\n
Freestanding-sdk target no longer builds
The freestanding-sdk target no longer builds on FreeBSD. Fails at the beginning with the message:
./cheribuild.py --cheri-128 --source-root=./src freestanding-sdk
Configuration file /home/hx242/.config/cheribuild.json does not exist, using only command line arguments.
Sources will be stored in /usr/home/hx242/cheribuild/src
Build artifacts will be stored in /usr/home/hx242/cheribuild/src/output
cc '-DLLVM_PATH="/usr/home/hx242/cheribuild/src/output/sdk128/bin/"' -x c - -o /usr/home/hx242/cheribuild/src/output/sdk128/bin/cheridis
Fatal error: CheriBSD build directory is missing! (Tried [PosixPath('/usr/home/hx242/cheribuild/src/build/cheribsd-obj-128/mips.mips64/usr/home/hx242/cheribuild/src/cheribsd'), PosixPath('/usr/home/hx242/cheribuild/src/build/cheribsd-obj-128/mips.mips64/usr/home/hx242/cheribuild/src/cheribsd')] )
The workaround is to build the full sdk, but can be painful and unnecessary as CheriOS work does not use CheriBSD bits at all
cheribuild assumes gdb folder exists when checking URL
When building gdb for the first time (no build folder exists for it), I get the following error:
cd /home/CTSRD/CHERI/cheri-build/gdb && git remote get-url origin
Fatal error:Command `git remote get-url origin` failed with non-zero exit code 2. Working directory was /home/CTSRD/CHERI/cheri-build/gdb
Qemu build needs pixman dependency
Hi,
I just tried to build the sdk via cheribuild.py sdk on FreeBSD 11
FreeBSD 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0
r306420: Thu Sep 29 01:43:23 UTC 2016
[email protected]:/usr/obj/usr/src/sys/GENERIC amd64
Apparently qemu is missing a dependency to pixman.
After installing it via pkg install pixman qemu compiled without any further issues.
Perhaps add it as an additional dependency in https://github.com/CTSRD-CHERI/cheribuild/blob/master/pycheribuild/projects/build_qemu.py
See attached console output.
Cloning into '/home/ar/cheri/qemu'...
remote: Counting objects: 337590, done.
remote: Total 337590 (delta 0), reused 0 (delta 0), pack-reused 337589
Receiving objects: 100% (337590/337590), 163.86 MiB | 2.05 MiB/s, done.
Resolving deltas: 100% (269961/269961), done.
Already up-to-date.
Configuring QEMU ...
cd /home/ar/cheri/build/qemu-build && /home/ar/cheri/qemu/configure --target-list=cheri256-softmmu,cheri128-softmmu,cheri128magic-softmmu --disable-linux-user --disable-bsd-user --disable-xen
--disable-docs --disable-rdma --disable-werror '--extra-cflags=-O3 -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/usr/local/include' --disable-stack-protector --disable-l
inux-aio --disable-kvm --prefix=/home/ar/cheri/output/sdk
Saving build log to /home/ar/cheri/build/qemu-build/configure.log
ERROR: pixman >= 0.21.8 not present. Your options:
(1) Preferred: Install the pixman devel package (any recent
distro should have packages as Xorg needs pixman too).
(2) Fetch the pixman submodule, using:
git submodule update --init pixman
Command "/home/ar/cheri/qemu/configure --target-list=cheri256-softmmu,cheri128-softmmu,cheri128magic-softmmu --disable-linux-user --disable-bsd-user --disable-xen --disable-docs --disable-rdm
a --disable-werror '--extra-cflags=-O3 -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include -I/usr/local/include' --disable-stack-protector --disable-linux-aio --disable-kvm --pref
ix=/home/ar/cheri/output/sdk" failed with exit code 1.
See /home/ar/cheri/build/qemu-build/configure.log for details.
'bsdtar has no installation candidate' on ubuntu 19.10
It might be handy to update the debian/ubuntu directions to include a different incantation for different versions. In 19.10, Eoan Ermine, there is no bsdtar; the installation aborts with an error : "has no installation candidate".
It appears that, after some poking, libarchive-tools replaces it, and was sufficient to continue installation.
Should error out if explicitly specified --config-file does not exist
Right now, it just raises a warning and continues on, which is the right idea if it hasn't been explicitly set, but is unexpected if it has.
Allow building a CHERI sdk
- Install host tools into /sdk and fix clang DEFAULT_SYSROOT
- Add the necessary sysroot files from cheribsd to /sdk/sysroot and fix symlinks
- Add cheridis utility to sdk
- Remove unecessary files from sdk
postgres target fails to do distclean in some cases
Differently from llvm or cheribsd targets, the postgres target doesn't use separate build directories for 128 and 256 bit build. This means that doing "./cheribuild --128 postgres" can result in previously built 256 binaries to be installed.
Add target to show useful .ssh/config
Something like
Host *cheribsd*
User root
AddressFamily inet
HostName localhost
IdentityFile ~/.ssh/insecure_id_ed25519
ControlPath ~/.ssh/controlmasters/%r@%h:%p
ControlMaster auto
PasswordAuthentication no
StrictHostKeyChecking no
UserKnownHostsFile /dev/null
Host cheribsd-mips-purecap
Port 12374
Host cheribsd-mips-hybrid
Port 12375
etc.
Seems like there is an Include directive so maybe this should be created as ~/.config/cheribuild_ssh_config and users can choose to include it.
Per-project options
Currently there is no way to specify certain build options on a per-project basis. One of those is --skip-update, which if specified per-project would allow us to do things such as upgrade LLVM and rebuild CheriBSD using the new LLVM.
[Ubuntu 18.04] Segfault while running qemu-system-cheri128
Hello!
I'm see a segmentation fault upon executing the following command:
$ ./cheribuild.py run-mips-purecap -d --config-file /home/bharadwaj/datadrive/github/src/CHERI/.config/cheribuild.json --source-root /home/bharadwaj/datadrive/github/src/CHERI
...
Listening for SSH connections on localhost:10005
...
Fatal error: Command `/home/bharadwaj/datadrive/github/src/CHERI/output/sdk/bin/qemu-system-cheri128 -M malta -kernel /home/bharadwaj/datadrive/github/src/CHERI/output/rootfs-purecap128/boot/kernel/kernel -m 2048 -nographic -drive file=/home/bharadwaj/datadrive/github/src/CHERI/output/cheribsd-mips-purecap128.img,format=raw,index=0,media=disk -net nic -net 'user,id=net0,ipv6=off,smb=/home/bharadwaj/datadrive/github/src/CHERI<<<source_root@ro:/home/bharadwaj/datadrive/github/src/CHERI/build<<<build_root:/home/bharadwaj/datadrive/github/src/CHERI/output<<<output_root@ro:/home/bharadwaj/datadrive/github/src/CHERI/output/rootfs-purecap128<<<rootfs,hostfwd=tcp::10005-:22' -device virtio-rng-pci` failed with non-zero exit code -11. Working directory was /mnt/datadrive/github/src/CHERI/cheribuild
Tip of cheribuild repo used:
commit 2a799388e87562913801cba33f5199d251bc596c (HEAD -> master, origin/master, origin/HEAD)
Author: Alex Richardson <[email protected]>
Date: Wed May 20 10:42:14 2020 +0100
Try to handle failures during --test better
Appreciate any suggestions to address this.
Thanks!
Make it possible to do a non-shallow git clone
By default, the git clone depth is 1 but one might want the full repo history.
Would be good to make this configurable.
EDIT:
a shallow clone can be converted to an unshallow one with this command:
git fetch --unshallow
Warning flags seem to leak from gdb.py, postgres.py and nginx.py
When building QtWebkit, the compiler flags seem have warning flags that are set in gdb.py, postgres.py and nginx.py:
-Wno-mismatched-tags -Wno-unknown-warning-option -Wno-mismatched-tags -Wno-unknown-warning-option -Wno-mismatched-tags -Wno-unknown-warning-option -pedantic -Wno-gnu-statement-expression -Wno-flexible-array-extensions -Wno-format-pedantic
Automatically generate symlinks for each configuration in .conf
makefs command fails because manifest file is missing entry
When building a disk image, makefs complains that firstly, the METALOG manifest file is missing the /usr/tests folder and secondly about the entry ./usr/tests/cheri/lib/libc/tls/.debug/ which should be ./usr/tests/cheri/lib/libc/tls/.debug
Cleaning build directories should use move + background delete
Otherwise it will block for a really long time (especially on vica) before doing anything useful.
Also cleaning the cheribsd rootfs should keep the root directory otherwise NFS reroot breaks
Properly setup ssh access to qemu
- used port on host machine must be configurable
- Install a ssh host key that doesn't change when disk image is rebuilt
- setup the right authorized_keys entries so passwordless login works?
- add mtree entries
- add PermitRootLogin without-password to sshd_config
Set CMake library suffix for static libraries
These CMake variables need to be set to ensure that CMake only finds static libraries when doing a static build:
set(CMAKE_SHARED_LIBRARY_SUFFIX ".a")
set(CMAKE_FIND_LIBRARY_SUFFIXES ".a")
set(CMAKE_EXTRA_SHARED_LIBRARY_SUFFIXES ".a")
Allow Projects to specify dependencies
CrossCompileCMakeProject missing some compiler flags
Due to a recent change, CrossCompileCMakeProject doesn't seem to pass the following compiler flags anymore:
-mabi=purecap -mcpu=mips4 -cheri=128|256
I noticed this with QtWebKit. Does qt5.py need to be updated or is it a bug in CrossCompileCMakeProject?
--skip-buildworld doesn't
... or rather, not entirely. It does skip building the world, but it still goes through it for some reason, which takes time:
--- obj_crunchdir_mount_cd9660 --- cd /usr/home/en322/cheri/cheribsd/rescue/rescue/../../sbin/mount_cd9660 && MK_AUTO_OBJ=no MK_TESTS=no UPDATE_DEPENDFILE=no _RECURSING_CRUNCH=1 MAKEOBJDIRPREFIX=/home/en322/cheri/build/cheribsd-obj-256/mips.mips64/usr/home/en322/cheri/cheribsd/rescue/rescue make DIRPRFX=rescue/rescue/mount_cd9660/ -DRESCUE CRUNCH_CFLAGS=-DRESCUE MK_AUTO_OBJ=no obj --- obj_crunchdir_kldunload --- --- obj --- --- obj_crunchdir_mount_msdosfs --- cd /usr/home/en322/cheri/cheribsd/rescue/rescue/../../sbin/mount_msdosfs && MK_AUTO_OBJ=no MK_TESTS=no UPDATE_DEPENDFILE=no _RECURSING_CRUNCH=1 MAKEOBJDIRPREFIX=/home/en322/cheri/build/cheribsd-obj-256/mips.mips64/usr/home/en322/cheri/cheribsd/rescue/rescue make DIRPRFX=rescue/rescue/mount_msdosfs/ -DRESCUE CRUNCH_CFLAGS=-DRESCUE MK_AUTO_OBJ=no obj --- obj_crunchdir_ldconfig --- --- obj ---
It then tars things up, which takes time as well.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.