open-quantum-safe / ci-containers Goto Github PK

As discussed on the dev call, we could add a travis test in the openssl projects to download and run the integration/oqs_openssl/run[2] script. Since the openssl project already has it's own code, it would be good split the script in subcomponents, for example: build-liboqs-master, build-liboqs-nist, build-openssl-1.0.2, build-openssl-1.1.1, run-openssl-1.0.2-tests, and run-openssl-1.1.1-tests. This way the openssl travis test could skip the build-openssl-* subscripts, and reuse the code to build liboqs and run the openssl tests.

Followon from open-quantum-safe/liboqs#1702

The openquantumsafe/ci-debian-custer-amd64:latest image is used as part of our ci process

When working on the above PR I noticed we weren't pinning the version spec of this image (though the tooling did not detect this).

I inspected the image with a scan on quay.io

This is used for testing/verification, rather than supplying images for consumers, but it looks as if it could do with being updated - any images/sw used for tests could be compromised to hide an injected vulnerability.

The current image has quite old java (1.11.0), and also older versions of qemu and other tools. See list

Supporting BoringSSL build & new parallel testing

• ubuntu
• alpine
• debian
• centos7
• centos8

In support of open-quantum-safe/liboqs#1046 (comment)

Write the equivalent of integration/oqs_openssl/run.sh for Windows.

In support of open-quantum-safe/liboqs#1007 . Maybe not go as far as creating multi-platform image but just one that can be properly referenced in a native machine runner test.

Would be helpful to be able to specify particular branches to run in the integration tests, so that developers can locally test a particular branch that hasn't yet been merged.

Add a run all script to:

• perform all documented pk install steps
• execute the run.sh scripts for openssl and openssh

Rationale: make it faster and less error prone to run the integration tests on target systems.

Implement .travis.yml and potentially some glue scripts.

Which platforms should be targeted in the travis matrix?

The current CI has the following problems:

• Still uses CircleCI that we want to get away from
• Still requires docker login during PR which excludes external contributors from running CI
• Still only builds single-platform images instead of multi-platform ones

This issue is to suggest fixing these shortcomings before/while upgrading to ubuntu:latest as per open-quantum-safe/liboqs#1780

To avoid "seesaw" open-quantum-safe/liboqs#1769 -> open-quantum-safe/liboqs#1715 -> ....

The repository presently contains dockerfiles which have not been updated in a while: centos-7 and centos-8. If these containers are no longer used we should remove them.

It may be worth tracking our container usage across OQS projects (using an automated tool? dependabot?) to ease future maintenance.

Line 177 repeats the build step at line 164:

https://github.com/open-quantum-safe/testing/blob/cee55b93cd8aa616ed1def3f7e3365e04973456e/integration/oqs_openssh/run.sh#L164
https://github.com/open-quantum-safe/testing/blob/cee55b93cd8aa616ed1def3f7e3365e04973456e/integration/oqs_openssh/run.sh#L177

With cutting CI runtime in mind, is there anything preventing us from keeping the SSL build parts for the liboqs-nist integration test?

OpenSSH can be built without OpenSSL (with the configure --without-openssl option). The OQS code ran into issues in the past while enabling this option (see e.g., PR21). If this is a recommendable deployement option, then we should add this to our integration test.

Adding an issue here for documentation/reference purposes:

Options to try:

• Caching dependencies
• Caching compiler outputs
• Reduce/share builds of components?
• Reduce number of containers in matrix (e.g. don't test all combos of Ubu 14, 16 vs 3 GCC versions)
• Cache brew step for Mac test containers?
• ...

Links

• https://docs.travis-ci.com/user/speeding-up-the-build/
• https://docs.travis-ci.com/user/installing-dependencies/
• https://docs.travis-ci.com/user/caching/

Tasks:

• Can we get consistent environments for the liboqs team?
• Automate provisioning/test environment setup
• Automate running and logging of the integration test suite

Other:

• Intended for local testing, before committing and running the Travis CI
• Partially duplicates some travis functionality

Our tests use root CA certs to instantiate the server. This does not invoke all code paths dealing with cert issuance (cert request generation and verification). Migrating to root-issued CA certs improve our coverage.