• how to configure Rancher OS on a new machine
• what each of the images do
• why Netdata is there
• how to admin Rancher OS
  • SSH key management

No locale is set causing a test failure when parsing UTF-8 as ASCII.

See here https://www.jenkins.io/doc/upgrade-guide/2.387/

Description:

• Add instructions for running a centos/ubuntu docker container from a windows environment.
• Add a .bat script (similar to mantid_development.sh) that will run the docker container from a windows environment. The current shell script doesn't work on windows.

Now that we have moved to conda packaging and are in the process of moving our Jenkins agents to new, slimmed down docker images it would be good to clean up the dockerfiles repo.

Major changes

• rename repo to jenkins-agents
• Create top level directory for Linux and move relevant folders
• Create top level directory for Windows and create relevant sub-folders
• Create top level directory for OSX

Remove directories that are no longer needed

• mantid/
• development/bin/

Delete any files not associated with new slim docker images from following directories

• development/docker/
• jenkins-node/mantid-builder/docker-images/
• development/docker/configure/

Assess function of following directories and their contents

• scripts/
• development/bin/
• jenkins-node/deployment/
• jenkins-node/mantid-builder/bin/

Update documentation

• Expand on top level readme.md to better describe the repo and it's contents
• Create readme docs for each OS folder and move exisiting information as appropriate

Tidy dockerfiles in packages

• Remove all but current and previous 2 docker images from the package repo - https://github.com/orgs/mantidproject/packages

Provide images with the nightly built packages installed, mostly the same as the release images.

See https://blog.kitware.com/ubuntu-cmake-repository-now-available/

For our Linux Docker images we push them to Github so that they can be called when needed.

We'd like to have a Windows image and then ensure the setup script for our Windows nodes use the latest version from our Github repo (https://github.com/orgs/mantidproject/packages).

Following on moving mantid to Conda, we no longer require separate images for specific Linux distributions.

Create a minimal Linux image with the required tools to build Mantid with inside Conda. I would suggest choosing CentOS 7 as the base image to match Qt on conda-forge.

Other tools that are required:

• git
• curl
• latex rpms:
  • texlive-latex
  • texlive-latex-bin
  • texlive-was
  • tex-preview

The recent updates (specifically commit 329090d) break mantid_development.sh for me, failing to mount readable volumes with the following output:

+ TARGET_USERNAME=abc
+ for rule in /etc/entrypoint.d/*.sh
+ env TARGET_USERNAME=abc /etc/entrypoint.d/10_change_user_ids.sh
+ PUID=1000
+ PGID=1000
+ groupmod --non-unique --gid 1000 abc
+ usermod --non-unique --uid 1000 abc
+ for rule in /etc/entrypoint.d/*.sh
+ env TARGET_USERNAME=abc /etc/entrypoint.d/20_abc_own_directories.sh
+ chown abc:abc /mantid_src
chown: changing ownership of '/mantid_src': Permission denied
+ chown abc:abc /mantid_build
chown: changing ownership of '/mantid_build': Permission denied
+ chown abc:abc /mantid_data
chown: changing ownership of '/mantid_data': Permission denied
+ chown abc:abc /ccache
+ CMD=bash
+ runuser -u abc -- bash
abc@ee926268edd1:/mantid_build$

The former behaviour is restored (without "permission denied" errors) if I reintroduce the --icp=host argument to docker within this script. This option is mentioned in the updated docs as being necessary for X windowing, but seems to have a wider impact?

A factor that is probably relevant is that I don't run docker as root but use a docker group instead. Perhaps the recent updates make an assumption that docker was run by root?

Windows build agents have typically been setup manually. Investigate creating a Windows container image with the required tools.

The tools that are required:

• Visual Studio 2019
• git/git bash
• miktex

It is possible that the memory being used on the isiscloudwin machines is gradually going up over time. We should investigate if this is the case, and find out why. Ideally, we shouldn't need to keep cleaning up the system manually when it runs out of memory.

Useful command docker system df and docker system prune -a. The memory used up by the containers is probably where the problem lies (if there is a problem).

Make a slimmed down image that contains all the analysis tools for Jenkins use.

Cppcheck might have to remain on Ubuntu Bionic as it has to actually have the dependencies, etc.

All the others may be possible to do using a miniconda image with the Jenkins agent on top.

• flake8==2.5.4
• pep8==1.6.2
• pyflakes==1.3.0
• mccabe==0.6.1
• doxygen 1.8.11
• clang-format 5.0.0

Currently done by hand which is error prone and time consuming. Adding this task might do the trick:
https://github.com/interactivereduction/k8s/blob/main/ansible/roles/update-upgrade/tasks/main.yml

The Jenkins build scripts are using killall to kill any existing Xvfb instances. It would appear that killall is not installed in the jenkins images. The build log shows: /jenkins_workdir/workspace/pull_requests-ubuntu-python3/buildconfig/Jenkins/buildscript: line 48: killall: command not found

I think there is only 80GB available to this node. Also investigate whether isicloudwin3 needs upgrading.

The remoting version of the Jenkins node image is now too old to be connected to Jenkins without specifying a parameter to allow unsupported versions. Update the image to a later version:

In this file also replace any mentions of the word slave to agent.

To fix an OpenSSL security vulnerability we need to update the Docker images for Linux.

We currently have to install OpenSSL directly as part of our Linux Docker containers. Eventually Mantid will be moving to v 3.0.8 or higher (see mantidproject/mantid#35194). At this point we will need to update our scripts for our Linux Docker containers to match.

The documentation on Linux container set up could probably do with a refresh

The mantid-development docker image for Linux setup (https://github.com/mantidproject/dockerfiles/blob/main/Linux/development/docker/CentOS7Slim.Dockerfile) could do with some tidying up.

In particular

• do we still need to install ccache?
• Do we still need cppcheck part included?

Also the setting of the latex paths seems to be in both the dockerfile and the latex shell file. Is this necessary?

Both the current running mac builders are on High Sierra (10.13). This hasn't been getting security updates since november 2020. We should be on at least Big Sur, which is still getting updates. We should avoid upgrading to ventura though, since there's been some issues with JAMF and the network at RAL.

The current running builders don't support anything higher than high-sierra, they're too old.
We need some way of building both an ARM and an x86 version of Mantid on an ARM machine.

Pros:

• Saves developers from having to build it on their machine
• Saves having to build it manually on build servers

Cons:

• Bigger Docker image

Currently changing installed packages required rebuilding Paraview (twice).

Change the order things are installed so that helper tools, static analysis tools, debuggers, etc. are installed towards the end of the Dockerfile.

Spin up more Linux and Windows nodes in the cloud and retire any physical machines

Include scripts that perform the initial CMake configuration in the images.

Saves having to copy them from the documentation in the repo or having to type them from memory.

Images with Mantid preinstalled with Python 3 support.

Need to verify the following will work:

• build
• test
• systemtests
• doctests

Once verified the Jenkins node images should work fine for build machines.

cc: @peterfpeterson

OSX build agents have typically been setup manually. We are unable to create docker containers for these machines but an ansible script should make setting up OSX machines easier. It will also ensure all machines are using the same setup.