These are automated bots and tools that work on Cockpit. This includes updating operating system images, testing changes, releasing Cockpit and more.

In order to test Cockpit-related projects, they are staged into an operating system image. These images are tracked in the images/ directory. For example, you might want to test a scenario where Cockpit on one machine talks to FreeIPA on another, and you want those two machines to use different images.

This is handled by passing a specific image to image-create and other scripts that work with test machine images. Available images include:

• fedora-, rhel-, debian-*, etc: Various operating systems for testing Cockpit related projects
• services: Auxiliary network services for tests which are independent from the OS where Cockpit runs: FreeIPA, Samba AD, candlepin, selenium containers
• openshift: An Openshift Origin server

These well known image names are expected to contain no . characters and have no file name extension.

Individual projects are expected to locally build their code into packages, and install them as overlay on top of these pristine images, with image-customize or using the machine Python API.

For managing these images:

• image-download: Download selected or all test images
• image-create: Create test machine images from scratch (usually through virt-install or downloading a cloud image), with common build and test dependencies for Cockpit projects preinstalled
• image-upload: Upload a locally built test image to the official image servers

For running and debugging the images:

• image-customize: Install packages, upload files, or run commands in a test machine image; this keeps the original image intact, and puts the changes into an image overlay into test/images/.
• vm-run: Run a test machine image; by default this happens in an ephemeral overlay. You can use the --maintain option to write into the persistent overlay in test/images/ instead.
• vm-reset: Remove all overlays from test/images/

If you use vm-run with the --network option and you get an error:

qemu-system-x86_64: -netdev bridge,br=cockpit1,id=bridge0: bridge helper failed

then please allow qemu-bridge-helper to access the bridge settings.

Downloaded images are stored into ~/.cache/cockpit-images/ by default. If you want to change that, you can set the cockpit.bots.images-data-dir variable with git config to a directory where to store the pristine virtual machine images. For example:

$ git config cockpit.bots.images-data-dir /srv/cockpit/images

The bots automatically run the tests as needed on pull requests and branches. To check when and where tests will be run, use the tests-scan tool:

$ ./tests-scan -vd

As eslint looks for additional configurations, eslintrc.(json|yaml) files, in parent directories, it is recommended to have "root": true in the eslint configuration of any project which is using eslint and is tested through cockpit-bots.

A number of machines are watching our GitHub repository and are executing tests for pull requests as well as making new images.

Most of this happens automatically, but you can influence their actions with the tests-trigger utility in this directory.

You need a GitHub token in ~/.config/github-token or from the GitHub CLI configuration in ~/.config/gh/config.yml. You can create one for your account at

https://github.com/settings/tokens

When generating a new personal access token, the scope only needs to encompass public_repo (or repo if you're accessing a private repo).

For describing tests which we want to run we use contexts. A context has the form:

os_image[/scenario][@bots#bots_pr][@owner/project/ref]

where items have the following meaning:

• os_image: Name of the image on which tests should run (e.g. 'fedora-testing').
• scenario: Name of a specific test. This is specific for each separate project and is passed verbatim to 'test/run' in $TEST_SCENARIO.
• bots_pr: Number of pull request that exists in bots repository. When specified, bots from this PR would be used instead of master.
• owner/project: Name of github project (e.g. 'cockpit-project/cockpit'). This part can be omitted when testing in the same project and no 'ref' is needed.
• ref: Reference in the project (usually branch) (e.g. 'rhel-8.2'). Default is 'master'.

For example, context for scenario 'firefox' on 'fedora-testing' is:

fedora-testing/firefox

If we want to trigger it on 'cockpit-project/cockpit':

fedora-testing/firefox@cockpit-project/cockpit

If we want to also not run it on master branch, but on 'rhel-8-0' branch:

fedora-testing/firefox@cockpit-project/cockpit/rhel-8-0

If we want to run tests on 'fedora-testing' but with bots from pull request '169':

fedora-testing@bots#169

If you want to run the "fedora-testing" testsuite again for pull request #1234 of cockpit-project/cockpit, run tests-trigger like so:

$ ./tests-trigger --repo cockpit-project/cockpit 1234 fedora-testing

You can also invoke bots/tests/trigger from any project checkout, in which case you don't need the explicit --repo -- it will default to the GitHub origin of the current directory's project.

If you want to run all tests on pull request #1234 that has been opened by someone who is not in our white-list, run tests-trigger with --allow:

$ ./tests-trigger --allow [...]

Of course, you should make sure that the pull request is proper and doesn't execute evil code during tests.

Test images are refreshed automatically once per week, and even if the last refresh has failed, the machines wait one week before trying again.

If you want the machines to refresh the fedora-testing image immediately, run image-trigger like so:

$ ./image-trigger fedora-testing

If as part of some new feature you need to change the content of some or all images, you can ask the machines to create those images.

If you want to have a new fedora-testing image for pull request #1234, add a bullet point to that pull request's description like so, and add the "bot" label to the pull request.

* [ ] image-refresh fedora-testing

The machines will post comments to the pull request about their progress and at the end there will be links to commits with the new images. You can then include these commits into the pull request in any way you like.

If you are certain about the changes to the images, it is probably a good idea to make a dedicated pull request just for the images. That pull request can then hopefully be merged to master faster. If instead the images are created on the main feature pull request and sit there for a long time, they might cause annoying merge conflicts.