eclipse-cbi / jiro Goto Github PK

Warning says

Builds in Jenkins run as the virtual SYSTEM user with full permissions by default. This can be a problem if some users have restricted or no access to some jobs, but can configure others. If that is the case, it is recommended to install a plugin implementing build authentication, and to override this default.
❌ No implementation of access control for builds is present. It is recommended that you install Authorize Project Plugin or another plugin implementing the QueueItemAuthenticator extension point

with a link to https://jenkins.io/doc/book/system-administration/security/build-authorization/

Some projects use a mix, some are only on a given kind of hosting. Permissions and installed plugins needs to be handled differently depending on the use case.

Determining which project does what could rely on projects.eclipse.org API.

We have installed an xvfb plugin, https://github.com/eclipse-cbi/jiro/blob/master/instances/technology.openj9/jenkins/plugins-list#L15

Based on eclipse-openj9/openj9#8376 (comment), it is not enough to just add the xvfb plugin, you additionally have to add a default installation in global tool configuration view in Jenkins.

To resolve this error on a Jenkins server we have admin access to, one would go to Manage Jenkins -> Global Tool Configuration (at the bottom of the page) and add a default xvfb installation (as shown):

See https://bugs.eclipse.org/bugs/show_bug.cgi?id=552525#c3

This preStop hook could trigger a safe shutdown of the instance

OMR is planning to launch PR builds in Docker containers: eclipse/omr#6525.

In order achieve the above goal, OMR needs to have the Docker plugin in its Jenkins pipeline.

The latest templates are pulled in, even though jenkins.master-base/latest does not point to the newest version (yet). In this case 2.176.1.

Workaround: Specifying the Jenkins version (2.176.1) explicitly in the config.json file.

During deployment the templates should be pulled in according to "actualVersion".

If there are issue during deployment, especially while waiting for a Jenkins instance to come online during/after a safe restart, it requires extra work to fix the routes, etc manually.
Since the benefit of "dropping the curtain" during a (in most cases) relatively short safe restart, I'd recommend that we only switch to maintenance mode on demand (e.g. if we work on a JIPP for a longer time and want to avoid interruptions).

https://plugins.jenkins.io/audit-trail/

Could be used as replacement to the greedy job-config-history plugin

Minimal requirement: name with link to instance + status

Features:

• Versions
• Some health status / historic
• Some workload info (current, short historic)
• Info about current quotas

Currently, agents are started in the same namespace and with the same service account as the master. It means several things that we would like to avoid:

• Secrets that should only be readable by masters are readable by agents.
• Quotas are set globally, so it's hard to track. Ideally, if we would run in separate namespaces, we would not need to set quotas on master's namespace, only specifying master resources requests/limits would be enough, and then only set limitrange/quotas on agent ns.
• Agents service account have more permissions than necessary (e.g. they can create other pods, it highly undesirable)

Resource quotas usable by project should be stored in the config.json. Currently, it's only the "sponsorshipLevel". It should be more explicit. Cluster quotas should be computed from these information (e.g. it will add the resources required for the jnlp container etc...).

They have been removed from the update center a while ago (https://issues.jenkins.io/browse/INFRA-2487) but some instances still have it and reports noisy warnings.

In order to do that, we first need to be able to remove plugin from jiro itself.

Depends on #59

Possibly related #145

Seems our infra team have lost permissions on omr jenkins

@rajdeepsingh1
@jdekonin
@AdamBrousseau

https://github.com/eclipse-cbi/jiro/blob/master/instances/technology.omr/config.jsonnet

Using the EF html templates, please provide a sample job that will build a static HTML web page suitable for hosting on download.eclipse.org (and archive.eclipse.org) from the contents of a directory.

https://plugins.jenkins.io/saml/
https://support.cloudbees.com/hc/en-us/articles/227202668-SAML-Plugin-Basics

This could replace direct ldap connection. Allows Jenkins setup outside of LAN where LDAP is reachable. Plugin is supported by Cloudbees whereas OPenIDConnect plugin (https://plugins.jenkins.io/oic-auth/) is more or less dead (no update in 2y).

It could replace the yaml merge and json merge operators.

This plugin was in the list of plugins listed in Prioritized TCK infra requirements, but must have been missed.

For accuracy purpose.

Allow uninstalling Jenkins plugins and handle associated permissions (e.g. Gerrit) to avoid JCasC startup errors.

This might become easier with the new Jenkins Plugins Installation Manager CLI Tool/Library: https://jenkins.io/projects/gsoc/2019/plugin-installation-manager-tool-cli/

The default Maven settings file is defined in a config map. When credentials for Nexus and/or OSSRH are required, then a secret should be used. To make the generation scripts aware which of the two should be used, an (boolean) option should be added to the config.json file.

In templates/jenkins/configuration.yml.hbs the following lines need to be removed:

security:
  remotingCLI:
    enabled: false

Gerrit is not utilized in the project and the plugin does not work

JCasC fails when it has a configuration with tools installation for plugins that are not installed.

We should find a way to only generates tools installations for instances that have the proper plugins installed.

When I tried to checkout the current master of this repo on my Windows computer, the checkout fails with the following message:

An internal error occurred during: "Checking out eclipse.cbi.jiro - refs/heads/pref".
Invalid path: performance-tests/results/local/Darth Plagueis/guava-2021-06-17T14:03:51+02:00.log

This is probably because the colon is forbidden as file name on Windows:
https://stackoverflow.com/a/31976060

Removing the files in performance-tests/results/ (via a GH codespace) allows me to checkout those files.
Since you already ignore new result files, I wonder if you are fine to delete the old once too?

Class Data Sharing may help improve startup time and memory consumption for both masters and agents (jnlp).

For masters, it would mean mounting a folder from the host (https://kubernetes.io/docs/concepts/storage/volumes/#hostpath) and use it as the cache. It requires some changes in scc/podsecuritypolicy as today hostPath volumes cannot be mounted by masters' service accounts. It's not wise to give more permissions to this service account until we split masters and agents as specified in #5.

For agents, it's TBD. Regarding security, it may be unsafe to allow agents to mount hostPath (e.g. /var/lib/docker).

Note:
PodSecurityPolicy offers fine grain policy that let specifies a whitelist of host paths that are allowed to be used by hostPath volumes. PodSecurityPolicies are not available on OpenShift 3.9 (but is a beta feature in 3.11). There is only SecurityContextConstraint which provide hostPath access on a all or nothing basis

The following lines need to be changed in templates/jenkins/configuration.yml.hbs:

   authorizationStrategy:
     projectMatrix:
-      grantedPermissions:
+      permissions:

This is a recurring issue where the jenkins build queue is stuck after a reboot.

Workaround: clear the queue with Jenkins.instance.queue.clear()

Related to https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/4306#note_1754550

Add option in config.json, default = America/Toronto

Used to set -Duser.timezone={{jenkins.timezone}} in Jenkins startup command.

Currently, all metadata about k8s resources are labels

org.eclipse.cbi.jiro/project.shortname: "{{project.shortName}}"
org.eclipse.cbi.jiro/project.fullName: "{{project.fullName}}"
org.eclipse.cbi.jiro/jenkins.version: "{{jenkins.version}}"
org.eclipse.cbi.jiro/jenkins.actualVersion: "{{jenkins.actualVersion}}"
org.eclipse.cbi.jiro/kubernetes.master.namespace: "{{kubernetes.master.namespace}}"
org.eclipse.cbi.jiro/project.sponsorshipLevel: "{{project.sponsorshipLevel}}"

We should only keep

org.eclipse.cbi.jiro/project.shortname: "{{project.shortName}}"
org.eclipse.cbi.jiro/project.fullName: "{{project.fullName}}"

and move other metadata as simple annotations as they will probably won't be use by any selectors.

Also, we may want to add org.eclipse.cbi.jiro/kind: "master" or org.eclipse.cbi.jiro/kind: "agent"
To differentiate agents' artifacts from masters'.

And try to find a way to declare docker repo name "eclipsecbijenkins" in a single location.

It's GA since https://www.jenkins.io/changelog-stable/#v2.263.1 and we can now grant it to committers. I think it would be helpful for them.

The Jiro default timezone is America/Toronto set in code here

The main problem with this setting is when it comes to deal with job based on a docker image with a different timezone.
So there is a difference between the UI and the job log generated by the docker image, which can quickly lead to confusion.

I.E: jenkins UI with job #46 started at 07:28

Job log start at: 13:28

It might be great to align the jenkins timezone with something more in common, and understandable by all like UTC-0 with no DST.

As part of the definition of "rbac.authorization.k8s.io/v1", namespace attribute doesn't exist on RoleRef and Subjects attributes definition. Instead it must be the attribut "apiGroup".

Example : https://github.com/eclipse-cbi/jiro/blob/master/instances/automotive.mdmbl/target/k8s/role-binding.json
should be :

"roleRef": {
      "kind": "Role",
      "name": "jenkins-master-owner",
      "apiGroup": "my_api_group"    <=== here
   },
   "subjects": [
      {
         "kind": "ServiceAccount",
         "name": "mdmbl",
         "apiGroup": "my_api_group"         <=== here
      }
   ]

And apiGroup should have a name: https://github.com/eclipse-cbi/jiro/blob/master/instances/automotive.mdmbl/target/k8s/role.json

   "rules": [
      {
         "apiGroups": [
            "my_api_group"             <=== here
         ],
         "resources": [
            "pods",
            "pods/exec"
         ],

I just want to understand where is it used and why is it needed. Any pointers will be very helpful, thank you.

They are old-fashion by today's standard

All our machines which connect using an "ssh command" rather than the standard ssh plugin are offline.

I assume because of this change
#249

Not sure why this isn't automatic.

cc @fredg02

Currently, docker cache is either used too much or not enough. We should force (no cache) rebuild images when system packages have been updated or when jenkins plugins have been upgraded.

To store local .m2 repos etc.

It's not yet defined wether projects will be responsible for creating job specific sub-folders in this cache

• Number of history entries to keep: 500
• Max number of days to keep history entries: 60
• Max number of history entries to show per page: 12
• Do not save duplicate history: unticked

Steps to reproduce:

1. Create a new jenkins-master-base version (e.g. 2.176.1)
2. Redeploy a Jiro instance with no explicit Jenkins version set (e.g. dash instance)
3. Expected version after redeploy: same as before (e.g. 2.164.1), actual version: 2.176.1

IMO eclipsecbijenkins is not satisfactory

3.29 is the new version for LTS 2.164.1

When a new version of Jenkins is released, the JCasC files sometime need to be adapted (field renaming, removal of deprecated options... e.g., see #23 and #24).

It should be possible to have version specific templates and a default fallback being the current location.

We can use supplemental groups to grant agents permissions to read/write from project owned folder (like downloads, archives etc...). See https://github.com/eclipse-cbi/sonatype-nexus/blob/master/repo.locationtech.org.yml for an example

It seems to be an issue with the OkHttp lib. See

Jetty describe a solution: add alpn to bootclasspath java -Xbootclasspath/p:libs/alpn-boot-8.1.9.v20160720.jar

List of matching versions are here https://www.eclipse.org/jetty/documentation/9.4.x/alpn-chapter.html#alpn-versions and binaries can be downloaded on maven central https://search.maven.org/search?q=g:org.mortbay.jetty.alpn%20a:alpn-boot

This won't be necessary anymore as soon as Jenkins will run on OpenJDK9+

If some jobs have been run with a version < 1.40.0, they are still affected by https://www.jenkins.io/security/advisory/2018-03-26/#SECURITY-261, so it's quite hard to know if we're at risk or not (apart from running https://github.com/jenkinsci-cert/SECURITY-261 on a regular basis).

Also, the plugin is for adoption and advise to switch to https://plugins.jenkins.io/github-branch-source/ which is preferable anyway.

@fredg02, what do you think?

This webhook should add common settings to all agent pods

• maven/gradle default configs
• resource request/limits
• jnlp agent configuration (image + resources etc.)

See

• https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook
• https://medium.com/ibm-cloud/diving-into-kubernetes-mutatingadmissionwebhook-6ef3c5695f74