Tooling to install clusters for testing via an on-prem Assisted Installer in the Red Hat Scale/Alias Lab and bare metal servers in IBMcloud.
Three separate layouts of clusters can be deployed:
- BM - Bare Metal - 3 control-plane nodes, X number of worker nodes
- RWN - Remote Worker Node - 3 control-plane/worker nodes, X number of remote worker nodes
- SNO - Single Node OpenShift - 1 OpenShift Master/Worker Node "cluster" per available hardware resource
Each cluster layout requires a bastion machine which is the first machine out of your lab "cloud" allocation. The bastion machine will host the assisted-installer service and serve as a router for clusters with a private machine network. BM and RWN layouts produce a single cluster consisting of 3 control-plane nodes and X number of worker or remote worker nodes. The worker node count can also be 0 such that your bare metal cluster is a compact 3 node cluster with schedulable control-plane nodes. SNO layout creates an SNO cluster per available machine after fulfilling the bastion machine requirement. Lastly, BM/RWN cluster types will allocate any unused machines under the hv ansible group which stands for hypervisor nodes. The hv nodes can host vms for additional clusters that can be deployed from the hub cluster. (For ACM/MCE testing)
Table of Contents
- Tested Labs/Hardware
- Prerequisites
- Cluster Deployment Usage
- Quickstart guides
- Tips and Troubleshooting
- Disconnected API/Console Access
- Jetlag Hypervisors
The listed hardware has been used for cluster deployments successfully. Potentially other hardware has been tested but not documented here.
Alias Lab
| Hardware | BM | RWN | SNO |
|---|---|---|---|
| 740xd | Yes | No | Yes |
| Dell r750 | Yes | No | Yes |
Scale Lab
| Hardware | BM | RWN | SNO |
|---|---|---|---|
| Dell r650 | Yes | No | Yes |
| Dell r640 | Yes | Yes | Yes |
| Dell fc640 | Yes | No | Yes |
| Supermicro 1029p | Yes | Yes | No |
| Supermicro 1029U | Yes | No | Yes |
| Supermicro 5039ms | Yes | No | Yes |
IBMcloud
| Hardware | BM | SNO |
|---|---|---|
| Supermicro E5-2620 | Yes | Yes |
| Lenovo ThinkSystem SR630 | Yes | Yes |
For guidance on how to order hardware on IBMcloud, see order-hardware-ibmcloud.md in docs directory.
Versions:
- Ansible 4.10+ (core >= 2.11.12) (on machine running jetlag playbooks)
- ibmcloud cli => 2.0.1 (IBMcloud environments)
- ibmcloud plugin install sl (IBMcloud environments)
- RHEL >= 8.6 (Bastion)
- podman 3 / 4 (Bastion)
Update to RHEL 8.9
[root@xxx-xxx-xxx-r640 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux release 8.2 (Ootpa)
[root@xxx-xxx-xxx-r640 ~]# ./update-latest-rhel-release.sh 8.9
...
[root@xxx-xxx-xxx-r640 ~]# dnf update -y
...
[root@xxx-xxx-xxx-r640 ~]# reboot
...
[root@xxx-xxx-xxx-r640 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux release 8.9 (Ootpa)Installing Ansible via bootstrap (requires python3-pip)
[root@xxx-xxx-xxx-r640 jetlag]# source bootstrap.sh
...
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]#Pre-reqs for Supermicro hardware:
- SMCIPMITool downloaded to jetlag repo, renamed to
smcipmitool.tar.gz, and placed underansible/
There are three main files to configure. The inventory file is generated but might have to be edited for specific scenario/hardware usage:
ansible/vars/all.yml- An ansible vars file (Sample providedansible/vars/all.sample.yml)pull_secret.txt- Your OCP pull secret, download from console.redhat.com/openshift/downloadsansible/inventory/$CLOUDNAME.local- The generated inventory file (Samples provided inansible/inventory)
Start by editing the vars
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# cp ansible/vars/all.sample.yml ansible/vars/all.yml
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# vi ansible/vars/all.ymlMake sure to set/review the following vars:
lab- eitheraliasorscalelablab_cloud- the cloud within the lab environment (Example:cloud42)cluster_type- eitherbm,rwn, orsnofor the respective cluster layoutworker_node_count- applies to bm and rwn cluster types for the desired worker count, ideal for leaving left over inventory hosts for other purposessno_node_count- applies to sno cluster type for the desired sno count, ideal for leaving left over inventory hosts for other purposesbastion_lab_interface- set to the bastion machine's lab accessible interfacebastion_controlplane_interface- set to the interface in which the bastion will be networked to the deployed ocp clustercontrolplane_lab_interface- applies to bm and rwn cluster types and should map to the nodes interface in which the lab provides dhcp to and also required for public routable vlan based sno deployment(to disable this interface)- More customization such as cluster_network and service_network can be supported as extra vars, check each ansible roles default vars file for variable names and options
Set your pull-secret in pull_secret.txt in repo base directory. Example:
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# cat pull_secret.txt
{
"auths": {
...Run create-inventory playbook
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# ansible-playbook ansible/create-inventory.ymlRun setup-bastion playbook
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# ansible-playbook -i ansible/inventory/cloud99.local ansible/setup-bastion.ymlRun deploy for either bm/rwn/sno playbook with inventory created by create-inventory playbook
Bare Metal Cluster:
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# ansible-playbook -i ansible/inventory/cloud99.local ansible/bm-deploy.ymlSee troubleshooting.md in docs directory for BM install related issues
Remote Worker Node Cluster:
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# ansible-playbook -i ansible/inventory/cloud99.local ansible/rwn-deploy.ymlSingle Node OpenShift:
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# ansible-playbook -i ansible/inventory/cloud99.local ansible/sno-deploy.ymlInteract with your cluster from your bastion machine:
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# export KUBECONFIG=/root/bm/kubeconfig
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# oc get no
NAME STATUS ROLES AGE VERSION
xxx-h02-000-r650 Ready control-plane,master,worker 73m v1.25.7+eab9cc9
xxx-h03-000-r650 Ready control-plane,master,worker 103m v1.25.7+eab9cc9
xxx-h05-000-r650 Ready control-plane,master,worker 105m v1.25.7+eab9cc9
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# cat /root/bm/kubeadmin-password
xxxxx-xxxxx-xxxxx-xxxxxAnd for SNO
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# export KUBECONFIG=/root/sno/xxx-h02-000-r650/kubeconfig
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# oc get no
NAME STATUS ROLES AGE VERSION
xxx-h02-000-r650 Ready control-plane,master,worker 30h v1.28.6+0fb4726
(.ansible) [root@xxx-xxx-xxx-r640 jetlag]# cat /root/sno/xxx-h02-000-r650/kubeadmin-password
xxxxx-xxxxx-xxxxx-xxxxx- Deploy a Bare Metal cluster via jetlag from a Scale Lab Bastion Machine quickstart
- Deploy a Bare Metal cluster on IBMcloud via jetlag quickstart
- Deploy Single Node OpenShift (SNO) clusters via jetlag quickstart guide
- Deploy Single Node OpenShift (SNO) clusters on IBMcloud via jetlag quickstart
See tips-and-vars.md in docs directory.
See troubleshooting.md in docs directory.
See disconnected-ipv6-cluster-access.md in docs directory.
See hypervisors.md in docs directory.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent