snowdrop / k8s-infra Goto Github PK
View Code? Open in Web Editor NEWInformation to bootstrap vms using dedicated server, local machine and setup using Ansible Playbooks
License: Apache License 2.0
Information to bootstrap vms using dedicated server, local machine and setup using Ansible Playbooks
License: Apache License 2.0
Due to the refactoring of theopenshift-ansible
project which does not use same paths to access the byo
playbook or a module playbook, then we should add a var to specify what is the root path to access it according to the openshift ansible release
E.g message that you will get if you git clone openshift-ansible release-3.9
and run our playbook
ansible-playbook -i inventory/cloud_host playbook/post_installation.yml -e openshift_admin_pwd=${OPENSHIFT_ADMIN_PWD} --tags "identity_provider,enable_cluster_admin"
ERROR! Unable to retrieve file contents
Could not find or access '/Users/dabou/Downloads/snowdrop-infra/ansible/openshift-ansible/playbooks/byo/openshift-cluster/service-catalog.yml'
Step
ansible-playbook -i inventory/cloud_host playbook/post_installation.yml \
> --tags install-launcher \
> -e launcher_catalog_git_repo=https://github.com/snowdrop/cloud-native-catalog.git \
> -e launcher_catalog_git_branch=master \
> -e launcher_github_username=YOUR_GIT_TOKEN \
> -e launcher_github_token=YOUR_GIT_USER
Error
TASK [launcher : Check if project/namespace exists] *************************************************************************************************************************************************************************************************
fatal: [192.168.99.50]: FAILED! => {"changed": true, "cmd": ["oc", "get", "project/devex"], "delta": "0:00:14.985096", "end": "2018-05-09 18:55:46.441224", "msg": "non-zero return code", "rc": 1, "start": "2018-05-09 18:55:31.456128", "stderr": "Error from server (Forbidden): projects.project.openshift.io \"devex\" is forbidden: User \"user2\" cannot get projects.project.openshift.io in the namespace \"devex\": User \"user2\" cannot get project \"devex\"", "stderr_lines": ["Error from server (Forbidden): projects.project.openshift.io \"devex\" is forbidden: User \"user2\" cannot get projects.project.openshift.io in the namespace \"devex\": User \"user2\" cannot get project \"devex\""], "stdout": "", "stdout_lines": []}
...ignoring
We should Include the istio playbook in order to install it we do for jaeger, nexus, jenkins.
Shoud we consider to use Ansible Galaxy in order to deploy istio playbook or to use git clone command ... ?
Times diverge between local machine and VM
14:01
[root@cloud ~]# date
Tue May 8 12:01:31 UTC 2018
Solution : Change timezone during vm creation timedatectl set-timezone Europe/Brussels
How : https://access.redhat.com/solutions/2996411
# vi user-data
#cloud-config
password: redhat
chpasswd: { expire: False }
ssh_pwauth: True
ssh_authorized_keys:
- ssh-ed25519 AAAAC3Naaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa0 [email protected]
runcmd:
- timedatectl set-timezone UTC
Do we still need playbook enable_rpm_packages.yml ?
Rename github repo from cloud-native-infra
to openshift-infra
Post creation of a new Openshift cluster using Ansible OpenShift playbooks, I had several times to delete this symbolic file within the linux vm
/etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt
in order to let the openshift server to download the openjdk1.8 image from the registry.access.redhat.com server
Remark : That was working pretty well 2 weeks ago
Steps
ansible-playbook playbook/generate_inventory.yml -e ip_address=192.168.99.50 -e type=simple
ansible-playbook -i inventory/simple_host playbook/cluster.yml --tags "up"
ansible-playbook -i inventory/simple_host openshift-ansible/playbooks/openshift-service-catalog/config.yml -e openshift_master_unsupported_embedded_etcd=true
Error
TASK [Evaluate groups - Fail if no etcd hosts group is defined] ******************************************************************************************************
fatal: [localhost]: FAILED! => {
"changed": false,
"msg": "Running etcd as an embedded service is no longer supported. If this is a new install please define an 'etcd' group with either one, three or five hosts.
These hosts may be the same hosts as your masters. If this is an upgrade please see https://docs.openshift.com/container-platform/latest/install_config/upgrading/migrating_embedded_etcd.html for documentation on how to migrate from embedded to external etcd.
"}
to retry, use: --limit @/Users/dabou/Code/snowdrop/cloud-native/temp/openshift-infra/ansible/openshift-ansible/playbooks/openshift-service-catalog/config.retry
We should use oc --config
wherever the admin users needs to perform an action
We should focus on Centos, 3.7
as the origin repo is added automatically when installation will take place.
yum repolist -v
Repo-id : centos-openshift-origin37
Repo-name : CentOS OpenShift Origin
Repo-revision: 1514193298
Repo-updated : Mon Dec 25 09:15:01 2017
Repo-pkgs : 44
Repo-size : 301 M
Repo-baseurl : http://mirror.centos.org/centos/7/paas/x86_64/openshift-origin37/
Repo-expire : 21600 second(s) (last: Sat Mar 17 09:46:41 2018)
Filter : read-only:present
Repo-filename: /etc/yum.repos.d/CentOS-OpenShift-Origin37.repo
and here are the packages deployed
yum list installed | grep origin
Failed to set locale, defaulting to C
origin.x86_64 3.7.0-1.0.7ed6862 @centos-openshift-origin37
origin-clients.x86_64 3.7.0-1.0.7ed6862 @centos-openshift-origin37
origin-docker-excluder.noarch 3.7.0-1.0.7ed6862 @centos-openshift-origin37
origin-excluder.noarch 3.7.0-1.0.7ed6862 @centos-openshift-origin37
origin-master.x86_64 3.7.0-1.0.7ed6862 @centos-openshift-origin37
Remark : as mentioned by Michael Gugino, we should stop to use openshift_repos_enable_testing
as finally it will mix different rpms during playbook execution
origin.x86_64 3.7.0-1.0.7ed6862 @centos-openshift-origin37
origin-clients.x86_64 3.7.0-1.0.7ed6862 @centos-openshift-origin37
origin-master.x86_64 3.7.0-1.0.7ed6862 @centos-openshift-origin37
[root@cloud ~]# yum list installed | grep origin
Failed to set locale, defaulting to C
...
[root@cloud ~]# yum list installed | grep origin
Failed to set locale, defaulting to C
origin.x86_64 3.7.1-1.el7.git.0.0a2d6a1 @centos-openshift-origin37-testing
origin-clients.x86_64 3.7.1-1.el7.git.0.0a2d6a1 @centos-openshift-origin37-testing
origin-master.x86_64 3.7.1-1.el7.git.0.0a2d6a1 @centos-openshift-origin37-testing
origin-node.x86_64 3.7.1-1.el7.git.0.0a2d6a1 @centos-openshift-origin37-testing
Tag project to 3.9.0
as our tests are passing and close Milestone-1.0
Currently, the pv and temp created are default to 3
volume:
defaults: #define the defaults for all Persistent Volumes
storage: 5Gi
persistentVolumeReclaimPolicy: Recycle
volumes:
pv001:
storage: "{{ volume.defaults.storage }}"
persistentVolumeReclaimPolicy: "{{ volume.defaults.persistentVolumeReclaimPolicy }}"
pv002:
storage: "{{ volume.defaults.storage }}"
persistentVolumeReclaimPolicy: "{{ volume.defaults.persistentVolumeReclaimPolicy }}"
pv003:
storage: "{{ volume.defaults.storage }}"
persistentVolumeReclaimPolicy: "{{ volume.defaults.persistentVolumeReclaimPolicy }}"
I suggest to add a var in order to let to override the value and remove the hard coded list as defined under volumes
and use instead a dynamic list generated
There is no filter option defined within our launcher
role. This role is used by the f8 launcher backend to filter the boosters within the UI
To be done : update role and command's doc of the md file
To ssh as root to the vm hetzner, local
, we use these commands to import our ssh_key as authorized key
sshpass -f pwd.txt ssh -o StrictHostKeyChecking=no [email protected] -p 5222 "mkdir ~/.ssh && chmod 700 ~/.ssh && touch ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"
sshpass -f pwd.txt ssh-copy-id -o StrictHostKeyChecking=no -i ~/.ssh/id_rsa.pub [email protected] -p 5222
We should improve the existing playbook to :
Interesting articles :
As the installation of the service catalog will certainly fail when the cluster is created
TASK [ansible_service_broker : Create the Broker resource in the catalog] **************************************************************************************************************************************************************************
fatal: [192.168.99.50]: FAILED! => {"changed": false, "failed": true, "msg": {"cmd": "/usr/bin/oc create -f /tmp/brokerout-dJmL1S -n default", "results": {}, "returncode": 1, "stderr": "error: unable to recognize \"/tmp/brokerout-dJmL1S\": no matches for servicecatalog.k8s.io/, Kind=ClusterServiceBroker\n", "stdout": ""}}
I propose that we install it separately and include it as one of the modules that we can install such as jaeger, nexus, jenkins, ...
ansible-playbook -i inventory openshift-ansible/playbooks/byo/openshift-cluster/service-catalog.yml
We need to generate these automatically and update the docs
When we stop the server and restart it, then we get this k8s service catalog error
Origin version : 3.9.0-alpha3
Command used:
ansible-playbook -i inventory/cloud_host playbook/cluster.yml -e openshift_node=masters -e openshift_release_tag_name=v3.9.0-alpha.3 --tags "down"
ansible-playbook -i inventory/cloud_host playbook/cluster.yml -e openshift_node=masters -e openshift_release_tag_name=v3.9.0-alpha.3 --tags "start"
Error
{
"changed": true,
"cmd": [
"oc",
"cluster",
"up",
"--version=v3.9.0-alpha.3",
"--host-config-dir=/var/lib/origin/openshift.local.config",
"--host-data-dir=/var/lib/openshift/data",
"--host-volumes-dir=/var/lib/openshift/volumes",
"--host-pv-dir=/var/lib/openshift/pv",
"--use-existing-config=True",
"--public-hostname=192.168.99.50",
"--routing-suffix=192.168.99.50.nip.io",
"--loglevel=1",
"--service-catalog=True"
],
"delta": "0:00:13.124961",
"end": "2018-03-28 10:00:57.619554",
"msg": "non-zero return code",
"rc": 1,
"start": "2018-03-28 10:00:44.494593",
"stderr": "I0328 10:00:46.889982 3674 helper.go:585] Copying OpenShift config to local directory /tmp/openshift-config211463473",
"stderr_lines": [
"I0328 10:00:46.889982 3674 helper.go:585] Copying OpenShift config to local directory /tmp/openshift-config211463473"
],
"stdout": "-- Checking OpenShift client ... \n-- Checking Docker client ... \n-- Checking Docker version ... \n-- Checking for existing OpenShift container ... \n-- Checking for openshift/origin:v3.9.0-alpha.3 image ... \n-- Checking Docker daemon configuration ... \n-- Checking for available ports ... \n-- Checking type of volume mount ... \n\n Using nsenter mounter for OpenShift volumes\n-- Creating host directories ... \n-- Finding server IP ... \n\n Using public hostname IP 192.168.99.50 as the host IP\n Using 192.168.99.50 as the server IP\n-- Checking service catalog version requirements ... \n-- Starting OpenShift container ... \n\n Starting OpenShift using container 'origin'\n Waiting for API server to start listening\n OpenShift server started\n-- Registering template service broker with service catalog ... \nFAIL\n Error: cannot register the template service broker\n Caused By:\n Error: cannot create objects from template openshift-infra/template-service-broker-registration\n Caused By:\n Error: unable to recognize servicecatalog.k8s.io/v1beta1, Kind=ClusterServiceBroker: no matches for servicecatalog.k8s.io/, Kind=ClusterServiceBroker",
"stdout_lines": [
"-- Checking OpenShift client ... ",
"-- Checking Docker client ... ",
"-- Checking Docker version ... ",
"-- Checking for existing OpenShift container ... ",
"-- Checking for openshift/origin:v3.9.0-alpha.3 image ... ",
"-- Checking Docker daemon configuration ... ",
"-- Checking for available ports ... ",
"-- Checking type of volume mount ... ",
"",
" Using nsenter mounter for OpenShift volumes",
"-- Creating host directories ... ",
"-- Finding server IP ... ",
"",
" Using public hostname IP 192.168.99.50 as the host IP",
" Using 192.168.99.50 as the server IP",
"-- Checking service catalog version requirements ... ",
"-- Starting OpenShift container ... ",
"",
" Starting OpenShift using container 'origin'",
" Waiting for API server to start listening",
" OpenShift server started",
"-- Registering template service broker with service catalog ... ",
"FAIL",
" Error: cannot register the template service broker",
" Caused By:",
" Error: cannot create objects from template openshift-infra/template-service-broker-registration",
" Caused By:",
" Error: unable to recognize servicecatalog.k8s.io/v1beta1, Kind=ClusterServiceBroker: no matches for servicecatalog.k8s.io/, Kind=ClusterServiceBroker"
]
}
When we install jenkins role top of openshift where identity provider htpasswd
is not installed, then we get this error :
TASK [install_jenkins : Get patch file] ***********************************************************************************************************************************************************************************************************
fatal: [192.168.99.50]: FAILED! => {"changed": true, "cmd": ["cat", "/tmp/htpwd_ip_patch.json"], "delta": "0:00:00.004362", "end": "2018-05-07 08:10:05.887029", "msg": "non-zero return code", "rc": 1, "start": "2018-05-07 08:10:05.882667", "stderr": "cat: /tmp/htpwd_ip_patch.json: No such file or directory", "stderr_lines": ["cat: /tmp/htpwd_ip_patch.json: No such file or directory"], "stdout": "", "stdout_lines": []}
Persistent volumes exist but jenkins role fails
oc get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pv001 5Gi RWO Recycle Bound openshift-ansible-service-broker/etcd 2h
pv002 5Gi RWO Recycle Available 2h
pv003 5Gi RWO Recycle Available 2h
pv004 5Gi RWO Recycle Available 2h
pv005 5Gi RWO Recycle Available 2h
pv006 5Gi RWO Recycle Available 2h
pv007 5Gi RWO Recycle Available 2h
pv008 5Gi RWO Recycle Available 2h
pv009 5Gi RWO Recycle Bound infra/jenkins 2h
pv010 5Gi RWO Recycle Available 2h
Error
TASK [install_jenkins : Install jenkins-persistent] ***********************************************************************************************************************************************************************************************
fatal: [192.168.99.50]: FAILED! => {"changed": true, "cmd": ["oc", "new-app", "JENKINS_PASSWORD=admin123", "jenkins-persistent", "-n", "infra"], "delta": "0:00:00.454552", "end": "2018-05-07 09:11:15.808450", "msg": "non-zero return code", "rc": 1, "start": "2018-05-07 09:11:15.353898", "stderr": " error: routes.route.openshift.io \"jenkins\" already exists\n error: persistentvolumeclaims \"jenkins\" already exists\n error: deploymentconfigs.apps.openshift.io \"jenkins\" already exists\n error: serviceaccounts \"jenkins\" already exists\n error: rolebindings.authorization.openshift.io \"jenkins_edit\" already exists\n error: services \"jenkins-jnlp\" already exists\n error: services \"jenkins\" already exists", "stderr_lines": [" error: routes.route.openshift.io \"jenkins\" already exists", " error: persistentvolumeclaims \"jenkins\" already exists", " error: deploymentconfigs.apps.openshift.io \"jenkins\" already exists", " error: serviceaccounts \"jenkins\" already exists", " error: rolebindings.authorization.openshift.io \"jenkins_edit\" already exists", " error: services \"jenkins-jnlp\" already exists", " error: services \"jenkins\" already exists"], "stdout": "--> Deploying template \"openshift/jenkins-persistent\" to project infra\n\n Jenkins\n ---------\n
Jenkins service, with persistent storage.\n \n NOTE: You must have persistent volumes available in your cluster to use this template.\n\n A Jenkins service has been created in your project. Log into Jenkins with your OpenShift account. The tutorial at https://github.com/openshift/origin/blob/master/examples/jenkins/README.md contains more information about using this template.\n\n * With parameters:\n * Jenkins Service Name=jenkins\n * Jenkins JNLP Service Name=jenkins-jnlp\n * Enable OAuth in Jenkins=true\n * Memory Limit=512Mi\n * Volume Capacity=1Gi\n * Jenkins ImageStream Namespace=openshift\n * Jenkins ImageStreamTag=jenkins:2\n\n--> Creating resources ...\n--> Failed", "stdout_lines": ["--> Deploying template \"openshift/jenkins-persistent\" to project infra", "", " Jenkins", " ---------", " Jenkins service, with persistent storage.", " ", " NOTE: You must have persistent volumes available in your cluster to use this template.", "", " A Jenkins service has been created in your project.
Log into Jenkins with your OpenShift account. The tutorial at https://github.com/openshift/origin/blob/master/examples/jenkins/README.md contains more information about using this template.", "", " * With parameters:", " * Jenkins Service Name=jenkins", " * Jenkins JNLP Service Name=jenkins-jnlp", " * Enable OAuth in Jenkins=true", " * Memory Limit=512Mi", " * Volume Capacity=1Gi", " * Jenkins ImageStream Namespace=openshift", " * Jenkins ImageStreamTag=jenkins:2", "", "--> Creating resources ...", "--> Failed"]}
Since Ansible 2.4, it is possible to use the oc module. Then I propose that we study the idea to use it instead of installing our own oc
client which is next used by our playbooks. That could resolve the issue on minishift
where it is more difficult to install a package within the centos or boot2docker image
Since Ansible 2.4, we can adopt this new convention as we can :
Define the task to be used from the role and then have one role to by example, install, uninstall
Pass condition
Old
---
- hosts: webservers
roles:
- { role: foo, tags: ["bar", "baz"] }
---
- hosts: webservers
tasks:
- import_role:
name: foo
tags:
- bar
- baz
When we nexus
role more than once (due to time out during step to configure nexus), then the following error will be reported
TASK [install_nexus : Enable persistence] ***********************************************************************************************************************************************************************************************************
fatal: [192.168.99.50]: FAILED! => {"changed": true, "cmd": "oc --config /etc/origin/master/admin.kubeconfig volumes dc/nexus --add --name 'nexus-volume-1' --type 'pvc' --mount-path '/sonatype-work/' --claim-name 'nexus-pv' --claim-size '5G' --overwrite", "delta": "0:00:00.300711", "end": "2018-05-09 18:51:21.346746", "msg": "non-zero return code", "rc": 1, "start": "2018-05-09 18:51:21.046035", "stderr": "error: persistentvolumeclaims \"nexus-pv\" already exists\ninfo: deploymentconfigs \"nexus\" was not changed", "stderr_lines": ["error: persistentvolumeclaims \"nexus-pv\" already exists", "info: deploymentconfigs \"nexus\" was not changed"], "stdout": "", "stdout_lines": []}
to retry, use: --limit @/Users/dabou/Code/snowdrop/cloud-native/lab/tmp/openshift-infra/ansible/playbook/post_installation.retry
This error occurs as nexus has already been installed and pvc/pv mounted
The playbook could be configurable
We still have internal playbooks that we could integrate here as they aren't specific to the Red Hat infrastructure and could be used to create/delete openstack VM, ....
Playbooks :
Add an Ansible parameter to install nexus role using either persistence or not
Generate inventory template doesn't allow to create inventory file for local deployment
and instructions doesn't mention to generate the inventory file
For HOL, it is required to create OpenShift's user, project and add them admin role (see remark hereafter nevertheless)
We have created a role to change the identityProvider
of Openshift to become httpPassword
[1] but we don't have anymore a role to create for each's user, its OpenShift's project and assign it to the role specified
So, I propose to split the existing role into 2 and that we create a new role
Role 1 : Install httpd-tools package if not there, create admin
user, patch master-config
to HTPasswdPasswordIdentityProvider
, restart cluster
Role 2 : Create htpasswd user's /password from a list OR using range user1 .... user99
Role 3: Create from a list or a range of users, an openshift project and assign it a role.
e.g
oc login -u {{ user }} -p pwd{{ pwd }}
oc new-project {{ user }}
oc login -u admin -p admin
oc adm policy add-role-to-user admin user
Remarks:
admin
as role for the moment but long term, we should certainly revisit that to give less rights on the machine.[1] https://goo.gl/cW1ChU
[2] https://docs.openshift.com/enterprise/3.2/admin_guide/limits.html#admin-guide-limits
For 3.7 and below, you need to do some manual preparation steps and
then the playbook you want to run is:
openshift-ansible/playbooks/byo/config.yml
For 3.9 (when the rpms will be ready)
Prerequisites: https://docs.openshift.org/latest/install_config/install/prerequisites.html
Host prep: https://docs.openshift.org/latest/install_config/install/host_preparation.html
I think some of the items in those pages are already done on atomic
host (such as installing docker).
Interesting project : https://github.com/michaelgugino/openshift-stuff/tree/master/centos
Currently when an all-in-one cluster is deployed via openshift-ansible, Openshift will be able deploy any pods, reporting 0/1 nodes are available: 1 MatchNodeSelector
.
The solution is probably to label the all-in-one
node
When we create by example the infra
project, our oc command is executed using the current logged user which has perhaps or not the appropriate role to create a project, configMap, serviceaccount ....
Nevertheless, if the role linked to the user is the one used, that means that he/she will be able to manage the content of the infra
project. This is not an use for the admin user which is cluster
wised but this is a problem for demo's users (user1, user2, ....)
To secure our platform in that case, the following parameter should be passed to the oc
command when a resource is created/deleted or edited
oc --config={{ openshift.common.config_base }}/admin.kubeconfig
where {{ openshift.common.config_base }} could be : /etc/origin/master
If we create the 'infra' project as such
- name: Create project
command: oc --config=/etc/origin/master/admin.kubeconfig new-project {{ infra_project }}
then the user can't access content of infra folder
Enhance inventory_cloud j2 template to support deployment on openstack vs local, hetzner.
A few modifications are required :
a) openshift_hostname
openshift_hostname=node_name -> openshift_hostname=ip
Should be good to see if we can also define a hostname for local, hetzner deployments.
If we use the Centos ISO created, then hostname
command executed on the terminal of the vm returns : cloud
and for hetzner
-> CentOS-74-64-minimal
We should perhaps specify it as a name
b) Node, master, etcd
[masters]
10.8.250.104 openshift_public_hostname=10.8.250.104 openshift_hostname=172.16.195.12
Change To : -->
192.168.99.50 openshift_public_hostname=192.168.99.50 openshift_ip=192.168.99.50
[etcd]
10.8.250.104
Change To : -->
192.168.99.50 openshift_ip=192.168.99.50
[nodes]
10.8.250.104 openshift_node_labels="{'region':'infra','zone':'default', 'node-role.kubernetes.io/compute': 'true'}" \
openshift_public_hostname=10.8.250.104 \
openshift_hostname=172.16.195.12
Change To : -->
192.168.99.50 openshift_node_labels="{'region':'infra','zone':'default', 'node-role.kubernetes.io/compute': 'true'}" \
openshift_public_hostname=192.168.99.50 \
openshift_ip=192.168.99.50
See diff file
Externalize config directory as it is different between oc cluster up
, mininishift
, ....
By example, the jenkins role uses a hard coded reference which is different from the directory used by minishift or oc cluster up
- name: Set config file
set_fact:
config_file: /etc/origin/master/master-config.yaml
- name: Update configuration file
shell: |
echo "jenkinsPipelineConfig:" >> {{ config_file }}
echo " autoProvisionEnabled: false" >> {{ config_file }}
oc get pods
NAME READY STATUS RESTARTS AGE
jaeger-210917857-x84ht 1/1 Running 0 19m
jenkins-1-jz6z8 1/1 Running 0 19m
nexus-1-cn6vg 1/1 Running 0 22m
oc rsh nexus-1-cn6vg
and within the bash shell
1) Fail
curl http://nexus.infra.svc:8081
curl: (6) Could not resolve host: nexus.infra.svc; Unknown error
2) Succeeded
sh-4.2$ curl http://172.17.0.2:8081
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 Not Found</title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /. Reason:
<pre> Not Found</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>
Convert install_package.yml
playbook to become a role in order to:
TASK [Gathering Facts] *******************************************************************************************************************************************************************************************************************************************
fatal: [10.8.241.7]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\r\n@ WARNING: UNPROTECTED PRIVATE KEY FILE! @\r\n@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\r\nPermissions 0644 for 'inventory/id_openstack.rsa' are too open.\r\nIt is required that your private key files are NOT accessible by others.\r\nThis private key will be ignored.\r\nLoad key \"inventory/id_openstack.rsa\": bad permissions\r\[email protected]: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).\r\n", "unreachable": true}
Have you committed something about that @geoand ? If yes, can you add the sha commit and close the ticket ?
Create 2 inventory files + templates
inventory
file + templateBut when when we will use oc cluster up
then a lightweight inventory is required with few vars + template too
Steps to reproduce the error
git clone https://github.com/snowdrop/openshift-infra.git
cd openshift-infra/ansible
git clone -b release-3.9 https://github.com/openshift/openshift-ansible.git
ansible-playbook -i inventory/cloud_host openshift-ansible/playbooks/prerequisites.yml
ansible-playbook -i inventory/cloud_host openshift-ansible/playbooks/deploy_cluster.yml
ansible-playbook -i inventory/cloud_host playbook/post_installation.yml -e openshift_admin_pwd=admin --tags enable_cluster_admin
ansible-playbook -i inventory/cloud_host playbook/post_installation.yml -e openshift_admin_pwd=admin --tags identity_provider
ansible-playbook -i inventory/cloud_host playbook/post_installation.yml --tags add_extra_users -e number_of_extra_users=2 -e first_extra_user_offset=1 -e openshift_admin_pwd=admin
Error
TASK [add_extra_users : Grant user admin priviledges] *****************************************************************************************************************************************************************************************
failed: [192.168.99.50] (item=1) => {"changed": true, "cmd": ["oc", "adm", "policy", "add-role-to-user", "admin", "user1"], "delta": "0:00:00.215537", "end": "2018-05-09 16:20:15.113077", "item": "1", "msg": "non-zero return code", "rc": 1, "start": "2018-05-09 16:20:14.897540", "stderr": "Error from server (Forbidden): rolebindings.authorization.openshift.io is forbidden: User \"admin\" cannot list rolebindings.authorization.openshift.io in the namespace \"default\": User \"admin\" cannot list rolebindings.authorization.openshift.io in project \"default\"", "stderr_lines": ["Error from server (Forbidden): rolebindings.authorization.openshift.io is forbidden: User \"admin\" cannot list rolebindings.authorization.openshift.io in the namespace \"default\": User \"admin\" cannot list rolebindings.authorization.openshift.io in project \"default\""], "stdout": "", "stdout_lines": []}
failed: [192.168.99.50] (item=2) => {"changed": true, "cmd": ["oc", "adm", "policy", "add-role-to-user", "admin", "user2"], "delta": "0:00:00.239512", "end": "2018-05-09 16:20:15.705065", "item": "2", "msg": "non-zero return code", "rc": 1, "start": "2018-05-09 16:20:15.465553", "stderr": "Error from server (Forbidden): rolebindings.authorization.openshift.io is forbidden: User \"admin\" cannot list rolebindings.authorization.openshift.io in the namespace \"default\": User \"admin\" cannot list rolebindings.authorization.openshift.io in project \"default\"", "stderr_lines": ["Error from server (Forbidden): rolebindings.authorization.openshift.io is forbidden: User \"admin\" cannot list rolebindings.authorization.openshift.io in the namespace \"default\": User \"admin\" cannot list rolebindings.authorization.openshift.io in project \"default\""], "stdout": "", "stdout_lines": []}
The OpenShift role assigned by our role add-to-users
is hard coded to admin
- name: Grant user admin priviledges
command: oc adm policy add-role-to-user admin user{{ item }}
with_sequence: start={{ first_extra_user_offset }} count={{ number_of_extra_users }} format=%d
As this role could be too "high" for users accessing the hetzner machine, I suggest to do 2 things
edit
role could be enoughFYI
Openshift's roles available are
oc describe clusterrole.rbac | grep Name:
Name: admin
Name: asb-access
Name: asb-auth
Name: basic-user
Name: cluster-admin
Name: cluster-debugger
Name: cluster-reader
Name: cluster-status
Name: edit
...
Name: view
Definition of the edit's role is
oc describe clusterrole.rbac/edit
Name: edit
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: openshift.io/description=A user that can create and edit most objects in a project, but can not update the project's membership.
rbac.authorization.kubernetes.io/autoupdate=true
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
appliedclusterresourcequotas [] [] [get list watch]
appliedclusterresourcequotas.quota.openshift.io [] [] [get list watch]
bindings [] [] [get list watch]
buildconfigs [] [] [create delete deletecollection get list patch update watch]
buildconfigs.build.openshift.io [] [] [create delete deletecollection get list patch update watch]
buildconfigs/instantiate [] [] [create]
buildconfigs.build.openshift.io/instantiate [] [] [create]
buildconfigs/instantiatebinary [] [] [create]
buildconfigs.build.openshift.io/instantiatebinary [] [] [create]
buildconfigs/webhooks [] [] [create delete deletecollection get list patch update watch]
buildconfigs.build.openshift.io/webhooks [] [] [create delete deletecollection get list patch update watch]
buildlogs [] [] [create delete deletecollection get list patch update watch]
buildlogs.build.openshift.io [] [] [create delete deletecollection get list patch update watch]
builds [] [] [create delete deletecollection get list patch update watch]
builds.build.openshift.io [] [] [create delete deletecollection get list patch update watch]
builds/clone [] [] [create]
builds.build.openshift.io/clone [] [] [create]
builds/details [] [] [update]
builds.build.openshift.io/details [] [] [update]
builds/log [] [] [get list watch]
builds.build.openshift.io/log [] [] [get list watch]
configmaps [] [] [create delete deletecollection get list patch update watch]
cronjobs.batch [] [] [create delete deletecollection get list patch update watch]
daemonsets.apps [] [] [get list watch]
daemonsets.extensions [] [] [get list watch]
deploymentconfigrollbacks [] [] [create]
deploymentconfigrollbacks.apps.openshift.io [] [] [create]
deploymentconfigs [] [] [create delete deletecollection get list patch update watch]
deploymentconfigs.apps.openshift.io [] [] [create delete deletecollection get list patch update watch]
deploymentconfigs/instantiate [] [] [create]
deploymentconfigs.apps.openshift.io/instantiate [] [] [create]
deploymentconfigs/log [] [] [get list watch]
deploymentconfigs.apps.openshift.io/log [] [] [get list watch]
deploymentconfigs/rollback [] [] [create]
deploymentconfigs.apps.openshift.io/rollback [] [] [create]
deploymentconfigs/scale [] [] [create delete deletecollection get list patch update watch]
deploymentconfigs.apps.openshift.io/scale [] [] [create delete deletecollection get list patch update watch]
deploymentconfigs/status [] [] [get list watch]
deploymentconfigs.apps.openshift.io/status [] [] [get list watch]
deployments.apps [] [] [create delete deletecollection get list patch update watch]
deployments.extensions [] [] [create delete deletecollection get list patch update watch]
deployments.apps/rollback [] [] [create delete deletecollection get list patch update watch]
deployments.extensions/rollback [] [] [create delete deletecollection get list patch update watch]
deployments.apps/scale [] [] [create delete deletecollection get list patch update watch]
deployments.extensions/scale [] [] [create delete deletecollection get list patch update watch]
endpoints [] [] [create delete deletecollection get list patch update watch]
events [] [] [get list watch]
horizontalpodautoscalers.autoscaling [] [] [create delete deletecollection get list patch update watch]
imagestreamimages [] [] [create delete deletecollection get list patch update watch]
imagestreamimages.image.openshift.io [] [] [create delete deletecollection get list patch update watch]
imagestreamimports [] [] [create]
imagestreamimports.image.openshift.io [] [] [create]
imagestreammappings [] [] [create delete deletecollection get list patch update watch]
imagestreammappings.image.openshift.io [] [] [create delete deletecollection get list patch update watch]
imagestreams [] [] [create delete deletecollection get list patch update watch]
imagestreams.image.openshift.io [] [] [create delete deletecollection get list patch update watch]
imagestreams/layers [] [] [get update]
imagestreams.image.openshift.io/layers [] [] [get update]
imagestreams/secrets [] [] [create delete deletecollection get list patch update watch]
imagestreams.image.openshift.io/secrets [] [] [create delete deletecollection get list patch update watch]
imagestreams/status [] [] [get list watch]
imagestreams.image.openshift.io/status [] [] [get list watch]
imagestreamtags [] [] [create delete deletecollection get list patch update watch]
imagestreamtags.image.openshift.io [] [] [create delete deletecollection get list patch update watch]
ingresses.extensions [] [] [create delete deletecollection get list patch update watch]
jenkins.build.openshift.io [] [] [edit view]
jobs.batch [] [] [create delete deletecollection get list patch update watch]
limitranges [] [] [get list watch]
namespaces [] [] [get list watch]
namespaces/status [] [] [get list watch]
networkpolicies.extensions [] [] [create delete deletecollection get list patch update watch]
networkpolicies.networking.k8s.io [] [] [create delete deletecollection get list patch update watch]
persistentvolumeclaims [] [] [create delete deletecollection get list patch update watch]
poddisruptionbudgets.policy [] [] [create delete deletecollection get list patch update watch]
podpresets.settings.k8s.io [] [] [create update delete get list watch]
pods [] [] [create delete deletecollection get list patch update watch]
pods/attach [] [] [create delete deletecollection get list patch update watch]
pods/exec [] [] [create delete deletecollection get list patch update watch]
pods/log [] [] [get list watch]
pods/portforward [] [] [create delete deletecollection get list patch update watch]
pods/proxy [] [] [create delete deletecollection get list patch update watch]
pods/status [] [] [get list watch]
processedtemplates [] [] [create delete deletecollection get list patch update watch]
processedtemplates.template.openshift.io [] [] [create delete deletecollection get list patch update watch]
projects [] [] [get]
projects.project.openshift.io [] [] [get]
replicasets.apps [] [] [create delete deletecollection get list patch update watch]
replicasets.extensions [] [] [create delete deletecollection get list patch update watch]
replicasets.apps/scale [] [] [create delete deletecollection get list patch update watch]
replicasets.extensions/scale [] [] [create delete deletecollection get list patch update watch]
replicationcontrollers [] [] [create delete deletecollection get list patch update watch]
replicationcontrollers/scale [] [] [create delete deletecollection get list patch update watch]
replicationcontrollers.extensions/scale [] [] [create delete deletecollection get list patch update watch]
replicationcontrollers/status [] [] [get list watch]
resourcequotas [] [] [get list watch]
resourcequotas/status [] [] [get list watch]
resourcequotausages [] [] [get list watch]
routes [] [] [create delete deletecollection get list patch update watch]
routes.route.openshift.io [] [] [create delete deletecollection get list patch update watch]
routes/custom-host [] [] [create]
routes.route.openshift.io/custom-host [] [] [create]
routes/status [] [] [get list watch]
routes.route.openshift.io/status [] [] [get list watch]
secrets [] [] [create delete deletecollection get list patch update watch]
serviceaccounts [] [] [create delete deletecollection get list patch update watch impersonate]
servicebindings.servicecatalog.k8s.io [] [] [create update delete get list watch patch]
serviceinstances.servicecatalog.k8s.io [] [] [create update delete get list watch patch]
services [] [] [create delete deletecollection get list patch update watch]
services/proxy [] [] [create delete deletecollection get list patch update watch]
statefulsets.apps [] [] [create delete deletecollection get list patch update watch]
templateconfigs [] [] [create delete deletecollection get list patch update watch]
templateconfigs.template.openshift.io [] [] [create delete deletecollection get list patch update watch]
templateinstances [] [] [create delete deletecollection get list patch update watch]
templateinstances.template.openshift.io [] [] [create delete deletecollection get list patch update watch]
templates [] [] [create delete deletecollection get list patch update watch]
templates.template.openshift.io [] [] [create delete deletecollection get list patch update watch]
Can you add the commit sha here and next close the ticket as you have implemented something @geoand ?
Step
ansible-playbook -i inventory/cloud_host playbook/post_installation.yml \
--tags install-launcher \
-e launcher_catalog_git_repo=https://github.com/snowdrop/cloud-native-catalog.git \
-e launcher_catalog_git_branch=master \
-e launcher_github_username=YOUR_GIT_TOKEN \
-e launcher_github_token=YOUR_GIT_USER
Error
TASK [openshift_env : Find yml files in {{ item }}] *************************************************************************************************************************************************************************************************
fatal: [192.168.99.50]: FAILED! => {"reason": "Unable to retrieve file contents\nCould not find or access '/Users/dabou/Code/snowdrop/cloud-native/lab/tmp/openshift-infra/ansible/playbook/determine_is_openshift_config_dir.yml'"}
fatal: [192.168.99.50]: FAILED! => {"reason": "Unable to retrieve file contents\nCould not find or access '/Users/dabou/Code/snowdrop/cloud-native/lab/tmp/openshift-infra/ansible/playbook/determine_is_openshift_config_dir.yml'"}
Steps executed
git clone https://github.com/snowdrop/openshift-infra.git
cd openshift-infra/ansible
ansible-playbook playbook/generate_inventory.yml -e ip_address=192.168.99.50 -e type=simple
ansible-playbook -i inventory/simple_host playbook/cluster.yml -e openshift_release_tag_name=v3.9.0 --tags "up"
ansible-playbook -i inventory/simple_host playbook/post_installation.yml -e openshift_admin_pwd=admin --tags "enable_cluster_admin"
ansible-playbook -i inventory/simple_host playbook/post_installation.yml --tags jenkins
Error
TASK [install_jenkins : Get Jenkins Service Account Token] *************************************************************************************************************
fatal: [192.168.99.50]: FAILED! => {"changed": true, "cmd": ["oc", "serviceaccounts", "get-token", "jenkins"], "delta": "0:00:00.205584", "end": "2018-05-08 11:55:32.617959", "msg": "non-zero return code", "rc": 1, "start": "2018-05-08 11:55:32.412375", "stderr": "error: could not find a service account token for service account \"jenkins\"", "stderr_lines": ["error: could not find a service account token for service account \"jenkins\""], "stdout": "", "stdout_lines": []}
Refactor the playbook add-users with role identity_provider in order to :
add_users.yml
playbook to a roleidentity_provider
the task to install htpasswd
package if not there like the steps to create the user/passwords to the new roleA declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.