Hi
molecule testing is showing the following error:
TASK [time : Install packages] ************************************************* fatal: [instance]: FAILED! => {"changed": false, "failures": ["No package chrony available."], "msg": "Failed to install some of the specified packages", "rc": 1, "results": []}

Diskless doc and tools must largely be improved.

In Multi Iceberg, an Ansible role to deploy Ansible itself is needed, and must be added into core.

Tumulus cluster is multi arch (arm64 and x86_64) and also multi distribs.

Check the stacks handle this properly.

Need to finish report role:

• plugins
• remove ansible-inventory-grapher and ansible-playbook-grapher
• add more patterns
• finish network graph generation
• add system map and racks map

Hi,

It is possible to add a feature allowing to access an external repos ?
For exemple : EPEL, Erratas, or others

internal_repo:
 - os
 - repo1

external_repo:
  - { name: epel, url: ... }
  - { name: errata, url: ... }

Best regards,

Missing equipment_naming support in pxe_stack task
When condition simplification for RH / CentOS test

PR submitted : #24

Hey ,

I'm trying to use bluebanquise to deploy a cluster stack. I'm not interested in using external servers at all. So I just removed values from example in my inventory. The file (/etc/ansible/inventory/gorup_vars/general_settings/external.yml) is like this :

external_time:
  time_server:
    server: # List of possible time servers
    pool: # List of possible time pools
  time_client:
    server: 
    pool: 

external_dns:
  dns_server: # set as forwarders in named.conf
  dns_client: # set directly on client side in resolv.conf

# Hosts defined here will be automatically added into /etc/hosts and into DNS configuration
external_hosts:

Then I got the following error on several roles (at least hostsfile, dns_client, dns_server) :

AnsibleError: Unexpected templating type error occurred on (#### Blue Banquise file ####
): 'NoneType' object is not iterable

It feels like templates need to check if lists length is more than 0.

Mathis

Template should prevent non server hosts to be included, and do not filter right now.

Should add:

{% for host in range %}
{% if hostvars[host]['equipment_profile']['equipment_type'] == "server" %}
...
{% endif %}
{% endfor %}

Hi,

The internal var "equipment_naming is not taken by the pxe_stack role.

PR submitted for RH/CentOS part : #22

This correction should be applied on other distributions

Issue to track the work to implement CI/CD unit tests of the roles with Molecule. Please refer to the instructions below. You can use PR #242 as an example.

Core roles

• bluebanquise (#221)
• conman (#170) (#463)
• dhcp_server (#413)
• display_tuning (#175)
• dns_client (#175)
• dns_server (#162)
• hosts_file (Cannot be tested with molecule, see: #462)
• log_client (#187)
• log_server (#187)
• nfs_client (#412)
• nfs_server (#412)
• nic (#512)
• pxe_stack (#439)
• repositories_client (#171)
• repositories_server (#462)
• set_hostname (#511)
• ssh_master (#242)
• ssh_slave (#462)
• time (#162)

Advanced core roles

• advanced_dhcp_server (#428)
• advanced_dns_server (#438)

Addons roles

• clone
• clustershell (#174)
• diskless
• nic_nmcli
• ofed
• ofed_sm
• openldap_client
• openldap_server
• prometheus_client (#430)
• prometheus_server (#430)
• report
• slurm
• users_basic (#174)

How to

For developers, install virtualenv in your environment:

$ pip install --user virtualenv

At the root of your git clone, create a new virtual environment in directory venv and activate it:

$ virtualenv venv-molecule
$ source venv-molecule/bin/activate
(venv-molecule) $ 

Install the required modules in the virtual environment:

(venv-molecule) $ pip install 'molecule[docker]>3' ansible-lint flake8

Change your current directory to the role you want to update and initialize the default molecule scenario:

(venv-molecule) $ cd roles/core/conman/
(venv-molecule) $ molecule init scenario --role-name conman
--> Initializing new scenario default...
Initialized scenario in /git/bluebanquise/roles/core/conman/molecule/default successfully.

Tune, commit, test, open a PR.

Once completed, deactivate your virtual environment:

(venv-molecule) $ deactivate 
$ 

Note: If you run Fedora 31, Docker will not run by default due to switch to cgroups v2, however it's still possible to run most of the tests locally using the podman driver (molecule create -d podman).

I'm concerned by the F821 undefined name 'node'.

$ flake8 . --statistics --ignore E501,E226
./roles/addons/diskless/files/disklessset.py:17:1: F401 'shutil.copy2' imported but unused
./roles/addons/diskless/files/disklessset.py:20:1: F401 'pwd' imported but unused
./roles/addons/diskless/files/disklessset.py:21:1: F401 'grp' imported but unused
./roles/addons/diskless/files/disklessset.py:22:1: F401 're' imported but unused
./roles/addons/diskless/files/disklessset.py:23:1: F401 'hashlib' imported but unused
./roles/addons/diskless/files/disklessset.py:25:1: F401 'base64.urlsafe_b64encode as encode' imported but unused
./roles/addons/diskless/files/disklessset.py:26:1: F401 'base64.urlsafe_b64decode as decode' imported but unused
./roles/addons/diskless/files/disklessset.py:27:1: F401 'getpass.getpass' imported but unused
./roles/addons/diskless/files/disklessset.py:405:78: F821 undefined name 'node'
8     F401 'shutil.copy2' imported but unused
1     F821 undefined name 'node'

Bluebanquise stack uses the same IP address in pxe chain boot for tftp service and http service: The pxe_ip variable in network definition.
networks:
ice1-1:
subnet: 10.10.0.0
prefix: 16
netmask: 255.255.0.0
broadcast: 10.10.255.255
dhcp_unknown_range: 10.10.254.1 10.10.254.254
gateway: 10.10.255.254
is_in_dhcp: true
is_in_dns: true
services_ip:
pxe_ip: 10.10.0.1

This is working fine when services are running on the same host/VM/container and so exposing the same IP address for both services.
When deploying services like dhcp, tftp, http, dns etc. inside containers running on various bare metal nodes, orchestrated bu Kubernetes, we have a service IP address per service. For that reason, we need to have a pxe chain boot solution for which we can configure a tftp_ip address and a pxe_ip address for respectively access to TFTP server and HTTP server.

Hi,

Can you provide a more reliable IPXE (mainly for the DHCP part) ? It works perfectly with good switches, but when it comes to old switches, the interface configuration command hangs out and timeout after 5 retries.

File : bluebanquise_standard.ipxe

Command to be changed :
ifconf --configurator dhcp || shell

Exemple provided by the IPXE website :
:retry
ifconf --configurator dhcp || goto retry

or

:retry
ifconf -c dhcp && isset ${filename} || goto retry

40 {% endif %}
41
42  include "/etc/dhcp/dhcpd.{{network}}.conf";
43
44 }

please add '}' after line 44

Molecule testing not working on ubuntu18.04:
TASK [time : Install packages] ************************************************* fatal: [instance]: FAILED! => {"changed": false, "msg": "No package matching 'chrony' is available"}

When upgrading tumulus cluster to 1.2.0, we found an issue with SELinux on logins nodes with /home for uses.

A boolean must be set.

In a kubernetes context, the DNS server is started in a container and only knows the kubernetes network (not managed by bluebanquise but by kubernetes)
In this case, we must delete the listen-on and allow_query section and keep the default behavior to be able to function

for example

cat /etc/ansible/kubernetes-dns.patch
 
--- roles/core/dns_server/templates/named.conf.j2.bb    2020-02-05 14:51:21.871668303 +0100
+++ roles/core/dns_server/templates/named.conf.j2       2020-02-05 14:52:41.327668303 +0100
@@ -12,14 +12,14 @@
 {% set main_ntw = (ntw | first | join | trim) %}
 
 options {
-  listen-on port 53 {
-    127.0.0.1;
-{% for network in (networks | select('match',(j2_current_iceberg_network+'-[0-9]+')) | list | unique | sort) %}
-{% if networks[network]['is_in_dns'] is defined and networks[network]['is_in_dns'] == true %}
-    {{networks[network]['services_ip']['dns_ip']}};
-{% endif %}
-{% endfor %}
-  };
+  #listen-on port 53 {
+    #127.0.0.1;
+#{% for network in (networks | select('match',(j2_current_iceberg_network+'-[0-9]+')) | list | unique | sort) %}
+#{% if networks[network]['is_in_dns'] is defined and networks[network]['is_in_dns'] == true %}
+    #{{networks[network]['services_ip']['dns_ip']}};
+#{% endif %}
+#{% endfor %}
+  #};
 
   listen-on-v6 port 53 { ::1; };
   directory    "/var/named";
@@ -27,14 +27,14 @@ options {
   statistics-file "/var/named/data/named_stats.txt";
   memstatistics-file "/var/named/data/named_mem_stats.txt";
 
-  allow-query {
-    localhost;
-{% for network in (networks | select('match',(j2_current_iceberg_network+'-[0-9]+')) | list | unique | sort) %}
-{% if (networks[network]['is_in_dns'] is defined and not none) and (networks[network]['is_in_dns'] == true) %}
-    {{networks[network]['subnet']}}/{{networks[network]['prefix']}};
-{% endif %}
-{% endfor %}
-  };
+  #allow-query {
+    #localhost;
+#{% for network in (networks | select('match',(j2_current_iceberg_network+'-[0-9]+')) | list | unique | sort) %}
+#{% if (networks[network]['is_in_dns'] is defined and not none) and (networks[network]['is_in_dns'] == true) %}
+    #{{networks[network]['subnet']}}/{{networks[network]['prefix']}};
+#{% endif %}
+#{% endfor %}
+  #};
 
 {% if external_dns.dns_server is defined and not none and external_dns.dns_server %}
   forwarders {

https://github.com/oxedions/bluebanquise/blob/48057e5f6173233f9799f9cf1b141600791bf2e2/roles/core/conman/tasks/main.yml#L5

Yo Ox,

As discussed, I will have a look on how to integrate RHEL 8 into BB.
Can you give me the rights, and create the branch for my devs ?
Thanks

JK

Yo Ox,

I found EFI is baldy managed: sanboot in menu.ipxe do not work in EFI, and iPXE usb and iso images do not contains EFI part.

OK if I take care of that ? The CINES SMP node is in EFI and this is broken this way.

Need to build all packages for 1.0, i.e. Centos 7 packages for now.

Also need to add online documentation, and remove build directory from git base.

Missing parts:

• Ansible training do not cover tasks (when, loop, with_items, etc)
• equipment_profiles values are not properly explained in the documentation, and available values are not provided.
• Web site upgrade with new documentation

Create default sample inventory from Tumulus Algoric cluster, replacing MAC and private elements by random ones.

Hi,
i find a new issue.
When I want to add a BMC entry with option82 match, the entry in dhcp file isn't added.
On the other hand, the entry with mac works.

          bmc:
            name: pm0-bmc05
            #mac: 08:00:27:0d:f8:b2
            option_match: |
              substring (option agent.remote-id, 0 , 64) = "0"
              and option agent.circuit-id = 01:00:10
              and (substring(option vendor-class-identifier, 0, 10) = "XXXX S BMC")
            ip4: 10.0.0.106
            network: ice1-1

Kind regards,

Accelerated mode POC must be benchmarked on Tumulus with thousands of hosts.

If performant, developments must be ended.

On many hardware, NIC carrier timeout too fast at dracut, leaving osdeploy or diskless boots orphan of kickstart or image.

A simple dhclient in dracut shell shows ip is easy to get, but dracut timed out too fast.

Need a simple way in bootset to add correct kernel parameters to ask dracut to wait and retry more.

Tumulus has 2 Windows 10 systems.

We are using clonezilla to backup/reinstall. Need support for basic imaging based on clonezilla live.

1.3.0 Milestone.

Hi
It could be great to have tag convention (a simple list of standard tags you are using) in CONTRIBUTING file.

Chrony roles are nearly the same between server and client. Merging them like for slurm would simplify the list.

The ifcfg.j2 template sets the NM_MANAGED flag to yes whatever the operating system installed on the host.
For nodes running in RHEL7, the NM_MANAGED MUST be set to no.

Hi,

I think python scripts (bootset) code should not be include in Ansible templates.

Moving this code to a more standard path and making the code "static" (by declaring your CLI in a setup.py file and remove sources from role templates) could be a great enhancement.

Configuration of the bootset executable can be done through an external configuration file (let's say /etc/bootset/bootset.conf) and templated by Ansible.

Need to finish 1.0 documentation.

Check syntax, check commands, and check URLs.

Value:
SERVER execpath="/usr/share/conman/exec"

Is not adapted to new path of conman.

These 3 lines are not needed and are leading the the dns_server role failing in chroot context execution.

diff -r roles/core/dns_server/templates/forward.j2 roles.bkp/core/dns_server/templates/forward.j212a13,15> @ IN A {{network_interfaces[j2_node_main_network_interface]['ip4']}}>> {{inventory_hostname}} IN A {{network_interfaces[j2_node_main_network_interface]['ip4']}}

Prometheus ADDON role need ability to define alerts on equipment_profiles granularity.

Need to move alerts from file to template, and use Jinja inside with jobs. Need to find how to define an alert for multiple jobs.

Would be also interesting do define a graphana role to bind to Prometheus and Alert Manager.

Hey,

From several weeks now, we're using the amazing Bluebanquise roles to manage our cluster and most of all to customize our overlayfs diskless images.
That allow us to have diskfull and diskless system configured on the same way, which really ease the way of managing clusters.

Some roles are using "service" Ansible task that do not work very well when using chroot ansible connection plugin. In fact it makes services starting in the diskless image chroot where we only expect the service to be enabled. And that can cause some troubles on the node we're generating this diskless.

That would be amazing if Bluebanquise core roles could take care of this need (and I think we need to think about this a little).

Mathis

Documentation about multiple icebergs is missing.

Need to add it.

Hi,

It could be great to have some contribution guidelines (contrib.md, role_skeleton for ansible-galaxy)

Skeleton example (you just need to provide a role directory template somewhere in this repo) : https://github.com/cjsteel/galaxy-role-skeleton
Usage : ansible-galaxy init --role-skeleton=<skeleton_dir> <role_name>

For now, bootset generates output configuration files on the local root file system.
This feature request is a requirement for deployment types who have their services running on a different node than the node hosting locally the configuration files.
As an example, we have dhcp, tftp, http services running inside containers on different hosts (orchestrated by kubernetes for instance). These services need to have access to pxe and repository data files from a shared storage file system.

Hi,

iPXE os deployment is not working on RHEL 7. in fact, inst.repo URL point to a BaseOS folder that is existent in Redhat 8 but not in Redhat 7. It causes pxe installation failure.

Mathis

We need to declare a nic in a new network but we can't set an IP as the IP will be mamanged by kubernetes.
I tried addind a test to skipped the ifcfg generation if no ipv4 is set in the role nic
For example

- name: Set NIC configuration
  template:
    src: ifcfg.j2
    dest: /etc/sysconfig/network-scripts/ifcfg-{{item}}
    owner: root
    group: root
    mode: 0644
  with_items: "{{network_interfaces}}"
  when: (network_interfaces[item]['ip4'] is defined and not none)

without the test, an ifcfg file is generate but with empty parameter.

Where is the nyancat ? Where is the display message ?

Error 'pool declared outside of network', this error is not explicit !

The real issue is in /etc/dhcp/dhcpd.{network}.conf
we can't define a pool in a group

I was reviewing the dns_client and dns_server roles and noticed there is a maximum number of one DNS server per network.

Could we consider adding support for secondary DNS servers?
We can discuss this in this thread then I can push a PR if accepted.

Hi @oxedions,

Just submitted a bootset refactoring PR here : #42

It can be improved, but i think that's a better base for improvment.

TODO :

• OO class
• Add error handling
• More flexibility (we should be able to develop additionnal plugins to add more boot methods)
• Use logging lib instead of print

When deploying services like dhcp, tftp, http, dns etc. inside containers running on various bare metal nodes, we used the ansible "chroot" connector to generate the configuration files on a shared file system mounted on all nodes.
We want to use the standard roles (dhcp, dns, pxe_stack ...) for that purpose, but we do NOT want to start the services on the bare metal management node. So we use the template tag , but we need also for pxe_stack role to create the output directories and to copy the bootswitch cgi script. So, a directory and a script tag are needed for that.

Yo Ox,

I was wondering, how should we organize tasks when dealing with multiple distros ?

You know, with centos_7.yml, ubuntu_18.yml etc files, because for some tasks packages only is not enough :-( you have to deal with paths or services also...

The role time defines both server and pool in chrony configuration if both are defined in the inventory.

[root@management1 ~]# head -6 /etc/chrony.conf 
#### Blue Banquise file ####
## Ansible managed: modified on 2020-04-07 03:52:27 by root on management1

# External servers/pool if asked
server 0.fr.pool.ntp.org
pool pool.ntp.org iburst maxsources 3

We should change this behaviour to define either server(s) or pool. I would default to pool.

See:

• https://chrony.tuxfamily.org/faq.html#_what_is_the_minimum_recommended_configuration_for_an_ntp_client
• https://help.fortinet.com/coyotepoint/10-3-2/Content/NetConfig/Selecting_an_NTP_Server.htm

Hi,

i submitted a PR (see #31) containing the following changes :

Major changes :

• Add boolean to disable all services (HA)
• Implement tags + tag uniformisation (package, template, service)
• Implement handlers

Minor changes :

• Use new package module syntax (using list in name instead of loops)
• Replace some "with_X" by loop
• Minor fixes

The issue #19 can also be updated (contribution guidelines should be updated before closing)