- Overview - What is the PuppetDB module?
- Module Description - What does the module do?
- Setup - The basics of getting started with PuppetDB module
- Upgrading - Guide for upgrading from older revisions of this module
- Usage - The classes and parameters available for configuration
- Implementation - An under-the-hood peek at what the module is doing
- Limitations - OS compatibility, etc.
- Development - Guide for contributing to the module
- Release Notes - Notes on the most recent updates to the module
By guiding puppetdb setup and configuration with a Puppet master, the PuppetDB module provides fast, streamlined access to data on puppetized infrastructure.
The PuppetDB module provides a quick way to get started using PuppetDB, an open source inventory resource service that manages storage and retrieval of platform-generated data. The module will install PostgreSQL and PuppetDB if you don't have them, as well as set up the connection to Puppet master. The module will also provide a dashboard you can use to view the current state of your system.
For more information about PuppetDB please see the official PuppetDB documentation.
What PuppetDB affects:
- package/service/configuration files for PuppetDB
- package/service/configuration files for PostgreSQL (optional, but set as default)
- Puppet master's runtime (via plugins)
- Puppet master's configuration
- note: Using the
puppetdb::master::configclass will cause your routes.yaml file to be overwritten entirely (see Usage below for options and more information )
- note: Using the
- system firewall (optional)
- listened-to ports
Introductory Questions
To begin using PuppetDB, you’ll have to make a few decisions:
- Should I run the database on the same node that I run PuppetDB on?
- Should I run PuppetDB on the same node that I run my master on?
The answers to those questions will be largely dependent on your answers to questions about your Puppet environment:
- How many nodes are you managing?
- What kind of hardware are you running on?
- Is your current load approaching the limits of your hardware?
Depending on your answers to all of the questions above, you will likely fall under one of these set-up options:
This approach assumes you will use our default database (PostgreSQL) and run everything (PostgreSQL, PuppetDB, Puppet master) all on the same node. This setup will be great for a testing or experimental environment. In this case, your manifest will look like:
node <hostname> {
# Configure puppetdb and its underlying database
class { 'puppetdb': }
# Configure the Puppet master to use puppetdb
class { 'puppetdb::master::config': }
}
You can provide some parameters for these classes if you’d like more control, but that is literally all that it will take to get you up and running with the default configuration.
This approach is for those who prefer not to install PuppetDB on the same node as the Puppet master. Your environment will be easier to scale if you are able to dedicate hardware to the individual system components. You may even choose to run the puppetdb server on a different node from the PostgreSQL database that it uses to store its data. So let’s have a look at what a manifest for that scenario might look like:
This is an example of a very basic 3-node setup for PuppetDB.
$puppetdb_host = 'puppetdb.example.lan'
$postgres_host = 'postgres.example.lan'
node 'master.example.lan' {
# Here we configure the Puppet master to use PuppetDB,
# telling it the hostname of the PuppetDB node
class { 'puppetdb::master::config':
puppetdb_server => $puppetdb_host,
}
}
node 'postgres.example.lan' {
# Here we install and configure PostgreSQL and the PuppetDB
# database instance, and tell PostgreSQL that it should
# listen for connections to the `$postgres_host`
class { 'puppetdb::database::postgresql':
listen_addresses => $postgres_host,
}
}
node 'puppetdb.example.lan' {
# Here we install and configure PuppetDB, and tell it where to
# find the PostgreSQL database.
class { 'puppetdb::server':
database_host => $postgres_host,
}
}
This should be all it takes to get a 3-node, distributed installation of PuppetDB up and running. Note that, if you prefer, you could easily move two of these classes to a single node and end up with a 2-node setup instead.
To use SSL connections for the single node setup, use the following manifest:
node <hostname> {
# Here we configure puppetdb and PostgreSQL to use ssl connections
class { 'puppetdb':
postgresql_ssl_on => true,
database_host => '<hostname>',
database_listen_address => '0.0.0.0'
}
# Configure the Puppet master to use puppetdb
class { 'puppetdb::master::config': }
To use SSL connections for the multiple nodes setup, use the following manifest:
$puppetdb_host = 'puppetdb.example.lan'
$postgres_host = 'postgres.example.lan'
node 'master.example.lan' {
# Here we configure the Puppet master to use PuppetDB,
# telling it the hostname of the PuppetDB node.
class { 'puppetdb::master::config':
puppetdb_server => $puppetdb_host,
}
}
node 'postgres.example.lan' {
# Here we install and configure PostgreSQL and the PuppetDB
# database instance, and tell PostgreSQL that it should
# listen for connections to the `$postgres_host`.
# We also enable SSL connections.
class { 'puppetdb::database::postgresql':
listen_addresses => $postgres_host,
postgresql_ssl_on => true,
puppetdb_server => $puppetdb_host
}
}
node 'puppetdb.example.lan' {
# Here we install and configure PuppetDB, and tell it where to
# find the PostgreSQL database. We also enable SSL connections.
class { 'puppetdb::server':
database_host => $postgres_host,
postgresql_ssl_on => true
}
}
Whether you choose a single node development setup or a multi-node setup, a basic setup of PuppetDB will cause: PostgreSQL to install on the node if it’s not already there; PuppetDB postgres database instance and user account to be created; the postgres connection to be validated and, if successful, PuppetDB to be installed and configured; PuppetDB connection to be validated and, if successful, the Puppet master config files to be modified to use PuppetDB; and the Puppet master to be restarted so that it will pick up the config changes.
If your logging level is set to INFO or finer, you should start seeing PuppetDB-related log messages appear in both your Puppet master log and your puppetdb log as subsequent agent runs occur.
It is worth noting that there are some cross-node dependencies, which means that the first time you add the module's configurations to your manifests, you may see a few failed puppet runs on the affected nodes.
PuppetDB handles cross-node dependencies by taking a sort of "eventual consistency" approach. There’s nothing that the module can do to control the order in which your nodes check in, but the module can check to verify that the services it depends on are up and running before it makes configuration changes--so that’s what it does.
When your Puppet master node checks in, it will validate the connectivity to the puppetdb server before it applies its changes to the Puppet master config files. If it can’t connect to puppetdb, then the puppet run will fail and the previous config files will be left intact. This prevents your master from getting into a broken state where all incoming puppet runs fail because the master is configured to use a puppetdb server that doesn’t exist yet. The same strategy is used to handle the dependency between the puppetdb server and the postgres server.
Hence the failed puppet runs. These failures should be limited to 1 failed run on the puppetdb node, and up to 2 failed runs on the Puppet master node. After that, all of the dependencies should be satisfied and your puppet runs should start to succeed again.
You can also manually trigger puppet runs on the nodes in the correct order (Postgres, PuppetDB, Puppet master), which will avoid any failed runs.
Significant parameter changes are listed below:
- The PuppetDB module defaults to Puppet 4 pathing and assumes
puppetserveris the master service by default - The PuppetDB module manages Postgres repos by default. To turn this behavior
off, set
manage_package_repotofalse. - To specify a specific version of PuppetDB to manage, you'll need to use the
puppetdb::globalsclass to set the version of PuppetDB you're using explicitly. The ability to configure the version in thepuppetdb::serverandpuppetdbclass have been removed.
For example if your config looked like this before:
class {'puppetdb':
puppetdb_version => '3.2.4-1.el7',
}
class { 'puppetdb::master::config': }
and you'd still like to use the module with PuppetDB 3.2.4, all you'd have to change would be:
class { 'puppetdb::globals':
version => '3.2.4-1.el7',
}
class { 'puppetdb' : }
class { 'puppetdb::master::config' : }
The globals class above takes into account the following PuppetDB 3 and Puppet
4 related changes:
* The puppetdb::master:puppetdb_conf class has added a $legacy_terminus
to support the PuppetDB 2.x terminus configuration.
* The default test_url for the PuppetDBConnValidator has also been
changed to /pdb/meta/v1/version but will default to /v3/version when
using a PuppetDB 2.x version.
* The configuration pathing for Puppet and PuppetDB has changed with Puppet
4 and PuppetDB 3, using PuppetDB 2.x or older assumes the old
configuration pathing.
See the CHANGELOG file for more detailed information on changes for each release.
For this release, all dependency versions have been bumped to their latest. Significant parameter changes are listed below:
- The PuppetDB module now only supports Puppet 3.7.1 or later
puppetlabs/postgresql4.0.0 or later is now requiredpuppetlabs/inifile1.1.3 or later is now requiredpuppetlabs/firewall1.1.3 or later is now requiredpuppetlabs/stdlib4.2.2 or later is now required- The parameter
manage_firewallfor the classpuppetdb::database::postgresqlhas now been removed, since the PostgreSQL module no longer supports this. - The parameter
open_postgres_portfor the classpuppetdbhas also been removed, due to PostgreSQL changes.
See the CHANGELOG file for more detailed information on changes for each release.
For this release a major dependency has changed. The module
pupppetlabs/postgresql must now be version 3.x. Upgrading the module should
upgrade the puppetlabs/postgresql module for you, but if another module has a
fixed dependency that module will have to be fixed before you can continue.
Some other changes include:
- The parameter
manage_redhat_firewallfor the classpuppetdbhas now been removed completely in favor ofopen_postgres_portandopen_ssl_listen_port. - The parameter
manage_redhat_firewallfor the classpuppetdb::database::postgresql, has now been renamed tomanage_firewall. - The parameter
manage_redhat_firewallfor the classpuppetdb::serverhas now been removed completely in favor ofopen_listen_portandopen_ssl_listen_port. - The internal class:
puppetdb::database::postgresql_dbhas been removed. If you were using this, it is now defunct. - The class
puppetdb::server::firewallhas been marked as private, do not use it directly. - The class
puppetdb::server::jetty_iniandpuppetdb::server::database_inihave been marked as private, do not use it directly.
A major dependency has been changed, so now when you upgrade to 2.0 the
dependency cprice404/inifile has been replaced with puppetlabs/inifile. This
may interfere with other modules as they may depend on the old
cprice404/inifile instead, so upgrading should be done with caution. Check
that your other modules use the newer puppetlabs/inifile module as
interoperation with the old cprice404/inifile module will no longer be
supported by this module.
Depending on how you install your modules, changing the dependency may require
manual intervention. Double check your modules contain the newer
puppetlabs/inifile after installing this latest module.
Otherwise, all existing parameters from 1.x should still work correctly.
PuppetDB supports a large number of configuration options for both configuring the puppetdb service and connecting that service to the Puppet master.
The puppetdb::globals class is intended to provide similar functionality to
the postgresql::globals class in the puppetlabs-postgresql module by
exposing a top-level entry-point into the module so that we can properly set
defaults for the puppetdb::params class based on the version of puppetdb you
are using. This setting defaults to present.
You must declare the class to use it:
class { 'puppetdb::globals': }
The puppetdb class is intended as a high-level abstraction (sort of an
'all-in-one' class) to help simplify the process of getting your puppetdb server
up and running. It wraps the slightly-lower-level classes puppetdb::server and
puppetdb::database::*, and it'll get you up and running with everything you
need (including database setup and management) on the server side. For maximum
configurability, you may choose not to use this class. You may prefer to use the
puppetdb::server class directly, or manage your puppetdb setup on your own.
You must declare the class to use it:
class { 'puppetdb': }
The puppetdb::server class manages the PuppetDB server independently of the
underlying database that it depends on. It will manage the PuppetDB package,
service, config files, etc., but will still allow you to manage the database
(e.g. PostgreSQL) however you see fit.
class { 'puppetdb::server':
database_host => 'pg1.mydomain.com',
}
The puppetdb::master::config class directs your Puppet master to use PuppetDB,
which means that this class should be used on your Puppet master node. It’ll
verify that it can successfully communicate with your PuppetDB server, and then
configure your master to use PuppetDB.
Using this class allows the module to manipulate the puppet configuration files
puppet.conf and routes.yaml. The puppet.conf changes are supplemental and should
not affect any of your existing settings, but the routes.yaml file will be
overwritten entirely. If you have an existing routes.yaml file, you will want to
take care to use the manage_routes parameter of this class to prevent the module
from managing that file, and you’ll need to manage it yourself.
class { 'puppetdb::master::config':
puppetdb_server => 'my.host.name',
puppetdb_port => 8081,
}
The puppetdb::database::postgresql class manages a PostgreSQL server for use
by PuppetDB. It can manage the PostgreSQL packages and service, as well as
creating and managing the PuppetDB database and database user accounts.
class { 'puppetdb::database::postgresql':
listen_addresses => 'my.postgres.host.name',
}
In addition to the classes and variables mentioned above, PuppetDB includes:
puppetdb::master::routes
Configures the Puppet master to use PuppetDB as the facts terminus. WARNING: the current implementation simply overwrites your routes.yaml file; if you have an existing routes.yaml file that you are using for other purposes, you should not use this.
class { 'puppetdb::master::routes':
puppet_confdir => '/etc/puppet'
}
The optional parameter routes can be used to specify a custom route configuration. For example to configure routes for masterless puppet.
class { 'puppetdb::master::routes':
routes => {
'apply' => {
'facts' => {
'terminus' => 'facter',
'cache' => 'puppetdb_apply',
}
}
}
}
puppetdb::master::storeconfigs
Configures the Puppet master to enable storeconfigs and to use PuppetDB as the storeconfigs backend.
class { 'puppetdb::master::storeconfigs':
puppet_conf => '/etc/puppet/puppet.conf'
}
puppetdb::server::validate_db
Validates that a successful database connection can be established between the node on which this resource is run and the specified PuppetDB database instance (host/port/user/password/database name).
puppetdb::server::validate_db { 'validate my puppetdb database connection':
database_host => 'my.postgres.host',
database_username => 'mydbuser',
database_password => 'mydbpassword',
database_name => 'mydbname',
}
puppetdb_conn_validator
Verifies that a connection can be successfully established between a node and the PuppetDB server. Its primary use is as a precondition to prevent configuration changes from being applied if the PuppetDB server cannot be reached, but it could potentially be used for other purposes such as monitoring.
Currently, PuppetDB is compatible with:
Puppet Version: 4.10+
Platforms:
- EL 5, 6, 7
- Debian 6, 7
- Ubuntu 10.04, 12.04, 14.04
Community Maintained Platforms:
- Archlinux
- OpenBSD 5.6-current and newer
- SLES 11 SP1
Puppet Labs modules on the Puppet Forge are open projects, and community contributions are essential for keeping them great. We can’t access the huge number of platforms and myriad of hardware, software, and deployment configurations that Puppet is intended to serve.
We want to keep it as easy as possible to contribute changes so that our modules work in your environment. There are a few guidelines that we need contributors to follow so that we can have a chance of keeping on top of things.
You can read the complete contribution guide.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent