Giter Club home page Giter Club logo

puppet-validation-tools's Introduction

puppet-validation-tools

The following is a general summary of code validation "self-service" tools/scripts which can be used to troubleshoot and validate your Puppet code. It is by no means intended as anything close to an exhaustive list. Feel free to contact me/submit a pull request for any inclusions you might like to see :)

Note: Remember to use the --noop flag if you don't want your changes immediately applied.

  1. puppet help -- Not to be overlooked, this command is often the best starting place for troubleshooting a significant number of issues.

  2. puppet apply -e -- For example, you might test a hiera lookup with, sudo puppet apply --environment=target environment -e "notice(hiera('message'))" . Also, just to give you an idea of how the interpolation works, here's an example using the exec resource:

    puppet apply -e 'exec { "/usr/bin/cowsay $::fqdn > /etc/motd": }'

    An example using an include: puppet apply -e "include pe_repo::platform::aix_71_power"

    How to use an inline_epp template to access elements in an array without a key:

puppet apply -e 'notice(inline_epp("<%= \$processors[models][0] %>"))'

Same thing, but with a ruby inline_template (and a few more options):

puppet apply -e 'notice(inline_template("<%= @processors[\"models\"][0] %>"))'
puppet apply -e 'notice(inline_template("<%= @processors[\"models\"].at(0) %>"))'
puppet apply -e 'notice(inline_template("<%= @processors[\"models\"].first %>"))'
puppet apply -e 'notice(inline_template("<%= @processors[\"models\"].last %>"))'

Here is a link to the Type reference: https://puppet.com/docs/puppet/5.3/type.html. Certainly a great way to learn about the various Types in Puppet, and how best to leverage them.

  1. hiera_explain -- Use of this tool is going to vary based on how you have your code and data structured in your Puppet implementation. Here's a link that provides a good starting place for experimenting/learning about how to interact with this module: https://github.com/binford2k/hiera_explain

  2. puppet-syntax -- A module that is worth considering for any CI/CD workflow. More info available here: https://github.com/voxpupuli/puppet-syntax.

  3. puppet parser validate Although many may be already familiar with it, it bears inclusion in this list -- as well as perhaps defining as a specific function in .zshrc/.bashrc ;)

  4. puppet-lint -- Another tried and true method for keeping things orderly :)

  5. The Puppet Enterprise Client Tools Suite. https://docs.puppet.com/pe/latest/install_pe_client_tools.html -- I highly recommend getting familiar with this suite of tools, as it makes much easier work of interacting with various PE services, previously relegated to lengthy curl commands ran against the various API endpoints (usually as root), and/or logging in directly to the console.

  6. Increasing the default logging level for your Puppet agents. -- While not specifically a "tool" for code validation, increasing the debug log level can be incredibly useful for troubleshooting issues where agents are inexplicably hanging, etc.. The default logging level is typically too low to determine why an agent run may be intermittently hanging.

To turn this on, edit the /etc/sysconfig/puppet file and uncomment the PUPPET_EXTRA_OPTS line, changing it to: PUPPET_EXTRA_OPTS="--debug --trace --logdest=/var/log/puppetlabs/puppet/puppet-debug.log"

Then create the log file with: $touch /var/log/puppetlabs/puppet/puppet-debug.log. Restart Puppet with service puppet restart. When the problem re-surfaces, look at the /var/log/puppetlabs/puppet/puppet-debug.log file for clues. This is basically the equivalent of running the following from the command line:

puppet agent -t --debug --trace 2>&1 | tee trace.txt
  1. Beginning with Puppet 4.4, the puppet lookup command may be used (this is now simply referred to as "Hiera 5", and is documented here: https://docs.puppet.com/puppet/4.9/hiera_intro.html and here: https://support.puppet.com/hc/en-us/articles/115005435168

  2. r10k puppetfile install -- This will look for a Puppetfile in the current working directory, and if found, will install all modules as listed in the file, pulling in all relevant dependencies. This is an indispensable tool when you need to test code, i.e. where you keep getting dependency related errors when using puppet apply.

  3. puppet module install -- This will install a module and bring in all dependencies. While the r10k puppetfile install is fine if you are testing on a PE Master, you might want/need to test some code on a Linux client system where you have your control repo, but aren't managing your module code there directly -- again to avoid dependency issue when using puppet apply. You can use puppet module install with the --modulepath flag to install these modules, again automatically bringing in all dependencies.

  4. While you really should be using https://github.com/dylanratcliffe/onceover, here is a somewhat primitive bash for loop to quickly validate .eyaml/.yaml (haven't tested this fully with .eyaml, but it should at least catch those pesky "hard tabs"):

for x in `find . -regex ".*[eyaml][yaml]?" -print`; do ruby -e "require 'yaml'; YAML.load_file('$x')"; echo $x; done 2>&1 |tee /dev/tty >> outfile.txt

Here's a script to retrieve all the parameters for the Puppet_enterprise class from the most recently compiled catalog on the agent

for x in `find $(puppet agent --configprint client_datadir) -name "*.json" -print`; do echo $x; jq '.resources[] | select(.type == "Class" and .title == "Puppet_enterprise").parameters' $x >> ~/pe_class_params.txt; done

Here's a command to get all the classes that were applied on the last run:

# cat $(puppet agent --configprint classfile)
pe_repo
pe_repo::platform::el_7_x86_64
puppet_enterprise
puppet_enterprise::profile::agent
puppet_enterprise::profile::amq::broker
puppet_enterprise::profile::certificate_authority
puppet_enterprise::profile::console
puppet_enterprise::profile::master
puppet_enterprise::profile::master::mcollective
[...]

And here's a grep command to grab something more specific:

# grep docker $(puppet agent --configprint classfile)
docker::params
docker::systemd_reload
docker
docker::repos
docker::install
docker::config
docker::service

This lists all resource titles enforced on the last run:

# cat $(puppet agent --configprint resourcefile)
service[postgresql]
file[/etc/puppetlabs/puppet/routes.yaml]
ini_setting[puppetdbserver]
service[pe-httpd]
firewall[5432 accept - postgres]
puppetdb_conn_validator[puppetdb_conn]
package[postgresql-server]
exec[/sbin/iptables-save > /etc/sysconfig/iptables]
[...]

Identify which yum repo's are being managed:

# grep yumrepo $(puppet agent --configprint resourcefile)
yumrepo[docker]
yumrepo[epel-testing]
yumrepo[epel-testing-debuginfo]
yumrepo[epel-testing-source]
yumrepo[epel]
yumrepo[epel-debuginfo]
yumrepo[epel-source]
  1. If you are looking to implement something a bit more sophisticated than the "one-off" command-line validation suggestions above, you might consider implementing rspec testing of your Puppet modules, as discussed in the excellent blog post by Joseph Oaks available here: https://puppet.com/blog/unit-testing-rspec-puppet-for-beginners and another post by Nick Walker here: https://puppet.com/blog/use-onceover-start-testing-rspec-puppet

  2. The validate_cmd Resource Types: https://puppet.com/docs/puppet/5.5/types/file.html#file-attribute-validate_cmd. These are great resource types to use in validating a file’s syntax before replacing it.

  3. Using a git pre-commit hook. Here's a link to a gist I've created for a useful git pre-commit hook that can be used to automatically validate/lint files before they are commited to your git repo: https://gist.github.com/rodtreweek/d1ce5ed8925b8d1af6ac90a2fc71d1e4#file-git-pre-commit-hook

The above list should offer a reasonable collection of tools for detecting errors in advance or validating your Puppet code prior to pushing this to production, as well as possibly clarifying various "points in the decision tree" within Hiera. For a more thorough explanation of how to best leverage each of the above suggestions, you should consult the relevant pages at https://puppet.com/docs, or try searching support.puppet.com for more specific information.

puppet-validation-tools's People

Contributors

rodtreweek avatar

Stargazers

 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.