Giter Club home page Giter Club logo

puppet-puppet's Introduction

Puppet-puppet

Build Status

100% free range, organic, pesticide free Puppet module for managing Puppet.

Table of Contents

  1. Overview
  2. Module Description - What puppet-puppet does and why it is useful
  3. Setup - getting started with puppet-puppet
  4. Usage - Configuration options and additional functionality
  5. Limitations - OS compatibility, etc.
  6. Development - Guide for contributing to puppet-puppet

Overview

The puppet-puppet module manages puppet masters and agents using puppet.

Module Description

Puppet masters are frequently the only hand-crafted part of puppet-managed infrastructure. This module seeks to make the experience of running a puppet master similar to running Apache, Nginx, or MySQL using puppet.

Setup

What puppet-puppet affects

Depending on how you use this module, it can touch any of:

  • Puppet configuration files
  • Nginx or Apache configurations needed for running a master
  • Unicorn and Passenger configurations and init scripts
  • PuppetDB and PostgreSQL

As far as possible, this module tries to use other general-purpose modules to configure required non-puppet systems.

Setup Requirements

Puppet-puppet does not manage SSL certificates. You can generate the appropriate puppet SSL certificates by starting the webrick-based puppetmaster before using puppet-puppet. If you don't generate those SSL certs first, the resulting master won't work. (but should if you generate the certs; it's not strictly order dependent)

This module also doesn't manage r10k or hiera. Look at zack/r10k or sharpie/r10k for r10k, and the hunner/hiera module for managing hiera. If this is all unfamiliar, read the Shit Gary Says blog, starting with Building a Functional Puppet Workflow Part 1: Module Structure.

Usage

There are two general areas managed with this module: masters and agents.

Puppetmaster Setup

Webrick master

At an absolute minimum, you need the following.

class { 'puppet::server':
  servertype => 'standalone',
  manifest   => '/etc/puppet/manifests/site.pp',
  ca         => true,
}

This should get you a puppetmaster running under webrick which might scale to about 10 nodes if the wind doesn't blow too hard.

If, however, the moon is in the next phase then you probably want to use something that scales a bit more. Your options are nginx/unicorn or apache/passenger.

Nginx/Unicorn Master

The most basic setup would look something like:

class { 'puppet::server':
  servertype => 'unicorn',
  ca         => true,
}

A similar Apache/Passenger server would be:

class { 'puppet::server':
  servertype => 'passenger',
  ca         => true,
}

Certificate authority proxy configuration

If you want to automatically relay the certificate requests to an other CA you can do the following :

class { 'puppet::server':
  servertype  => 'unicorn',
  ca          => false,
  external_ca => 'https://my_puppet_ca_server:8140',
}

NB: This is only implemented for Nginx/Unicorn configuration so far.

Master with PuppetDB, PostgreSQL, and reports

Running a puppet master without PuppetDB loses much of the utility of Puppet, so you probably want it. As a convenience, this module will install puppetdb and postgresql using the puppetlabs/puppetdb module if manage_puppetdb => true is set.

If you want a more complex setup with PuppetDB and/or PostgreSQL on a different server, don't enable that option; use the puppetlabs/puppetdb module directly because it has many more configuration options that aren't exposed here.

class { 'puppet::server':
  directoryenvs    => true,
  basemodulepath   => '$confdir/modules:$confdir/secure',
  environmentpath  => '$confdir/environments',
  default_manifest => 'site/site.pp',
  manage_puppetdb  => true,
  reporturl        => "https://${::fqdn}/reports",
  servertype       => 'unicorn',
  ca               => true,
  reports          => [
    'https',
    'store',
    'puppetdb',
  ],
}

# in a real environment, you'll probably populate parameters on these
# report classes from hiera. For this example, it's specified inline so that
# the manifest works as-is.

class { 'puppet::reports::graphite':
  server => $::fqdn,
  port   => 2003,
  prefix => 'puppetmaster'
}

class { 'puppet::reports::irccat':
  host      => $::fqdn,
  githuburl => 'https://github.com/example/foo',
  dashboard => 'https://dashboard.example.com',
}

Puppet Agent Configuration

At the most basic, simply:

include puppet::agent

That will configure a cron job to run the puppet agent. If that's not what you want, one of the following may be more to your liking:

Running the agent service instead of via cron

class { 'puppet::agent':
  method => 'service',
}

Agent using cron, with more configuration options set:

Note that although the parameters correspond with puppet configuration file option names, only a relatively subset can currently be managed with this module.

class { 'puppet::agent':
  server        => 'puppet.example.com',
  ca_server     => 'puppetca.example.com',
  report_server => 'puppet_reports.example.com',
  method        => 'cron',
  configtimeout => 900,
}

In a production environment, you should probably use include puppet::agent and populate parameters using hiera automatic parameter lookup instead of hardcoding these values into manifests.

Limitations

This module is (basically) only tested on Debian Wheezy. The maintainers also care about FreeBSD and OpenBSD support. A token gesture of EL support exists in params.pp but that's about it; this probably won't do much on CentOS/RedHat. You'll see remnants of support for Windows, Gentoo, Solaris etc in the codebase but there's no testing or ongoing support for those platforms. They probably don't work at all. Pull requests welcome if you're interested.

Bootstrapping an all-in-one (master, puppetdb, postgresql) puppetmaster with puppet-puppet is relatively straightforward. However, building a usable puppet infrastructure with it requires additional steps, such as figuring out how you want to deploy manifests and modules to your master. (tip: use r10k)

You should definitely not use this module on an existing production puppetmaster unless you've tested it extensively. This is a tool developed by sysadmins, not developers, and testing is very incomplete.

Development

Read CONTRIBUTING.md to see instructions on running beaker and rspec tests.

puppet-puppet's People

Contributors

actown avatar adrienthebo avatar antaflos avatar barn avatar danieldreier avatar geoffgarside avatar gloppasglop avatar haus avatar igalic avatar jtopjian avatar juasiepo avatar justinstoller avatar ody avatar sschneid avatar tampakrap avatar ttres avatar zachfi avatar ziaunys avatar zxjinn avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

puppet-puppet's Issues

warning: You cannot collect without storeconfigs being set

The module doesn't work out of the box. It reports an error with the $storeconfigs variable as follows:

warning: You cannot collect without storeconfigs being set on line 152 in file /tmp/vagrant-puppet/modules-0/puppet/manifests/server.pp

warning: You cannot collect without storeconfigs being set on line 163 in file /tmp/vagrant-puppet/modules-0/puppet/manifests/server.pp

warning: You cannot collect without storeconfigs being set on line 173 in file /tmp/vagrant-puppet/modules-0/puppet/manifests/server.pp

warning: You cannot collect without storeconfigs being set on line 184 in file /tmp/vagrant-puppet/modules-0/puppet/manifests/server.pp

By default, $storeconfigs is undef, but setting the value to true or false seems to make no difference. The error remains the same during provisioning.

Add puppetserver support

This somewhat duplicates the PR I submitted, but I wanted to track work, outstanding tasks, and resources of use toward adding puppetserver support.

Relevant Documentation

Known Issues
Puppet Server vs. Apache/Passenger Puppet Master
Puppet Server Configuration
Configuring The Webserver Service
Puppet Server and Gems
External CA configuration
External SSL termination

Known Challenges

  • The /etc/puppetserver/bootstrap.cfg config file uses an odd syntax; stdlib file_line is probably an easier way to handle this than writing a new augeas lens.
  • Gems in the new master are in a different place to isolate them; see Puppet Server and Gems for details. We may want to add a new package provider to wrap that command to allow us to manage gems in the new environment.
  • The new master service takes longer to start, but the init scripts are non-blocking, so testing with beaker will require workarounds to watch log files or wait a fixed amount of time.

Features To-Do

  • Determine expected paths on supported platforms for files that correspond to /etc/puppetserver/bootstrap.cfg, /etc/default/puppetserver, /etc/puppetserver/conf.d, etc.
  • Add ability to configure puppetserver webserver
  • CSR signing proxying
  • Handle directory environments deprecation
  • Manage static file serving via embededed webserver to serve puppet explorer
  • Need to identify which puppet.conf features are not supported in puppetserver and handle those specially

What else? I'm adding this for discussion / tracking rather than trying to dictate how we handle puppetserver.

Add test coverage for Gentoo codebase

Based on issue #136 and @tampakrap's feedback on PR #140 we ought to fix Gentoo support rather than dropping it.

Gentoo Support To-Do List

  • Add rspec-puppet test coverage given the facts that would be expected in a Gentoo environment
  • Add a Gentoo beaker nodeset (what version(s) of Gentoo need coverage?)
  • Run tests and fix the inevitable issues that are discovered
  • Can we move gentoo_use and gentoo_keywords parameters from puppet::agent and puppet::server to puppet::params? I don't feel like we should have distro/OS specific parameters outside of params unless absolutely necessary. (but I don't understand what those parameters do, so that may be a bad idea)
  • Add gentoo test coverage to the other puppet modules we depend on. This is a ton of work, but our ability to support a platform (and the troubleshooting effort involved in it) is massively higher when dependencies break

Anything else people can think of? I don't mean to be overly project-manager-y, just hoping that making a more specific list of tasks will make it easier to support the platform.

Overview of issues blocking normal forge-based installs

I'm creating this issue to track the tasks needed to make the puppet-puppet module install in a usable form from the forge, like people expect modules to work. I'm currently working on these and would appreciate feedback.

The key issues I'm currently aware of are:

  • ploperations/unicorn depends on ploperations/rack but the latter is not on the forge at all. Fix pending in puppetlabs-operations/puppet-unicorn#14.
  • manifests/server/thin.pp requires a thin::app class but the Modulefile doesn't document any dependencies. What module is this supposed to be coming from? I can't find one on the forge that matches the parameters this is passing.
  • switch from ploperations/nginx to jfryman/nginx or publish a newer version of ploperations/nginx, because the released ploprations module doesn't seem to handle SSL correctly (I have a dev branch of the switch to jfryman/ngin working, just blocking on figuring out the thin module)

These extras would be nice but not absolutely essential:

  • add documentation about (or fix) the need to generate SSL / CA certs manually before setting up a non-standalone CA via this module (Fix pending in PR #133)
  • add metadata.json to clarify that RHEL/CentOS isn't really supported (for now) (Fix pending in PR #131)

Any other issues people are aware of that get in the way of doing a normal install? I don't mean this as an overall wishlist, just for stuff needed to do another forge release that works something like people reasonably expect it to out of the box.

release 1.0.0

i think we should make a release of 1.0.0

then we can branch off 2.0.x and start whacking away at #205 and #116.

manage_repos should not be on by default on all distros

$puppet::agent::manage_repos currently is on for all distros by default. Since there are no puppetlabs provided repos for many of the supported distros, I would like to propose either of the followings:

  1. Leave it on only for Debian/Ubuntu and Centos/Fedora/Red Hat
  2. Default it to off

Let me know if any of the above solutions is acceptable and I'll work on the PR

(optionally) configure environmentpath

Starting puppet 3.5.0, we can now use a single directive to configure a common pattern of modules and manifests in different environments: environmentpath

Experience shows that this setting should be in the [main] section, or else things start getting wonky.1

When setting environmentpath, we have to neglect the modulepath and manifestpath settings.

It should not be within the scope of this new directive to extend to extend config_version_cmd.

An optional environment.conf is presumed to be provided by the user, via git.


Footnotes

  1. That's a technical term for saying: It didn't work, and we didn't have time to analyse what exactly and how it didn't

PuppetLabs APT/YUM repos should be optional

The configuration of apt and yum PuppetLabs repos should be optional (enabled as class parameter and/or hiera). My situation is that we have a company mirror of PuppetLabs repos, and the servers are behind our proxy, so the module fails to install with:

Error: wget -q 'http://apt.puppetlabs.com/pubkey.gpg' -O- | apt-key add - returned 2 instead of one of [0]

feasibility of adding future parser, facter 2 dependencies

How do the existing users of puppet-puppet feel about using future parser and structured facts? I'd like to use things like "$processors[count] + 2" but that requires facter 2 and enabling structured facts, which isn't the default.

@igalic in particular I want to make sure this wouldn't cause problems for you, since you've contributed significantly to this module.

Use a real boolean after hiera 1.0 is out

from puppet::server

# FIXME
# http://projects.puppetlabs.com/issues/10590
# err: Could not retrieve catalog from remote server: Error 400 on SERVER: can't clone TrueClass
#
# Use a real boolean after hiera 1.0 is out

I guess that's pretty much the case now

master should refresh when modulepath is changed

Just ran into this - I updated my modulepath via hiera, re-ran puppet on the master, and the master service didn't refresh. Obviously this should be the case for other settings as well.

(I'm planning on fixing this, just wanted to track it in the meantime)

puppetlabs/apt 2.x support

Currently our tests fail because of the changes in puppetlabs/apt apt::source class. Two of our dependencies currently fail, one is ploperations/puppetlabs_apt and the other is jfryman/nginx. I will submit a PR for the first one. The nginx module is already fixed upstream, we are waiting for a forge release for it.

$master_service is wrong for centos/rhel

$puppet::params::master_service = 'puppetmasterd' under the centos/rhel case; it should be 'puppetmaster'. I think very much older versions of the rpm installed the service as puppetmasterd, but this hasn't been the case for quite some time I think.

Remove support for thin

In an effort to make a more targeted approach to this module and its capabilities on the server, we have discussed internally about the possibility of removing the support for thin. I'd like to get some feedback from the user(s) of thin if this is something that would cause pain, or if unicorn/nginx is just as workable a solution for them. To do this, I'd like to put a notice in the module to ask those users to voice their support here.

If you have reached this issue and are in support of removing or keeping thin, please note that here.

metadata.json is out of date

we should

  • get rid the dependencies we no longer use
  • restrict the version requirements on the dependencies we do use.

Remove support for puppet dashboard

Puppet Explorer and Puppet Board are under active development, whereas Puppet Dashboard is deprecated by Puppet Labs and gets relatively little maintenance.

How do other users/developers of puppet-puppet feel about removing dashboard support? I'd be open to removing entirely, switching to one of the other dashboards, or keeping it if there actually are active users.

request for 0.13.1

The bug that e32f849 is solving is quite important to me, thus I hereby request a new patch release 0.13.1 of this module

Does the dashboard.pp work with Apache?

The dashboard.pp says
Actions:
Install puppet-dashboard packages
Write the database.yml
Install the apache vhost
Installs logrotate

But only code applicable to nginx an unicorn can be seen

Duplicate declaration: Class[Puppet ::Agent] is already declared; cannot redeclare

Hi,

I am getting the following error:
"
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate declaration: Class[Puppet::Agent] is already declared; cannot redeclare at /etc/puppet/modules/site/manifests/roles/puppet/master.pp:22 on node server3.site.com
"
My declaration is like this:
class site::roles::puppet::master {

anchor { '::site::roles::puppet::master': }

Class {
require => Anchor['::site::roles::puppet::master']
}

class { '::puppet::server':
modulepath => ['$confdir/modules'],
manifest => '/etc/puppet/manifests/site.pp',
servertype => 'passenger',
reports => 'puppetdb',
servername => $::fqdn,
config_version_cmd => false,
#monitor_server => false,
backup_server => false,
reporturl => '',
parser => 'future',
ca => true,
} ->
class { '::puppet::agent':
server => hiera('puppet_server'),
method => 'service',
manage_repos => false,
} ->

class { '::puppetdb': } ->
class { '::puppetdb::master::config': }
}

The puppet::agent class is only declared once on my code, so I think there must be any kind of dependence/internal declaration in the puppet::server class as If I delete the puppet::agent class and run the code I can see that It handles the puppet agent run on the cron, etc.

Any idea how to fix this

Thanks for your time

Best regards

PS: I am following this tutorial http://terrarum.net/administration/puppet-infrastructure.html

Drop Gentoo support?

I spoke with @actown and he said he's no longer using this module on Gentoo. We don't have test coverage for it, and it's not high on my list. My thinking is that it'd be better to remove probably-broken gentoo code than to let people waste time trying to make it work, until we get a maintainer who's interested in putting the time in to keep it updated.

Could not find dependency Package[puppetmaster] for File_line[/etc/default/puppetmaster START]

When trying to include puppet::server with the following configuration:

puppet::server::servertype            : passenger
puppet::server::storeconfigs          : puppetdb
puppet::server::servername            : ppmaster.esat
puppet::storeconfig::puppetdb::server : ppmaster.esat

puppet::agent::server: ppmaster.esat
puppet::agent::manage_repos: false

and this code:

class roles::ppmaster inherits roles::server::internal {
  $role = 'puppetmaster'
  contain repos::config::phusion
  contain puppet::server
  contain puppetdb::server
  contain r10k

  Class[repos::config::phusion] -> Class[puppet::server]
}

We're getting the above error:

Error: Could not find dependency Package[puppetmaster] for File_line[/etc/default/puppetmaster START] at /etc/puppet/environment/production/modules/puppet/manifests/server/standalone.pp:23

The cause seems to be:

Warning: Scope(Class[Puppet::Package]): Could not look up qualified variable 'puppet::server::master'; class puppet::server has not been evaluated
Warning: Scope(Class[Puppet::Package]): Could not look up qualified variable 'puppet::server::master'; class puppet::server has not been evaluated

unicorn beaker test fails

The unicorn beaker tests fail with:

Error: /Stage[main]/Nginx::Service/Service[nginx]: Could not restart Service[nginx]: Execution of '/etc/init.d/nginx restart' returned 1: Restarting nginx: nginxnginx:
[emerg] BIO_new_file("/etc/nginx/puppetmaster.crt") failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/nginx/puppetmaster.crt','r') error:20
06D080:BIO routines:BIO_new_file:no such file)

I haven't investigated the cause yet, but figured I'd file an issue to track it. Since we don't have automated beaker test runs this sort of thing is expected.

unicorn / nginx support broken

I'm working on fixing this, but putting the issue up for the sake of anybody trying to use the module in the meantime.

On nginx and unicorn, the wrong headers are apparently getting set, resulting in SSL validation failing and a 403 permission error on all agent runs.

The reason is that jfryman/nginx defaults to adding a default location in a vhost, and that default location defaults to having three headers set that we also set explicitly later.

Pull request?

Hello,

I found this module and like it a lot better than the puppetlabs/puppet module. I've made several small changes and was wondering if you'd be interested in a pull request.

https://github.com/jtopjian/puppet-puppet

Since this is the -operations area of Puppet, I understand if these modules are more internally maintained than the puppetlabs modules.

Thanks,
Joe

Rotate logs, reports

This module should set up cron jobs to compress/delete old reports, because that's a problem common to nearly all puppet masters.

(I'm planning on adding this eventually, just want to add a note so I remember)

Version bump and forge update?

Definitely needed. Latest version on the forge uses ripienaar/concat which conflicts with modern modules use of puppetlabs/concat.

Thanks!

standalone bindaddress '::' results in v6 only master

On systems where IPv6 sockets are V6ONLY by default the default 'bindaddress' used for the servertype => 'standalone' results in an unreachable master when IPv6 is not configured on the host or AAAA records are not configured for the name. It might be better to stick with the Puppet default of '0.0.0.0' to be less surprising and where needed this can be overridden to '::' when IPv6 is desired.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.