Basic Puppet...

Puppet

A Configuration Management Tool
A framework for Systems Automation
A Declarative Domain Specific Language ()
An OpenSource software in written Ruby
Works on Linux, Unix (Solaris, AIX, *BSD), MacOS, Windows
Developed by Labs
Used by (http://puppetlabs.com/customers/companies/) ...
... and many others

Configuration Management advantages

Infrastructure as Code: Track, Test, Deploy, Reproduce, Scale
Code commits log shows the history of change on the infrastructure
Reproducible setups: Do once, repeat forever
Scale quickly: Done for one, use on many
Coherent and consistent server setups
Aligned Environments for devel, test, qa, prod nodes

References and Ecosystem

Labs - The Company behind
- The OpenSource version
Enterprise - The commercial version
The Community - Active and vibrant
Documentation - Main and Official reference
Modules on: Module Forge and GitHub
Software related to :
MCollective - Infrastructure Orchestration framework
Hiera - Key-value lookup tool where data can be placed
PuppetDB - An Inventory Service and StoredConfigs backend
DashBoard - A Web frontend and External Node Classifier (ENC)
The Foreman - A well-known third party provisioning tool and ENC
Geppetto - A IDE based on Eclipse

Installation

Debian, Ubuntu
Available by default

# On clients (nodes):
apt-get install puppet

# On server (master):
apt-get install puppetmaster

RedHat, Centos, Fedora
Add EPEL repository or RHN Extra channel

# On clients (nodes):
yum install puppet

# On server (master):
yum install puppet-server

Use PuppetLabs repositories for latest updates
Installation Instructions for different OS

Puppet configuration: puppet.conf

It's main configuration file.
On opensource is generally in:

/etc/puppet/puppet.conf

On Enterprise:

/etc/puppetlabs/puppet/puppet.conf

When running as a normal user can be placed in the home directory:

/home/user/.puppet/puppet.conf

Configurations are divided in [stanzas] for different sub commands
Common for all commands: [main]
For puppet agent (client): [agent] (Was [puppetd] in pre 2.6)
For puppet apply (client): [user] (Was [puppet])
For puppet master (server): [master] (Was [puppetmasterd] and [puppetca])
Hash (#) can be used for comments.

Configuration options

To view all or a specific configuration setting:

puppet config print all
puppet config print modulepath

Important options under [main] section:
vardir: Path where stores dynamic data.
ssldir: Path where SSL certifications are stored.
Under [agent] section:
server: Host name of the PuppetMaster. (Default: puppet)
certname: Certificate name used by the client. (Default is the hostname)
runinterval: Number of minutes between runs, when running as service. (Default: 30)
report: If to send runs' reports to the **report_server. (Default: true)
Under [master] section:
autosign: If new clients certificates are automatically signed. (Default: false)
reports: How to manage clients' reports (Default: store)
storeconfigs: If to enable store configs to support exported resources. (Default: false)
Full configuration reference

Common command-line parameters

All configuration options can be overriden by command-line options.
Run puppet agent in foreground and verbose mode.
A very common option used when you want to see immediately the effect of a run.
It's actually the combination of: -onetime, -verbose, -ignorecache, -no-daemonize, -no-usecacheonfailure, -detailed-exit-codes, -no-splay, and -show_diff

puppet agent --test

Run puppet agent in foreground and debug mode

puppet agent --test --debug

Run puppet without making any change to the system

puppet agent --test --noop

Run puppet using an environment different from the default one

puppet agent --environment testing

Wait for certificate approval (by default 120 seconds) in the first Run
Useful during automated first fime installation if PuppetMaster's autosign is false

puppet agent --test --waitforcert 120

Other configuration files:

auth.conf
Defines ACLs to access 's REST interface. Details
fileserver.conf
Used to manage ACL on files served from sources different than modules Details
puppetdb.conf
Settings for connection to PuppetDB, if used. Details
tagmail.conf , autosign.conf , device.conf , routes.yaml
These are other configuration files for specific functions. Details

Puppet Language

A Declarative Domain Specific Language ()
Defines STATES (Not procedures)
code stays in manifests (files .pp)
Code contains resources that affects elements of the systme (file, package, service ...)
Resources are often grouped in classes which are generally organized in modules
Variables may be defined nodes and can be Facts (generated from the node) or User defined
On the Master are defined the resources or classes to include on the nodes (clients)
All the resources to apply on a node are defined in the catalog, generated by the Master

Resource Types (Types)

Resources are single units of configuration composed by:
A type (package, service, file, user, mount, exec ...)
A title (how is called and referred)
One or more arguments

type { 'title':
  argument  => value,
  other_arg => value,
}

Example for a file resource type:

file { 'motd':
  path    => '/etc/motd',
  content => 'Tomorrow is another day',
}

Complete Type Reference Online or at the command line

puppet describe file

Give a glance to code for the list of native resource types:

ls $(facter rubysitedir)/puppet/type

Simple samples of resources

Installation of OpenSSH package

package { 'openssh':
  ensure => present,
}

Creation of /etc/motd file

file { 'motd':
  path => '/etc/motd',
}

Start of httpd service

service { 'httpd':
  ensure => running,
  enable => true,
}

More Complex examples of resources

Installation of Apache package with the correct name for different OS

package { 'apache':
  ensure => present,
  name   => $operatingsystem ? {
    /(?i:Ubuntu|Debian|Mint)/ => 'apache2',
    default                   => 'httpd',
  }
}

Management of nginx service with parameters defined in module's variables

service { 'nginx':
  ensure     => $nginx::manage_service_ensure,
  name       => $nginx::service,
  enable     => $nginx::manage_service_enable,
}

Creation of nginx.conf with content retrived from different sources (first found is served)

file { 'nginx.conf':
  ensure  => present,
  path    => '/etc/nginx/nginx.conf',
  source  => [ "puppet:///modules/site/nginx/nginx.conf--${fqdn}",
               "puppet:///modules/site/nginx/nginx.conf-${role}",
               "puppet:///modules/site/nginx/nginx.conf" ],
  }
}

Resource Abstraction Layer

Resources are abstracted from the underlining OS
Resource Types have different providers for different OS
Package type is known for the great number of providers

ls $(facter rubysitedir)/puppet/provider/package

Use puppet resource to interrogate the RAL:

puppet resource user

puppet resource user root

puppet resource package

puppet resource service

Or to directly modify resources:

puppet resource service httpd ensure=running enable=true

Classes

Classes are containers of different resources. Since 2.6 they can have parameters
Example of a class definition:

class mysql {
  package { 'mysql-server':
    ensure => present,
  }
  service { 'mysql':
    ensure    => running,
  }
  [...]
}

Usage (declaration) of "old style" classea (without parameters):
Even if a class is a singleton, you can include it multiple times: it's applied only once.

include mysql

Usage (declaration) of a parametrized classes
You can declare a parametrized class only once for a node

class { 'mysql':
  required_param => 'my_value',
  optional_param => 'dont_like_defaults',
}

Defines

Also called: Defined resource types or defined types
Similar to parametrized classes but can be used multi times, with different parameters
Definition example:

define apache::virtualhost (
  $template = 'apache/virtualhost.conf.erb' ,
  [...] ) {

  file { "ApacheVirtualHost_${name}":
    ensure  => $ensure,
    content => template("${template}"),
  }
}

Usage example (declaration):

apache::virtualhost { 'www.example42.com':
  template => 'site/apache/www.example42.com-erb'
}

Variables

You need them to provide different configurations for different kind of servers
Can be provided by client nodes as facts
Facter runs on clients and collects facts that the server can use as variables

al$ facter

architecture => x86_64
fqdn => Macante.example42.com
hostname => Macante
interfaces => lo0,eth0
ipaddress => 10.42.42.98
ipaddress_eth0 => 10.42.42.98
kernel => Linux
macaddress => 20:c9:d0:44:61:57
macaddress_eth0 => 20:c9:d0:44:61:57
memorytotal => 16.00 GB
netmask => 255.255.255.0
operatingsystem => Centos
operatingsystemrelease => 6.3
osfamily => RedHat
virtual => physical

Or can be defined by users

User Variables

You can define custom variables in different ways:
In manifests:

$role = 'mail'

$package = $operatingsystem ? {
  /(?i:Ubuntu|Debian|Mint)/ => 'apache2',
  default                   => 'httpd',
}

In an External Node Classifier ( DashBoard, the Foreman, Enterprise)
In an Hiera backend

$syslog_server = hiera(syslog_server)

Nodes

A node is identified by the PuppetMaster by its hostname or certname
You can decide what resources, classes and variables to assign to a node in 2 ways:
Using language ( Starting from /etc/manifests/site.pp )

node 'web01' {
  include apache
}

Using an External Node Classifier (DashBoard, Foreman or custom scripts)
When a client connects a PuppetMaster builds a catalog with all the resources to apply on the client
The catalog is in Pson format (a version of Json)

Modules

Self Contained and Distributable recipes contained in a directory with a predefined structure
Used to manage an application, system's resources, a local site or more complex structures
Modules must be placed in the Master's modulepath

puppet config print modulepath
/etc/puppet/modules:/usr/share/puppet/modules

module tool to interface with Modules Forge

puppet help module
[...]
ACTIONS:
  build        Build a module release package.
  changes      Show modified files of an installed module.
  generate     Generate boilerplate for a new module.
  install      Install a module from the Puppet Forge or an archive.
  list         List installed modules
  search       Search the Puppet Forge for a module.
  uninstall    Uninstall a puppet module.
  upgrade      Upgrade a puppet module.

GitHub, also, is full of modules

Paths of a module

Modules have a standard structure:

mysql/            # Main module directory

mysql/manifests/  # Manifests directory. Puppet code here. Required.
mysql/lib/        # Plugins directory. Ruby code here
mysql/templates/  # ERB Templates directory
mysql/files/      # Static files directory
mysql/spec/       # Puppet-rspec test directory
mysql/tests/      # Tests / Usage examples directory

mysql/Modulefile  # Module's metadata descriptor

This layout enables useful conventions

Modules paths conventions

Classes and defines autoloading:

include mysql
# Main mysql class is placed in: $modulepath/mysql/manifests/init.pp

include mysql::server
# This class is defined in: $modulepath/mysql/manifests/server.pp

mysql::conf { ...}
# This define is defined in: $modulepath/mysql/manifests/conf.pp

include mysql::server::ha
# This class is defined in: $modulepath/mysql/manifests/server/ha.pp

Provide files based on Erb Templates (Dynamic content)

content => template('mysql/my.cnf.erb'),
# Template is in: $modulepath/mysql/templates/my.cnf.erb

Provide static files (Static content). Note you can use content OR source for the same file.

source => 'puppet:///modules/mysql/my.cnf'
# File is in: $modulepath/mysql/files/my.cnf

Erb templates

Files provisioned by can be Ruby ERB templates
In a template all the variables (facts or user assigned) can be used :

# File managed by Puppet on <%= @fqdn %>
search <%= @domain %>

But also more elaborated Ruby code

<% @dns_servers.each do |ns| %>
nameserver <%= ns %>
<% end %>

The computed template content is placed directly inside the catalog
(Sourced files, instead, are retrieved from the puppetmaster during catalog application)

Principes behind a Reusable Module

Data Separation

- Configuration data is defined outside the module (or even Puppet manifests)
- Module's behavior is managed via APIs
- Allow module's extension and override via external data

Reusability

- Support different OS. Easily allow new additions.
- Customize behavior without changing module code
- Do not force author's idea on how configurations should be provided

Standardization

- Follow PuppetLabs style guidelines (puppet-lint)
- Have coherent, predictable and intuitive interfaces
- Provide contextual documentation (puppet-doc)

Interoperability

- Limit cross-module dependencies
- Allow easy modules' cherry picking
- Be self contained, do not interfere with other modules' resources

What's a Good Module anyway?

The most reusable and customizable
The one full of features
The one that works for you
The most essential, and optimized (but not reusable) one
The quickest one to do now
as usual... your mileage may vary
In 's world the concept of "Best Practices" is somehow fluid... :-)
(It follows the language's features evolution and the blog post of the moment...)

Modules documentation with Puppet Doc

Puppetdoc generates documentation from manifests comments:

$ puppetdoc [--all] --mode rdoc [--outputdir ] [--debug|--verbose] [--trace]
  [--modulepath ] [--manifestdir ] [--config ]

Comment classes as below:

# Class: apache
#
# This module manages Apache
#
# Parameters:
#
# Actions:
#
# Requires:
#
# [Remember: No empty lines between comments and class definition]
class apache {
  ...
}