Kevin Ashcraft

Linux & Radio Tutorials

Setup Puppet 5

Puppet is a server management tool used to automatically configure users, services, and more. It's fairly easy to setup and takes the guesswork out of server configuration, giving you an authority to reference for the current state of your machines as well as a central location to update everything at once.

In this tutorial we'll walk through an introduction to Puppet, how to install Puppet server, and then setting up the site manifest, hieradata, an example module to configure linux packages and users, an example module to configure an apache web server, and then some best practices.

Puppet Intro

Have you ever learned something new and then wanted to implement it across all of your servers, like SSL Best Practices, hardening an SSH server, or maybe just updating the motd? Or have you ever wondered what state you left the firewall in 6 months ago the last time you were working on a box? Or have you ever gotten tired of using a handful of scripts to maintain more servers than you can count? That's why I like puppet.

Puppet provides one location for all of your user passwords, default packages, virtual host configurations, cron jobs, everything. And, best yet, if you keep your puppet config revision controlled then you always have control over the state of your servers, including the ability to rollback configuration changes.

Puppet consists of Puppet Server running on one machine, and Puppet Agent running on all of the others. The agents can be configured to automatically retrieve new changes, or manually with a trigger. All of the communication is encrypted and certificates (automatically generated by puppet) are used to ensure the server and agents are known. The only open port is 8140 on the master.

Directory Structure

/etc/puppetlabs/code/environments/production

./manifests/site.pp  #list of hosts and their roles

./data/common.yaml   #default configuration values
./data/nodes/        #host-specific config values

./modules/           #custom modules
./modules/$module/manifests/init.pp   #module entry point
./modules/$module/files/              #static files
./modules/$module/templates/          #template files
    

Install Pupppet Server

To install Puppet 5, add the official repo and then use your package manager.

Install Puppet on CentOS 7

rpm -Uvh https://yum.puppet.com/puppet5/puppet5-release-el-7.noarch.rpm
yum install puppetserver -y
systemctl start puppetserver
systemctl enable puppetserver
    

Install Puppet on Ubuntu 16.04

wget https://apt.puppetlabs.com/puppet5-release-xenial.deb
dpkg -i puppet5-release-xenial.deb
apt update
apt install puppetserver
    

Here's a list of all of the Official Puppet Repositories for other operating systems.

Set PATH for /opt/puppetlabs/bin

echo 'export PATH=/opt/puppetlabs/bin:$PATH' >> ~/.bashrc
export PATH=/opt/puppetlabs/bin:$PATH
    

Site Manifest

Let's begin with the site.pp which defines the hosts and the corresponding modules to apply to each.

./manifests/site.pp

node default {
  include linux
}

node 'webserver.example.com' {
  include linux
  include httpd
}
    

Here we're stating that every undefined node should have the linux module applied to it and the `webserver.example.com` host should get both the linux and httpd modules.

You could also use '*' as a wildcard to define hosts so `webserver.*` would work as well as the explicit hostname.

Hieradata

Hieradata is the recommended way to define the variables used by Puppet modules. We'll start with the hiera.yaml file which defines how Puppet loads the rest of the data.

./hiera.yaml

---
version: 5
defaults:
  datadir: data
  data_hash: yaml_data
hierarchy:
  - name: "Per-node data (yaml version)"
    path: "nodes/%{::trusted.certname}.yaml"
  - name: "Other YAML hierarchy levels"
    paths:
      - "common.yaml"
    

This states that we'll be using the yaml file format, the data will be stored in ./data, and it will be loaded by first checking if a node-specific file exists (such as ./data/nodes/webserver.example.com.yaml) and if not then loading the ./data/common.yaml file.

Next we'll look at an example common.yaml file.

./data/common.yaml

linux::root_pw: '$1$xyz$LeL/QNMIaRT9H99RmIsgy/'
linux::nameserver: 8.8.8.8
    

Generate Linux Password Hash

openssl passwd -1 -salt xyz
    

This defines the root_pw and nameserver arguments for the linux module.

Before we move on let's look at one more example that demonstrates an advantage of using the yaml format. We can define variables once and then reuse them, which comes in handy as your configurations grow.

./data/common.yaml

global::root_pw: &global_root_pw
  '$1$xyz$LeL/QNMIaRT9H99RmIsgy/'
global::nameserver: &global_nameserver
  8.8.8.8
linux::root_pw: *global_root_pw
linux::nameserver: *global_nameserver
    

Here we've first defined the root_pw and nameserver variables, including an &identifier next to them which we're able to use later.

An Example Linux Module

A Puppet Module is a group of configuration parameters which can include Packages to install, Services to start, Files to copy, and Users to create.

To get started, go to ./modules and run puppet module generate $USER-linux. The generation script will create the essential files needed for the module. Most of the files created are for metadata, tests, and documentation. We'll focus on manifests/init.pp as it contains the module definition.

Let's look at the basic structure of the module class first.

./manifests/init.pp

class linux (
  // arguments
) {
  // actions
}
    

The argument values will be coming from the hieradata and the actions are Puppet actions such as user, package, service, and file. Ruby can be used to work with the arguments to combine or loop through arrays, create conditional statements, etc.

Let's create a basic class to set the root user's password.

./manifests/init.pp

class linux (
  String $root_pw
) {
  user { 'root':
    ensure   => present,
    password => $root_pw
  }
}
    

That's easy enough, right? We've added a String argument that will be coming from the hieradata, and used the user definition to tell Puppet that this user should 1) exist, and 2) have the given password. If either of those conditions are not met, Puppet will remedy the situation.

The structure of a definition is as follows:

Module Statement Syntax

type { subject:
  parameter => value
}
    

Let's add the ability to install a package to this module.

./manifests/init.pp

class linux (
  String $root_pw
) {
  user { 'root':
    ensure   => present,
    password => $root_pw
  }

  package { 'vim-enhanced':
    ensure => present
  }
}
    

Again, we're stating the type (package), and then telling Puppet to ensure it's present. If it's not already present, then it'll be installed.

Next let's add the apache package and enable it as a service.

./manifests/init.pp

class linux (
  String $root_pw
) {
  user { 'root':
    ensure   => present,
    password => $root_pw
  }

  package { 'vim-enhanced':
    ensure => present
  }

  package { 'httpd':
    ensure => present
  }

  service { 'httpd':
    ensure => running,
    enable => true
  }
}
    

There we've added the httpd package and then told Puppet to ensure that it's both running and enabled.

But what if we're not on a Red Hat system and apache goes by a different name? Those commands would fail. So let's update the module with some conditional statements to ensure that it'll work on Debian systems as well.

./manifests/init.pp

class linux (
  String $root_pw
) {
  user { 'root':
    ensure   => present,
    password => $root_pw
  }

  $vim = $::osfamily == 'redhat' ? 'vim-enhanced' : 'vim'

  package { $vim:
    ensure => present
  }

  $apache = $::osfamily == 'debian' ? 'apache2' : 'httpd'

  package { $apache:
    ensure => present
  }

  service { $apache:
    ensure => running,
    enable => true
  }
}
    

There we've added two ternary expressions to set both the vim and apache package names. You could also use a switch if you wanted to check for additional options.

The $::osfamily fact is a global variable in Puppet that contains, wait for it, the os family. Others are also available such as $::operatingsystem and more.

Okay, we're about there. Let's do two more things with this module. First let's make sure a file is present.

For simplicities sake we're going to empty out this module example and start fresh.

./manifests/init.pp

class linux () {
  file { '/etc/resolv.conf':
    source => 'puppet://modules/linux/resolv.conf'
  }
}
    

That will ensure that the file ./modules/linux/resolv.conf on the Puppet server also exists at /etc/resolv.conf on the client.

Then to include a non-static file, we'd use a template.

./manifests/init.pp

class linux ( String $nameserver ) {
  file { '/etc/resolv.conf':
    content => template('linux/resolv.conf.erb')
  }
}
    

./modules/linux/templates/resolv.conf.erb

nameserver <%= @nameserver %> 

That will pass the $nameserver argument to the resolv.conf.erb template file and render it to /etc/resolv.conf on the client.

Those are the basics for creating a module.

An Example Httpd Module

Next we'll look through an httpd module. To begin, we'll add the following to the common.yaml file.

./hieradata/common.yaml

httpd::admin_email: webmaster@example.com
httpd::docroot: /srv
httpd::listen_on: 10.17.0.7
httpd::maint_user: kevin
httpd::sites:
  - domain: www.example.com
    aliases:
      - example.com
    docroot: /srv/example.com/www/dist
    ssl: example.com
  - domain: blog.example.com
    aliases:
      - www.blog.example.com
    docroot: /srv/example.com/blog/dist
    ssl: example.com
  - domain: tutorials.example.com
    aliases:
      - www.tutorials.example.com
    docroot: /srv/example.com/tutorials/dist
    ssl: example.com
    

Here we've defined the admin_email, docroot, maint_user, and listen_on variables as well as an array of sites.

./modules/httpd/manifests/init.pp

class httpd (
  $sites,
  $admin_email,
  $maint_user,
  $docroot,
  $listen_on,
  ) {

  case $::osfamily {
    'debian': {
      $httpd = 'apache2'
      $httpd_user = 'http'
      $letsencrypt = 'certbot'
    }
    'redhat': {
      $httpd = 'httpd'
      $httpd_user = 'apache'
      $letsencrypt = 'certbot'

      package { 'mod_ssl':
        ensure  => installed,
        require => Package[$httpd],
      }
    }
    default: {
      $httpd = 'httpd'
      $httpd_user = 'apache'
      $letsencrypt = 'letsencrypt'
    }
  }

  package { $httpd:
    ensure => installed,
  }

  package { $letsencrypt:
    ensure  => installed,
    require => Package[$httpd],
  }

  service { $httpd:
    ensure     => running,
    enable     => true,
    hasrestart => true,
    hasstatus  => true,
    require    => Package[$httpd],
  }

  file { "/etc/$httpd/conf/httpd.conf":
    content => template('httpd/httpd.conf.erb'),
    notify  => Service['httpd'],
  }

  file { "/etc/$httpd/conf.d":
    ensure => directory,
  }

  file { $docroot:
    ensure  => directory,
    owner   => $maint_user,
    group   => $httpd_user,
    selrole => 'object_r',
    seluser => 'system_u',
    seltype => 'httpd_sys_content_t',
  }

  $sites.each |$index, $site| {
    file { "/etc/httpd/conf.d/${index}-${site['domain']}.conf":
      content => template('httpd/virtual_host.conf.erb'),
      notify  => Service['httpd'],
      require => File['/etc/httpd/conf.d'],
    }
  }
}
    

Walking through this module, we're first setting the package named with the case $::osfamily switch, setting the name of the webserver package, user and group, as well as installing mod_ssl if we're on a redhat system.

Next we're installing the webserver package, and then the letsencrypt package.

After that we're enabling the webserver service. Note that we've included arguments for require to make sure that the package is installed before we attempt to start the service. We've also told Puppet that this service has the ability to restart and a status check to assist whenever it needs to be checked or restarted.

Next we're setting up the master configuration file, httpd.conf. This is being populated by the httpd.conf.erb file.

./modules/httpd/manifests/init.pp partial

file { "/etc/$httpd/conf/httpd.conf":
  content => template('httpd/httpd.conf.erb'),
  notify  => Service['httpd'],
}
    

Then we're ensuring that the root web directory exists with the proper owner and roles.

./modules/httpd/manifests/init.pp partial

file { $docroot:
  ensure  => directory,
  owner   => $maint_user,
  group   => $httpd_user,
  selrole => 'object_r',
  seluser => 'system_u',
  seltype => 'httpd_sys_content_t',
}
    

And finally we're creating the virtual host config files. Note here that we've added the notify parameter which will restart the service any time one of the config files changes, as well as the require param to ensure this is all done after the conf.d directory is created.

./modules/httpd/manifests/init.pp partial

  $sites.each |$index, $site| {
    file { "/etc/httpd/conf.d/${index}-${site['domain']}.conf":
      content => template('httpd/virtual_host.conf.erb'),
      notify  => Service['httpd'],
      require => File['/etc/httpd/conf.d'],
    }
  }
    

Best Practices

Puppet-Lint

Puppet has a set standard for formatting module files and there's a gem to help you follow that standard.

gem install puppet-lint

This will install puppet-lint and then you can run it in your module directory with puppet-lint to see (and fix) any standardization issues.

Revision Control (git)

Revision Control Your Configurations!

The best part, to me, about Puppet is that it lets me see how I last left my servers configured. If you implement revision control with git then you can also see where they've been over time and make it easy to rollback any changes.

Next

Run an example