Create Puppet modules with solid foundations

Over the last year , the team at PuppetLabs have done a great job making the forge a better place. Also, during this time, they have been pushing puppet module authors to create better modules. By better, three characteristics can be highlighted here; testable – thanks to rspec-puppet, style compliant – thanks to puppet-lint and input-validated – thanks to the validation functions on the stdlib puppetlabs’ module. This post walk you through the process of creating a puppet module taking advantage of these.

Note : It is assumed that you have Puppet, gems and bundler already installed

Create the module

Simply run the following command. If you are root by default the folder will be on /etc/puppet/modules/, if you are logged in as a regular user you’ll find it on ~/.puppet/modules (It relies on $modulepath)

puppet module generate yguenane-mkdir

This will generates boilerplate for a new mkdir module by creating the directory structure and files. Note : the module name needs to have the pattern forgeusername-modulename.

Since the forge last update, new files got pushed as first class citizen file :

  • README : This one still gets displayed as the home page of your module profile
  • CHANGELOG : The changelog gets it’s own tab. One can quickly see the activity of a module without going to the module project page
  • LICENSE : The license file gets it’s own tab also. The license tab contains – as you would guess – the license text

Bring a set of extra features to your module

Gemfile

Create a .gemfile file at the root of your module with the following content

source 'https://rubygems.org'

puppetversion = ENV['PUPPET_VERSION']
gem 'puppet', puppetversion, :require => false
gem 'puppet-lint'
gem 'rspec-puppet'
gem 'puppetlabs_spec_helper', '>= 0.1.0'

Install the bundled gems with the following command

bundle install --gemfile .gemfile

Some of you might ask why .gemfile instead of the traditional and conventional Gemfile, this is PuppetLabs explanation :

Aside: Gemfiles and the Puppet Module Tool. In our modules, we name our Gemfiles .gemfile instead of Gemfile. Dotfiles are automatically ignored when packaging with the Module Tool. We recommend this practice in order to avoid clutter on end-users’ systems, but it is not strictly required.

Everything we need to start working on a puppet module has been installed. Now time for configuration

Rakefile

First of all, a Rakefile needs to be created at the root of your module with the following content

require 'rubygems'
require 'puppetlabs_spec_helper/rake_tasks'
require 'puppet-lint'

Then execute BUNDLE_GEMFILE=.gemfile bundle exec rake help to see what task one can run

Note: Here we preceded our bundle command by setting the BUNDLE_GEMFILE environment variable to tell bundle which file to use. You can rename your .gemfile to Gemfile, since this is the default file you will not need to specify it, but remember to rename it before packaging the module.

rake build            # Build puppet module package
rake clean            # Clean a built module package
rake coverage         # Generate code coverage information
rake help             # Display the list of available rake tasks
rake lint             # Check puppet manifests with puppet-lint
rake spec             # Run spec tests in a clean fixtures directory
rake spec_clean       # Clean up the fixtures directory
rake spec_prep        # Create the fixtures directory
rake spec_standalone  # Run spec tests on an existing fixtures directory

rspec-puppet

rspec-puppet is a project that brings rspec feature to test puppet module. It is lead by one of the puppet community member @rodjek. The necessary gems has been installed when you ran bundle install earlier.

rspec-puppet provide the rspec-puppet-init binary to help you getting started with. Simply run rspec-puppet-init at the root of your module and a bunch of directory will be created under your spec/ folder.

The only thing to configure here is the spec/spec_helper.rb file that should contain this content

require 'rubygems'
require 'puppetlabs_spec_helper/module_spec_helper'

.fixtures.yml

Most probably than not, your module will depend on other modules. Thus your rspec tests will need to know about those modules in order to pass. This is the exact purpose of the .fixtures.yml file located at the root of your module. Before running tests, it will download the module it depends on and set them as fixtures, so your test can actually be run against them.

A .fixtures.yml file looks like this

fixtures:
  repositories:
    stdlib: "git://github.com/puppetlabs/puppetlabs-stdlib"
    dep2: "git://github.com/dep2author/dep2author-dep2module"
  symlinks:
    mymodule: "#{source_dir}"

.travis.yml

This file aims to tell travis-ci.org how to run your test suite. For those who does not know travis-ci it is a hosted continuous integration service that you can use freely to test your open source project. Refer to travis-ci.org official website for more information on how to use it.

This is the travis configuration I use for my own modules

language: ruby
rvm:
  - 1.8.7
  - 1.9.3
  - ruby-head
script:
  - "rake spec SPEC_OPTS='--format documentation'"
env:
  - PUPPET_VERSION="~> 2.7.0"
  - PUPPET_VERSION="~> 3.0.0"
  - PUPPET_VERSION="~> 3.1.0"
matrix:
  allow_failures:
    - rvm: ruby-head
  exclude:
    - rvm: 1.9.3
      env: PUPPET_GEM_VERSION="~> 2.7.0"
    - rvm: ruby-head
      env: PUPPET_GEM_VERSION="~> 2.7.0"
gemfile: .gemfile

Please refer to the official documentation for technical details, explanation of the keywords and key concept to extend this configuration file.

Use them all together

For the following test I will be using yguenane-mkdir module available on github. This sample module has been configured as mention above.

Lint

With the current version of this module executing the following command

BUNDLE_GEMFILE=.gemfile bundle exec rake lint

will output the following result

manifests/dir.pp - WARNING: defined type not documented on line 1
tests/init.pp - WARNING: line has more than 80 characters on line 5
tests/init.pp - WARNING: line has more than 80 characters on line 6
tests/init.pp - WARNING: line has more than 80 characters on line 9

Thanks to this feature you can make sure your module is conformed to PuppetLabs Style Guide. Some constraints can be too much restrictive for some situation (ie. 80chars) in order for puppet-lint to not complain about it you can disable this check by adding this line in your Rakefile – PuppetLint.configuration.send(“disable_check“) – where check is the condition being checked.

For example for the 80 characters constraint the line would be

PuppetLint.configuration.send("disable_80chars")

List of checks is available here

Rspec

With the current version of this module executing the following command

BUNDLE_GEMFILE=.gemfile bundle exec rake spec

will output the following result

..

Finished in 0.28395 seconds
2 examples, 0 failures

It ran the tests under spec/{classes,defines} folders, and 2 tests out of 2 passed.

Once your module is puppet-lint compliant and passes all tests, the module is ready to be uploaded to the forge. Simply use puppet module build to create the tarball and upload it.
Done ! A new puppet style-guide compliant and testable module is on the forge !

Extra

If you are using travis-ci do not forget to add the travis badge on your README that let people now the status of the builds. Information here

Conclusion

Even if it can be seen as a long process at first, it is definitely worth it :

  • Contribution to the module becomes simpler, by running the tests a contributor knows the module status during contributions
  • Testing on multiple puppet and ruby version becomes trivial using travis-ci.org
  • This point is more subjective, but when I see a puppet module with tests and changelog, I have a better feeling about the dedication one is putting into h(er|is) module

The more puppet module authors will do module this way, better the quality of the forge and the quality of the modules you can find there will be. QED

PS: The actual module is available at http://forge.puppetlabs.com/yguenane/mkdir

Advertisements

4 Comments on “Create Puppet modules with solid foundations”

  1. Michael says:

    Received an error:
    The source :rubygems is depreciated because HTTP requests are insecure. Use https://rubygems.org instead.

    Changed:
    source :rubygems
    to:
    source “https://rubygems.org/”

  2. Yop,

    Petite typo, remplacer :

    PuppetLint.configuration.send(“disable 80chars”)

    Par :

    PuppetLint.configuration.send(“disable_80chars”)

    Tchuss 🙂

  3. […] As our code base grows, and modules are dependant on other modules, the case of testing our code becomes more and more important, this is not only related to Puppet code, this is relevant to other languages as well. To get you started on rspec, go to this great blog posting […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s