Puppet29 April, 2015

Managing Puppet modules on Vagrant with Librarian-Puppet

When it comes to source code control I'm a purist: I avoid storing any derived files (ex. generated CSS) or third-party code. So when I started using PuPHPet to generate Vagrant and Puppet configurations, I was dismayed by the gigantic /puphpet/puppet/modules directory. 3,000 files totaling 5 MB?! There must be a better way.

Instead of keeping the Puppet dependencies in our repository, librarian-puppet lets us automatically load these modules during setup of the Vagrant box. In our Vagrantfile we add a new setup script:

config.vm.provision :shell, :path => 'puphpet/shell/install-ruby.sh'
config.vm.provision :shell, :path => 'puphpet/shell/install-puppet.sh'
config.vm.provision :shell, run: 'always' do |s|    s.path = 'puphpet/shell/install-puppet-modules.sh'    s.args = '/vagrant/puphpet/puppet'endconfig.vm.provision :shell, run: 'always' do |s|
    s.path = 'puphpet/shell/windows-path-limit-fix.sh'
    s.args = '/vagrant/node_modules'
end

The setup script is based on the bootstrap script from vagrant-puppet-librarian.

#!/bin/sh

# Temporary directory in which to install puppet modules.
# Avoids issues with writing to a VirtualBox shared folder.
TMP_FOLDER="/var/tmp/install-puppet-modules/"

install()
{
    # Skip install if the temp folder already exists
    if [ -d "$TMP_FOLDER" ]; then

        echo "Skipping puppet dependencies because the temporary installation folder already exists"

    else

        # Create the temp folder and populate it with the Puppetfile, Puppetfile.lock, etc.
        mkdir -p $TMP_FOLDER && cd $TMP_FOLDER
        cp -r $1/* ./

        # NB: librarian-puppet might need git installed. If it is not already installed
        # in your basebox, this will manually install it at this point using apt or yum
        GIT=/usr/bin/git
        APT_GET=/usr/bin/apt-get
        YUM=/usr/sbin/yum
        if [ ! -x $GIT ]; then
            if [ -x $YUM ]; then
                echo "Installing git via yum"
                yum -q -y install git-core
            elif [ -x $APT_GET ]; then
                echo "Installing git via apt-get"
                apt-get -q -y install git-core
            else
                echo "No package installer available. You may need to install git manually."
            fi
        fi

        if [ `gem query --local | grep librarian-puppet | wc -l` -eq 0 ]; then
          echo "Installing the librarian-puppet gem"
          gem install librarian-puppet
        fi

        echo "Installing puppet dependencies"
        librarian-puppet install

        # Copy the installed modules back to the project
        rsync -Lr --inplace $TMP_FOLDER/.librarian $1/
        rsync -Lr --inplace $TMP_FOLDER/modules $1/
        rsync -Lr --inplace $TMP_FOLDER/Puppetfile.lock $1/
    fi
}

if [ ! "$1" -o -z "$1" ]; then
    echo "You must specify a destination for the Puppet module installation (recommend an absolute path)"
elif [ ! -d "$1" ]; then
    echo "Puppet module installation destination $1 must already exist"
else
    install $1
fi

librarian-puppet reads from our Puppetfile -- also provided by PuPHPet -- for a list of all the Puppet dependencies to download. It behaves similar to the Windows path limit fix:

The modules install to a temporary directory because I had issues running librarian-puppet install inside the /vagrant directory, presumably because it's shared with Windows. After installation, the modules are copied to /vagrant/puphpet/puppet/modules so Vagrant's setup can continue as usual.
If the temp directory already exists during setup, the script assumes that the Puppet dependencies are ready and aborts.
Temporary files are stored in /var/tmp to persist during restarts, and the run: 'always' parameter in the Vagrantfile ensures that the script will fire during both vagrant reload and vagrant up.

Finally we add two entries to our .gitignore.

puphpet/puppet/.tmp/
puphpet/puppet/modules/

Now we can keep our repository lean while leveraging the power of third-party Puppet modules!

Drew