Using puppet in UEC/EC2: puppet support in Ubuntu images

One of the focus for the Lucid release cycle in the Ubuntu Server team is to improve the integration between puppet and UEC/EC2. I'll discuss in a series of articles how to setup a puppet infrastructure to manage Ubuntu Lucid instances running on UEC/EC2. I'll focus on the bootstrapping process rather than writing puppet recipes.

Today we'll look at configuring a puppetmaster into an instance and how to start instances that will register automatically with the puppetmaster.

We'll work with the Lucid Beta1 image on EC2. All the instances started through out this article will be based on this AMI.

Puppetmaster setup

Let's start by creating a puppetmaster running on EC2. We'll setup all the puppet configuration via ssh using a bzr branch on Launchpad: lp:~mathiaz/+junk/uec-ec2-puppet-config-tut1.

Start an instance of the Lucid Beta1 AMI using an ssh key. Once it's running write down its public and private DNS addresses. The public DNS address will be used to setup the puppetmaster via ssh. The private DNS address will be used as the puppetmaster hostname given out to puppet clients.

We'll actually install the puppetmaster using puppet itself.

Log on the started instance via ssh to install and setup the puppet master:

1. Update apt files:

   
   

   
   

   sudo apt-get update

   
   

   
   



2. Install the puppet and bzr packages:

   
   

   
   

   sudo apt-get install puppet bzr

   
   

   
   



3. Change the ownership of the puppet directory so that the ubuntu user can directly edit the puppet configuration files:

   
   

   
   

   sudo chown ubuntu:ubuntu /etc/puppet/

   
   

   
   



4. Get the puppet configuration branch:

   
   

   
   

   bzr branch --use-existing-directory lp:~mathiaz/+junk/uec-ec2-puppet-config-tut1 /etc/puppet/

   
   

   
   

   Before doing the actual configuration let's have a look at the content of the /etc/puppet/ directory created from the bzr branch.

   
   

   The layout follows the recommended puppet practices. The puppet module available in the modules directory defines a puppet::master class. The class makes sure that the puppetmaster package is installed and that the puppetmaster service is running. The manifests/puppetmaster.pp file defines the default node to be configured as a puppetmaster.

   
   



5. We'll now run the puppet client to setup the instance as a puppetmaster:

   
   

   
   

   sudo puppet /etc/puppet/manifests/puppetmaster.pp

   
   

   
   

Starting a new instance

Now that we have puppetmaster available in our cloud we'll have look at how a new instances of the Lucid Beta1 AMI can be started and automatically setup to register with the puppetmaster.

We're going to use the cloud-config puppet syntax to boot an instance and have it configure itself to connect to the puppetmaster using its user data information:

1. On the puppetmaster instance create a user-data.yaml file to include the relevant puppetmaster configuration:

   
   

   
   

   cp /usr/share/doc/cloud-init/examples/cloud-config-puppet.txt user-data.yaml

   
   

   
   



2. Update the server setting to point to the puppetmaster private dns hostname. I also strongly recommend to include the puppmaster ca certificate as the ca_cert setting.

   
   

   The example certname setting uses a string extrapolation to make each puppet client certificate unique: for now %i is replace by the instance Id while %f is replaced by the FQDN of the instance.

   
   

   The sample file has extensive comments about the format of the file. One of the key point is that you can set any of the puppet configuration options via the user data passed to the instance.

   
   

   Note that you can remove all the comments to make the user-data.yaml file easier to copy and paste. However don't remove the first line (#cloud-config) as this is used by the instance boot process to start the puppet installation.

   
   



3. Launch a new instance using the content of the user-data.yaml file you've just created as the user-data option passed to the new instance.

   
   



4. You can watch the puppetmaster log on the puppetmaster instance to see when the new instance will request a new certificate:

   
   

   
   

   tail -f /var/log/syslog

   
   

   
   



5. After some time you should see a request coming in:

   
   

   
   

   puppetmasterd[2637]: i-fdb31b96.ip-10-195-18-227.ec2.internal has a waiting certificate request

   
   

   
   

   During the boot process of the new instance the puppet cloud-config plugin used the user-data information to automatically install the puppet package, generate the /etc/puppet/puppet.conf file and start the puppetd daemon.

   
   



6. You can then approve the new instance:

   
   

   
   

   sudo puppetca -s i-fdb31b96.ip-10-195-18-227.ec2.internal

   
   

   
   



7. Watching the puppetmaster log you'll see that after some time the new instance will connect and get its new manifest compiled and sent:

   
   

   
   

   puppetmasterd[2637]: Compiled catalog for i-fdb31b96.ip-10-195-18-227.ec2.internal in 0.03 seconds

   
   

   
   

In conclusion we now have an instance acting as a puppetmaster and have a single user-data configuration for the whole puppet infrastructure. That user data can be passed to new instances which will automatically register with our puppetmaster.

Even though we're able to make all our instances automatically register with our puppetmaster we still need to manually sign each request as outlined in step 6 above. We'll have a look at automating this step in the next article.