Using the FreeIPA PKI with Puppet

At my cur­rent employer, I’ve setup FreeIPA to deal with the whole DNS/LDAP/Kerberos/PKI ker­fuf­fle. At pre­vi­ous firms I’ve done a DIY setup for this: CentOS 5, tied into OpenLDAP, MIT Kerberos, and Cobbler—which required I back­port OpenLDAP 2.4 and the ver­sion of MIT Kerberos that’s capa­ble of using LDAP as a back­end data­base to CentOS 5. On the SSL side, I let Puppet man­age it’s own PKI and just used gnoMint for ser­vice certificates.

It worked pretty well, but I never got around to writ­ing a web GUI that was sup­posed to sit in front of all of it — mainly because FreeIPA already existed and I secretly wished it would use OpenLDAP rather than FedoraDS (since renamed to “389”). Alas, three years later and it still hasn’t hap­pened, and now I’m at another new finan­cial indus­try startup. I could screw around for a few months get­ting OpenLDAP, Kerberos, Puppet, SSSD, and Foreman all talk­ing together, or I could setup FreeIPA.

Aside from not using my LDAP direc­tory of choice, it also doesn’t do DHCP or Kickstart, and it doesn’t have any sort of built-in sup­port for Puppet, so I’ve got to hack it to work together. The first step is to setup Puppet to use the FreeIPA-integrated PKI with Passenger. Because I’m try­ing to do things the FreeIPA way, I’m also going to use mod_nss.

So the first ques­tion is always the same: why do I want to do all this? There are really only two rea­sons: the first is ease of scal­ing, the sec­ond is ease of man­age­ment. If I’m using the builtin PKI that ships with pup­pet, I’ve got a scal­a­bil­ity prob­lem because the CA cert is tied to the host­name of the first pup­pet­mas­ter. There are newer fea­tures in pup­pet that let you work around this, some of which we’ll use to ensure we’re using FreeIPA’s PKI prop­erly, but I haven’t played with them.

The sec­ond issue is ease-of-management. FreeIPA is already main­tain­ing a secure cer­tifi­cate dis­tri­b­u­tion chan­nel via cert­mon­ger, which allows users to pull down per-service cer­tifi­cates tied to Kerberos ser­vice prin­ci­pals. Is it really worth hav­ing a sec­ond PKI tree that pup­pet is man­ag­ing itself when FreeIPA is already doing this for you out of the box? Only if you’re lazy about it.

On the down­side, there is the issue of secu­rity. FreeIPA out of the box only sup­ports a sin­gle toplevel CA, which means that all your cer­tifi­cates (IPA host certs, pup­pet certs, Website certs, etc.) are all in a sin­gle secu­rity domain — there’s no built-in way to restrict this access to pup­pet. Users can’t invent certs, of course, but any cert with the right host­name can be used to authen­ti­cate to pup­pet, because they share the same trust hier­ar­chy (a “third way” option would be to have FreeIPA sign a puppet-specific CA, which is then used to sign the client certs, restrict­ing the domain to just the pup­pet CA and the certs it has signed).

Lets assume that you’re OK with han­dling the issue of a sin­gle hard­coded autho­riza­tion domain shared by mul­ti­ple ser­vices (in this case, any­thing using an SSL cert from FreeIPA that has the DNS host­name in it), if for no other rea­son than this blog post would be a com­plete waste of time with­out that assump­tion. I’m also going to assume that you already have a work­ing FreeIPA setup. Roughly, the steps we’re going to look into imple­ment­ing are:

1. Create ser­vice cer­tifi­cate for puppetmaster
2. Setup the pup­pet­mas­ter to use Apache with Passenger and mod_nss
3. Create the ser­vice cer­tifi­cates for the pup­pet clients
4. Setup the pup­pet clients
5. Verify every­thing is working

The first step is to get get your pup­pet server into FreeIPA and cre­ate the ser­vice cer­tifi­cate, a fairly triv­ial process. On the FreeIPA server (or at least some­thing with the admin tools installed), replace the IP address and host­name as appro­pri­ate, and sea­son to taste:

ipa host-add puppet.example.com --ip-address=10.10.10.10 --password=secret
ipa service-add puppetmaster/puppet.example.com
ipa service-add puppet/puppet.example.com

On your puppetmaster:

yum --nogpgcheck --localinstall
   http://passenger.stealthymonkeys.com/fedora/16/passenger-release.noarch.rpm
yum install mod_nss mod_passenger
ipa-client-install --password=secret
systemctl stop puppetmaster.service
ipa-getcert -K puppetmaster/puppet.example.com
            -d /etc/httpd/alias
            -n puppetmaster/puppet.example.com
ipa-getcert -K puppet/puppet.example.com
            -D puppet.example.com
            -k /etc/puppet/ssl/private_keys/puppet.example.com.pem
            -f /etc/puppet/ssl/public_keys/puppet.example.com.pem
mkdir -p /var/www/puppet/public
cp /usr/share/puppet/ext/rack/files/config.ru /var/www/puppet

So now you’ve joined your pup­pet­mas­ter to IPA, installed the rel­e­vant soft­ware, and got­ten the SSL keys you need to iden­tify the server to clients installed. Next step is to con­fig­ure puppet.conf — it should look like this:

[main]
    logdir = /var/log/puppet
    rundir = /var/run/puppet

[agent]
    classfile = $vardir/classes.txt
    localconfig = $vardir/localconfig

[master]
    ca = false
    certificate_revocation = false

So what all does this do? Well, firstly we’re remov­ing the ssldir option, which will make pup­pet use /etc/puppet/ssl by default (i.e. where we had IPA install the SSL certs). The rest of it is pretty stan­dard, save for the mas­ter con­fig, which has two options — the first one dis­ables the built-in PKI, and the sec­ond which dis­ables cer­tifi­cate revo­ca­tion (which we may be able to turn back on later).

That’s it for puppet’s con­fig­u­ra­tion, lets move on to the HTTPd con­fig­u­ra­tion. Firstly, lets clean up mod_nss, which installs a whole bunch of weird­ness in it’s ini­tial con­fig­u­ra­tion. Suffice to say, you only need this:

LoadModule			nss_module modules/libmodnss.so
AddType				application/x-x509-ca-cert .crt
AddType				application/x-pkcs7-crl    .crl
NSSPassPhraseDialog		builtin
NSSPassPhraseHelper		/usr/sbin/nss_pcache
NSSSessionCacheSize		10000
NSSSessionCacheTimeout		100
NSSSession3CacheTimeout		86400
NSSRandomSeed			startup builtin
NSSRenegotiation		off
NSSRequireSafeNegotiation	off

The rest of the stuff in that file (includ­ing the bizarre deci­sion to lis­ten on port 8443 by default) is unnec­es­sary, so you can sim­ply cut it out. With that out of the way, we can move on to the puppetmaster-via-apache con­fig­u­ra­tion. One thing to note here is that Puppet is writ­ten in ruby, and it oper­ates via HTTP, so there’s lit­tle rea­son it can’t run just like any other ruby web­site, even though it’s not strictly intended for user-facing content.

Anyways, on to the configuration:

Listen 8140
<VirtualHost _default_:8140>
	ServerName	puppet.example.com
	ServerAdmin	puppetmaster@example.com

	NSSEngine		on
	NSSCertificateDatabase	/etc/httpd/alias
	NSSNickname		"puppetmaster/puppet.example.com"
	NSSOptions		+StdEnvVars
	NSSEnforceValidCerts	on
	NSSVerifyClient		require
	NSSProtocol		SSLv3,TLSv1

	RequestHeader set X-SSL-Subject %{SSL_CLIENT_S_DN}e
	RequestHeader set X-Client-DN "/CN=%{SSL_CLIENT_S_DN_CN}e"
	RequestHeader set X-Client-Verify %{SSL_CLIENT_VERIFY}e

	PassengerHighPerformance	on
	PassengerStatThrottleRate	120
	PassengerUseGlobalQueue		on

	RackAutoDetect	off
	RailsAutoDetect	off
	RackBaseURI	/

	DocumentRoot	/var/www/puppet/public
	<Directory /var/www/puppet>
		Options		None
		AllowOverride	None
		Order		allow,deny
		Allow		from all
	</Directory>
</VirtualHost>

So the first bits are obvi­ous: setup Apache to lis­ten on port 8140 (the pup­pet­mas­ter port), and setup SSL as pro­vided by mod_nss. Of par­tic­u­lar note is the fact that we’re cre­at­ing envi­ron­ment vari­ables from the con­tents of the SSL cer­tifi­cate that we’re being pre­sented. My under­stand­ing (or at least hope) is that mod_nss will ver­ify the cer­tifi­cate has been signed, then gen­er­ate some envi­ron­ment vari­ables for the actual pup­pet appli­ca­tion based on that. In par­tic­u­lar, you need the +StdENvVars NSSOption to be set.

The next bit, set­ting the RequestHeaders, han­dles mung­ing the SSL cer­tifi­cate names in such a way as to let pup­pet val­i­date them. After that it’s set­ting some pas­sen­ger con­fig­u­ra­tion which I haven’t looked up, and point­ing it at the proper direc­tory. After that comes set­ting up the direc­tory, which is fairly sim­ple and already done — just cre­ate the direc­to­ries and copy the con​fig​.ru file that ships with pup­pet to the appli­ca­tion base.

And that’s it. Assuming you’ve setup every­thing prop­erly, your clients (in this case, just the local pup­pet dae­mon run­ning on your pup­pet­mas­ter) should be able to con­nect and start pulling their cat­a­logs (even if the cat­a­log is empty you’ll still get an suc­cess­ful run notice for the client in /var/log/messages)