Foreman and Puppet 5 – the Hard Way

As of this writing Puppet 5 isn’t supported by Foreman’s latest version (1.15.2) yet. I saw some discussions or tickets in the community about Puppet 5 but I can’t wait for a new release. I want to integrate Foreman with a separately installed Puppet 5 for several reasons:

  • I don’t like foreman installer’s idea of encapsulating declarative configuration management code (under the hood, the installer uses Puppet code to do the job) into an complicated imperative script. You may argue that it’s not so imperative since it saves parameters to a location and it’s idempotent – alright if you like and understand the long list of options available. It’s an anti-pattern of configuration management. It’s a tool for sysadmins and modern sysadmins should be familiar with configuration management tools. (Update 3 months later – I just discovered foreman’s deployment tool – forklift. It wraps foreman installer with Ansible playbooks. Hooray! Finally it’s complex enough to be enterprise ready!)
  • Why Puppet 4 when 5 is out? The changes are not so radical like from 3.8 to 4 so API should be compatible.
  • I’m new to Foreman and some manual steps helps me understand it. I don’t know what are smart proxies and how various proxies work by just reading the manual.

Install Foreman

I won’t elaborate on Puppet 5 installation here since it’s very straightforward. Let’s start right from the foreman-installer. I used the following parameters:

foreman-installer \
--puppet-server=false \
--foreman-proxy-puppet=false \
--foreman-proxy-puppetca=false \
--no-enable-puppet \
--foreman-db-type=postgresql \
--foreman-db-manage=false \
--foreman-db-host=localhost \
--foreman-db-database=foreman \
--foreman-db-username=foreman \


  • I don’t want the installer to do anything Puppet related so I added a lot of safety nets. The manual recommends Foreman still installs a Puppet and proxy its CA even if you’re integrating with a standalone Puppet. But in my experience this is not a hard requirement even if you have no Puppet server running on Foreman’s host.
  • The database needs to be created beforehand. I installed PuppetDB along with Puppet and it had a PostgreSQL instance setup so I just created a database in it for convenience. Here comes the annoyance of foreman-installer – although I set –foreman-db-manage=false, it still installed a PostgreSQL of its own and tried to start it. It failed to start because port 5432 had already been bound by PuppetDB’s instance and I like that fact.
  • Things turns out to be a little easier. In a lot of Foreman’s configuration files, the certificate signed by Puppet (the host is acting as both master and minion) is correctly found and configured to secure Foreman and proxy’s communication with Puppet master.

Proxy Puppet and Puppet CA

If you look into Infrastructure – Smart Proxies, there’s already one on the same node’s port 8443.

A Smart-Proxy is located on or near a machine that performs a specific function and helps foreman orchestrate the process of commissioning a new host. Placing the proxy on or near to the actual service will also help reduce latencies in large distributed organizations.

In other words, a smart proxy is a web API that wraps some function. Foreman server sends it instructions to do some specific job, or get information from a source.

In the case of Puppet, Foreman can import Puppet classes via the proxy and do a lot of other jobs if you enable the functions. As for Puppet CA, the smart proxy must be on the same host as Puppet because it needs to use the puppet executable to sign certificates.

Smart proxy settings are located in /etc/foreman-proxy/settings.d/

After enabling Puppet proxy in puppet.yml (the installer got puppet_version right even if it’s 5.0.1!), I tried to import Puppet classes in Foreman but got this:

2017-07-27 07:14:34,893 ERROR [qtp1085091490-83] [p.t.a.rules] Forbidden request: access to /puppet/v3/environment_classes (method :get) (authenticated: true) denied by rule 'puppetlabs deny all'.

I had to add this into /etc/puppetlabs/puppetserver/conf.d/auth.conf:

            match-request: {
            path: "/puppet/v3/environment_classes"
               type: path
               method: get
            allow: "*"
            sort-order: 500
            name: "puppetlabs environment classes"

When I enabled Puppet CA proxy in puppetca.yml, I got this:

E, [2017-07-27T07:41:07.985454 ] ERROR -- : Failed to list certificates: Execution of puppetca failed, check log files

So I set loglevel to DEBUG in /etc/foreman-proxy/settings.yml and restarted the proxy to see what’s happening:

D, [2017-07-27T07:43:44.309887 ] DEBUG -- : Found puppetca at /opt/puppetlabs/bin/puppet
D, [2017-07-27T07:43:44.309962 ] DEBUG -- : Found sudo at /usr/bin/sudo
D, [2017-07-27T07:43:44.310026 ] DEBUG -- : Executing /usr/bin/sudo -S /opt/puppetlabs/bin/puppet cert --ssldir /etc/puppetlabs/puppet/ssl --list --all
W, [2017-07-27T07:43:44.319046 ] WARN -- : Failed to run puppetca:
E, [2017-07-27T07:43:44.319213 ] ERROR -- : Failed to list certificates: Execution of puppetca failed, check log files
D, [2017-07-27T07:43:44.319240 ] DEBUG -- : Failed to list certificates: Execution of puppetca failed, check log files

Looks like a problem with the proxy process (user is foreman-proxy) running sudo. To fix, I created /etc/sudoers.d/foreman with content:

foreman-proxy ALL = NOPASSWD: /opt/puppetlabs/bin/puppet cert *
Defaults:foreman-proxy !requiretty

Facts and Reports

A very nice outcome of connecting Foreman with Puppet is that we can have a list of hosts managed by Puppet, and view all facts of them. For this you need to look at Puppet Reports and Facts and the ENC.

Puppet 5 has a slightly different directory structure to store facts, so I had to do the following:

mkdir -p /etc/puppetlabs/puppet/yaml/foreman
cd /opt/puppetlabs/server/data/puppetserver/yaml
mv facts /etc/puppetlabs/puppet/yaml
ln -s /etc/puppetlabs/puppet/yaml/facts facts
chown -R puppet:puppet /etc/puppetlabs/puppet/yaml

# Run puppet agent to send report:
puppet agent -t

Now you should be able to see the host appearing in Foreman and you can see all facts of it.


While Foreman has a lot of nice and powerful tools, I would avoid assigning Puppet classes to nodes in Foreman and using it as an external node classifier. Configuration should be in a central place so that you don’t have to search both code and Foreman (where it seems hard to search for such assignments?) to troubleshoot a problem.

Next I’ll look into bare-metal/VM lifecycle management and see if there’s anything worth noting.

Leave a comment

Your email address will not be published. Required fields are marked *

Prove your intelligence before hitting * Time limit is exhausted. Please reload CAPTCHA.