Saturday, February 4, 2017

Troubleshooting some Common Errors in Running Puppet Agent

Here I am going to guide you on how to troubleshoot some common errors in running puppet agent(client).
1. SSL Certificate Error

Puppet uses self signed certificates to communicate between Master(server) and Agent(client). When there is mismatch or verification failure, following error logs may display on the puppet agent.

Error log in Agent:
 
Warning: Setting templatedir is deprecated. See http://links.puppetlabs.com/env-settings-deprecations
  (at /usr/lib/ruby/vendor_ruby/puppet/settings.rb:1139:in `issue_deprecation_warning')
Warning: Unable to fetch my node definition, but the agent run will continue:
Warning: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA: puppetmaster.openstacklocal]
Info: Loading facts
Error: Could not retrieve catalog from remote server: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA: puppetmaster.openstacklocal]
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
Error: Could not send report: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA: puppetmaster.openstacklocal]

Error log may be displayed as following too.

Error: Could not request certificate: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA: puppetmaster.openstacklocal]

Solution:   

Following is the simplest solution. (recommended only if you are using a single Agent node).
Enter the following commands with root permissions,
1) on agent>> 
  • rm -rf /var/lib/puppet/ssl/
2) on master>> 
  • puppet cert clean --all
  • service puppetmaster restart 
Then try to run agent again and the error should have been resolved.

A more elegant solution:

Usually when you encounter this kind of ssl issue, what you can do is first delete the ssl directory in the Agent.
   
     rm -rf /var/lib/puppet/ssl/

Then try to run Agent again and then the puppet will show you exactly what to do; something similar to below..

On the master:
  puppet cert clean node2-apim-publisher.openstacklocal
On the agent:
  1a. On most platforms: find /home/ubuntu/.puppet/ssl -name node2-apim-publisher.openstacklocal.pem -delete
  1b. On Windows: del "/home/ubuntu/.puppet/ssl/node2-apim-publisher.openstacklocal.pem" /f
  2. puppet agent -t


Do what puppet says as above and start puppet agent again.

I recommend to follow this solution as so here you are not deleting all the certificates related to each puppet agent. You are deleting only the relevant agent's certificate only.


2. "<unknown>" Error due to hira data file syntax error

Error log in Agent:
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: (<unknown>):


Solution:

This error log with message “<unknown>” is mostly occurred due to a syntax error in a related hiera data .yaml file. So go through your hiera data files again. May be you can use some .yaml hiera data file validation online tools to validate your .yaml files. (eg. http://www.yamllint.com/)

3.  Agent node not defined on Master

Error log in Agent:

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not find default node or by name with 'node2-apim-publisher.openstacklocal, node2-apim-publisher' on node node2-apim-publisher.openstacklocal
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run


("'node2-apim-publisher" is the hostname of my agent)

Solution:

This error occurs when you have not defined your Agent, in your master's related agent-node-defining  .pp file. This file exists usually in /etc/puppet/manifests/ of the Master and it's name can be site.pp or node.pp. You have to define the agent nodes using their hostnames in this file.

Sample node definition is as follows.

node "host-name-of-agent" {
 
}