I don’t know why, but it took me some time to figure out, and wrap my head around the fact that EVERYTHING in ICINGA is a kind of object. And I do mean Everything. a Host? It’s a Host Object. A service? a Service Object. a check command? a CheckCommand Object. To me, at least, it took a while to figure this out.In my head. An object was something physical, like a Host and CheckCommands were, well, commands. In my mind they were not even faintly named in the same manner. This lead to much struggling, frustration and  ore, but once that concept crawled into my thick skull, things were much easier to do. a Cool thing about ICINGA 2 and onwards are ‘Configuration Apply Rules’ . This allows you to, for example, create a Service let’s say it checks the status of Ubuntu servers’ ‘apt’ package management system and the use apply rules to apply it to all Ubuntu servers. This can  save you a lot of configuration, both in time and work, as it is now applied to all relative hosts. In my opinion, this is a much better way to do things.

Say, for instance, you wanted to check if your home server’s SAMBA is running and that a share can be accessed. To configure this is quite easy (I will be referring to a standard installation’ files with this. You can, however, do things in a way that is more comfortable for you, as I have done.

Step 1

Make sure that the necessary CheckCommand objects exists. This can be added later as well, but for the tutorial I will be handling it in this order. We use the CheckCommand objects to test for basically anything that can be tested with a script. To do this, you must give the object a name, along with a path to the file/script that is to be executed and the neccesary arguments for said script, as well as any environment variables where applicable.

(Call me stupid, and maybe you can clarify this for me, but I do not know the difference between a runtime or environment variable. I have tried to figure it out, I have scoured google, but unfortunately, I still do not know the difference. Or maybe I do know it, just not the official names for them, as was the case with a class’ method and so on)

The format for this:

object CheckCommand "commandname" {
	import "plugin-check-command"
	command = [ “<parameter>”, “<Path/To/Script>” ]
	arguments = {
		"-H" = "$AddressToLookup$"
		"-s" = "$ServerToUse$"
	}
}

First, at the beginning of the definition, we must tell ICINGA that the definition is for a checkcommand (object CheckCommand).

Inside (in between) the quotation marks is the name that the Host or service will use to reference the command.

Next the template to use for the command is imported. this sets many default values, that can be overridden in the definition. there are templates for all object types. For CheckCommands, hosts, services and the rest as well.

Next we have the line command = [ “<parameter>”, “</Path/To/Script>”, “<parameter>” ].

This is by far the longest line in this definition, and it will be quite a lengthy explanation as well.

Firstly, you may have already noticed, that the command is not a string. It took me quite a while to figure out, and I cannot remember exactly how I figured it out.

The line is in the format of an array. That was very funny at first, but I got used to it, and it does fit. The command will be built in the same order that it was given here. So, basically, this command will end up looking something like this:

<parameter> <Path/To/Script>

The first parameter can be the interpreter, as in my case ‘php’ to have the script run/executed as PHP (or another interpreter). You can also add something like “sudo”. It is basically a prefix.

The next part is the full path to the script to be executed. Plain and simple. I would recommend using the full path, as it then causes less confusion.

Next in the definition an ‘arguments’ array is visible. This is, you guessed it, the list of arguments that is to be passed on to the script (like `u Username p Password’). They are added to the end of the script. Each keyvalue pair is the argument identifier and the argument value. The format for this is: “ArgumentIdentifier” = “ArgumentValue”

In the example above there are placeholders for variables that can be passed to the command, hence the dollar signs in the arguments array. More on this later.

In the end the command will look for example something like this:

“sudo /path/to/scipt -a argument -or two”

And that is something that can be executed. That is a Check Command object. They, I think, can be used to check the status of either a host or a service.

NOTE: If using sudo as prefix, ensure that the permissions are set correctly (that, for the necessary file, the user nagios can execute it as root without providing a password). This is done with the sudoers file.

Step 2

Defining the Host. (Your server at home).

This is actually quite easy, at least it is easier than you might think.

The format for this:

object Host "machinename" {
	# Import the default host template defined in `Templates.conf`.
	import "generic-host"
	display_name = "A Well Formed String"
	address = "host.name"
	vars.os = "Linux"
	vars.notification.mail = {
		groups = [ "support" ]
	}
}

 

This is, once again, it. I will explain everything. Just remember that, for both hosts and services, and maybe the rest of the objects as well, there are much more options that can be configured. Let us go through this line by line, as with the CheckCommand explanation.

First, at the beginning of the definition, we must tell ICINGA that the definition is for a Host (object Host).

Inside (in between) the quotation marks is the name that the Host or service will use to reference the command by the machine. This needs to be a string, as plain as possible and not necessarily easy for a human to read (that is coming later in the definition)

As with the CheckCommand, we need to import the necessary template, generi-host in this case. This sets a whole lot of default values so the it is not necessary to set them for every host. The values can then be overridden as needed in the host definition, as is done with the display name and so on.

Next is setting the display_name (the Human Readable Name) for the host. This can be anything as long as it’s a valid string. It should be placed in quotation marks so that it is a well formed string. This will (as the name suggests) be displayed everywhere that the host is to be displayed, such as on the web interface.

The address line is used to specify the IPv4 address or a hostname for the host, address6 is alternate for IPv6 addresses and hostnames resolving to a IPv6 address.

Another awesome thing about ICINGA is that you can now specify custom variables in a host, service or just about any definition. This is done in/by placing them in the vars array in the following format:

vars. = “Value”

Think about the possibilities that this, along with the apply rules open up. This is also the way to pass variables to command, but more about that later. Although I haven’t used it yet, you can also place scripts here. there are many possibilities to this.

This host definition has 3 custom variables set.

  • It has vars.os set to Linux because it is a linux server and this will make it easier to apply a hostgroup, if we have more than one active Linux host.
  • It also has a value set that is also an array, the notification.mail key. We are going to use this to set notifications for the service.

Step 3

The next step is to define the services to be checked on a host. a Service is anything that can be checked with a script. This makes it a quite broad definition, as there are many thing that can be checked with a script.

a Few examples:

  • a Wireless router. This needs to be on the network and is therefore able to be reached VIA a ping request and that can be scripted.
  • a Web server. a Script can be written to check if the web server is reachable and therefore can be connected to. It can also be made to ensure that the expected page, that should be there, is served, along with many more tests. That makes it an ideal thing that can be tested
  • an SSH server on something can be checked to be sure it can be connected to.

Luckily ICINGA 2 comes with many checks (known as plugins). That means that there are many checks that can be done straight away and that you do not have to write your own for everyday/mundane things. There is also a project dedicated to these plugins, available at https://www.monitoringplugins.org/ that can be downloaded and used quite easily.

The format to define a service:

	apply Service "machinename" {
		import "generic-service"
		display_name = "Human Readable Name"
		check_command = "CheckCommand"
		assign where host.name == "machinename"
	}

We start the definition by telling ICINGA that this is a ServiceObject definition with the apply Service part on the first line. In between the quotation marks comes the machinename for the service. This needs to be unique per host . You can have multiple services with the same name, as long as each is applied to a different host.

Once again, the display_name value is exactly the same as with the host definition. This can be anything as long as it’s a valid string. It should be placed in quotation marks so that it is a well formed string. This will (as the name suggests) be displayed everywhere that the host is to be displayed, such as on the web interface.

The check_command value is a CheckCommand object that is available, or, to put it differently, a CheckCommand object that you have specified, or was provided with the ICINGA 2 package, such as the hostalive command. See how things are starting to work together? It is beautiful.

The next step is to apply it to a the correct hosts. This is done with the ‘Configuration Apply Rules’ that I mentioned previously. In the example above, the service will be applied to the host with the macine name machinename (not very original I know).

That is all that is necessary for a very basic monitoring setup. The next step is to configure notifications. I’ll post that next.

Feel free to contact me with queries, question, suggestions and so on that might help and I’ll investigate and if necessary post an update.

NOTE: this configuration was not tested, so it might contain a few bugs. If you encounter any, please let me know so that I can update the information.

Enjoy and happy monitoring!