Mike Weber's presentation on using Nagios with NRPE. The presentation was given during the Nagios World Conference North America held Sept 25-28th, 2012 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/nwcna

1. NRPE
Nagios Remote Plugin Executor
Mike Weber
mweber@spidertools.com

2. NRPE Concepts
Agent Required on Client
* plugin or script executes locally
* daemon runs on client (port 5666)
Nagios
* check_nrpe connects to client
Direct Checks
* connect and collect data from client
Indirect Checks
* connect to client, execute plugin to connect to second client
2012 2

3. Basic Concepts
2012 3

4. NRPE: Active and Indirect Checks
2012 4

5. NRPE Components: Client
NRPE Daemon
* daemon managed by xinetd
* security based on allowed_hosts
* runs on port 5666
* open firewall
NRPE Commands File (nrpe.cfg)
* check_nrpe connects to client
* command definitions
Plugins
* connect and collect data from client
2012 5

6. NRPE Components: Server
One Plugin
* check_nrpe (installed by default in XI)
Host and Service Definitions
* define the host
* define each service
2012 6

7. NRPE Variables

8. NRPE Variables: Client
Agent Install
* repository (CentOS, SUSE,Ubuntu, Debian, etc.)
* XI agent from wizard
* compile
Daemon Install
* repository (CentOS, SUSE,Ubuntu, Debian, etc.)
* compile
nrpe.cfg
* repository (CentOS, SUSE,Ubuntu, Debian, etc.)
* compile
Command Definitions
2012 8

9. Client Variables: Daemon and Config
Program
/usr/local/nagios/bin/nrpe
/usr/sbin/nrpe
Daemon Config
/etc/xinet.d/nrpe
/etc/nagios/nrpe
/etc/nagios3/nrpe (Ubuntu)
2012 9

10. NRPE: Daemon Config - nrpe
# default: on
# description: NRPE (Nagios Remote Plugin Executor)
service nrpe
{
flags = REUSE
socket_type = stream
port = 5666
wait = no
user = nagios
group = nagios
server = /usr/local/nagios/bin/nrpe
server_args = -c /usr/local/nagios/etc/nrpe.cfg --inetd
log_on_failure += USERID
disable = no
only_from = 127.0.0.1 192.168.5.99
}
2012 10

11. Client Variables: Commands File
Location
/usr/local/nagios/etc/nrpe.cfg (compile)
/usr/local/nagios/etc/nrpe/common.cfg (XI Agent)
/etc/nagios/nrpe.cfg (SUSE)
/etc/nagios3/nrpe.cfg (Ubuntu)
Definitions
* able to use arguments
* dont_blame_nrpe=1
command[check_users]=/usr/local/nagios/libexec/check_users -w $ARG1$ -c $ARG2$
* unable to use arguments
* dont_blame_nrpe=0
command[check_users]=/usr/local/nagios/libexec/check_users -w 2 -c 4
2012 11

12. Client Variables: Plugins Location
/usr/local/nagios/libexec
/usr/lib/nagios/plugins
/usr/lib64/nagios/plugins
2012 12

13. NRPE Compile

14. Why Compile NRPE and Plugins?
Eliminate Variables Across Distribution
* same daemon settings and location
* same nrpe definitions
* same commands locations and configuration (nrpe.cfg)
* same location for plugins
* same versions
Uniform Troubleshooting
* standards across distributions
* easier for multiple administrators
Uniform Upgrades
* method of upgrade the same
* upgrades will be the latest
2012 14

15. NRPE: Install Script
#!/bin/bash
if [[ $EUID -ne 0 ]]; then
echo "This script must be run as root" 1>&2
exit 1
fi
cd /tmp
wget http://sourceforge.net/projects/nagios/files/nrpe-2.x/nrpe-2.13/nrpe-2.13.tar.gz/download
tar zxvf nrpe-2.13.tar.gz
cd nrpe-2.13
yum install -y mod_ssl openssl-devel xinetd gcc make
./configure --with-ssl=/usr/bin/openssl --with-ssl-lib=/usr/lib
make
make install
make install-daemon
make install-daemon-config
make install-xinetd
cp sample-config/nrpe.cfg /usr/local/nagios/etc/
2012 15

16. Plugins: Install Script
#!/bin/bash
if [[ $EUID -ne 0 ]]; then
echo "This script must be run as root" 1>&2
exit 1
fi
cd /tmp
wget http://sourceforge.net/projects/nagiosplug/files/nagiosplug/1.4.16/nagios-plugins-
1.4.16.tar.gz/download
yum install -y gcc make mysql-devel mysql net-snmp
useradd nagios
groupadd nagcmd
usermod -a -G nagcmd nagios
tar zxvf nagios-plugins-1.4.16.tar.gz
cd nagios-plugins-1.4.16
./configure --with-nagios-user=nagios --with-nagios-group=nagios
make
make install
exit 0
2012 16

17. NRPE: Locations When Compiled
Daemon Configuration
/etc/xinetd.d/nrpe
Daemon Binary
/usr/local/nagios/bin/nrpe
Configuration File
/usr/local/nagios/etc/nrpe.cfg
Plugins
/usr/local/nagios/libexec
2012 17

18. NRPE: XI Agent

19. CentOS Agent: XI
cd /tmp
wget http://assets.nagios.com/downloads/nagiosxi/agents/linux-nrpe-agent.tar.gz
tar zxvf linux-nrpe-agent.tar.gz
cd linux-nrpe-agent
./fullinstall
Allow from: 192.168.5.184
Stopping xinetd: [ OK ]
Starting xinetd: [ OK ]
Subcomponents installed OK
RESULT=0
##########################################################
### ###
### Nagios XI Linux Agent Installation Complete! ###
### ###
##########################################################
2012 19

20. NRPE: Locations XI Agent
Daemon Configuration
/etc/xinetd.d/nrpe
Daemon Binary
/usr/local/nagios/bin/nrpe
Configuration File
/usr/local/nagios/etc/nrpe.cfg
include_dir=/usr/local/nagios/etc/nrpe
dont_blame_nrpe=1
/usr/local/nagios/etc/nrpe/common.cfg
Plugins
/usr/local/nagios/libexec
2012 20

21. common.cfg
### GENERIC SERVICES ###
command[check_init_service]=sudo /usr/local/nagios/libexec/check_init_service $ARG1$
command[check_services]=/usr/local/nagios/libexec/check_services -p $ARG1$
### MISC SYSTEM METRICS ###
#command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
command[check_users]=/usr/local/nagios/libexec/check_users $ARG1$
command[check_load]=/usr/local/nagios/libexec/check_load $ARG1$
command[check_swap]=/usr/local/nagios/libexec/check_swap $ARG1$
command[check_cpu_stats]=/usr/local/nagios/libexec/check_cpu_stats.sh $ARG1$
command[check_mem]=/usr/local/nagios/libexec/custom_check_mem $ARG1$
### YUM UPDATES ###
command[check_yum]=/usr/local/nagios/libexec/check_yum
### DISK ###
command[check_disk]=/usr/local/nagios/libexec/check_disk $ARG1$
command[check_ide_smart]=/usr/local/nagios/libexec/check_ide_smart $ARG1$
### PROCESSES ###
command[check_all_procs]=/usr/local/nagios/libexec/custom_check_procs
command[check_procs]=/usr/local/nagios/libexec/check_procs $ARG1$
### OPEN FILES ###
command[check_open_files]=/usr/local/nagios/libexec/check_open_files.pl $ARG1$
### NETWORK CONNECTIONS ###
command[check_netstat]=/usr/local/nagios/libexec/check_netstat.pl -p $ARG1$ $ARG2$
2012 21

22. Commands: nrpe.cfg
No Variables Allowed
dont_blame_nrpe=0
command <service_check> <path_to_plugin> <plugin> <arguments>
command[check_hda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/hda1
Using Variables
dont_blame_nrpe=1
command <service_check> <path_to_plugin> <plugin> <arguments>
command[check_hda1]=/usr/local/nagios/libexec/check_disk -w $ARG1$ -c $ARG2$ -p
$ARG3$
2012 22

23. Downside of Arguments
CentOS
check_cpu_stats!-a '-w 85 -c 95'
SUSE Enterprise
check_cpu_stats!-a -w 80 -a -c 90
2012 23

24. NRPE: Nagios Core

25. NRPE: Active Check
2012 25

26. Nagios Core: Service Checks
No Arguments
define service{
use generic-service
host_name centos
service_description Users
check_command check_nrpe!check_users
}
define service{
use generic-service,srv-pnp
host_name centos
service_description Mailq
check_command check_nrpe!check_mailq
}
Arguments
define service{
use generic-service
host_name centos
service_description Disk
check_command check_nrpe_arg!check_disk!80!90!/
}
2012 26

27. Nagios Core: Command Definition
No Arguments
define command{
command_name check_nrpe
command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
}
define command{
command_name check_nrpe_60
command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -t 60 -c $ARG1$
}
Arguments
define command{
command_name check_nrpe_arg
command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$ -a
$ARG2$ $ARG3$ $ARG4$
}
2012 27

28. Summary
No Arguments
define command{
command_name check_nrpe
command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
}
define service{
use generic-service
host_name centos
service_description Users
check_command check_nrpe!check_users
}
command[check_users]=/usr/local/nagios/libexec/check_users -w 2 -c 4
Arguments
define command{
command_name check_nrpe_arg
command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$ -a $ARG2$
$ARG3$ $ARG4$
}
define service{
use generic-service
host_name centos
service_description Disk
check_command check_nrpe_arg!check_disk!80!90!/
}
command[check_disk]=/usr/local/nagios/libexec/check_disk $ARG1$
2012 28

29. AIDE: Advanced Intrusion Detection Environment
Script
!/bin/bash
# Create 4 Hour Cron Job With AIDE
logfile=/tmp/aide
x=$(grep "Looks okay" $logfile | wc -l)
if [ $x -eq 1 ]
then
echo "All Systems Look OK"
stateid=0
else
echo "$(egrep "added|changed" /tmp/aide)"
stateid=2
fi
exit $stateid
Command in nrpe.cfg
command[check_aide]=/usr/local/nagios/libexec/check_aide
Service Definition
define service{
use generic-service
host_name bash
service_description AIDE
check_command check_nrpe!check_aide
}
2012 29

30. NRPE: Nagios XI

31. NRPE: Linux Server
2012 31

32. NRPE: Linux Server
2012 32

33. NRPE: Linux Server
2012 33

34. NRPE: Linux Server
2012 34

35. NRPE: Linux Server
2012 35

36. /etc/init.d
ls /etc/init.d
crond gpm iscsi lvm2-monitor named nscd rawdevices
snmpd sysstat dc_client haldaemon iscsid mcstrans netconsole
ntpd rdisc snmptrapd vsftpd dc_server halt killall
messagebus netfs portmap restorecond snmptt winbind dovecot
httpd kudzu multipathd netplugd postfix saslauthd sshd
xinetd functions iptables lm_sensors mysqld network postgresql-9.0
single syslog
2012 36

37. NRPE: Linux Server
2012 37

38. NRPE: Linux Server
2012 38

39. NRPE: Linux Server
2012 39

40. NRPE: Linux Server
2012 40

41. NRPE: Additional Checks
2012 41

42. NRPE: Additional Checks
2012 42

43. NRPE: Additional Checks
2012 43

44. NRPE: Questions?