Onboard Automation with EEM

1. Onboard Automation with
Embedded Event Manager
Shaila Sharmin
Senior Engineer, Core & IP Network
Banglalion Communications LTD

2. Index
• Embedded Event Manager (EEM) Overview
• Understanding EEM Event Detectors
• EEM Policies with sample scripts
• EEM Demos

3. Introduction: EEM(Embedded Event manager
)
EEM is a software component of cisco IOS, XR, and NX-OS that allows
you to run a script or a set of commands upon an event and makes life
easier for administrators by tracking and classifying events that take
place on a router and providing notification options for those events. .
There are two independent pieces: Applets and Scripting
-> Applets are a collection of CLI commands
-> Scripts are actions coded up in TCL(interpreter language)

4. EEM Core Event Detectors & Components
EEM detectors can be
1) Syslog
2) CLI events
3) Counter
4) Timers
5) SNMP
6) IP SLA and Netflows events.
7) None : simply "event manager run"
command.
EEM Components:
EEM server
EEM publisher (detector)
EEM subscriber (policy)

5. Determining the Version of EEM
CISCO ACCESS ROUTERS - Current Models
CISCO ACCESS ROUTERS - Old Models

6. Determining the Version of EEM
CISCO SERVICE AGGREGATION/CORE ROUTERS
CISCO CATALYST SWITCHES

7. Creating an EEM applet
There are three steps to creating this EEM applet.
1: Create the applet and give it a name
R6(config)#event manager applet Intf_Management
2: Tell the applet what to look out for
R6(config-applet)#event syslog pattern "%LINK-5-CHANGED: Interface Loopback0, changed state to administratively down“
3: What do you want the router to do when it sees what you have defined in step 2 – simple!
R6(config-applet)#action 1.0 cli command “enable”
R6(config-applet)#action 2.0 cli command “conf t”
R6(config-applet)#action 3.0 cli command "interface loopback 0"
R6(config-applet)#action 4.0 cli command “no shut”
R6(config-applet)#action 5.0 cli command "end"
R6(config-applet)#action 6.0 cli command "who"
R6(config-applet)#action 7.0 mail server "58.97.254.49" to "s.sharmin@banglalionwimax.com" from "s.sharmin@banglalionwimax.com" subject
"ISP1_Interface_loopback 0 SHUTDOWN" body "Current users $_cli_result"
R6(config-applet)#end

8. Sample EEM applet

9. EEM Event Detector – Syslog ED
Functionality
Triggers Event on Matches for Syslog Messages Based on Regular Expression
Example
event syslog pattern "%LINK-5-CHANGED: Interface Loopback0, changed state to
administratively down“
Use Case
Troubleshooting, Automatic Fault Detection and Alert

10. Syslog ED Example
Syslog messages are the messages that show up by default on console. This example shows the
syslog event detector.
Configuration:
SMTP Server is reachable, Loopback0 is up.

11. Syslog ED Example continue..
When the Loopback0interface has been shutdown, the below applet automatically runs to turn on the
interface and send the alert to specific email address including logged in user information.
To display the Embedded Event Manager events that have been triggered
in R1, use the following command:
Sample mail received by network administrator

12. EEM Event Detector – CLI ED
Functionality
Triggers Synchronous or Asynchronous Events When Certain CLI Is Executed. Allow
Custom CLI Creation (EEM 3.0).
Example
event cli pattern "reload" sync no skip yes occurs 1
Use Case
Config Management, Security, Feature Customization

13. CLI ED ED Example
It can take action based on commands that are used on the CLI
Configuration:
Instead of looking for a pattern in syslog, this time we’re waiting for a pattern entered onto the CLI.
break down :
event cli pattern: Defines the event criteria to initialize the EEM applet.
sync: Specifies if the policy should be executed synchronously before the CLI commands executes
skip: Indicates if the CLI commands should be executed
occurs: Indicates the number of occurrences before the EEM applet is triggers.
When we attempt to reload the router, the results are as expected

14. EEM Event Detector – Interface ED
Functionality
Triggers Event When Interface Counters Cross Threshold. 22 Counters Supported,
Including input_error, interface_reset, transmit_rate, etc.
Example
event tag if_1 interface name Fa0/0 parameter input_errors_crc entry-op ge entry-
val 10 entry-type increment poll-interval 60
Use Case
Real Time Alert and Recovery of Interface Error

15. Interface ED Example
Monitor CRC errors on multiple WAN interfaces and notify the operator (via e-mail) when an
interface has more than two errors per minute.
Configuration:
event manager applet multiple_if
event tag if_1 interface name Fa0/0 parameter input_errors_crc entry-op ge entry-val 2 entry-type increment poll-interval 60
event tag if_2 interface name Gi01/0 parameter input_errors_crc entry-op ge entry-val 2 entry-type increment poll-interval 60
trigger
correlate event if_1 or event if_2
action 1.0 syslog msg "CRC failure leased line $_interface_name"
action 2.0 mail server "58.97.254.49" to "s.sharmin@banglalionwimax.com" from "s.sharmin@banglalionwimax.com" subject "CRC problems on
$_info_routername interface $_interface_name" body "CRC failures have exceeded the threshold“
To view the registered policies on router R1, you can use the following command:

16. EEM Event Detector – Timer ED
Functionality
Triggers Events on Watchdog, Count Down, cron and Absolute Timer
Example
event timer cron cron-entry "0 19 * * 0-7"
event timer watchdog time 300
Use Case
System Monitoring via Periodic Action, Periodic Data Collection and Reporting

17. Timer ED Example
This applet is put into Cron, which will be triggered at the exact specific time. It is composed of 5
values separated by a space. Minutes hours day month {day of week (0-6, 0 is Sunday)}
Configuration:

18. EEM Event Detector – SNMP ED
Functionality
Triggers Event Based on SNMP OID Value Crossing Predefined Threshold
Example
event snmp oid " 1.3.6.1.4.1.9.9.109.1.1.1.1.5 " get-type exact entry-op ge entry-val
50 exit-op le exit-val 5 poll-interval 5
Use Case
System Stats Monitoring and Alerting, e.g. CPU and Memory Utilization

19. SNMP ED Example
Following EEM script run the command when the CPU goes above a certain value.
Configuration:
event manager applet highcpu
event snmp oid " 1.3.6.1.4.1.9.9.109.1.1.1.1.5 " get-type exact entry-op ge entry-val 50 exit-op le exit-val 5 poll-interval 5
action 1.0 cli command "enable"
action 2.0 cli command "show proc cpu sorted"
action 3.0 mail server "58.97.254.49" to "s.sharmin@banglalionwimax.com" from "s.sharmin@banglalionwimax.com" subject "High CPU Alert" body
"$_cli_result"
End
This will poll the five second CPU utilization of the route processor every five seconds. If the
utilization is at or above 50%, the event will fire. The event will not fire again until the CPU drops
below 5%, then goes back to 50%.
The definitions of variables are:
highcpu - name of the event manager applet/script
1.3.6.1.4.1.9.9.109.1.1.1.1.5 / cpmCPUTotal5min - Object identifier (OID) for polling the total CPU utilization of the route processor (RP)
entry-val 50 - CPU utilization that triggers the script
poll-interval 0.5 - Frequency (every 0.5 seconds) the script monitors the CPU

20. EEM Event Detector – IPSLA ED
Functionality
Trigger Events When IPSLA Test Results Cross Certain Threshold. Integrated with Auto
IPSLA Group to Monitor Large Number of IPSLA Operation Results
Example
event manager applet watch-jitter
event ipsla operation-id 1 reaction-type jitterAvg
action 001 cli command "enable"
action 002 if $_ipsla_measured_threshold_value > $_ipsla_threshold_rising
action 003 cli command "config t"
action 004 cli command "ip route 10.10.20.0 255.255.255.0 192.168.15.1"
action 005 cli command "end“
Use Case
Link Failure Detection, Diagnostics and Recovery
The definitions of variables are:
operation-id - Specifies the IP SLAs operation ID.
operation-id-value - Number in the range from 1 to 2147483647.
reaction-type - Specifies the reaction to be taken for the specified IP SLAs operation.
jitterAvg Jitter Average in both the directions

21. EEM TCL-Based Policy Example
EEM scripts are written using TCL. TCL
(Tool Control Language) is a scripting
language used by Cisco for testing and
automating of various functions in the
IOS. In this example, small TCL script
configured to check reachability of few
IP from the Core router.

22. EEM Demos

23. The Problem : An Enterprise network connected with two ISP. While load sharing
traffic with both ISP , if one link to ISP fails then traffic should shift to another ISP.
But NAT translations are not clearing after the primary link fails. When the primary
link recovers , traffic still going over the back-up link.
The Solution : Using IP SLA and EEM applet to failover the traffic using NAT.
1. Dual ISP: NAT Problem

24. Topology

25. EEM Action
event manager applet link-ISP-1-Down
event syslog pattern "1 ip sla 1 reachability Up->Down"
action 1.0 cli command "enable"
action 1.1 cli command "configure terminal"
action 1.2 cli command "no ip nat inside source list 101 interface GigabitEthernet1/0 overload"
action 1.3 cli command "no ip nat inside source list 102 interface GigabitEthernet2/0 overload"
action 1.4 cli command "ip nat inside source list 100 interface GigabitEthernet2/0 overload"
event manager applet link-ISP-1-UP
event syslog pattern "1 ip sla 1 reachability Down->Up"
action 1.0 cli command "enable"
action 1.1 cli command "configure terminal"
action 1.2 cli command "no ip nat inside source list 100 interface GigabitEthernet2/0 overload"
action 1.3 cli command "ip nat inside source list 102 interface GigabitEthernet2/0 overload"
action 1.4 cli command "ip nat inside source list 101 interface GigabitEthernet1/0 overload"
event manager applet link-ISP-2-Down
event syslog pattern "2 ip sla 2 reachability Up->Down"
action 1.0 cli command "enable"
action 1.1 cli command "configure terminal"
action 1.2 cli command "no ip nat inside source list 101 interface GigabitEthernet1/0 overload"
action 1.3 cli command "no ip nat inside source list 102 interface GigabitEthernet2/0 overload"
action 1.4 cli command "ip nat inside source list 100 interface GigabitEthernet1/0 overload"
event manager applet link-ISP-2-UP
event syslog pattern "2 ip sla 2 reachability Down->Up"
action 1.0 cli command "enable"
action 1.1 cli command "configure terminal"
action 1.2 cli command "no ip nat inside source list 100 interface GigabitEthernet1/0 overload"
action 1.3 cli command "ip nat inside source list 102 interface GigabitEthernet2/0 overload"
action 1.4 cli command "ip nat inside source list 101 interface GigabitEthernet1/0 overload"
!

26. Resources
• Support forums for this technology are GREAT
• “Living” documentat https://supportforums.cisco.com/docs/DOC-12757 Contains helpful tips
and tricks to get the most out of EEM .
• For reading material and further resources for this session,visit www.pearson-
books.com/CLMilan2014.
• https://networklessons.com/network-management/cisco-ios-embedded-event-manager/.
• http://www.techtutsonline.com/cisco-ios-embedded-event-manager/
• http://www.cisco.com/c/en/us/td/docs/ios/netmgmt/configuration/guide/12_2sx/nm_12_2sx_bo
ok/nm_eem_overview.htmlfor basic info
• http://www.cisco.com/c/en/us/td/docs/ios/netmgmt/configuration/guide/12_2sx/nm_12_2sx_bo
ok/nm_eem_policy_cli.htmlfor Policies Using the Cisco IOS CLI
• http://www.cisco.com/c/en/us/td/docs/ios/netmgmt/configuration/guide/12_2sx/nm_12_2sx_bo
ok/nm_eem_policy_tcl.htmlfor Policies Using Tcl

27. Questions?