SlideShare ist ein Scribd-Unternehmen logo
1 von 57
Downloaden Sie, um offline zu lesen
Replacing NSClient++ for
Windows Monitoring
Sven Nierlein
09.11.2023
Beyond NSClient++ 2
whoami
• Name: Sven Nierlein
• E-Mail: sven@consol.de
• Location: Munich, Germany
• Work: consol.de
• We might have met before here in Nuremberg:
• 2022: Thruk
• 2021: Naemon password vault
• 2019: Naemon
• 2018: Thruk
• 2013: Thruk
• 2012: Mod-Gearman
Introduction
Beyond NSClient++ 4
Recap: NSClient++
• Windows Agent (and Linux)
• Supports several protocols
• NRPE
• REST
• and more …
• Lots of built-in checks
• Extendable
• Customisable
Beyond NSClient++ 5
Status Quo
• Basically all our customers use NSClient++ in some way
• Largest setup ~15k 250k installations
• Usually happy with NSClient++
Beyond NSClient++ 6
NSClient++ - So what’s the issue?
• TLS <= 1.2
• Last release was years ago
Beyond NSClient++ 7
NSClient++
• But besides that, it just works…
• Anyway, we needed a solution for the SSL issue…
Beyond NSClient++
• Just update SSL and rebuild, how hard could it be…
8
Going the easy way…
Beyond NSClient++ 9
Looking for alternatives
• There are a couple of options:
• https://github.com/infraweavers/monitoring-agent
• https://github.com/continentale/sma-monitoring-agent/
• https://github.com/continentale/monitoring-agent
• https://github.com/it-novum/openitcockpit-agent-go
• https://github.com/Icinga/icinga-powershell-framework
• Probably missed some more…
Alternative by Nick Youngson CC BY-SA 3.0 Alpha Stock Images
Beyond NSClient++ 10
Requirements
• Drop-In Replacement
• Support nsclient.ini Format
• Must work on Windows and Linux
• Modern SSL/TLS
• Checks format/arguments compatible to NSClient
• No Change to Naemon configuration
• Bonus Points
• Prometheus
SNClient+
Beyond NSClient++ 12
Introducing SNClient+
• Rewrite NSClient++ in Go
• Compatible with nsclient.ini
• Multiplatform
• Windows
• Linux
• OSX, Freebsd…
• Dual Stack
• NSClient++
• Prometheus
Beyond NSClient++ 13
What does SNClient mean?
• Let’s ask the chatbot of your choice:
• Security/Networking Client
• Software Name Client
• Server Ninja Client
• The actual meaning is:
• Secure Naemon Client
Beyond NSClient++ 14
SNClient Dual-Stack
NRPE
HTTP(S) /
REST
NSClient++ /
Naemon / Icinga /
Nagios
Prometheus
HTTP(S)
Checks
Protocol
Naemon Stack
Beyond NSClient++ 16
NSClient / Naemon Stack
NRPE HTTP(S) / REST
NSClient++ / Naemon / Icinga
Prometheus
Checks
Ext. Scripts
Plugins
Checks
Protocol
Beyond NSClient++
SNClient
Beyond NSClient++ 18
Installation
• .MSI Installer for windows
• DEB/RPM repository for linux:
https://labs.consol.de/repo/
• Single go binary
Beyond NSClient++ 19
Checks
• Built in check commands, ex.:
• check_cpu
• check_drivesize
• check_memory
• check_process
• check_service
• …
• Get full list with
• .snclient.exe run check_index
Beyond NSClient++ 20
Checks
• Checks can be executed via check_nsc_web utilising the REST API.
• NRPE is available for compatibility reasons, check_nsc_web is the recommended way.
https://github.com/ConSol-Monitoring/check_nsc_web
Beyond NSClient++
• Some checks need to be enabled in the [/modules]” section.
21
Checks
Beyond NSClient++
• Default snclient.ini contains all available values with their defaults
• Best practice: Put changes into a snclient_local.ini which is included by default
• Included files overwrite existing settings
22
Configuration
Beyond NSClient++
• The order of inheritance for the example to the right:
• /settings/sub1/other (most significant)
• /settings/sub1/default
• /settings/sub1
• /settings/default (least significant)
23
Configuration - Inheritance
[/settings/sub1/other]
key = value
[/settings/sub1/default]
; fallback if the above is not set
key = value
[/settings/sub1]
; fallback if the above is not set
key = value
[/settings/default]
; fallback if the above is not set
key = value
Beyond NSClient++
• Default macros:
• ${exe-path}
• ${shared-path}
• ${scripts}
• ${certificate-path}
• ${hostname}
• ${goos} and ${goarch}
• %(macroname) is also supported
• Example:
[/settings/log]
file name = ${shared-path}/snclient.log
24
Configuration - Macros
Beyond NSClient++
• On demand macros can be used to access values from other sections
• Example:
[/settings/external scripts/alias]
alias_allowed_hosts = check_dummy 0 "${/settings/WEB/server/allowed hosts}”
• Those macros are only valid in the .ini file, not as check arguments.
25
Configuration - On Demand Macros
Beyond NSClient++
• Use macro operators to alter existing values:
• Example:
[/settings/external scripts/alias]
alias_hostname = check_dummy 0 "host:${hostname:lc}"
26
Configuration - Macros Operators
https://omd.consol.de/docs/snclient/configuration/#macro-operators
Beyond NSClient++
• Each check builds an internal table for detail data:
.snclient.exe run -vvv check_service filter=“name=snclient”
OK: All 1 service(s) are ok.
• Filter are simple “<key> <op> <value>” sets combined by logical operators
27
Checks - Filter
classification cpu delayed desc name pid service start_type state
service-shared-process 0 % 0 AssignedAccessManager AssignedAccessManagerSvc 0 AssignedAccessManagerSvc demand stopped
service-own-process 1 % 0 Windows Audio Audiosrv 1700 Audiosrv auto running
service-own-process 0.2% 0 snclient snclient 8336 snclient Auto Running
…
https://omd.consol.de/docs/snclient/checks/filter/
Beyond NSClient++
• Thresholds work like filter but instead of removing an entry, the according state will be set.
.snclient.exe run check_service filter="name=snclient" "warn=cpu>90%" "crit=cpu>95" show-all
OK: snclient=running (auto) |'snclient'=4 'snclient rss'=32395264B 'snclient vms'=29831168B 'snclient cpu’=0.1%
• Use multiple warn/crit to create ranges
.snclient.exe run check_drivesize drive=c: warn="used > 10% && used < 30%"
OK: All 1 drive(s) are ok |… 'c: used %'=57.9%;@10:30;90;0;100
28
Checks - Thresholds
Beyond NSClient++ 29
External Scripts
• External scripts need to enabled in the “[/modules]” section first:
[/modules]
CheckExternalScripts = enabled
• Concise Format:
[/settings/external scripts]
my_check1 = check_custom.bat
my_check2 = myscriptscheck_custom.bat
• Verbose Format:
[/settings/external scripts/scripts/my_check1]
my_check1 = check_custom.bat
[/settings/external scripts/scripts/my_check2]
my_check2 = myscriptscheck_custom.bat
Beyond NSClient++ 30
External Scripts
• Don’t:
cmd /c echo scriptscheck_something.ps1 $ARG1$; exit($lastexitcode) | powershell.exe -command -
• Instead:
scriptscheck_something.ps1 $ARG1$
Beyond NSClient++ 31
Plugins
• Plugins are built in checks, but work more like the official monitoring-plugins
• Currently there are:
• check_nsc_web
• check_http
• check_tcp
• check_dns
• Plugins need to be enabled:
[/modules]
CheckBuiltinPlugins = enabled
https://github.com/mackerelio/go-check-plugins
Beyond NSClient++
• Example: use internal check_http to check reachability of services:
.snclient.exe run check_http -H company-service:8443 -S
HTTP OK: HTTP/1.1 307 Temporary Redirect - 202 bytes in 0.073 second response time | time=0.072689s...
.../check_nsc_web -p 123 -u https://hostname:8443 check_http -H labs.consol.de -S
• Example: use internal check_nsc_web to check other snclients:
.snclient.exe run check_nsc_web -p test -u https://192.123.123.2:8443 -k check_os_version
OK: Microsoft Windows 11 Pro 10.0.22631.2428 Build 22631.2428 (arch: amd64)
32
Plugins
Beyond NSClient++ 33
Security
• Native SSL/TLS Support
• Strong Encryption and Security Standards
• Secure Ciphers by Default
• Hashed Passwords:
[/settings/default]
password = SHA256:9f86d081...
• “Allowed Hosts” are used for the webserver as well (all listeners)
Prometheus Stack
Beyond NSClient++ 35
Prometheus Stack
NRPE HTTP(S) / REST
Prometheus
Protocol
Int. Metrics
Node Exp.
Windows E.
Exp Exp
Checks
NSClient++ /
Naemon / Icinga /
Nagios
Beyond NSClient++ 36
Prometheus Stack
• Exporter Exporter
• Basically a reverse proxy for prometheus exporters
Beyond NSClient++ 37
Prometheus Stack
• Exporter Exporter
• Uses .yaml format for compatibility reasons in the “modules dir”
• Example .exporter_modulesclient.yaml:
method: http
http:
scheme: http
port: 9999
path: /metrics
tls_insecure_skip_verify: true
• Does not verify metrics and just passes through
https://github.com/QubitProducts/exporter_exporter
Beyond NSClient++ 38
Prometheus Stack
• Managed Exporter
• Starts and controls exporters
• Controls memory usage
• Restarts exporter in case of errors
• Enable with
“ManagedExporterServer = enabled”
in the modules section
• URL Prefix /example usually results in
/example/metrics as path to metrics
Beyond NSClient++ 39
Prometheus Stack
• Windows Exporter
• Managed Exporter
• Started and controlled by SNClient+
• Checks memory usage and restarts the exporter
Beyond NSClient++ 40
Prometheus Stack
Beyond NSClient++ 41
Prometheus Stack
• Node Exporter
• Same as windows exporter but for linux
Beyond NSClient++ 42
Prometheus Stack
Beyond NSClient++ 43
Prometheus Stack
• Like every good tool in the prometheus eco system
• Provides internal metrics
• Must be enabled with “PrometheusServer” in the modules section.
Updates
Beyond NSClient++ 45
Updates
Examples
Beyond NSClient++ 47
Replace NRPE
Performance
Beyond NSClient++ 49
Performance
• Best practice:
• Naemon Core
• check_nsc_web
• Mod-Gearman (Go) Worker
• But everything else is fine as well
Beyond NSClient++ 50
Performance
Naemon check_nsc_web
exec
SNClient
http(s)
Classical Approach:
Naemon
tcp/aes
Mod-Gearman
Worker
SNClient
http(s)
Mod-Gearman:
The Mod-Gearman Go Worker uses a builtin check_nsc_web
Thruk Agents
Beyond NSClient++
• This is fine, but…
52
Thruk Agents
Beyond NSClient++ 53
Thruk Agents
Beyond NSClient++ 54
Thruk Agents
Roadmap
Beyond NSClient++ 56
Roadmap
• Improve defaults for builtin checks
• Add more builtin checks
• Add configuration validator
• Maybe add support for opentelemetry
Beyond NSClient++ 57
Thanks
• https://github.com/ConSol-Monitoring/snclient
• https://omd.consol.de/docs/snclient/
• mailto:sven.nierlein@consol.de
Slides: https://labs.consol.de/_slides/snclient-2023-11-09.pdf

Weitere ähnliche Inhalte

Was ist angesagt?

Presentation fortinet securing the cloud
Presentation   fortinet securing the cloudPresentation   fortinet securing the cloud
Presentation fortinet securing the cloud
xKinAnx
 

Was ist angesagt? (20)

CloudStack Architecture
CloudStack ArchitectureCloudStack Architecture
CloudStack Architecture
 
From Three Nines to Five Nines - A Kafka Journey
From Three Nines to Five Nines - A Kafka JourneyFrom Three Nines to Five Nines - A Kafka Journey
From Three Nines to Five Nines - A Kafka Journey
 
Cryptography ppt
Cryptography pptCryptography ppt
Cryptography ppt
 
Network Security - Fortinet, Dublin June 2017
Network Security - Fortinet, Dublin June 2017Network Security - Fortinet, Dublin June 2017
Network Security - Fortinet, Dublin June 2017
 
Janus/SIP @ OpenSIPS 2019
Janus/SIP @ OpenSIPS 2019Janus/SIP @ OpenSIPS 2019
Janus/SIP @ OpenSIPS 2019
 
CloudStack - Top 5 Technical Issues and Troubleshooting
CloudStack - Top 5 Technical Issues and TroubleshootingCloudStack - Top 5 Technical Issues and Troubleshooting
CloudStack - Top 5 Technical Issues and Troubleshooting
 
Presentation fortinet securing the cloud
Presentation   fortinet securing the cloudPresentation   fortinet securing the cloud
Presentation fortinet securing the cloud
 
Philip young current state of mainframe hacking - vanguard - 101016
Philip young   current state of mainframe hacking - vanguard - 101016Philip young   current state of mainframe hacking - vanguard - 101016
Philip young current state of mainframe hacking - vanguard - 101016
 
Cryptography ppt
Cryptography pptCryptography ppt
Cryptography ppt
 
Fixing SCADA: How Ignition Reduces Frustration
Fixing SCADA: How Ignition Reduces FrustrationFixing SCADA: How Ignition Reduces Frustration
Fixing SCADA: How Ignition Reduces Frustration
 
Advanced mainframe hacking
Advanced mainframe hackingAdvanced mainframe hacking
Advanced mainframe hacking
 
Homomorphic Encryption
Homomorphic EncryptionHomomorphic Encryption
Homomorphic Encryption
 
Understanding private blockchains
Understanding private blockchainsUnderstanding private blockchains
Understanding private blockchains
 
Ipfs : InterPlanetary File System
Ipfs : InterPlanetary File SystemIpfs : InterPlanetary File System
Ipfs : InterPlanetary File System
 
Red Hat Insights
Red Hat InsightsRed Hat Insights
Red Hat Insights
 
Docker and WASM
Docker and WASMDocker and WASM
Docker and WASM
 
Windows Debugging with WinDbg
Windows Debugging with WinDbgWindows Debugging with WinDbg
Windows Debugging with WinDbg
 
Turtlebot3: VxWorks running ROS2 as a real-time guest OS on Hypervisor
Turtlebot3: VxWorks running ROS2 as a real-time guest OS on HypervisorTurtlebot3: VxWorks running ROS2 as a real-time guest OS on Hypervisor
Turtlebot3: VxWorks running ROS2 as a real-time guest OS on Hypervisor
 
Lets Encrypt!
Lets Encrypt!Lets Encrypt!
Lets Encrypt!
 
Twitter Finagle
Twitter FinagleTwitter Finagle
Twitter Finagle
 

Ähnlich wie OSMC 2023 | Replacing NSClient++ for Windows Monitoring by Sven Nielein

Ähnlich wie OSMC 2023 | Replacing NSClient++ for Windows Monitoring by Sven Nielein (20)

CNIT 152 10 Enterprise Service
CNIT 152 10 Enterprise ServiceCNIT 152 10 Enterprise Service
CNIT 152 10 Enterprise Service
 
Owning computers without shell access 2
Owning computers without shell access 2Owning computers without shell access 2
Owning computers without shell access 2
 
Sensu and Sensibility - Puppetconf 2014
Sensu and Sensibility - Puppetconf 2014Sensu and Sensibility - Puppetconf 2014
Sensu and Sensibility - Puppetconf 2014
 
OSMC 2022 | Icinga for Windows in the Monitoring of Madness by Christian Stein
OSMC 2022 | Icinga for Windows in the Monitoring of Madness by Christian SteinOSMC 2022 | Icinga for Windows in the Monitoring of Madness by Christian Stein
OSMC 2022 | Icinga for Windows in the Monitoring of Madness by Christian Stein
 
OSMC 2010 | NSClient++ - what's new? And what's coming! by Michael Medin
OSMC 2010 |  NSClient++ - what's new? And what's coming! by Michael MedinOSMC 2010 |  NSClient++ - what's new? And what's coming! by Michael Medin
OSMC 2010 | NSClient++ - what's new? And what's coming! by Michael Medin
 
Private cloud networking_cloudstack_days_austin
Private cloud networking_cloudstack_days_austinPrivate cloud networking_cloudstack_days_austin
Private cloud networking_cloudstack_days_austin
 
Using MCollective with Chef - cfgmgmtcamp.eu 2014
Using MCollective with Chef - cfgmgmtcamp.eu 2014Using MCollective with Chef - cfgmgmtcamp.eu 2014
Using MCollective with Chef - cfgmgmtcamp.eu 2014
 
Jenkins Pipelines Advanced
Jenkins Pipelines AdvancedJenkins Pipelines Advanced
Jenkins Pipelines Advanced
 
OSSEC @ ISSA Jan 21st 2010
OSSEC @ ISSA Jan 21st 2010OSSEC @ ISSA Jan 21st 2010
OSSEC @ ISSA Jan 21st 2010
 
OSMC 2009 | Windows monitoring - Going where no man has gone before... by Mic...
OSMC 2009 | Windows monitoring - Going where no man has gone before... by Mic...OSMC 2009 | Windows monitoring - Going where no man has gone before... by Mic...
OSMC 2009 | Windows monitoring - Going where no man has gone before... by Mic...
 
Over-the-Air: How we Remotely Compromised the Gateway, BCM, and Autopilot ECU...
Over-the-Air: How we Remotely Compromised the Gateway, BCM, and Autopilot ECU...Over-the-Air: How we Remotely Compromised the Gateway, BCM, and Autopilot ECU...
Over-the-Air: How we Remotely Compromised the Gateway, BCM, and Autopilot ECU...
 
Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min!
Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min!Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min!
Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min!
 
iland Internet Solutions: Leveraging Cassandra for real-time multi-datacenter...
iland Internet Solutions: Leveraging Cassandra for real-time multi-datacenter...iland Internet Solutions: Leveraging Cassandra for real-time multi-datacenter...
iland Internet Solutions: Leveraging Cassandra for real-time multi-datacenter...
 
Leveraging Cassandra for real-time multi-datacenter public cloud analytics
Leveraging Cassandra for real-time multi-datacenter public cloud analyticsLeveraging Cassandra for real-time multi-datacenter public cloud analytics
Leveraging Cassandra for real-time multi-datacenter public cloud analytics
 
Distributed Automation(2018) - London Test Automation in Devops Meetup
Distributed Automation(2018) - London Test Automation in Devops MeetupDistributed Automation(2018) - London Test Automation in Devops Meetup
Distributed Automation(2018) - London Test Automation in Devops Meetup
 
Alexei Vladishev - Zabbix - Monitoring Solution for Everyone
Alexei Vladishev - Zabbix - Monitoring Solution for EveryoneAlexei Vladishev - Zabbix - Monitoring Solution for Everyone
Alexei Vladishev - Zabbix - Monitoring Solution for Everyone
 
"Ускорение сборки большого проекта на Objective-C + Swift" Иван Бондарь (Avito)
"Ускорение сборки большого проекта на Objective-C + Swift" Иван Бондарь (Avito)"Ускорение сборки большого проекта на Objective-C + Swift" Иван Бондарь (Avito)
"Ускорение сборки большого проекта на Objective-C + Swift" Иван Бондарь (Avito)
 
Automating Post Exploitation with PowerShell
Automating Post Exploitation with PowerShellAutomating Post Exploitation with PowerShell
Automating Post Exploitation with PowerShell
 
Linux Systems Performance 2016
Linux Systems Performance 2016Linux Systems Performance 2016
Linux Systems Performance 2016
 
Versioning for Developers
Versioning for DevelopersVersioning for Developers
Versioning for Developers
 

Kürzlich hochgeladen

Unlocking Exploration: Self-Motivated Agents Thrive on Memory-Driven Curiosity
Unlocking Exploration: Self-Motivated Agents Thrive on Memory-Driven CuriosityUnlocking Exploration: Self-Motivated Agents Thrive on Memory-Driven Curiosity
Unlocking Exploration: Self-Motivated Agents Thrive on Memory-Driven Curiosity
Hung Le
 
Uncommon Grace The Autobiography of Isaac Folorunso
Uncommon Grace The Autobiography of Isaac FolorunsoUncommon Grace The Autobiography of Isaac Folorunso
Uncommon Grace The Autobiography of Isaac Folorunso
Kayode Fayemi
 
Jual obat aborsi Jakarta 085657271886 Cytote pil telat bulan penggugur kandun...
Jual obat aborsi Jakarta 085657271886 Cytote pil telat bulan penggugur kandun...Jual obat aborsi Jakarta 085657271886 Cytote pil telat bulan penggugur kandun...
Jual obat aborsi Jakarta 085657271886 Cytote pil telat bulan penggugur kandun...
ZurliaSoop
 
Proofreading- Basics to Artificial Intelligence Integration - Presentation:Sl...
Proofreading- Basics to Artificial Intelligence Integration - Presentation:Sl...Proofreading- Basics to Artificial Intelligence Integration - Presentation:Sl...
Proofreading- Basics to Artificial Intelligence Integration - Presentation:Sl...
David Celestin
 
Bring back lost lover in USA, Canada ,Uk ,Australia ,London Lost Love Spell C...
Bring back lost lover in USA, Canada ,Uk ,Australia ,London Lost Love Spell C...Bring back lost lover in USA, Canada ,Uk ,Australia ,London Lost Love Spell C...
Bring back lost lover in USA, Canada ,Uk ,Australia ,London Lost Love Spell C...
amilabibi1
 

Kürzlich hochgeladen (17)

Unlocking Exploration: Self-Motivated Agents Thrive on Memory-Driven Curiosity
Unlocking Exploration: Self-Motivated Agents Thrive on Memory-Driven CuriosityUnlocking Exploration: Self-Motivated Agents Thrive on Memory-Driven Curiosity
Unlocking Exploration: Self-Motivated Agents Thrive on Memory-Driven Curiosity
 
Report Writing Webinar Training
Report Writing Webinar TrainingReport Writing Webinar Training
Report Writing Webinar Training
 
lONG QUESTION ANSWER PAKISTAN STUDIES10.
lONG QUESTION ANSWER PAKISTAN STUDIES10.lONG QUESTION ANSWER PAKISTAN STUDIES10.
lONG QUESTION ANSWER PAKISTAN STUDIES10.
 
My Presentation "In Your Hands" by Halle Bailey
My Presentation "In Your Hands" by Halle BaileyMy Presentation "In Your Hands" by Halle Bailey
My Presentation "In Your Hands" by Halle Bailey
 
in kuwait௹+918133066128....) @abortion pills for sale in Kuwait City
in kuwait௹+918133066128....) @abortion pills for sale in Kuwait Cityin kuwait௹+918133066128....) @abortion pills for sale in Kuwait City
in kuwait௹+918133066128....) @abortion pills for sale in Kuwait City
 
AWS Data Engineer Associate (DEA-C01) Exam Dumps 2024.pdf
AWS Data Engineer Associate (DEA-C01) Exam Dumps 2024.pdfAWS Data Engineer Associate (DEA-C01) Exam Dumps 2024.pdf
AWS Data Engineer Associate (DEA-C01) Exam Dumps 2024.pdf
 
Dreaming Marissa Sánchez Music Video Treatment
Dreaming Marissa Sánchez Music Video TreatmentDreaming Marissa Sánchez Music Video Treatment
Dreaming Marissa Sánchez Music Video Treatment
 
Uncommon Grace The Autobiography of Isaac Folorunso
Uncommon Grace The Autobiography of Isaac FolorunsoUncommon Grace The Autobiography of Isaac Folorunso
Uncommon Grace The Autobiography of Isaac Folorunso
 
Digital collaboration with Microsoft 365 as extension of Drupal
Digital collaboration with Microsoft 365 as extension of DrupalDigital collaboration with Microsoft 365 as extension of Drupal
Digital collaboration with Microsoft 365 as extension of Drupal
 
Jual obat aborsi Jakarta 085657271886 Cytote pil telat bulan penggugur kandun...
Jual obat aborsi Jakarta 085657271886 Cytote pil telat bulan penggugur kandun...Jual obat aborsi Jakarta 085657271886 Cytote pil telat bulan penggugur kandun...
Jual obat aborsi Jakarta 085657271886 Cytote pil telat bulan penggugur kandun...
 
Zone Chairperson Role and Responsibilities New updated.pptx
Zone Chairperson Role and Responsibilities New updated.pptxZone Chairperson Role and Responsibilities New updated.pptx
Zone Chairperson Role and Responsibilities New updated.pptx
 
Proofreading- Basics to Artificial Intelligence Integration - Presentation:Sl...
Proofreading- Basics to Artificial Intelligence Integration - Presentation:Sl...Proofreading- Basics to Artificial Intelligence Integration - Presentation:Sl...
Proofreading- Basics to Artificial Intelligence Integration - Presentation:Sl...
 
Dreaming Music Video Treatment _ Project & Portfolio III
Dreaming Music Video Treatment _ Project & Portfolio IIIDreaming Music Video Treatment _ Project & Portfolio III
Dreaming Music Video Treatment _ Project & Portfolio III
 
SOLID WASTE MANAGEMENT SYSTEM OF FENI PAURASHAVA, BANGLADESH.pdf
SOLID WASTE MANAGEMENT SYSTEM OF FENI PAURASHAVA, BANGLADESH.pdfSOLID WASTE MANAGEMENT SYSTEM OF FENI PAURASHAVA, BANGLADESH.pdf
SOLID WASTE MANAGEMENT SYSTEM OF FENI PAURASHAVA, BANGLADESH.pdf
 
Introduction to Artificial intelligence.
Introduction to Artificial intelligence.Introduction to Artificial intelligence.
Introduction to Artificial intelligence.
 
Bring back lost lover in USA, Canada ,Uk ,Australia ,London Lost Love Spell C...
Bring back lost lover in USA, Canada ,Uk ,Australia ,London Lost Love Spell C...Bring back lost lover in USA, Canada ,Uk ,Australia ,London Lost Love Spell C...
Bring back lost lover in USA, Canada ,Uk ,Australia ,London Lost Love Spell C...
 
ICT role in 21st century education and it's challenges.pdf
ICT role in 21st century education and it's challenges.pdfICT role in 21st century education and it's challenges.pdf
ICT role in 21st century education and it's challenges.pdf
 

OSMC 2023 | Replacing NSClient++ for Windows Monitoring by Sven Nielein

  • 1. Replacing NSClient++ for Windows Monitoring Sven Nierlein 09.11.2023
  • 2. Beyond NSClient++ 2 whoami • Name: Sven Nierlein • E-Mail: sven@consol.de • Location: Munich, Germany • Work: consol.de • We might have met before here in Nuremberg: • 2022: Thruk • 2021: Naemon password vault • 2019: Naemon • 2018: Thruk • 2013: Thruk • 2012: Mod-Gearman
  • 4. Beyond NSClient++ 4 Recap: NSClient++ • Windows Agent (and Linux) • Supports several protocols • NRPE • REST • and more … • Lots of built-in checks • Extendable • Customisable
  • 5. Beyond NSClient++ 5 Status Quo • Basically all our customers use NSClient++ in some way • Largest setup ~15k 250k installations • Usually happy with NSClient++
  • 6. Beyond NSClient++ 6 NSClient++ - So what’s the issue? • TLS <= 1.2 • Last release was years ago
  • 7. Beyond NSClient++ 7 NSClient++ • But besides that, it just works… • Anyway, we needed a solution for the SSL issue…
  • 8. Beyond NSClient++ • Just update SSL and rebuild, how hard could it be… 8 Going the easy way…
  • 9. Beyond NSClient++ 9 Looking for alternatives • There are a couple of options: • https://github.com/infraweavers/monitoring-agent • https://github.com/continentale/sma-monitoring-agent/ • https://github.com/continentale/monitoring-agent • https://github.com/it-novum/openitcockpit-agent-go • https://github.com/Icinga/icinga-powershell-framework • Probably missed some more… Alternative by Nick Youngson CC BY-SA 3.0 Alpha Stock Images
  • 10. Beyond NSClient++ 10 Requirements • Drop-In Replacement • Support nsclient.ini Format • Must work on Windows and Linux • Modern SSL/TLS • Checks format/arguments compatible to NSClient • No Change to Naemon configuration • Bonus Points • Prometheus
  • 12. Beyond NSClient++ 12 Introducing SNClient+ • Rewrite NSClient++ in Go • Compatible with nsclient.ini • Multiplatform • Windows • Linux • OSX, Freebsd… • Dual Stack • NSClient++ • Prometheus
  • 13. Beyond NSClient++ 13 What does SNClient mean? • Let’s ask the chatbot of your choice: • Security/Networking Client • Software Name Client • Server Ninja Client • The actual meaning is: • Secure Naemon Client
  • 14. Beyond NSClient++ 14 SNClient Dual-Stack NRPE HTTP(S) / REST NSClient++ / Naemon / Icinga / Nagios Prometheus HTTP(S) Checks Protocol
  • 16. Beyond NSClient++ 16 NSClient / Naemon Stack NRPE HTTP(S) / REST NSClient++ / Naemon / Icinga Prometheus Checks Ext. Scripts Plugins Checks Protocol
  • 18. Beyond NSClient++ 18 Installation • .MSI Installer for windows • DEB/RPM repository for linux: https://labs.consol.de/repo/ • Single go binary
  • 19. Beyond NSClient++ 19 Checks • Built in check commands, ex.: • check_cpu • check_drivesize • check_memory • check_process • check_service • … • Get full list with • .snclient.exe run check_index
  • 20. Beyond NSClient++ 20 Checks • Checks can be executed via check_nsc_web utilising the REST API. • NRPE is available for compatibility reasons, check_nsc_web is the recommended way. https://github.com/ConSol-Monitoring/check_nsc_web
  • 21. Beyond NSClient++ • Some checks need to be enabled in the [/modules]” section. 21 Checks
  • 22. Beyond NSClient++ • Default snclient.ini contains all available values with their defaults • Best practice: Put changes into a snclient_local.ini which is included by default • Included files overwrite existing settings 22 Configuration
  • 23. Beyond NSClient++ • The order of inheritance for the example to the right: • /settings/sub1/other (most significant) • /settings/sub1/default • /settings/sub1 • /settings/default (least significant) 23 Configuration - Inheritance [/settings/sub1/other] key = value [/settings/sub1/default] ; fallback if the above is not set key = value [/settings/sub1] ; fallback if the above is not set key = value [/settings/default] ; fallback if the above is not set key = value
  • 24. Beyond NSClient++ • Default macros: • ${exe-path} • ${shared-path} • ${scripts} • ${certificate-path} • ${hostname} • ${goos} and ${goarch} • %(macroname) is also supported • Example: [/settings/log] file name = ${shared-path}/snclient.log 24 Configuration - Macros
  • 25. Beyond NSClient++ • On demand macros can be used to access values from other sections • Example: [/settings/external scripts/alias] alias_allowed_hosts = check_dummy 0 "${/settings/WEB/server/allowed hosts}” • Those macros are only valid in the .ini file, not as check arguments. 25 Configuration - On Demand Macros
  • 26. Beyond NSClient++ • Use macro operators to alter existing values: • Example: [/settings/external scripts/alias] alias_hostname = check_dummy 0 "host:${hostname:lc}" 26 Configuration - Macros Operators https://omd.consol.de/docs/snclient/configuration/#macro-operators
  • 27. Beyond NSClient++ • Each check builds an internal table for detail data: .snclient.exe run -vvv check_service filter=“name=snclient” OK: All 1 service(s) are ok. • Filter are simple “<key> <op> <value>” sets combined by logical operators 27 Checks - Filter classification cpu delayed desc name pid service start_type state service-shared-process 0 % 0 AssignedAccessManager AssignedAccessManagerSvc 0 AssignedAccessManagerSvc demand stopped service-own-process 1 % 0 Windows Audio Audiosrv 1700 Audiosrv auto running service-own-process 0.2% 0 snclient snclient 8336 snclient Auto Running … https://omd.consol.de/docs/snclient/checks/filter/
  • 28. Beyond NSClient++ • Thresholds work like filter but instead of removing an entry, the according state will be set. .snclient.exe run check_service filter="name=snclient" "warn=cpu>90%" "crit=cpu>95" show-all OK: snclient=running (auto) |'snclient'=4 'snclient rss'=32395264B 'snclient vms'=29831168B 'snclient cpu’=0.1% • Use multiple warn/crit to create ranges .snclient.exe run check_drivesize drive=c: warn="used > 10% && used < 30%" OK: All 1 drive(s) are ok |… 'c: used %'=57.9%;@10:30;90;0;100 28 Checks - Thresholds
  • 29. Beyond NSClient++ 29 External Scripts • External scripts need to enabled in the “[/modules]” section first: [/modules] CheckExternalScripts = enabled • Concise Format: [/settings/external scripts] my_check1 = check_custom.bat my_check2 = myscriptscheck_custom.bat • Verbose Format: [/settings/external scripts/scripts/my_check1] my_check1 = check_custom.bat [/settings/external scripts/scripts/my_check2] my_check2 = myscriptscheck_custom.bat
  • 30. Beyond NSClient++ 30 External Scripts • Don’t: cmd /c echo scriptscheck_something.ps1 $ARG1$; exit($lastexitcode) | powershell.exe -command - • Instead: scriptscheck_something.ps1 $ARG1$
  • 31. Beyond NSClient++ 31 Plugins • Plugins are built in checks, but work more like the official monitoring-plugins • Currently there are: • check_nsc_web • check_http • check_tcp • check_dns • Plugins need to be enabled: [/modules] CheckBuiltinPlugins = enabled https://github.com/mackerelio/go-check-plugins
  • 32. Beyond NSClient++ • Example: use internal check_http to check reachability of services: .snclient.exe run check_http -H company-service:8443 -S HTTP OK: HTTP/1.1 307 Temporary Redirect - 202 bytes in 0.073 second response time | time=0.072689s... .../check_nsc_web -p 123 -u https://hostname:8443 check_http -H labs.consol.de -S • Example: use internal check_nsc_web to check other snclients: .snclient.exe run check_nsc_web -p test -u https://192.123.123.2:8443 -k check_os_version OK: Microsoft Windows 11 Pro 10.0.22631.2428 Build 22631.2428 (arch: amd64) 32 Plugins
  • 33. Beyond NSClient++ 33 Security • Native SSL/TLS Support • Strong Encryption and Security Standards • Secure Ciphers by Default • Hashed Passwords: [/settings/default] password = SHA256:9f86d081... • “Allowed Hosts” are used for the webserver as well (all listeners)
  • 35. Beyond NSClient++ 35 Prometheus Stack NRPE HTTP(S) / REST Prometheus Protocol Int. Metrics Node Exp. Windows E. Exp Exp Checks NSClient++ / Naemon / Icinga / Nagios
  • 36. Beyond NSClient++ 36 Prometheus Stack • Exporter Exporter • Basically a reverse proxy for prometheus exporters
  • 37. Beyond NSClient++ 37 Prometheus Stack • Exporter Exporter • Uses .yaml format for compatibility reasons in the “modules dir” • Example .exporter_modulesclient.yaml: method: http http: scheme: http port: 9999 path: /metrics tls_insecure_skip_verify: true • Does not verify metrics and just passes through https://github.com/QubitProducts/exporter_exporter
  • 38. Beyond NSClient++ 38 Prometheus Stack • Managed Exporter • Starts and controls exporters • Controls memory usage • Restarts exporter in case of errors • Enable with “ManagedExporterServer = enabled” in the modules section • URL Prefix /example usually results in /example/metrics as path to metrics
  • 39. Beyond NSClient++ 39 Prometheus Stack • Windows Exporter • Managed Exporter • Started and controlled by SNClient+ • Checks memory usage and restarts the exporter
  • 41. Beyond NSClient++ 41 Prometheus Stack • Node Exporter • Same as windows exporter but for linux
  • 43. Beyond NSClient++ 43 Prometheus Stack • Like every good tool in the prometheus eco system • Provides internal metrics • Must be enabled with “PrometheusServer” in the modules section.
  • 49. Beyond NSClient++ 49 Performance • Best practice: • Naemon Core • check_nsc_web • Mod-Gearman (Go) Worker • But everything else is fine as well
  • 50. Beyond NSClient++ 50 Performance Naemon check_nsc_web exec SNClient http(s) Classical Approach: Naemon tcp/aes Mod-Gearman Worker SNClient http(s) Mod-Gearman: The Mod-Gearman Go Worker uses a builtin check_nsc_web
  • 52. Beyond NSClient++ • This is fine, but… 52 Thruk Agents
  • 56. Beyond NSClient++ 56 Roadmap • Improve defaults for builtin checks • Add more builtin checks • Add configuration validator • Maybe add support for opentelemetry
  • 57. Beyond NSClient++ 57 Thanks • https://github.com/ConSol-Monitoring/snclient • https://omd.consol.de/docs/snclient/ • mailto:sven.nierlein@consol.de Slides: https://labs.consol.de/_slides/snclient-2023-11-09.pdf