There are many tools and frameworks for monitoring. Usually when you think of an Open Source solution, you don’t think to implement it in a COTS product. Nevertheless, this session will tell you how you can implement tools such as Prometheus, Grafana and ELK into such an Enterprise application platform. Monitoring performance, throughput and error rate is important to be in control of your transactions. If you use a Service Bus or SOA/BPM suite product there are a lot out of the box diagnostics waiting for you. The puzzle here is how to get it out in a useful way. Besides of the many commercial solutions also Open Source tools can help you out with it. You can export runtime diagnostics out of the Diagnostics framework, monitor your SOA Composites and trace down Service Bus statistics using Prometheus and Grafana. The session will elaborate how to set up a proper monitoring using these tools, also in a proactive way where automated monitoring is a must for every application environment.
WSO2CON 2024 - Building the API First Enterprise – Running an API Program, fr...
OSMC 2021 | Use OpenSource monitoring for an Enterprise Grade Platform
1. A presentation by Michel Schildmeijer,
11 October 2021
Open Source Monitoring
Of an Enterprise Grade Platform
2. Me & My “careerpath” evolution
Kindergarten
1970
Junior
Highschool
1972
Highschool
1977
Study
Clinical
Pharmaco
therapy
1981
Royal
Airforce &
Clinical
Pharmacist
1987
Careerswitch
to IT
2000
• From Amsterdam, the Netherlands
• Work @ Qualogy
• Oracle ACE since 2012, ACED 2019
• Authored 2 books WebLogic in 2010
• IT Career started with UNIX in 1994
• Former BEA technology in 2000
• Focus on containerization strategies,
• DevOps etc
Born
1966
6. Huge amounts of data
•Functional data
□ Application functional
data
□ User related data
•Non Functional Data
□ Metadata
□ Log data
□ Runtime data
7. Operational control
•Get in control of all IT systems
□ By knowing and interpret this data
•Do predictive analysis
□ Know when to expect peak loads
□ Know what trends are going on
□ Know what end users experiencing
□ Know the weaknesses and vulnerabilities of
Systems, servers, network, databases
Application and tiers
11. Oracle WebLogic Foundation
Cloud Application Foundation for Java applications
WebLogic Server
Java EE: Reliability, Availability, Scalability & Performance
Coherence EE
Data Grid Scale Out for Java, C++, .NET
Java Virtual Machines
Oracle and OpenSource JVM’s
Development
Tools
Jdeveloper
/
Eclipse
Enterprise
Manager
Admin
and
Operations
WebLogic Suite
Custom App
Packaged
App
Fusion
Middleware
12. Building Blocks Java EE WLS Containerization
• WebLogic Docker certification
□ Docker images, Dockerfiles, GitHub
• WebLogic OCI certification
□ Open Container Initiative CRI-O / KATA
• Integration
□ Management: Operator
□ Monitoring: Exporter for Prometheus
□ Migration: Deploy tooling
□ Logging: Exporter for Elastic Stack
□ Image: Tool management
Kubernetes
Operator
WebLogic WebLogic WebLogic
kubectl
13. WebLogic in Containers
Base Image Linux
Image JDK & WebLogic bins
Image WebLogic Domain
Image WebLogic Deployed app
Container Runtime
Container Software
libvirt LXC
Linux Kernel
15. Kubernetes Operators
• Extension on K8S api
• Configure & manage more complex instances
• Why Operators ?
□ Stateless (web ) apps are easy to manage, scale
□ But thougher when it is with stateful
□ To leverage more experience based knowledge to k8s
The operators from the field in a software driven solution!
16. WebLogic Kubernetes Operator
• Simpler WebLogic management in Kubernetes
• Kubernetes resources are allocated for WebLogic domain(s)
• Manages overall WebLogic environment through Kubernetes APIs
□ Load Balancer, Network,
□ Ingress Controllers,
□ Security,
□ HA restart, upgrade, scaling
□ Persistent storage
• Ensures WebLogic best practices are followed
20. Process Services
Orchestration
System BPM
Business Logic
Databases
Data
Services
Enterprise Service Bus (ESB)
data
Web Portals
Human Business Process Management (BPM)
Security
Registry
and
Repository
Manage
and
monitor
SOA generic landscape
21. SOA Suite Platform
WebLogic Server
Managed Server(S)
Java EE deployed Apps for
SOA
Webapps
EJB/MDB
RAR
Java/WLS/SOA Software
Linux
SOA WebLogic Cluster
Operating System
Installation & Config
Domain Configuration
JDBC
JMS
HTTP(S
)
JMX
23. WebLogic REST Management API
• REST API for managing WebLogic
• Covers all of WebLogic management
• Configure, Start/Stop, Deploy,
Monitor...
• Kubernetes Operator makes use of
it for lifecycle
• HTTP, no WebLogic client required
• Well-defined, fully documented
• Consistent with existing management
• Generated from WebLogic MBeans
WebLogic Domain
Admin Server
REST
WebApp
Managed Server
REST
WebApp
Managed Server
REST
WebApp
REST
Start/stop
REST REST
Operator
25. Diagnostic Frameworks & Diagnostic Tools in FMW
• WLDF – WebLogic Diagnostic Framework
• DFW – Diagnostic Framework
o Dynamic Monitoring Service (DMS)
o Oracle Diagnostic Logging (ODL)
o Incident Packaging System (IPS or ADRCI)
• Additional Tools
o Selective Tracing
26. WebLogic Diagnostic Framework
Watch & Notification Instrumentation
Data Collectors Image Capture
Data Providers
Archive
Accessor
Monitoring Dashboard
• Included with WebLogic
• Define Watches to monitor
runtime conditions and notify
in case of failure.
• Collect configuration and
runtime data for analysis.
• Capture images to preserve
system state in failure
situation.
28. Diagnostic Tools for SOA and FMW
• Predefined Watches and Diagnostic Dumps to detect,
• Diagnose & resolve problems with SOA MBeans & DMS Metrics.
• Detect critical failures and collect diagnostic dumps
o Relevant diagnostic information like logs, metrics, server images,
• Create incidents in the ADR (Automatic Diagnostic Repository)
29. WLDF, DFW and Diagnostic Tools for SOA
Monitor
System
• Predefined Watches
• Predefined Diagnostic Dumps
Critical
Failure
• Collect Diagnostic Dumps
• Create Incident
Collect
• Collect Data for processing
30. Dynamic Monitoring Service (DMS)
• DMS for runtime metrics applications and
components
• Can be monitored through WLDF Watches to trigger
data collections through DFW.
32. • Logstash & Kibana
• WebLogicLoggingExporter.yaml
• weblogic-logging-exporter-<version>.jar
• snakeyaml-1.23.jar - parser and emitter for Java
□ From Maven Central Repository
Components
33. Actions
• Install ELK in your Kubernetes Cluster – Or separate
• Copy jar files to shared location of you K8S cluster(not to pods)
• WLS Startup Class
• Set Classpath in setDomainEnv
• Setup config for export
# Export for ELK Logging
CLASSPATH="/scratch/domains/soadomain/weblogic-logging-exporter-
1.0.0.jar:/scratch/domains/soadomain//snakeyaml-1.23.jar:${CLASSPATH}"
export CLASSPATH
40. • Prometheus & Grafana Install
• wlsexporter.war & get script
□ Downloads the war & repackage with proper config
• JSON Dashboard imports
• DB Exporter deployment for querying SOA Repository
Components
42. Prometheus integration
• Generate exporter wars
• Deploy warfiles to Admin and Managed Servers
• Create Rolebinding, Role and Service Account
□ wls-exporter-ServiceMonitor.yaml
46. Possibilities
• Query with SQL on SOA Infra Dehydration tables
• Database exporter for Prometheus / Grafana
• Status monitoring of long running instance
• Toml config with custom metrics
• No commercial Oracle Grafana plugin needed
• Still under investigation