Presentation section from Splunk Live content

1. Splunk Java Agent
Damien Dallimore
Developer Evangelist

2. Copyright©2013,SplunkInc.
Splunk APM Current State
2
• Several apps on Splunkbase that can be used together to form part of an overall APM solution
• AppDynamics integration available on Splunkbase
• Dynatrace integration work in progress
• Extrahop integration for network packet capture
• I think that Splunk is well suited to deliver an integrated APM solution in its own right
• Data Collection
• Searching , Correlation, Analysis,Transactions – using Splunk Search Language
• Alerting and Reporting
• Data Security
• Processing large volumes of APM data , scales with increased APM data volumes
• Visualization Capabilitys

3. Copyright©2013,SplunkInc.
Breaking down a solution
3
• Data Collection
• Agents that can be injected into target application (invasive)
• JVM , .NET, Server Side scripts, Browser scripts
• Wire capture (non-invasive)
• Search and Visualization
• SplunkBase app that can be used in conjunction with the data collected from the agent
• Out of the box Splunk UI components
• Custom UI components (d3.js, three.js, Google Charts)
• Use our developer SDKs to integrate with the collected data in Spunk ie: during dev/test

4. Copyright©2013,SplunkInc.
Splunk Java Agent
4
An instrumentation agent for tracing code level metrics via bytecode injection, JMX
attributes/operations/notification and decoded HPROF records and streaming these events directly
into Splunk
https://github.com/damiendallimore/SplunkJavaAgent
• class loading
• method execution
• method timings (cumulative, min, avg, max, std deviation)
• method call tracing(count of calls, group by app/app node(for clustered systems)/thread/class/package)
• method parameter and return value capture (in progress)
• application/thread stalls , thread dumps and stacktraces
• errors/exceptions/throwables
• JVM heap analysis, object/array allocation count/size,class dumps, leak detection, stack traces, frames
• JMX attributes/operations/notifications from the JVM or Application layer MBean Domains
By default , collected data is streamed to Splunk over TCP , but this is configurable/extensible

5. Copyright©2013,SplunkInc.
Design goals
5
• Just pull out the raw APM metrics , then let Splunk perform the crunching
• Format APM events in best practice semantic , well defined key value pairs , tagged events help
correlation across distributed APM environment
• Low impact to the instrumented application
• No code changes required
• Flexible configuration
• Extensible
• Generic open source APM agent , I may have used some Splunk terms in the naming
conventions, but it is still completely generic.
• Intelligence , can self throttle metric gathering based on application load (feature coming soon)

6. Copyright©2013,SplunkInc.
Setup should be as simple as possible
6
This is all you pass to the JVM at startup :
-javaagent:splunkagent.jar
Everything required by the agent is built into the one single jar file

7. Copyright©2013,SplunkInc.
Configuration should allow for flexibility
7
• Config file lives inside the agent jar
• Granular controls for precisely which metrics you want to trace
• Automate with deployment tools such as Puppet and Chef

8. Copyright©2013,SplunkInc.
Raw APM events streamed into Splunk
8

9. Copyright©2013,SplunkInc.
Use Splunk to these into APM insights
9

10. Demo

11. Copyright©2013,SplunkInc.
Contact me
11
Email : ddallimore@splunk.com
Twitter : @damiendallimore
Skype : damien.dallimore
Github : damiendallimore
Splunkbase : damiend
Slideshare : http://www.slideshare.net/damiendallimore
Blogs : http://blogs.splunk.com/dev
Web : http://dev.splunk.com