Software Profiling: Understanding Java Performance and how to profile in Java

Software Profiling
Understanding Java Performance and how to profile in Java
M. Isuru Tharanga Chrishantha Perera
Technical Lead at WSO2, Co-organizer of Java Colombo Meetup

What’s Software Profiling?
Wikipedia definition:
In software engineering, profiling ("program profiling", "software profiling") is a form
of dynamic program analysis that measures, for example, the space (memory) or
time complexity of a program, the usage of particular instructions, or the frequency
and duration of function calls. Most commonly, profiling information serves to aid
program optimization.
https://en.wikipedia.org/wiki/Profiling_(computer_programming)
2

What’s Software Profiling?
Wikipedia definition:
Profiling is achieved by instrumenting either the program source code or its binary
executable form using a tool called a profiler (or code profiler). Profilers may use a
number of different techniques, such as event-based, statistical, instrumented, and
simulation methods.
https://en.wikipedia.org/wiki/Profiling_(computer_programming)
3

Measuring Performance
5
We need a way to measure the performance:
● To understand how the system behaves
● To see performance improvements after doing any optimizations
There are two key performance metrics.
● Response Time/Latency
● Throughput

Throughput
Throughput measures the number of messages that a server processes during a
specific time interval (e.g. per second).
Throughput is calculated using the equation:
Throughput = number of requests / time to complete the requests
6

Response Time/Latency
Response time is the end-to-end processing time for an operation.
7

Benchmarking Tools
● Apache JMeter
● Apache Benchmark
● wrk - a HTTP benchmarking tool
8

Tuning Java Applications
● We need to have a very high throughput and very low latency values.
● There is a tradeoff between throughput and latency. With more concurrent
users, the throughput increases, but the average latency will also increase.
● Usually, you need to achieve maximum throughput while keeping latency
within some acceptable limit. For eg: you might choose maximum throughput
in a range where latency is less than 10ms
9

Throughput and Latency Graphs
10
Source: https://www.infoq.com/articles/Tuning-Java-Servers

Response Time/Latency Distribution
When measuring response time, it’s important to look at the the whole distribution:
min, max, avg, median, 75th percentile, 98th percentile, 99th percentile etc.
11

Longtail latencies
When high percentiles have values much greater
than the average latency
Source:
https://engineering.linkedin.com/performance/
who-moved-my-99th-percentile-latency
12

Latency Numbers Every Programmer Should
Know
L1 cache reference 0.5 ns
Branch mispredict 5 ns
L2 cache reference 7 ns 14x L1 cache
Mutex lock/unlock 25 ns
Main memory reference 100 ns 20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy 3,000 ns 3 us
Send 1K bytes over 1 Gbps network 10,000 ns 10 us
Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD
Read 1 MB sequentially from memory 250,000 ns 250 us
Round trip within same datacenter 500,000 ns 500 us
Read 1 MB sequentially from SSD* 1,000,000 ns 1,000 us 1 ms ~1GB/sec SSD, 4X memory
Disk seek 10,000,000 ns 10,000 us 10 ms 20x datacenter roundtrip
Read 1 MB sequentially from disk 20,000,000 ns 20,000 us 20 ms 80x memory, 20X SSD
Send packet CA->Netherlands->CA 150,000,000 ns 150,000 us 150 ms
13

Why do we need Profiling?
Improve throughput (Maximizing the transactions processed per second)
Improve latency (Minimizing the time taken to for each operation)
Find performance bottlenecks
14

Java Garbage Collection
16
Java automatically allocates memory for our applications and automatically
deallocates memory when certain objects are no longer used.
"Automatic Garbage Collection" is an important feature in Java.
As Java Developers, we don't have to worry about memory
allocations/deallocations as Java takes care of the task to manage memory for us

Marking and Sweeping Away Garbage
GC works by first marking all used objects in the heap and then deleting unused
objects.
GC also compacts the memory after deleting unreferenced objects to make new
memory allocations much easier and faster.
17

GC roots
JVM references GC roots, which refer the application objects in a tree structure.
There are several kinds of GC Roots in Java.
1. Local Variables
2. Active Java Threads
3. Static variables
4. JNI references
When the application can reach these GC roots, the whole tree is reachable and
GC can determine which objects are the live objects.
18

Java Heap Structure
Java Heap is divided into generations based on the object lifetime.
Following is the general structure of the Java Heap. (This is mostly dependent on
the type of collector).
19

Young Generation
Young Generation usually has Eden and Survivor spaces.
All new objects are allocated in Eden Space.
When this fills up, a minor GC happens.
Surviving objects are first moved to survivor spaces.
When objects survives several minor GCs (tenuring threshold), the relevant objects
are eventually moved to the old generation.
20

Old Generation
This stores long surviving objects.
When this fills up, a major GC (full GC) happens.
A major GC takes a longer time as it has to check all live objects.
21

Permanent Generation
This has the metadata required by JVM.
Classes and Methods are stored here.
This space is included in a full GC.
22

Java 8 and PermGen
In Java 8, the permanent generation is not a part of heap.
The metadata is now moved to native memory to an area called “Metaspace”
There is no limit for Metaspace by default
23

"Stop the World"
For some events, JVM pauses all application threads. These are called
Stop-The-World (STW) pauses.
GC Events also cause STW pauses.
We can see application stopped time with GC logs.
24

GC Logging
There are JVM flags to log details for each GC.
-XX:+PrintGC - Print messages at garbage collection
-XX:+PrintGCDetails - Print more details at garbage collection
-XX:+PrintGCTimeStamps - Print timestamps at garbage collection
-XX:+PrintGCApplicationStoppedTime - Print the application GC stopped time
-XX:+PrintGCApplicationConcurrentTime - Print the application GC concurrent
time
The GCViewer is a great tool to view GC logs
25

Java Memory Usage
● Init - initial amount of memory that the JVM requests from the OS for memory
management during startup.
● Used - amount of memory currently used
● Committed - amount of memory that is guaranteed to be available for use by
the JVM
● Max - maximum amount of memory that can be used for memory
management.
26

JDK Tools and Utilities
● Basic Tools (java, javac, jar)
● Security Tools (jarsigner, keytool)
● Java Web Service Tools (wsimport, wsgen)
● Java Troubleshooting, Profiling, Monitoring and Management Tools (jcmd,
jconsole, jmc, jvisualvm)
28

Java Troubleshooting, Profiling, Monitoring and
Management Tools
● jcmd - JVM Diagnostic Commands tool
● jconsole - A JMX-compliant graphical tool for monitoring a Java application
● jvisualvm – Provides detailed information about the Java application. It
provides CPU & Memory profiling, heap dump analysis, memory leak
detection etc.
● jmc – Tools to monitor and manage Java applications without introducing
performance overhead
29

Java Experimental Tools
Monitoring Tools
● jps – JVM Process Status Tool
● jstat – JVM Statistics Monitoring Tool
Troubleshooting Tools
● jmap - Memory Map for Java
● jhat - Heap Dump Browser
● jstack – Stack Trace for Java
30

Java Ergonomics and JVM Flags
31

Java Ergonomics and JVM Flags
Java Virtual Machine can tune itself depending on the environment and this smart
tuning is referred to as Ergonomics.
When tuning Java, it's important to know which values were used as default for
Garbage collector, Heap Sizes, Runtime Compiler by Java Ergonomics
32

Printing Command Line Flags
We can use "-XX:+PrintCommandLineFlags" to print the command line flags used
by the JVM.
This is a useful flag to see the values selected by Java Ergonomics.
eg:
$ java -XX:+PrintCommandLineFlags -version
-XX:InitialHeapSize=126592960 -XX:MaxHeapSize=2025487360 -XX:+PrintCommandLineFlags
-XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseParallelGC
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
33

Printing Initial & Final JVM Flags
Use following command to see the default values
java -XX:+PrintFlagsInitial -version
Use following command to see the final values.
java -XX:+PrintFlagsFinal -version
The values modified manually or by Java Ergonomics are shown with “:=”
java -XX:+PrintFlagsFinal -version | grep ':='
34

Java Flags
Java has a lot of tuning options:
$ java -XX:+UnlockCommercialFeatures -XX:+UnlockDiagnosticVMOptions -XX:+UnlockExperimentalVMOptions
-XX:+PrintFlagsFinal -version | head -n 10
[Global flags]
uintx AdaptiveSizeDecrementScaleFactor = 4 {product}
uintx AdaptiveSizeMajorGCDecayTimeScale = 10 {product}
uintx AdaptiveSizePausePolicy = 0 {product}
uintx AdaptiveSizePolicyCollectionCostMargin = 50 {product}
uintx AdaptiveSizePolicyInitializingSteps = 20 {product}
uintx AdaptiveSizePolicyOutputInterval = 0 {product}
uintx AdaptiveSizePolicyWeight = 10 {product}
uintx AdaptiveSizeThroughPutPolicy = 0 {product}
uintx AdaptiveTimeWeight = 25 {product}
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
35

Java Profiling Tools
37
Survey by RebelLabs in 2016:
http://pages.zeroturnaround.com/RebelLabs-Developer-Productivity-Report-2016.html

Java Profiling Tools
38
Java VisualVM - Available in JDK
Java Mission Control - Available in JDK
JProfiler - A commercially licensed Java profiling tool developed by
ej-technologies
Honest Profiler - Open Source Sampling CPU profiler

Measuring Methods for CPU Profiling
Sampling: Monitor running code externally and check which code is executed
Instrumentation: Include measurement code into the real code
39

Sampling
40
main()
foo()
bar()

Instrumentation
41
main()
foo()
bar()

How Java Profilers Work?
Generic profilers rely on the JVMTI spec
JVMTI offers only safepoint sampling stack trace collection options
42

Safepoints
A safepoint is a moment in time when a thread’s data, its internal state and
representation in the JVM are, well, safe for observation by other threads in the
JVM.
● Between every 2 bytecodes (interpreter mode)
● Backedge of non-’counted’ loops
● Method exit
● JNI call exit
43

Sampling vs. Instrumentation
Sampling
Overhead depends on the sampling
interval
Stable Overhead
Can see execution hotspots
Can miss methods, which returns faster
than the sampling interval.
Can discover unknown code
Instrumentation
Precise measurement for execution
times
No stable overhead
More data to process
44

Sampling vs. Instrumentation
45
Java VisualVM uses both sampling and instrumentation
Java Flight Recorder uses sampling for hot methods
JProfiler supports both sampling and instrumentation

Problems with Profiling
● Runtime Overhead
● Interpretation of the results can be difficult
● Identifying the "crucial“ parts of the software
● Identifying potential performance improvements
46

Profiling Applications with Java VisualVM
47
CPU Profiling: Profile the performance of the application.
Memory Profiling: Analyze the memory usage of the application.

Java Mission Control
A set of powerful tools running on the Oracle JDK to monitor and manage Java
applications
Free for development use (Oracle Binary Code License)
Available in JDK since Java 7 update 40
Supports Plugins
Two main tools
1. JMX Console
2. Java Flight Recorder
48

Java Flight Recorder (JFR)
A profiling and event collection framework built into the Oracle JDK
Gather low level information about the JVM and application behaviour without
performance impact (less than 2%)
Always on Profiling in Production Environments
Engine was released with Java 7 update 4
Commercial feature in Oracle JDK
50

JFR Events
JFR collects data about events.
JFR collects information about three types of events:
1. Instant events – Events occurring instantly
2. Sample (Requestable) events – Events with a user configurable period to
provide a sample of system activity
3. Duration events – Events taking some time to occur. The event has a start
and end time. You can set a threshold.
51

Java Flight Recorder Architecture
JFR is comprised of the following components:
1. JFR runtime - The recording engine inside the JVM that produces the
recordings.
2. Flight Recorder plugin for Java Mission Control (JMC)
52

Enabling Java Flight Recorder
Since JFR is a commercial feature, we must unlock commercial features before
trying to run JFR.
So, you need to have following arguments.
-XX:+UnlockCommercialFeatures
-XX:+FlightRecorder
53

Dynamically enabling JFR
If you are using Java 8 update 40 (8u40) or later, you can now dynamically enable
JFR.
This is useful as we don’t need to restart the server.
Sometimes a restart solves the problem anyway. :) But that’s just temporary and
it’s always good to analyze the root cause of the problem.
54

Improving the accuracy of JFR Method Profiler
An important feature of JFR Method Profiler is that it does not require threads to
be at safe points in order for stacks to be sampled.
Generally, the stacks will only be walked at safe points.
HotSpot JVM doesn’t provide metadata for non-safe point parts of the code. Use
following to improve the accuracy.
-XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints
55

JFR Event Settings
There are two event settings by default in Oracle JDK.
Files are in $JAVA_HOME/jre/lib/jfr
1. Continuous - default.jfc
2. Profiling - profile.jfc
56

JFR Recording Types
Time Fixed Recordings
● Fixed duration
● The recording will be opened automatically in JMC at the end (If the recording
was started by JMC)
Continuous Recordings
● No end time
● Must be explicitly dumped
57

Running Java Flight Recorder
There are few ways we can run JFR.
1. Using the JFR plugin in JMC
2. Using the command line
3. Using the Diagnostic Command
58

Running Java Flight Recorder
You can run multiple recordings concurrently and have different settings for each
recording.
However, the JFR runtime will use same buffers and resulting recording contains
the union of all events for all recordings active at that particular time.
This means that we might get more than we asked for. (but not less)
59

Running JFR from JMC
Right click on JVM and select “Start Flight Recording”
Select the type of recording: Time fixed / Continuous
Select the “Event Settings” template
Modify the event options for the selected flight recording template (Optional)
Modify the event details (Optional)
60

Running JFR from Command Line
To produce a Flight Recording from the command line, you can use “-
XX:StartFlightRecording” option. Eg:
-XX:StartFlightRecording=delay=20s,duration=60s,name=Test,fi
lename=recording.jfr,settings=profile
Use following to change log level
-XX:FlightRecorderOptions=loglevel=info
61

The Default Recording (Continuous Recording)
You can also start a continuous recording from the command line using
-XX:FlightRecorderOptions.
-XX:FlightRecorderOptions=defaultrecording=true,disk=true,re
pository=/tmp,maxage=6h,settings=default
Default recording can be dumped on exit. Only the default recording can be used
with the dumponexit and dumponexitpath parameters
-XX:FlightRecorderOptions=defaultrecording=true,dumponexit=t
rue,dumponexitpath=/tmp/dumponexit.jfr
62

Running JFR using Diagnostic Commands
The command “jcmd” can be used.
Start Recording Example:
jcmd <pid> JFR.start delay=20s duration=60s name=MyRecording
filename=/tmp/recording.jfr settings=profile
Check recording
jcmd <pid> JFR.check
Dump Recording
jcmd <pid> JFR.dump filename=/tmp/dump.jfr name=MyRecording
63

Analyzing Flight Recordings
JFR runtime engine dumps recorded data to files with *.jfr extension
These binary files can be viewed from JMC
There are tab groups showing certain aspects of the JVM and the Java application
runtime such as Memory, Threads, I/O etc.
64

JFR Tab Groups
General – Details of the JVM, the system, and the recording.
Memory - Information about memory & garbage collection.
Code - Information about methods, exceptions, compilations, and class loading.
Threads - Information about threads and locks.
I/O: Information about file and socket I/O.
System: Information about environment
Events: Information about the event types in the recording
65

Allocation Profiling
Finding out where the allocations happen in your application.
If there are more allocations, JVM will have to run garbage collection more often
66

Java Just-In-Time (JIT) compiler
67

Java code is usually compiled into platform independent bytecode (class files)
The JVM is able to load the class files and execute the Java bytecode via the Java
interpreter.
Even though this bytecode is usually interpreted, it might also be compiled into
native machine code using the JVM's Just-In-Time (JIT) compiler.
68

Unlike the normal compiler, the JIT compiler compiles the code (bytecode) only
when required. With JIT compiler, the JVM monitors the methods executed by the
interpreter and identifies the “hot methods” for compilation. After identifying the
Java method calls, the JVM compiles the bytecode into a more efficient native
code.
In this way, the JVM can avoid interpreting a method each time during the
execution and thereby improves the runtime performance of the application.
69

JIT Optimization Techniques
● Dead Code Elimination
○ Null Check Elimination
● Branch Prediction
● Loop Unrolling
● Inlining Methods
70

JITWatch
The JITWatch tool can analyze the compilation logs generated with the
“-XX:+LogCompilation” flag.
The logs generated by LogCompilation are XML-based and has lot of information
related to JIT compilation. Hence these files are very large.
https://github.com/AdoptOpenJDK/jitwatch
71

Premature Optimizations
“We should forget about small efficiencies, say
about 97% of the time: premature optimization
is the root of all evil. Yet we should not pass up
our opportunities in that critical 3%."
- Donald Knuth
72
Image is from: http://wiki.c2.com/?DonKnuth

Flame Graphs
Flame graphs are a visualization of profiled software, allowing the most frequent
code-paths to be identified quickly and accurately.
Flame Graphs can be generated using
https://github.com/brendangregg/FlameGraph
This creates an interactive SVG
http://www.brendangregg.com/flamegraphs.html
74

Flame Graph: Definition
The x-axis shows the stack profile population, sorted alphabetically
The y-axis shows stack depth
The top edge shows what is on-CPU, and beneath it is its ancestry
Each rectangle represents a stack frame.
Box width is proportional to the total time a function was profiled directly or its
children were profiled
The colors are usually not significant, picked randomly to differentiate frames.
76

Types of Flame Graphs
CPU
Memory
Off-CPU
Hot/Cold
Differential
77

Flame Graphs with Java Flight Recordings
We can generate CPU Flame Graphs from a Java Flight Recording
Program is available at GitHub: https://github.com/chrishantha/jfr-flame-graph
The program uses the (unsupported) JMC Parser
78

Generating a Flame Graph from a JFR dump
JFR has Method Profiling Samples
You can view those in “Hot Methods” and “Call Tree” tabs
A Flame Graph can be generated using these Method Profilings Samples
79

Profiling a Sample Program
Get Sample “highcpu” program from
https://github.com/chrishantha/sample-java-programs
Get a Profiling Recording
java -XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints
-XX:+UnlockCommercialFeatures -XX:+FlightRecorder
-XX:StartFlightRecording=delay=5s,duration=1m,name=Profiling,filename=highcp
u_profiling.jfr,settings=profile -jar target/highcpu.jar
Using jfr-flame-graph
create_flamegraph.sh -f highcpu_profiling.jfr -i > flamegraph.svg
80

Java Mixed-Mode Flame Graphs
With Java Profilers, we can get information about Java process only.
However with Java Mixed-Mode Flame Graphs, we can see how much CPU time is
spent in Java methods, system libraries and the kernel.
Mixed-mode means that the Flame Graph shows profile information from both
system code paths and Java code paths.
81

Linux Profiling
We can use “perf”, which is a Linux Profiler with performance counters to profile
system code paths.
Linux perf command is also called perf_events
Some perf commands:
perf stat: obtain event counts
perf record: record events for later reporting
perf report: break down events by process, function, etc.
perf top: see live event count
82

Installing “perf_events” on Ubuntu
On terminal, type perf
sudo apt install linux-tools-generic
83

The Problem with Java and Perf
perf needs the Java symbol table
JVM doesn’t preserve frame pointers by default
Run sample program
java -jar target/highcpu.jar --exit-timeout 600
Run perf record
sudo perf record -F 99 -g -p `pgrep -f highcpu`
Display trace output
sudo perf script
84

Preserving Frame Pointers in JVM
Run java program with the JVM flag "-XX:+PreserveFramePointer"
java -XX:+PreserveFramePointer -jar target/highcpu.jar
--exit-timeout 600
This flag is working only on JDK 8 update 60 and above.
85

How to generate Java symbol table
Use a java agent to generate method mappings to use with the linux `perf` tool
Clone & Build https://github.com/jrudolph/perf-map-agent
Create symbol map
./create-java-perf-map.sh `pgrep -f highcpu`
86

Generate Java Mixed Mode Flame Graph
Run perf
sudo perf record -F 99 -g -p `pgrep -f highcpu` -- sleep 60
Create symbol map
Generate Flame Graph
sudo perf script > out.stacks
$FLAMEGRAPH_DIR/stackcollapse-perf.pl out.stacks |
$FLAMEGRAPH_DIR/flamegraph.pl --color=java --hash --width
1680 > java-mixed-mode.svg
87

Java Mixed-Mode Flame Graphs
Helps to understand Java CPU Usage
With Flame Graphs, we can see both java and
system profiles
Can profile GC as well
88

Does profiling matter?
Yes!
Most of the performance issues are in the application code.
Early performance testing is key. Fix problems while developing.
89

Software Profiling: Understanding Java Performance and how to profile in Java

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Software Profiling: Understanding Java Performance and how to profile in Java

Ähnlich wie Software Profiling: Understanding Java Performance and how to profile in Java (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Software Profiling: Understanding Java Performance and how to profile in Java