Empfohlen
Weitere ähnliche Inhalte
Ähnlich wie Memory and runtime analysis
Ähnlich wie Memory and runtime analysis (10)
Mehr von adm_exoplatform
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Memory and runtime analysis
- 1. Click to edit the outline text format Java Performance Second Outline Level − Third Outline Level Memory and Runtime Analysis format Click to edit the outline text Fourth Outline Level Second Outline Outline Level − Fifth Level − Third − Sixth Outline Level Outline Level − Seventh Outline Level Fourth Outline Level − Eighth Outline Level Viet Nguyen, eXo CT − Fifth Ninth Outline LevelClick toOutline Level name, title edit presenter − Sixth Outline Level Click to edit the outline text format March 2012 − Seventh Outline Level Second Outline Level − Eighth Outline Level − Third Outline Level Ninth Outline LevelClick to edit subtitle Fourth Outline Level − Fifth Outline Level − Sixth Outline Level − Seventh Outline Level − Eighth Outline Level
- 2. Agenda 1 − Introduction the outline text format Click to edit − Java Memory Model Second Outline Level − Heap dump − Third Outline Level − 2 Java Memorythe outline text format Problems Click to editFourth Outline Level − Common problems − − Memory Leak Fifth Outline Level Second Outline Level − Sixth Outline Level − Third Outline Level − Seventh Outline Level 3 − Analysing Heap Dumps with Eclipse Memory Analyzer (MAT) Click to editFourth Outline Level the outline text format − Eighth Outline Level − Shallow & Retained Size, Dominator Tree − Fifth Outline Level − Second Outline Level item Ninth Outline LevelAgendaMAT 1 − Analysing Heap dumps with − − Second level Sixth Outline Level − Third Outline Level − Seventh Outline Level 4 − − Java runtimeFourth Outline Level profiling − JvisualVM & − − Eighth Outline Level JProfiler Fifth Outline Level Ninth Outline LevelAgenda item 2 − eXo JCR statistic tool − − Second level Sixth Outline Level − Seventh Outline Level − Eighth Outline Level − Ninth Outline LevelAgenda item 3 eXo Platform www.exoplatform.com - Copyright 2012 2
- 3. Code Performance Questions − Click to edit the outline text Did you choose the right data structure ? − format Are you using the right sorting algorithm ? − Is your recursive method TOO recursive ? Outline Level Second − Are you throwing away a computation that could prove useful − Third Outline Level later ? − Are you creating too many objects unnecessarily or otherwise Fourth Outline Level wasting memory ? − Fifth Outline Level − Sixth Outline Level − Seventh Outline Level − Eighth Outline Level − Ninth Outline LevelClick to edit first level • Second level • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 3
- 4. Optimization Metrics − Runtime / CPU usage Click to edit the outline text format − What line of code the program is spending the most time in − What call / invocation paths were used to get to these lines Second Outline Level − Third Outline Level − Memory usage Fourth Outline Level − What kind of objects are sitting on the heap − Fifth Outline − Where were they allocated Level − Who is pointing to them now − Sixth Outline − “Memory leak” Level − Seventh Outline Level − Eighth Outline Level − Ninth Outline LevelClick to edit first level • Second level • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 4
- 5. Java's Memory Model Click to edit the outline text format Second Outline Level − Third Outline Level Fourth Outline Level − Fifth Outline Level − Sixth Outline Level − Seventh Outline Level − Eighth Outline Level − Ninth Outline LevelClick to edit first level • Second level • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 5
- 6. Java's Memory Model 2 Click to edit the outline text format Second Outline Level − Third Outline Level Fourth Outline Level − Fifth Outline Level − Sixth Outline Level − Seventh Outline Level − Eighth Outline Level − Ninth Outline LevelClick to edit first level • Second level • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 6
- 7. The JVM Heaps freelist Native Heap Click to edit the outline text Size Java Heap format Next Second OutlinefreeLevel storage Size Next − Third Outline Level Null free storage Fourth Outline Level JIT Compiled Code − Fifth Outline Free List Level Motif structures − Sixth Outline ‘ Level Thread Stacks Xms - Active Area of Heap − Seventh Outline Level Buffers − Eighth Outline Level − Ninth Outline LevelClick to edit first level • Second level • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 7
- 8. The “Correct” Java Heap Size • GC will adapt heap size to keep occupancy between Click to edit the outline text 40% and 70% format –Heap occupancy over 70% causes frequent GC cycles Second Outline Level • Which generally means reduced performance − –Heap occupancy below 40% means Third OutlineGC cycles, infrequent Level but cycles longer than they needs to beFourth Outline Level • Which means longer pause times that necessary − Fifth Outline • Which generally means reduced performance Level Sixth Outline − Level • The maximum heap size setting should therefore be − Seventh Outline 43% larger than the maximum occupancy of the application Level − Eighth Outline –Eg. For 70MB occupancy, 100MB Max heap is Level required (which is 70MB Ninth Outline LevelClick to edit − + 43% of 70MB) first level • Second level • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 8
- 9. Binary Heap Dump A heap dump is a snapshot of objects that are alive at one point in time. It contains: IOIOII IOIIOI OIOIO IIOIIOI Objects: fields, references, primitive values, … Classes: class loader, super class, static fields, … IOIOII A heap dump does not contain IOIIOI OIOIO IIOIIOI where the objects have been created which objects have been garbage collected www.exoplatform.com - Copyright 2012 eXo Platform 9
- 10. Heap Dumps Are Useful for Analysis − Heap dumps come “for free” Click to edit the outline text − They are suitable for production format − The comprehensive data allows detail analysis Second Outline Level − There is a wide platform coverage − Third − HPROF dumps from HotSpot based JVMs Outline Level Fourth Outline Level − DTFJ and PHD dumps from IBM JVMs − Fifth Outline Level − Sixth Outline Level − Seventh Outline Level − Eighth Outline Level − Ninth Outline LevelClick to edit first level • Second level • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 10
- 11. How To Get A Heap Dump Non-Interactive -XX:+HeapDumpOnOutOfMemoryError On Demand JDK1.4.2_12 and -XX:+HeapDumpOnCtrlBreak JDK6 and Jconsole More… http://wiki.eclipse.org/index.php/MemoryAnalyzer#Getting_a_Heap_Dump www.exoplatform.com - Copyright 2012 eXo Platform 11
- 12. Java Memory Problems Common to edit the outline text format Click problems Memory Leak Second Outline Level − Third Outline Level Fourth Outline Level − Fifth Outline Level − Sixth Outline Level − Seventh Outline Level − Eighth Outline Level Ninth Outline LevelClick to edit subtitle
- 13. Java Memory Problems − Memory leaks − Unnecessarily high memory usage Click to edit the outline text • Referencing objects that are no format • Implementations consuming too longer used much memory • Developer forget clear them • Second Outline Level Large amount of state information • Unbound or inefficiently configured − Third Outline Level caches − Inefficient object creation • Large temporary objects: bigLevel Fourth Outline • GC must constantly clean up the − Fifth Outline documents (XML, PDF, images...) heap are read and processed Level • High CPU consumption by GC − Sixth Outline • Application response times Level − Inefficient garbage collector increases behaviour − Seventh Outline Level • Missing or wrong configuration of GC − Eighth Outline Level − Ninth Outline LevelClick to edit first level • Second level • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 13
- 14. Most Common Memory Leaks − Thread Local Variables − Mutable static fields and text Click to edit the outline • ThreadLocals are often used to track collections format state • Most common reason • Threads are often never garbage Second Outline Level • Statics often used to hold caches or collected shareThird Outline Level − states • Memory leaks may happen if thread • Memoryleak happens easily if this is local variable is not clean carefully not done diligently Outline Level Fourth − Fifth Outline Level − Circular and complex bi- − Wrong implementation of − Sixth Outline directional references equals/hashcode Level • All objects will not be garbage • Lead to memory leaks when used as − collected a key in a map Seventh Outline Level − Eighth Outline Level − Ninth Outline LevelClick to edit first level • Second level • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 14
- 15. Most Common Memory Leaks − Classloader Leaks − JNI memory leaks Click to edit the outline text • Classes are referenced by their format is used to call native code • JNI class loader • Native code can handle, call, or • They will not get garbage collected create javaOutline Level Second objects until classloader itself is collected • These objects are referenced until − Third Outline Level • First leak form: an object whose the native method returns class belongs to the class loader is • Fourth Outline Level First leak form: native method run still referenced by a cache, a thread forever − Fifth Outline local … • Level Second leak form: You keep created • Second leak form: Some objects after − Sixth Outline native code has ended frameworks allow dynamic creation of new classes. If they are forgotten, Level PermGen leaks happen − Seventh Outline Level − Eighth Outline Level − Ninth Outline LevelClick to edit first level • Second level • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 15
- 16. How to interpret the Memory Leak Analysis Results Click to edit the outline text What is leaking? format – What is the object (e.g. a HashMap) holding all the leaking objects i.e. leak container. Second Outline Level − Third Outline Level – What are the objects getting added to the leak container i.e. leak unit. Fourth Outline Level − Fifth Outline – Who is holding the leak container in memory? What are the object types and package names of objects on the chain of Level references − Sixth from a root object to the leak container i.e. owner chain. Outline Level − Seventh Outline Level − Eighth Outline Level − Ninth Outline LevelClick to edit first level • Second level • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 16
- 17. View of the leaking data structure Significant entities Leak Root Click to edit the outline text MyClass format Owner – An owner chain Chain Container Second HashSet Outline Level – A leak root − Third Outline Level – A container Leaking HashMap Fourth Outline Level Unit – The unit of the leak − Fifth Outline HM$Entry Level HM$Entry HM$Entry – Leak contents − Sixth Outline Level String String String − Seventh Outline Level Contents Char[] Char[] Char[] − Eighth Outline Level − Ninth Outline LevelClick to edit first level • Second level • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 17
- 18. Memory Leak Example public class MyClass { Leak Click to edit the outline text Root format static HashSet myContainer = new HashSet(); Second Outline Level public void leak(int numObjects) { − Third Outline Level Leak Container for (int i = 0; i < numObjects; ++i) { Fourth Outline Level String leakingUnit = new String("this is leaking object: " + i); − Fifth Outline myContainer.add(leakingUnit); Level } − Sixth Outline } Leaking public static void main(String[] args) throws Exception { Level Units − Seventh Outline { Level MyClass myObj = new MyClass(); − Eighth Outline myObj.leak(1000000); // 1 million Level } − Ninth Outline LevelClick to edit System.gc(); first level } • Second level } • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 18
- 19. Analyzing Heap dumps with MAT Retained to edit the outline text format Click size, Dominator Tree MATfeatures Demos Second Outline Level − Third Outline Level Fourth Outline Level − Fifth Outline Level − Sixth Outline Level − Seventh Outline Level − Eighth Outline Level Ninth Outline LevelClick to edit subtitle
- 20. Tools For Analysing Heap Dumps Click to edit the outline text format Second Outline Level − Third Outline Level Fourth Outline Level − Fifth Outline Level − Sixth Outline Level − Seventh Outline Level − Eighth Outline Level − Ninth Outline LevelClick to edit first level • Second level • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 20
- 21. Eclipse Memory Analyzer Introduction The number of memory-related problems is very high They are extremely difficult to analyze Analysis requires expertise in the analyzed code Eclipse Memory Analyzer: Simplifies memory analysis Extensible Free for download @ www.eclipse.org/mat www.exoplatform.com - Copyright 2012 eXo Platform 21
- 22. A Little Bit of History Problem #1: multi-GB heap dumps from 64-bit machines Quickly open and re-open a heap dump on 32-bit machines Problem #2: millions of objects. Where is the „needle“? Dominator Tree: fast calculation of retained memory Meaningful names for class loaders: see components SQL-like language: create custom queries Problem #3: complexity of analysis Reports on memory leak suspects and checks for known antipatterns Problem #4: HPROF heap dumps only Support for IBM system dumps www.exoplatform.com - Copyright 2012 eXo Platform 22
- 23. Shallow And Retained Size Shallow heap is the memory consumed by one object Retained set of X is the set of objects that will be garbage collected if X is garbage collected Retained heap of X is the sum of shallow sizes of all objects in the retained set of X, i.e. memory kept alive by X Set of elements Retained Set C C, F, G, J K K C, K C, F, G, J, K, I www.exoplatform.com - Copyright 2012 eXo Platform 23
- 24. Dominator Tree The Dominator Tree is a Transformation of the Cyclic Object Graph into a “Keep-Alive” Tree: Every node in the Tree is directly responsible for keeping alive its children Object X dominates object Y if all paths from the roots to Y run through X Benefits: Fast calculation of the retained size (sum all children) List of distinct big objects (first level of the tree) Fast identification of responsible objects (just go up the tree) www.exoplatform.com - Copyright 2012 eXo Platform 24
- 25. MAT Features Memory Leak Hunter Automatically detect memory leak suspects Discover if the issue is known (and a fix available) Collect details for in depth analysis by the code experts www.exoplatform.com - Copyright 2012 eXo Platform 25
- 26. MAT Report Overview One big object: memory leak suspect Any up-to-date architecture loads components with separate class loaders, be it OSGi or JEE application servers. Extensible to display meaningful names. Search by keywords: identify if problem is known Classification for trouble ticket system: less ping- pong of trouble tickets. www.exoplatform.com - Copyright 2012 eXo Platform 26
- 27. MAT Report Details The chain of objects and references which keep the suspect alive A significant drop in the retained sizes shows the accumulation point Accumulated objects www.exoplatform.com - Copyright 2012 eXo Platform 27
- 28. MAT Features Top Components Report Analyze components, that occupy more memory, than a certain threshold Give hints where memory footprint can be optimized Check for known antipatterns www.exoplatform.com - Copyright 2012 eXo Platform 28
- 29. MAT Report Details www.exoplatform.com - Copyright 2012 eXo Platform 29
- 30. MAT Report Details www.exoplatform.com - Copyright 2012 eXo Platform 30
- 31. How to Analyze Memory Leaks − Find the biggest objects Click to edit the outline text − Analyze why they are kept in memory format − Analyze what make them big Second Outline Level − Third Outline Level Use MAT To Analyze Leaks Fourth Outline Level Fifth Outline − Level − Get a “good” heapdump − Sixth Outline − Find the biggest objects (in the dominator tree) Level − Analyze who kept them alive (using paths) − Seventh Outline − Analyze what made them big (looking at their retain set) Level − Use “Leak report” to automate the analysis − Eighth Outline Level − Ninth Outline LevelClick to edit first level • Second level • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 31
- 32. How to Analyze a Heavy Request − Look at the thread attributes – name, class, etc... Click to edit the outline text − Look at the local variables format − Look at the stack of the thread Second Outline Level − Third Outline Level Use MAT To Analyze Thread Activity Fourth Outline Level Fifth Outline − Level − Inspect threads attributes and local variables (in object − Sixth Outline explorer) Level − Analyze stack trace − Seventh Outline Level − Eighth Outline Level − Ninth Outline LevelClick to edit first level • Second level • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 32
- 33. Find Where Memory Footprint Can Be Optimized − Inspect the set of retained objectsClick to edit the outline text − Search for inefficiently used data format structures − Look for redundant data Second Outline Level − Third Outline Level Use MAT To Analyze Memory FootprintFourth Outline Level Fifth Outline − Level − Analyze retained objects (in dominator tree, retained set) − Sixth Outline − Use “Group by Value” to find redundant objects Level − Use the commands from the “Collections” group − Seventh Outline − Use “Component Report” to automate the analysis Level − Eighth Outline Level − Ninth Outline LevelClick to edit first level • Second level • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 33
- 34. Java Runtime Profiling JVisualVM edit the outline text format Click to JProfiler eXo JCR statistic tool Level Second Outline − Third Outline Level Fourth Outline Level − Fifth Outline Level − Sixth Outline Level − Seventh Outline Level − Eighth Outline Level Ninth Outline LevelClick to edit subtitle
- 35. What to do with profiler results − Observe which methods are being callededit the outline text Click to the most format − These may not necessarily be the “slowest” methods! Second Outline Level − Observe which methods are taking the most time relative to − Third Outline Level others Fourth Outline Level − Common problem: inefficient unbuffered I/O − Fifth Outline − Common problem: poor choice of data structure Level − Common problem: recursion call overhead − Sixth Outline − Common problem: unnecessary re-computation of expensive Level information, or unnecessary multiple I/O of same data − Seventh Outline Level − Eighth Outline Level − Ninth Outline LevelClick to edit first level • Second level • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 35
- 36. Q&A Click to edit the outline text format Second Outline Level − Third Outline Level http://kohlerm.blogspot.com/search/label/memory Fourth Outline Level http://visualvm.java.net/docindex.html − Fifth Outline Level http://docs.exoplatform.org/PLF35/index.jsp?topic=/org.exoplatform.do − Sixth Outline Level − Seventh Outline Level − Eighth Outline Level Ninth Outline LevelClick to edit subtitle
- 37. How will this session do − Present common memory & thread-related issues Click to edit the outline text − Give hints how to solve them using heap dumps and Eclipse format Memory Analyzer Second Outline Level − Show demos using real-life heap dumps − Third Outline − Runtime Java profiling with JvisualVM, Jprofiler, eXoLevel JCR statistics tool (exo.jcr.component.statistics) Fourth Outline Level − Fifth Outline Level − Sixth Outline Level − Seventh Outline Level − Eighth Outline Level − Ninth Outline LevelClick to edit first level • Second level • Third level www.exoplatform.com - Copyright 2012 eXo Platform • Fourth level 37
Hinweis der Redaktion
- How will this session do Present common memory & thread-related issues Give hints how to solve them using heap dumps and Eclipse Memory Analyzer Show demos using real-life heap dumps Runtime Java profiling with JvisualVM, Jprofiler, eXo JCR statistics tool (exo.jcr.component.statistics)
- http://blog.dynatrace.com/2009/08/13/java-memory-problems/ http://blog.dynatrace.com/2011/04/20/the-top-java-memory-problems-part-1/ http://blog.dynatrace.com/2011/12/15/the-top-java-memory-problems-part-2/ Memory Leaks and other memory related problems are among the most prominent performance and scalability problems in Java. Reason enough to discuss this topic in more detail. The Java memory model- or more specifically the garbage collector – has solved many memory problems. At the same time new ones have been created. Especially in J2EE Environments with a large number of parallel users, memory is more and more becoming a critical ressource. In times with cheap memory available, 64bit JVMs and modern garbage collection algorithms this might sound strange at first sight. So let us now take a closer look at Java memory problems. Problems can be categorized into four: groups Memory leaks in Java are created by referencing objects that are no longer used. This easily happens when multiple references to objects exist and developer forget to clear them, when the object is no longer needed. Unnecessarily high memory usage caused by implementations consuming to much memory. This is very often a problem in web applications where a large amount of state information is managed for “user comfort”. When the number of active users increases, memory limits are reached very fast. Unbound or inefficiently configured caches are another source of constant high memory usage. Inefficient object creation easily results in a performance problem when user load increases, as the garbage collector must constantly clean up the heap. This leads to unnecessarily high CPU consumption by the garbage collector. As the CPU is blocked by garbage collection, application response times increases often already under moderate load. This behaviour is also referred to as GC trashing. Inefficient garbage collector behaviour is caused by missing or wrong configuration of the garbage collector. The garbage collector will take care that object are cleaned up. How and when this should happen must however by configured by the programmer or system architect. Very often people simple “forget” to properly configure and tune the garbage collecotr. I was involved in a number of performance workshops where a “simple” parameter change resulted in a performance improvement of up to 25 percent. In most cases memory problems affect not only performance but also scalability. The higher the amount of consumed memory per request, user or session the less parallel transactions can be executed. In some cases memory problems also affect availabilty. When the JVM runs out of memory or it is close to memory limits it will quit with an OutOfMemory error. This is when management enters your office and you know you are in serious trouble. Memory problems are often difficult to resolve for two reasons: In some case analysis will get complex and difficult – especially if you are missing the right methodology to resolve them. Secondly their foundation is often in the architecture of the application. Simple code changes will not help to resolve them. In order to make life easier I present a couple of memory antipatterns which are often found in real world world applications. Those patterns should help to be able to already avoid memory problems during development.
- http://blog.dynatrace.com/2009/08/13/java-memory-problems/ http://blog.dynatrace.com/2011/04/20/the-top-java-memory-problems-part-1/ http://blog.dynatrace.com/2011/12/15/the-top-java-memory-problems-part-2/ Memory Leaks and other memory related problems are among the most prominent performance and scalability problems in Java. Reason enough to discuss this topic in more detail. The Java memory model- or more specifically the garbage collector – has solved many memory problems. At the same time new ones have been created. Especially in J2EE Environments with a large number of parallel users, memory is more and more becoming a critical ressource. In times with cheap memory available, 64bit JVMs and modern garbage collection algorithms this might sound strange at first sight. So let us now take a closer look at Java memory problems. Problems can be categorized into four: groups Memory leaks in Java are created by referencing objects that are no longer used. This easily happens when multiple references to objects exist and developer forget to clear them, when the object is no longer needed. Unnecessarily high memory usage caused by implementations consuming to much memory. This is very often a problem in web applications where a large amount of state information is managed for “user comfort”. When the number of active users increases, memory limits are reached very fast. Unbound or inefficiently configured caches are another source of constant high memory usage. Inefficient object creation easily results in a performance problem when user load increases, as the garbage collector must constantly clean up the heap. This leads to unnecessarily high CPU consumption by the garbage collector. As the CPU is blocked by garbage collection, application response times increases often already under moderate load. This behaviour is also referred to as GC trashing. Inefficient garbage collector behaviour is caused by missing or wrong configuration of the garbage collector. The garbage collector will take care that object are cleaned up. How and when this should happen must however by configured by the programmer or system architect. Very often people simple “forget” to properly configure and tune the garbage collecotr. I was involved in a number of performance workshops where a “simple” parameter change resulted in a performance improvement of up to 25 percent. In most cases memory problems affect not only performance but also scalability. The higher the amount of consumed memory per request, user or session the less parallel transactions can be executed. In some cases memory problems also affect availabilty. When the JVM runs out of memory or it is close to memory limits it will quit with an OutOfMemory error. This is when management enters your office and you know you are in serious trouble. Memory problems are often difficult to resolve for two reasons: In some case analysis will get complex and difficult – especially if you are missing the right methodology to resolve them. Secondly their foundation is often in the architecture of the application. Simple code changes will not help to resolve them. In order to make life easier I present a couple of memory antipatterns which are often found in real world world applications. Those patterns should help to be able to already avoid memory problems during development.
- http://blog.dynatrace.com/2011/04/20/the-top-java-memory-problems-part-1/ Thread Local Variables ThreadLocals are used to bind a variable or a state to a thread. Each thread has its own instance of the variable. They are very useful but also very dangerous. They are often used to track a state, like the current transaction id, but sometimes they hold a little more information. A thread local variable is referenced by its thread and as such its lifecycle is bound to it. In most application servers threads are reused via thread pools and thus are never garbage collected. If the application code is not carefully clearing the thread local variable you get a nasty memory leak. These kind of memory leaks can easily be discovered with a heap dump. Just take a look at the ThreadLocalMap in the heap dump and follow the references. The heapdump shows that over 4K objects which amount to about 10MB are held by thread locals You can then also look at the name of the thread to figure out which part of your application is responsible for the leak. Mutable static fields and collections The most common reason for a memory leak is the wrong usage of statics. A static variable is held by its class and subsequently by its classloader. While a class can be garbage collected it will seldom happen during an applications lifetime. Very often statics are used to hold cache information or share state across threads. If this is not done diligently it is very easy to get a memory leak. Especially static mutable collections should be avoided at all costs for just that reason. A good architectural rule is not to use mutable static objects at all, most of the time there is a better alternative. Circular and complex bi-directional references This is my favorite memory leak. It is best explained by example: org.w3c.dom.Document doc = readXmlDocument(); org.w3c.dom.Node child = doc.getDocumentElement().getFirstChild(); doc.removeNode(child); doc = null; At the end of the code snippet we would think that the DOM Document will be garbage collected. That is however not the case. A DOM Node object always belongs to a Document. Even when removed from the Document the Node Object still has a reference to its owning document. As long as we keep the child object the document and all other nodes it contains will not be garbage collected. I’ve see this and other similar issues quite often. Wrong implementation of equals/hashcode It might not be obvious on the first glance, but if your equals/hashcode methods violate the equals contract it will lead to memory leaks when used as a key in a map. A HashMap uses the hashcode to lookup an object and verify that it found it by using the equals method. If two objects are equal they must have the same hashcode, but not the other way around. If you do not explicitly implement hashcode yourself this is not the case. The default hashcode is based on object identity. Thus using an object without a valid hashcode implementation as a key in a map, you will be able to add things but you will not find them anymore. Even worse if you re-add it, it will not overwrite the old item but really add a new one. And just like that you have memory leak. You will find it easily enough as it is growing, but the root cause will be hard to determine unless you remember this one. The easiest way to avoid this is to use unit testcases and one of the available frameworks that tests the equals contract of your classes (e.g. http://code.google.com/p/equalsverifier/).
- Classloader Leaks When thinking about memory leaks we think mostly about normal java objects. Especially in application servers and OSGi containers there is another form of memory leak, the class loader leak. Classes are referenced by their classloader and normally they will not get garbage collected until the classloader itself is collected. That however only happens when the application gets unloaded by the application server or OSGi container. There are two forms of Classloader Leaks that I can describe off the top of my head. In the first an object whose class belongs to the class loader is still referenced by a cache, a thread local or some other means. In that case the whole class loader, meaning the whole application cannot be garbage collected. This is something that happens quite a lot in OSGi containers nowadays and used to happen in JEE Application Servers frequently as well. As it is only happens when the application gets unloaded or redeployed it does not happen very often. The second form is nastier and was introduced by bytecode manipulation frameworks like BCEL and ASM. These frameworks allow the dynamic creation of new classes. If you follow that thought you will realize that now classes, just like objects, can be forgotten by the developer. The responsible code might create new classes for the same purpose multiple times. As the class is referenced in the current class loader you get a memory leak that will lead to an out of memory in the permanent generation. The real bad news is that most heap analyzer tools do not point out this problem either, we have to analyze it manually, the hard way. This form or memory leak became famous due to an issue in an older version of hibernate and its usage of CGLIB. JNI memory leaks This is a particularly nasty form or memory leak. It is not easily found, unless you have the right tool and it is also not known to a lot of people. JNI is used to call native code from java. This native code can handle, call and also create java objects. Every java object created in a native method begins its life as a so called local reference. That means that the object is referenced until the native method returns. We could say the native method references the java object. So you don’t have a problem unless the native method runs forever. In some cases you want to keep the created object even after the native call has ended. To achieve this you can either ensure that it is referenced by some other java object or you can change the local reference into a global reference. A global reference is a GC root and will never be garbage collected until explicitly deleted by the native code. The only way to discover such a memory leak is to use a heap dump tool that explicitly shows global native references. If you have to use JNI you should rather make sure that you reference these objects normally and forgo global references altogether. You can find this sort of leak when your heap dump analysis tool explicitly marks the GC Root as a native reference, otherwise you will have a hard time.
- Classloader Leaks When thinking about memory leaks we think mostly about normal java objects. Especially in application servers and OSGi containers there is another form of memory leak, the class loader leak. Classes are referenced by their classloader and normally they will not get garbage collected until the classloader itself is collected. That however only happens when the application gets unloaded by the application server or OSGi container. There are two forms of Classloader Leaks that I can describe off the top of my head. In the first an object whose class belongs to the class loader is still referenced by a cache, a thread local or some other means. In that case the whole class loader, meaning the whole application cannot be garbage collected. This is something that happens quite a lot in OSGi containers nowadays and used to happen in JEE Application Servers frequently as well. As it is only happens when the application gets unloaded or redeployed it does not happen very often. The second form is nastier and was introduced by bytecode manipulation frameworks like BCEL and ASM. These frameworks allow the dynamic creation of new classes. If you follow that thought you will realize that now classes, just like objects, can be forgotten by the developer. The responsible code might create new classes for the same purpose multiple times. As the class is referenced in the current class loader you get a memory leak that will lead to an out of memory in the permanent generation. The real bad news is that most heap analyzer tools do not point out this problem either, we have to analyze it manually, the hard way. This form or memory leak became famous due to an issue in an older version of hibernate and its usage of CGLIB. JNI memory leaks This is a particularly nasty form or memory leak. It is not easily found, unless you have the right tool and it is also not known to a lot of people. JNI is used to call native code from java. This native code can handle, call and also create java objects. Every java object created in a native method begins its life as a so called local reference. That means that the object is referenced until the native method returns. We could say the native method references the java object. So you don’t have a problem unless the native method runs forever. In some cases you want to keep the created object even after the native call has ended. To achieve this you can either ensure that it is referenced by some other java object or you can change the local reference into a global reference. A global reference is a GC root and will never be garbage collected until explicitly deleted by the native code. The only way to discover such a memory leak is to use a heap dump tool that explicitly shows global native references. If you have to use JNI you should rather make sure that you reference these objects normally and forgo global references altogether. You can find this sort of leak when your heap dump analysis tool explicitly marks the GC Root as a native reference, otherwise you will have a hard time.
- Classloader Leaks When thinking about memory leaks we think mostly about normal java objects. Especially in application servers and OSGi containers there is another form of memory leak, the class loader leak. Classes are referenced by their classloader and normally they will not get garbage collected until the classloader itself is collected. That however only happens when the application gets unloaded by the application server or OSGi container. There are two forms of Classloader Leaks that I can describe off the top of my head. In the first an object whose class belongs to the class loader is still referenced by a cache, a thread local or some other means. In that case the whole class loader, meaning the whole application cannot be garbage collected. This is something that happens quite a lot in OSGi containers nowadays and used to happen in JEE Application Servers frequently as well. As it is only happens when the application gets unloaded or redeployed it does not happen very often. The second form is nastier and was introduced by bytecode manipulation frameworks like BCEL and ASM. These frameworks allow the dynamic creation of new classes. If you follow that thought you will realize that now classes, just like objects, can be forgotten by the developer. The responsible code might create new classes for the same purpose multiple times. As the class is referenced in the current class loader you get a memory leak that will lead to an out of memory in the permanent generation. The real bad news is that most heap analyzer tools do not point out this problem either, we have to analyze it manually, the hard way. This form or memory leak became famous due to an issue in an older version of hibernate and its usage of CGLIB. JNI memory leaks This is a particularly nasty form or memory leak. It is not easily found, unless you have the right tool and it is also not known to a lot of people. JNI is used to call native code from java. This native code can handle, call and also create java objects. Every java object created in a native method begins its life as a so called local reference. That means that the object is referenced until the native method returns. We could say the native method references the java object. So you don’t have a problem unless the native method runs forever. In some cases you want to keep the created object even after the native call has ended. To achieve this you can either ensure that it is referenced by some other java object or you can change the local reference into a global reference. A global reference is a GC root and will never be garbage collected until explicitly deleted by the native code. The only way to discover such a memory leak is to use a heap dump tool that explicitly shows global native references. If you have to use JNI you should rather make sure that you reference these objects normally and forgo global references altogether. You can find this sort of leak when your heap dump analysis tool explicitly marks the GC Root as a native reference, otherwise you will have a hard time.
- Classloader Leaks When thinking about memory leaks we think mostly about normal java objects. Especially in application servers and OSGi containers there is another form of memory leak, the class loader leak. Classes are referenced by their classloader and normally they will not get garbage collected until the classloader itself is collected. That however only happens when the application gets unloaded by the application server or OSGi container. There are two forms of Classloader Leaks that I can describe off the top of my head. In the first an object whose class belongs to the class loader is still referenced by a cache, a thread local or some other means. In that case the whole class loader, meaning the whole application cannot be garbage collected. This is something that happens quite a lot in OSGi containers nowadays and used to happen in JEE Application Servers frequently as well. As it is only happens when the application gets unloaded or redeployed it does not happen very often. The second form is nastier and was introduced by bytecode manipulation frameworks like BCEL and ASM. These frameworks allow the dynamic creation of new classes. If you follow that thought you will realize that now classes, just like objects, can be forgotten by the developer. The responsible code might create new classes for the same purpose multiple times. As the class is referenced in the current class loader you get a memory leak that will lead to an out of memory in the permanent generation. The real bad news is that most heap analyzer tools do not point out this problem either, we have to analyze it manually, the hard way. This form or memory leak became famous due to an issue in an older version of hibernate and its usage of CGLIB. JNI memory leaks This is a particularly nasty form or memory leak. It is not easily found, unless you have the right tool and it is also not known to a lot of people. JNI is used to call native code from java. This native code can handle, call and also create java objects. Every java object created in a native method begins its life as a so called local reference. That means that the object is referenced until the native method returns. We could say the native method references the java object. So you don’t have a problem unless the native method runs forever. In some cases you want to keep the created object even after the native call has ended. To achieve this you can either ensure that it is referenced by some other java object or you can change the local reference into a global reference. A global reference is a GC root and will never be garbage collected until explicitly deleted by the native code. The only way to discover such a memory leak is to use a heap dump tool that explicitly shows global native references. If you have to use JNI you should rather make sure that you reference these objects normally and forgo global references altogether. You can find this sort of leak when your heap dump analysis tool explicitly marks the GC Root as a native reference, otherwise you will have a hard time.
- Classloader Leaks When thinking about memory leaks we think mostly about normal java objects. Especially in application servers and OSGi containers there is another form of memory leak, the class loader leak. Classes are referenced by their classloader and normally they will not get garbage collected until the classloader itself is collected. That however only happens when the application gets unloaded by the application server or OSGi container. There are two forms of Classloader Leaks that I can describe off the top of my head. In the first an object whose class belongs to the class loader is still referenced by a cache, a thread local or some other means. In that case the whole class loader, meaning the whole application cannot be garbage collected. This is something that happens quite a lot in OSGi containers nowadays and used to happen in JEE Application Servers frequently as well. As it is only happens when the application gets unloaded or redeployed it does not happen very often. The second form is nastier and was introduced by bytecode manipulation frameworks like BCEL and ASM. These frameworks allow the dynamic creation of new classes. If you follow that thought you will realize that now classes, just like objects, can be forgotten by the developer. The responsible code might create new classes for the same purpose multiple times. As the class is referenced in the current class loader you get a memory leak that will lead to an out of memory in the permanent generation. The real bad news is that most heap analyzer tools do not point out this problem either, we have to analyze it manually, the hard way. This form or memory leak became famous due to an issue in an older version of hibernate and its usage of CGLIB. JNI memory leaks This is a particularly nasty form or memory leak. It is not easily found, unless you have the right tool and it is also not known to a lot of people. JNI is used to call native code from java. This native code can handle, call and also create java objects. Every java object created in a native method begins its life as a so called local reference. That means that the object is referenced until the native method returns. We could say the native method references the java object. So you don’t have a problem unless the native method runs forever. In some cases you want to keep the created object even after the native call has ended. To achieve this you can either ensure that it is referenced by some other java object or you can change the local reference into a global reference. A global reference is a GC root and will never be garbage collected until explicitly deleted by the native code. The only way to discover such a memory leak is to use a heap dump tool that explicitly shows global native references. If you have to use JNI you should rather make sure that you reference these objects normally and forgo global references altogether. You can find this sort of leak when your heap dump analysis tool explicitly marks the GC Root as a native reference, otherwise you will have a hard time.
- Classloader Leaks When thinking about memory leaks we think mostly about normal java objects. Especially in application servers and OSGi containers there is another form of memory leak, the class loader leak. Classes are referenced by their classloader and normally they will not get garbage collected until the classloader itself is collected. That however only happens when the application gets unloaded by the application server or OSGi container. There are two forms of Classloader Leaks that I can describe off the top of my head. In the first an object whose class belongs to the class loader is still referenced by a cache, a thread local or some other means. In that case the whole class loader, meaning the whole application cannot be garbage collected. This is something that happens quite a lot in OSGi containers nowadays and used to happen in JEE Application Servers frequently as well. As it is only happens when the application gets unloaded or redeployed it does not happen very often. The second form is nastier and was introduced by bytecode manipulation frameworks like BCEL and ASM. These frameworks allow the dynamic creation of new classes. If you follow that thought you will realize that now classes, just like objects, can be forgotten by the developer. The responsible code might create new classes for the same purpose multiple times. As the class is referenced in the current class loader you get a memory leak that will lead to an out of memory in the permanent generation. The real bad news is that most heap analyzer tools do not point out this problem either, we have to analyze it manually, the hard way. This form or memory leak became famous due to an issue in an older version of hibernate and its usage of CGLIB. JNI memory leaks This is a particularly nasty form or memory leak. It is not easily found, unless you have the right tool and it is also not known to a lot of people. JNI is used to call native code from java. This native code can handle, call and also create java objects. Every java object created in a native method begins its life as a so called local reference. That means that the object is referenced until the native method returns. We could say the native method references the java object. So you don’t have a problem unless the native method runs forever. In some cases you want to keep the created object even after the native call has ended. To achieve this you can either ensure that it is referenced by some other java object or you can change the local reference into a global reference. A global reference is a GC root and will never be garbage collected until explicitly deleted by the native code. The only way to discover such a memory leak is to use a heap dump tool that explicitly shows global native references. If you have to use JNI you should rather make sure that you reference these objects normally and forgo global references altogether. You can find this sort of leak when your heap dump analysis tool explicitly marks the GC Root as a native reference, otherwise you will have a hard time.
- Classloader Leaks When thinking about memory leaks we think mostly about normal java objects. Especially in application servers and OSGi containers there is another form of memory leak, the class loader leak. Classes are referenced by their classloader and normally they will not get garbage collected until the classloader itself is collected. That however only happens when the application gets unloaded by the application server or OSGi container. There are two forms of Classloader Leaks that I can describe off the top of my head. In the first an object whose class belongs to the class loader is still referenced by a cache, a thread local or some other means. In that case the whole class loader, meaning the whole application cannot be garbage collected. This is something that happens quite a lot in OSGi containers nowadays and used to happen in JEE Application Servers frequently as well. As it is only happens when the application gets unloaded or redeployed it does not happen very often. The second form is nastier and was introduced by bytecode manipulation frameworks like BCEL and ASM. These frameworks allow the dynamic creation of new classes. If you follow that thought you will realize that now classes, just like objects, can be forgotten by the developer. The responsible code might create new classes for the same purpose multiple times. As the class is referenced in the current class loader you get a memory leak that will lead to an out of memory in the permanent generation. The real bad news is that most heap analyzer tools do not point out this problem either, we have to analyze it manually, the hard way. This form or memory leak became famous due to an issue in an older version of hibernate and its usage of CGLIB. JNI memory leaks This is a particularly nasty form or memory leak. It is not easily found, unless you have the right tool and it is also not known to a lot of people. JNI is used to call native code from java. This native code can handle, call and also create java objects. Every java object created in a native method begins its life as a so called local reference. That means that the object is referenced until the native method returns. We could say the native method references the java object. So you don’t have a problem unless the native method runs forever. In some cases you want to keep the created object even after the native call has ended. To achieve this you can either ensure that it is referenced by some other java object or you can change the local reference into a global reference. A global reference is a GC root and will never be garbage collected until explicitly deleted by the native code. The only way to discover such a memory leak is to use a heap dump tool that explicitly shows global native references. If you have to use JNI you should rather make sure that you reference these objects normally and forgo global references altogether. You can find this sort of leak when your heap dump analysis tool explicitly marks the GC Root as a native reference, otherwise you will have a hard time.