Today higher heaps and bigger RAM amounts are typical for standard Java server applications. We often then get to the limit of performance, predictability or GC pause times and using Off-Heap technologies we can lower that pains.

1. www.hazelcast.com

2. WHO AM I
Christoph Engelbert(@noctarius2k)
8+ years of professionalJavadevelopment
Specialized to performance, GC, traffic
topics
Apache DirectMemoryPMC
Previous companies incl. Ubisoftand HRS
OfficialInhabitantof Hazelcastia
CastMapRMapReduce for Hazelcast3
Co-Author JavaOff-heap JEP Proposal
www.hazelcast.com

3. TOPICS
JavaMemoryLayout
Definition of Off-Heap?
DirectMemoryAccess?
Whydo we need Off-Heap Memory?
Options for Off-Heap Storage
Advantages and Disadvantages of (Off-
)Heap
Available Frameworks
(Notso) SecretFuture
Smalldemonstration
www.hazelcast.com

4. JAVA MEMORY LAYOUT
A FAST WALK-THROUGH
www.hazelcast.com

5. JAVA MEMORY LAYOUT (1/5)
www.hazelcast.com

6. JAVA MEMORY LAYOUT (2/5)
YOUNG GENERATION - EDEN SPACE
Onlyused for Object-Allocation
TLAB¹ allocation available
Affected byMinor and Major GC
Objects moved to Survivor on Minor GC
¹TLAB:ThreadLocalAllocationBuffer
www.hazelcast.com

7. JAVA MEMORY LAYOUT (3/5)
YOUNG GENERATION - SURVIVOR SPACES
Always one From-Space and one To-Space
From-Space contains livingobjects on lastGC
Minor GC switches the Survivor Spaces
Alive Objects are moved to To-Space
Affected byMinor and Major GC
Longlivingobjects eventuallymoved to Tenured
www.hazelcast.com

8. JAVA MEMORY LAYOUT (4/5)
TENURED SPACE
Contains onlylonglivingobjects
Affected onlybyMajor GC
More objects means more GC spend time
Concurrentobjectinspectation available
LongStop-The-World pauses possible
Notoptimalfor bigdatasets
www.hazelcast.com

9. JAVA MEMORY LAYOUT (5/5)
PERMGEN / META SPACE
Class bytecodes
Class metadata
Runtime information
Code Compilation Cache
etc.
www.hazelcast.com

10. G1 (GARBAGE FIRST COLLECTOR)
GenerationalGC
Heap splitted into same sized regions
Everyregion is either Eden, Survivor or Tenured space
www.hazelcast.com

11. DEFINITION OF
OFF-HEAP?
www.hazelcast.com

12. Off-Heap is a(continuously) self allocated,
managed and freed directmemoryregion.
Itis notunder controlof the JavaGarbageCollector and needs
custom written allocation and cleanup of data-regions.
Off-Heap can notbe used for allocation of Javaobjects.
DEFINITION OF OFF-HEAP?
www.hazelcast.com

13. I'M DUKE SKYWALKER
I'M HERE TO RESCUE YOU!
www.hazelcast.com

14. DIRECT MEMORY
ACCESS?
GIVE YOURSELF TO THE DARK SIDE
www.hazelcast.com

15. DIRECT MEMORY ACCESS?
ARE YOU SERIOUS?
Fastmemoryarea
Notaffectingthe GC
Officiallyavailable since Java1.4
Serialization overhead when storingobjects
Limited by-XX:MaxDirectMemorySize
www.hazelcast.com

16. WHY DO WE NEED
OFF-HEAP MEMORY?
www.hazelcast.com

17. WHY DO WE NEED OFF-HEAP MEMORY?
Storingof huge datasets
Zero-Copywrites to channels
Lower pressure on GC /less GC pauses
Storage Space onlylimited byRAM
Compactdatarepresentation possible
IPC SharedMemorywith Memory-Mapped-Files
www.hazelcast.com

18. OPTIONS FOR
OFF-HEAP STORAGE
www.hazelcast.com

19. OPTIONS FOR OFF-HEAP STORAGE (1/3)
JNI (Java Native Interface)
int*buffer=(int*)malloc(1024* sizeof(int));
if(buffer==NULL)throwOutOfMemoryException();
for(inti=0;i<1024;i++)
buffer[sizeof(int)*i]=i;
free(buffer);
www.hazelcast.com

20. OPTIONS FOR OFF-HEAP STORAGE (2/3)
(Direct)ByteBuffer
ByteBufferbuffer=ByteBuffer.allocateDirect(1024*4);
for(inti=0;i<1024;i++)
buffer.putInt(i);
//bufferisautomaticallyfreedbyGC
www.hazelcast.com

21. OPTIONS FOR OFF-HEAP STORAGE (3/3)
sun.misc.Unsafe
Unsafeunsafe=trickToRetrieveUnsafe();
longaddress=unsafe.allocateMemory(1024*4);
for(inti=0;i<1024;i++)
unsafe.putInt(address+4*i,i);
unsafe.freeMemory(address);
www.hazelcast.com

22. ADVANTAGES
AND
DISADVANTAGES
OF (OFF-)HEAP
www.hazelcast.com

23. ADVANTAGES ON-HEAP
No messingwith malloc
Automatic Garbage Collection
No need for Serialization
www.hazelcast.com

24. DISADVANTAGES ON-HEAP
Slowdown on high allocation rate
Bigoverhead on smallobjects (header)
Heavilydependingon GC Combination
www.hazelcast.com

25. ADVANTAGES OFF-HEAP
No limitation*in size
Compactdatalayout
Zero-Copysocketread/write possible
Frameworks available to manage memory
www.hazelcast.com

26. DISADVANTAGES OFF-HEAP
Allocation is up to you
Deallocation is up to you
Datalayoutis up to you
Serialization maybe required
JNI requires native library
www.hazelcast.com

27. AVAILABLE
FRAMEWORKS
www.hazelcast.com

28. AVAILABLE FRAMEWORKS (1/5)
Apache DirectMemory
TerracottaBigMemory
HazelcastElasticMemory
MapDB.org
etc.
www.hazelcast.com

29. AVAILABLE FRAMEWORKS (2/5)
Apache DirectMemory
CacheService<String,String>cacheService=newDirectMemory<...>()
.setNumberOfBuffers(10).newCacheService();
cacheService.put("MyKey","SomeValue");
Stringvalue=cacheService.get("MyKey");
www.hazelcast.com

30. AVAILABLE FRAMEWORKS (3/5)
Terracotta BigMemory
<ehcachexml:noNamespaceSchemaLocation="..."
name="MyCacheDefinition">
<cachename="MyOffheapCache"maxBytesLocalOffHeap="2G"/>
</ehcache>
CacheManagercacheManager=newCacheManager();
CachedataStore=cacheManager.get("MyOffheapCache");
Elementelement=newElement("MyKey","SomeValue");
dataStore.put(element);
Stringvalue=(String)dataStore.get("MyKey").getObjectValue();
www.hazelcast.com

31. AVAILABLE FRAMEWORKS (4/5)
Hazelcast Offheap
<hazelcastxml:noNamespaceSchemaLocation="...">
<mapname="MyMap">
<storage-type>OFFHEAP</storage-type>
</map>
</hazelcast>
HazelcastInstancehz=Hazelcast.newInstance();
//ReturnsaIMapextendsConcurrentMap
Map<String,String>map=hz.getMap("MyMap");
map.put("MyKey","SomeValue");
Stringvalue=map.get("MyKey");
www.hazelcast.com

32. AVAILABLE FRAMEWORKS (5/5)
MapDB.org
DBdb=DBMaker.newDirectMemoryDB().sizeLimit(2).make();
//ReturnsaHTreeMapextendsConcurrentMap
Map<String,String>map=db.createHashMap("cache").make();
map.put("MyKey","SomeValue");
Stringvalue=map.get("MyKey");
www.hazelcast.com

33. FURTHER INFORMATION AVAILABLE
The Must-Read for Off-Heap
http://bit.ly/must-read-off-heap
PacketObjectDescription from IBM
http://bit.ly/packet-objects
CompactDatastructures (Martin Thompson)
http://bit.ly/compact-datastructures
C++ Like Performance for Serialization
http://bit.ly/serialization-performance
Tricks with DirectMemoryAccess in Java
http://bit.ly/direct-memory-tricks
www.hazelcast.com

34. THE
OFF-HEAP JEP PROPOSAL
MAY THE FORCE BE WITH YOU.
THE FORCE IS STRONG WITH THIS ONE.
www.hazelcast.com

35. OFF-HEAP JEP PROPOSAL
ByteBuffer like interface
Fully64-bitsizes and offsets
Compare-And-Swap operations
Volatile and ordered operations
Optionalboundingchecks
Supports Memorymapping
SupportForeign Function Interface (FFI) JEP
www.hazelcast.com

36. OFF-HEAP JEP PROPOSAL - CODE EXAMPLE
DISCLAIMER: PROVISIONAL API
importjavax.direct.*;
BytesFactoryfactory=createBytesFactory();
Bytesbytes=factory.boundsChecking(false).deallocationChecks(false)
.freeOnGC(false).create(ByteOrder.LITTLE_ENDIAN,1024*4);
for(inti=0;i<1024;i++)
bytes.putVolatileInt(i);
bytes.release();
www.hazelcast.com

37. OFF-HEAP JEP PROPOSAL
ProposalDiscussion Group
http://bit.ly/offheap-ml
ProposalText
http://bit.ly/proposal-text
FFI JEP 191
http://bit.ly/ffi-jep
www.hazelcast.com

38. @noctarius2k
@hazelcast
http://www.sourceprojects.com
http://github.com/noctarius
THANK YOU!
ANY QUESTIONS?
Images:www.clipartist.info,GnomeNebulaTheme,KDEtheme,www.grabsteine-klnt.de
www.hazelcast.com