What is Heap-off memory and how use cache heap off with Apache DirectMemory

1. Heap Off Memory
WTF ??
Reducing Heap memory stress

2. Abstract
* Java memory fundamental
* heap off memory principles
* heap-off cache with Apache DirectMemory

3. /me
Olivier Lamy
* Open Source Architect @Talend Apache Team
* Apache Member : Maven, Archiva, Tomcat,
DirectMemory, Build Infra, etc..
* Jenkins
* Add bugs in various OpenSource projects
* twitter.com/olamy olamy.blogspot.com

4. Java Memory Fundamental

5. Java Memory Fundamental
The cool part
* Automatic memory allocation (do you remember
when you used malloc)
* Garbage collector (GC) (no more free)

6. Java Memory Fundamental
The bad part
* stop-the-world mode
* proportionnal to memory's size
* unpredictable unresponsive application :
complicated with tight SLA

7. Heap Off Memory
Not in the Heap process
Limited GC stop
« Unlimited storage »

8. Heap Off Memory
How to use that ?
Native libraries with JNA
https://github.com/twall/jna
ByteBuffer @since 1.4 with nio
ByteBuffer.allocate( size ) (not off heap)
ByteBuffer.allocateDirect( size )
You can use sun.misc.Unsafe but can cause issue
and not portable

9. Heap Off Memory
How to use that ?
You can manipulate only Buffer (ByteBuffer byte[]
etc..) !
So you must serialize/deserialize your datas
Maximum size with -XX:MaxDirectMemorySize=

10. Our use case : Cache
RAM : 10-60 ns
Network : 10000-30000 ns
SSD Disk : 70000-120000 ns
Disk : 3000000-10000000 ns
Cache must use memory !

11. On-Heap Cache
Objects stored by reference (no
de/serialisation)
GC storm effect when refreshing/removing objects!

12. Off-Heap Cache
de/serialisation overhead (hopefully some very
performant libraries exist for that)
Cache objects payload no more affecting GC

13. Solutions :
* Terracotta BigMemory (off-heap storage on
top of ehcache)
* Infinispan (by Jboss)
* Huge collections
* Apache DirectMemory

14. Apache Direct Memory
Goals :
Apache Direct Memory is a multi layered cache
implementation featuring off-heap memory
storage to enable caching of java objects without
degrading jvm performance.

15. Apache Direct Memory
Joined Apache Incubator end 2011
12 developpers ATM
Under development :
Memory allocation service just rewrite
APIs are subject to be changed and bugs to be
found !

16. Design & principles
ByteBuffer.allocateDirect is the foundation of the
cache
ByteBuffers are allocated in big chunk and
splitted for internal use

17. Design & principles
Build on layers
CachingService : serialize object (pluggable)
MemoryManagerService: Compute
ByteBuffer access
ByteBufferAllocatorService: Eventually deals
with ByteBuffer

18. ByteBuffers Allocation
Merging Strategy
No memory wasted
Suffers from fragmentation
Need synchronization at de/allocation
Fixed Size buffers allocation
Memory wasted if size not correctly configured
No fragmentation

19. Use case (1)
Multi layers cache
Most used objects are cached on heap, the rest
off-heap (maybe overflow to disk)
Sounds like ehcache with BigMemory
Hard coded class to use :
net.sf.ehcache.store.offheap.OffHeapStore
So same package class name in Apache
DirectMemory
(https://jira.terracotta.org/jira/browse/EHC-940)
Demo

20. Use case (2)
Cache Server
« à la » memcache tru http(s)
But with a REST Api and written in Java
Client API available
Demo

21. > PUT /dm/cache/bordeaux HTTP/1.1
> Content-Type:text/plain
{"millesime":"2003","description":"so good so
good"}
< HTTP/1.1 200 OK
< X-DirectMemory-SerializeSize: 58
< Content-Length: 0
> GET /dm/cache/bordeaux HTTP/1.1
> Accept:text/plain
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 51
{"millesime":"2003","description":"so good so
good"}
> DELETE /dm/cache/foo HTTP/1.1

22. Next Steps
JSR 107 ?
Benchmarks
Components Integration (Cassandra, Tomcat etc..)
Dynamic cache size modification
Management/Monitoring

23. Links
http://incubator.apache.org/directmemory/
https://issues.apache.org/jira/browse/DIRECTMEMORY
Demo project : https://bitbucket.org/olamy/heap-off-memory-
wtf
Slides : http://www.slideshare.net/olamy

24. Thanks
&&
Questions