Discussing IBM's experiments in utilizing OMR technologies in MRI

1. Experiments in utilizing OMR technologies in MRI
Charlie Gracie
January 31, 2016
Ruby and OMR

2. Who am I?
Charlie Gracie
charlie_gracie@ca.ibm.com
@crgracie
2
• Garbage Collection Architect
for IBM J9 JVM
• Current focus on developing
technologies for the OMR
project

3. An open source toolkit for
language runtime
technologies.
OMR

4. Open Source Toolkit for Building Runtimes
• Eclipse OMR project proposal
• https://goo.gl/ZTBoeu
• Will be open sourced under EPL
• Derived from the source code of IBM’s
production runtimes
4

5. Open Source Toolkit for Building Runtimes
• Implements language-agnostic parts of a
managed runtime
• Bootstraps development of new runtimes
5

6. Open Source Toolkit for Building Runtimes
• Allows incremental enablement of advanced
functionality
• Consume new features when available
• New platforms
• GPU
• FPGA
6

7. Open Source Toolkit for Building Runtimes
• Ships as part of IBM SDK for Java 8
• Development of IBM SDK for Java 9
consuming OMR daily
• Proof-of-concept integration with Ruby MRI,
CPython, SOM++
7

8. Execution Environment
Language-Agnostic Components
Platform Abstraction Layer
Garbage Collector
Diagnostic Services
Source Code Bytecode/AST
Compiler
8
Just-In-Time Compiler
InterpreterSource Bytecode
Compiler
Interpreter

9. Execution Environment
Language-Agnostic Components
Platform Abstraction Layer
Garbage Collector
Diagnostic Services
Source Code Bytecode/AST
Compiler
9
Just-In-Time Compiler
InterpreterSource Bytecode
Compiler
Interpreter

10. Gluing it Together
Interpreter
10
Bytecode
/AST
Compiler
Garbage
Collector
GC Glue
Just-In-Time
Compiler
JIT Glue
Diagnostic
Services
D.
Glue

11. Ruby + OMR Preview

12. Ruby + OMR Preview
12
• 2 Presentations at Ruby Kaigi 2015
• The OMR GC Talk
• http://goo.gl/mA4IUG
• IBM’s Ruby JIT Talk
• http://goo.gl/K5ZHY0

13. Ruby + OMR Preview
13
• GitHub location for the preview
• https://goo.gl/P3yXuy
• Available as Docker images
• Ruby git clone + OMR changes included
• /home/rubyomr/ruby
• No OMR or GLUE source

14. Ruby + OMR Preview
14
• Currently available on 3 platforms
• Linux on x86
• Linux on Z
• Linux on OpenPOWER

15. Ruby + OMR Preview
15
• Based on Ruby 2.2.3
• Working to merge with the master branch
• Successfully passes “make test”
• Runs rails apps

16. Ruby + OMR Preview
16
• Integrated OMR components include
• Garbage Collection
• Just In Time Compilation
• Diagnostic Tooling

17. Garbage Collection

18. Our
Strategy:
• Be 100% compatible
18

19. Our
Strategy:
• Be 100% compatible
• Decrease pause times
19

20. Our
Strategy:
• Be 100% compatible
• Decrease pause times
• Increase allocation speed
20

21. Our
Strategy:
• Be 100% compatible
• Decrease pause times
• Increase allocation speed
• Improve object locality
21

22. 100% Compatible
22
• Do not break C extensions
• Pass all tests that ship with MRI
• Test 3rd party applications

23. Decrease Pause Times
23
• Use helper threads to:
• Mark roots in parallel
• Use OMR work stack to complete scanning
in parallel
• Perform object finalization in parallel
• Perform sweep operations in parallel

24. Increase Allocation Speed
24
• Use TLH mechanism from OMR
• Threads reserve blocks of heap memory
for exclusive allocation
• Threads bump allocate out of these blocks
alloc(rbthread_t thread, int size)
if (thread->tlhAlloc < thread->tlhTop - size)
object = thread->tlhAlloc
thread->tlhAlloc = object + size
else
object = OMR_GC_Allocate(thread, size);

25. Improve Object Locality
25
• Create new OMRBuffer object type
• Allocate OMRBuffers on heap
• Data is regularly adjacent to object in heap
• OMRBuffers are automatically reclaimed
during collection

26. Decrease Pause Times
26
• OMRBuffers do not require calls to obj_free
• Significant reduction in pause times

27. Future Work
27
• Improve heap fragmentation
• Add support for concurrent marking

28. Just In Time Compilation

29. Our
Strategy:
• Mimic interpreter
for maximum
compatibility.
• Implement
simple opcodes
directly in IL
29
IL

30. Our
Strategy:
• Build callbacks to
VM for complex
opcodes.
• Automatically
generate
callbacks from
the instruction
definition file.
30
IL
callback

31. Our
Strategy:
• Fast-path
particular
patterns
• e.g trace
31
IL
callback
IL
IL
callback

32. Our
Strategy:
• Don’t try to
handle
everything – let
interpreter
handle hard
cases!
32
IL
callback
IL
IL
callback

33. Current Status
• Supports almost all opcodes
• Compile iseq after they are executed N times
• Dispatch to JIT code if the iseq has been
compiled

34. Future Work
34
• Speculative optimizations powered by
decompilation and code-patching
• Class Hierarchy Based Optimization
• Guarded inlining
• Type Specialization
• Recompilation
• Interpreter and JIT Profiling
• Asynchronous Compilation
• More optimization!
• OMR’s Ruby JIT uses only 10 / 100+ optimizations.

35. Diagnostic Tooling

36. Garbage Collection and Memory Visualizer
36
• Provides a graphical details on GC events
post mortem from verbose:gc logs
• Works with IBM JDK, IBM Node.js and Ruby
OMR Preview
• https://goo.gl/YwNrmI

37. Garbage Collection and Memory Visualizer
37

38. Health Center
38
• Provides a live view of runtime details
• Works with IBM JDK, IBM Node.js and Ruby
OMR Preview
• http://goo.gl/u3VITI

39. Health Center – GC Statistics
39

40. Health Center – Ruby Method Profiling
40

41. Performance Results

42. 42
112.84
122.57
0
20
40
60
80
100
120
140
Ruby 2.2.2 Ruby 2.2.2 + OMR
Is Ruby Fast Yet?

43. 43
SpeedupRelativetoInterpreter
Micro Benchmarks
3x
2x
1x

44. 27.8
16.6
5.3
0
5
10
15
20
25
30
Ruby 2.2.2 Ruby 2.2.2 + OMR Ruby 2.2.2 + OMR +JIT
pow(2,n)

45. Future?!?!

46. Future
• More feedback from the community
• Is this work interesting?
• We would like to get more involved with the
Ruby community now that this work is public.
• If there are parts of OMR that the Ruby
community believes would be beneficial we
would like to be part of the Ruby 3x3 plan

47. Possible Ideas For Improvements
• Work on removing the GIL
• Improved scaling
• But how much code would be broken?
• Improved C extension API
• Allow object movement?
• Improve JITs ability to apply optimizations
• Again how much code would be broken?
• Implement key class libraries in Ruby
• Significant JIT improvements
• How much would this hurt short running
applications?

48. Thank you!

49. John Duimovich
CTO IBM Runtimes
duimovic@ca.ibm.com
@jduimovich
Contact Info
Mark Stoodley
OMR Project Lead
mstoodle@ca.ibm.com
@mstoodle
49
Charlie Gracie
OMR GC Architect
crgracie@ca.ibm.com
@crgracie

50. Trademarks, Copyrights, Disclaimers
50
IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without
notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our
general product direction and it should not be relied on in making a purchasing decision. The information
mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver
any material, code or functionality. Information about potential future products may not be incorporated into
any contract. The development, release, and timing of any future features or functionality described for our
products remains at our sole discretion.
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp.,
registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other
companies. A current list of other IBM trademarks is available on the web at "Copyright and trademark information" at
http://www.ibm.com/legal/copytrade.shtml
Other company, product, or service names may be trademarks or service marks of others.
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY.
WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION
CONTAINED IN THIS PRESENTATION, IT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY,
WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY
DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER
DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE
EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR
LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE
USE OF IBM PRODUCTS OR SOFTWARE.
© Copyright International Business Machines Corporation 2015. All rights reserved.

51. Additional Important Disclaimers
• THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL
PURPOSES ONLY.
• WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE
INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY
OF ANY KIND, EXPRESS OR IMPLIED.
• ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A
CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE,
SOFTWARE OR INFRASTRUCTURE DIFFERENCES.
• ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE.
• IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’S CURRENT
PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE.
• IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING
OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER
DOCUMENTATION.
• NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF:
• - CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS
OR THEIR SUPPLIERS AND/OR LICENSORS
51